One Repo to Rule Them All

5 min read —

— Sep 18, 2015

This week it's been a hot topic of discussion about how Google has (almost) all of their code within a single, massive, 2-billion lines of code repository.

There was a quote by Justin Searls, which was most definitely him joking around, but it got me thinking about how we share code at theScore.

There's the common misconception that you either have a "Monolith" or series of "Microservices". Or as somebody else put it on Twitter: "A swarm of loosely coupled nanoservices".

Microliths.
— Justin Searls (@searls) September 17, 2015

We certainly have a number of repos (iOS app, Android app, website, data and API, user accounts and authentication API, etc...), but the "data and API" repo in particular has a number of related Rails apps along with some local gems.

Sharing code across Rails apps

The repository I mentioned has 4 different Rails apps inside of it. It has an app for the API, an app for an internal admin panel, an app for processing incoming data, and an app for handling push alerts.

All of these apps share 2 gems which are also found inside the same repository: a gem for the "data" layer (basically the models and objects for interacting with the database), and a support gem which has a few utility classes that are shared, along with some things like lists of country codes and whatnot.

The repository is structured like this:

data_processor/ (rails app)
admin/ (rails app)
api/ (rails app)
pusher/ (rails app)
thescore-data/ (gem)
thescore-support/ (gem)
test (script)

Each of the 4 Rails apps can have their own gems and dependencies, two of which being the local data and support gems.

# Gemfile
gem 'the_score_support', path: '../thescore-support'
gem 'the_score_data', path: '../thescore-data'

Decreasing coupling without increasing complexity

Monoliths are great in certain respects: They allow you to have all of your code inside a single app. If you need to "communicate" to another part of the system, you can do so via a method call (fast, simple) rather than by communicating over HTTP with a JSON request (slow, complex). There is also no need to attempt to version your API... there is a single version and that is the app itself.

But there are obvious drawbacks as well: You can have seemingly unrelated things all packed into the same place (the API along with the admin panel for example), which both may require a number of gems that the other doesn't need.

Microservices aim to solve some of the problems brought on by a monolith, but end up causing an increase in complexity by having to coordinate APIs across the different services, versioning them and deploying them at the same time.

By combining related "Microservices" into a single repository, you end up getting the benefits from both approaches, but without the drawbacks that each one provides.

Benefits from testing

By having all of your apps in a single service you are also able to easily test all of your applications together. You can either run them individually, because each one has their own set of tests, or you can write a simple script which will test each app along with each gem, so that you know that your changes to the data gem hasn't broken anything in one of the Rails apps.

#!/bin/bash

export RAILS_ENV=test
export  RACK_ENV=test

trap exit 1 SIGINT SIGTERM

# The BASE_DIR is the absolute path of the directory of this script file
BASE_DIR=$(cd $(dirname ${BASH_SOURCE[0]}) && pwd)
FAILED=0

for project in admin api thescore-data pusher thescore-support data_processor; do
  cd $BASE_DIR/$project
  echo "Running specs for $project with seed $SEED"

  bin/rspec

  if [ $? -ne 0 ]; then
    FAILED=1
  fi
done

exit $FAILED

Better yet, you can set up your continuous integration tools to tets your entire project each time you push to the repository, helping to guarantee that all apps in this repo are functioning correctly.

A single pull request

Another benefit of having everything within a single repo is that only a single pull request is needed for a change which might span multiple apps and/or gems.

If you've ever had to coordinate multiple PRs which must be approved, merged, and deployed in sync, you know how difficult and annoying it can be. It's not just difficult for you though, it's difficult for whoever is having to review the PR as well. The reviewers need to be made aware that there are multiple PRs in different repositories, and keep that in mind when reviewing each one individually.

Combining apps and gems into a single repo helps to solve this, and PRs become a lot easier to manage.

Final thoughts

You don't have to go as far as Google, where everything your organization done is in a single repository. But by combining separate apps in a single repository, you're able to get some of the advantages of having Microservices while not throwing out all the advantages of a Monolith. Or, as Justin Searls said, Microliths.