Jbuilder to AMS - A tested journey

Rails 5 is coming and ships with the --api option. This will generate a fresh but slimmed down Rails app whose job is to serve as a JSON API. With this option you'll get Active Model Serializers - AMS installed by default. This is a great library that is only getting better with all of the work that the AMS is doing on it.

Not everyone is creating a new Rails app though... most of us have existing ones, which may use RABL, Jbuilder, or a number of other options to generate JSON responses.

Knock knock, who's there? Legacy code.

Let's say we have a JSON API written in Jbuilder (or RABL) and we want to switch it over to using AMS... what's the first step? And what about after that? In this article I'll be walking us through converting our app to use AMS through a series of steps to make sure that the actual JSON response doesn't change.

Our App

Our app for this demo will consist of 3 models: League, Team, Player. We'll be looking at the teams#show endpoint which includes information about the team, the league it is apart of, along with a list of that team's players.

Our Team model looks like this:

class Team < ActiveRecord::Base
  belongs_to :league
  has_many :players
end

Our controller is pretty bare bones right now but it does the trick:

class TeamsController < ApplicationController
  def show
    @team = Team.find(params[:id])
  end
end

Now that we've got the model and controller out of the way, let's visit the path /teams/1 to take a look at the JSON that is produced:

{
  "id": 1,
  "name": "Toronto FC",
  "updated_at": "2015-11-27T19:01:49.846Z",
  "players": [
    {
      "id": 1,
      "name": "Sebastian Giovinco",
      "position": "Forward",
      "nickname": "Atomic Ant",
      "jersey_number": "10",
      "nationality": "Italian",
      "updated_at": "2015-11-27T19:01:49.852Z"
    },
    {
      "id": 2,
      "name": "Sebastian Giovinco",
      "position": "Forward",
      "nickname": "Atomic Ant",
      "jersey_number": "10",
      "nationality": "Italian",
      "updated_at": "2015-11-27T19:01:49.853Z"
    }
  ],
  "league": {
    "id": 1,
    "name": "MLS",
    "slug": "mls",
    "updated_at": "2015-11-27T19:01:49.835Z"
  }
}

Yes there are 2 Giovincos... it's a test app! Here is what our current Jbuilder view looks like to generate that response:

json.(@team, :id, :name, :updated_at)

json.league do
  json.id @team.league.id
  json.name @team.league.name
  json.slug @team.league.slug
  json.updated_at @team.league.updated_at
end

json.players @team.players do |player|
  json.(player, :id, :name, :nationality, :nickname, :jersey_number, :position, :updated_at)
end

Testing the endpoints

There will be some serious refactoring involved to migrate from Jbuilder to AMS.

Code refactoring is the process of restructuring existing computer code - changing the factoring - without changing its external behavior.

According to that definition that I took off of Wikipedia, refactoring involves changing the code, but not the external behavour. So what is the external behaviour in the case of an endpoint? It is the JSON that the endpoint returns. You could also argue that the speed in which it generates that JSON is also its external behaviour, but for today we'll be focusing on just making sure the JSON remains the same.

To do this we'll write some tests! There are many ways to ensure that our JSON responses remain the same, but this is a similar approach to how we've been testing our endpoints at theScore. What we'll do is hit the endpoint, record the result, and from then on test to ensure that what the API is returning is equal to the saved response we have.

Our test will create a few database objects, call the endpoint, and then verify the response:

require 'rails_helper'

RSpec.describe TeamsController, type: :request do
  it 'returns correct output for single team' do
    team = create(:team)
    create(:player, team: team)
    create(:player, team: team)

    verify_endpoint "/teams/#{team.id}"
  end
end

The verify_endpoint method isn't one that comes with rspec... it comes from a small module I wrote to help automate recording the endpoint responses and comparing pre-recorded responses with the current response.

module RequestHelpers
  FILTERED_FIELDS = ['updated_at'].freeze
  CAPTURE_BASE_PATH = 'spec/captures'.freeze

  def verify_endpoint(path)
    get path
    raw_body = response.body
    raw_expected = read_or_write_capture(path, raw_body)
    compare_response(raw_body, raw_expected)
  end

  private

  def read_or_write_capture(path, raw_body)
    file_name = "#{path.parameterize}.json"
    begin
      read_capture(file_name)
    rescue Errno::ENOENT
      write_capture(file_name, raw_body)
    end
  end

  def read_capture(file_name)
    File.read(File.join(CAPTURE_BASE_PATH, file_name))
  end

  def write_capture(file_name, raw_body)
    parsed_body = JSON.parse(raw_body)
    JSON.pretty_generate(parsed_body).tap do |pretty_json|
      File.write(File.join(CAPTURE_BASE_PATH, file_name), pretty_json)
    end
  end

  def compare_response(raw_body, raw_expected)
    parsed_body = filter(JSON.parse(raw_body))
    parsed_expected = filter(JSON.parse(raw_expected))
    expect(parsed_body).to eq(parsed_expected)
  end

  # Remove filtered fields and order keys to make comparisons easier
  def filter(hash)
    hash.keys.sort.each_with_object({}) do |key, hsh|
      value = hash[key]

      hsh[key] = if value.is_a?(Hash)
        filter(value)
      elsif value.is_a?(Array)
        value.map do |elem|
          filter(elem)
        end
      elsif FILTERED_FIELDS.include?(key)
        :filtered
      else
        value
      end
    end
  end
end

The most complicated method here is the filter method, whose job is basically to filter out fields which will end up changing from test to test, like the updated_at field.

Just to make sure I'm explaining all the code in this example, I'm using FactoryGirl to generate the data used in this test. The factories look like this:

FactoryGirl.define do
  factory :league do
    name 'MLS'
    slug 'mls'
  end
end

FactoryGirl.define do
  factory :player do
    name 'Sebastian Giovinco'
    nickname 'Atomic Ant'
    jersey_number '10'
    nationality 'Italian'
    position 'Forward'
    team
  end
end

FactoryGirl.define do
  factory :team do
    name 'Toronto FC'
    league
  end
end

Time for AMS

Now that we can verify the responses that our endpoints give, we can create AMS classes. Here are the 3 serializers I have, which should produce the same JSON result.

class LeagueSerializer < ActiveModel::Serializer
  attributes :id,
    :name,
    :updated_at
end

class PlayerSerializer < ActiveModel::Serializer
  attributes :id,
    :name,
    :position,
    :nickname,
    :jersey_number,
    :nationality,
    :updated_at
end

class TeamSerializer < ActiveModel::Serializer
  attributes :id,
    :name,
    :updated_at
  has_many :players
  belongs_to :league
end

We'll have to make a small change to the controller so that it uses the AMS classes instead of attempting to find a view.

class TeamsController < ApplicationController
  def show
    @team = Team.find(params[:id])
    render json: @team, serializer: TeamSerializer
  end
end

Now it's time to run the test bundle exec rspec, and to verify that they continue to pass after making our change. It's easy to forget a field... so let's leave one out on purpose just to make sure that the tests are working as expected. Plus there is something quite satisfying about making the test turn from red to green.

 -"league" => {"id"=>1, "name"=>"MLS", "slug"=>"mls", "updated_at"=>:filtered},
 +"league" => {"id"=>1, "name"=>"MLS", "updated_at"=>:filtered},

We can see that we forgot the slug field on the league... so we'll add that to the LeagueSerializer and now the test is passing again.

Preparing for failure

We now basically have 2 working ways to generate the JSON in our API... we will eventually get rid of the Jbuilder code but for now let's leave it. If AMS code gives us problems that we didn't plan for (performance for example), all it takes is changing our controller to switch back to the previous version while we work out the kinks. After it's been running in production for a few days and you're sure there are no issues, feel free to go back and remove the Jbuilder view that is no longer needed.

Concluding thoughts

In this post we changed our code from using Jbuilder to AMS, but that wasn't really the main point. The main point I wanted to get across was both how to test your JSON endpoints, but also how to approach code refactoring in general. Without tests, how do you know the code you refactored still works as expected?

You can get the entire source code used in this article at https://github.com/leighhalliday/soccer_app