Setting up CI for a Multi-Engine Rails 3.2 Application

I recently setup continuous integration (via CircleCI, which I’m really impressed with). It ended up being a bit more tricky for a couple of reasons:

• I needed to pull semi-static seed data into the application. For this app, this was product data that isn’t changed often and is pulled from an external system. I wanted to use seed data from the production database because the integrity of the data in the external DB couldn’t be trusted and I wanted to ensure that if the tests passed it was a real representation of if the app would function in the live environment.
• There were a number of custom rails engines that app was compromised of. Each of these engines had separate unit tests, but not separate integration (feature) tests. The view layer had too many dependencies on the host app to separate out the feature tests.
• The migrations for each of the engines were inside the engine folder using this trick.
• I wanted the ability to visually debug feature test via the selenium capybara driver, but I wanted the speed of `capybara-webkit`
• For whatever reason, the `seed_dump` and `seed_dumper` gem was not working on this 3.2.x app
• There is a bunch of persistent app-wide configuration settings I needed to set via a rake task

Use SQL Dumps for Seed Data

After not being able to get seed_dump extracting data from the development database, I decided on using SQL dumps for seed data. Here are the benefits:

• It’s easy to update the data, just run a DB table export
• The relationships between various models are maintained (this was very important for this specific project, the eCommerce product model was fairly complex)
• Not dependent on a gem to update the data in the future

The engineer in me doesn’t like the fact that running tests are dependent on non-factory test data, but when dealing with an external data source that has data integrity issues, it’s the most robust way to test that your app will hold up in the wild.

I found a seeds.rb SQL import snippet and expanded upon it to dynamically load in any SQL files in db/seeds/:

unless Rails.env.production?
  # country and state seed data
  Spree::Core::Engine.load_seed if defined?(Spree::Core)

  # https://github.com/spree/spree_auth_devise/blob/1-3-stable/db/default/users.rb
  ENV['AUTO_ACCEPT'] = 'true'
  Spree::Auth::Engine.load_seed if defined?(Spree::Auth)

  # don't export table structures or schema_migrations table; that is handled by load:schema

  Dir[File.join(Rails.root, "db/seeds/*.sql")].each do |sql_file|
    sql = File.read(sql_file)

    puts "Reading seed: #{sql_file}"

    connection = ActiveRecord::Base.connection

    # AR can only manually execute one statement at a time
    statements = sql.split(/;$/)
    statements.pop

    ActiveRecord::Base.transaction do
      statements.each do |statement|
        connection.execute(statement)
      end
    end
  end

  # nil out the user generated paper clip attribute
  Spree::Author.all.each do |a|
    a.update_attribute :photo, nil
  end
end

Rake Task Execution Order for Database Setup

I had a couple custom rake tasks I needed to run to setup my test environment. However, some of those rake tasks required the database schema to be setup for some of rails accessors that are determined by the database schema to be created.

I found that running multiple rake tasks in a single rake command does not trigger a model refresh if the DB schema changes. Run the rake task in a separate bash command to pick up the updated schema:

rake db:drop db:create db:schema:load --trace RAILS_ENV=test
rake db:seed custom:configure RAILS_ENV=test --trace

Add Engine Migration Paths to ActiveRecord::Migrator

I’ve been adding the db/migrate path in my child engines to my parent engine in order to keep my codebase DRY.

With the following snippet in place running db:schema:load would trigger migration errors. The database schema/creation logic wasn’t picking the migrations inside the engines although db:migrate ran them just fine.

After some digging I found the magical configuration tweak to fix the issue:

initializer :append_migrations do |app|
  unless app.root.to_s.match root.to_s
    app.config.paths["db/migrate"] += config.paths["db/migrate"].expanded
    ActiveRecord::Migrator.migrations_paths += config.paths["db/migrate"].expanded
  end
end

That’s it for now, stay tuned for part 2!