The buzz cut

Ramblings from the barbershop

A Month With Vim: Introduction

| Comments

For years, I coded in Perl, PHP, and Java using Pico/Nano. I was pretty damn productive in those days, but simple refactoring was tedious and time consuming. When I shifted to Ruby, I started using TextMate, and eventually Sublime Text. Both provided huge improvements to my programming happiness and overall productivity. Fast forward 5-6 years and now, I consider myself a fairly advanced Sublime Text user. I know the majority of the shortcuts, leverage plenty of plugins, and can move around with ease. I don’t NEED it — I could still go back to Nano, if necessary — but it has certainly made me a better programmer.

Lately, the Vim community has been conspiring against me. Everywhere I turn, there’s another developer I respect gushing over Vim. I see daily references to it in screencasts, conference talks, and blog posts. What’s all the hype about? Will keeping my fingers on the home row, pecking at a new key combinations, provide me similar productivity gains to my Nano –> SublimeText shift?

It’s time to find out.

Vim has a steep enough learning curve that it’s impossible to do a fair assessment in a week, so I’ve decided to commit myself to it, as my only text editor, for the month of July. I’m documenting the experience for others who are starting at zero.

Read more about my first week with Vim

Welcome, Matt Brand

| Comments

Today, I’m excited to welcome Matt Brand to Barbershop Labs.

I’ve known Matt for nearly 20 years, and have worked with him on multiple occasions. Matt brings backend and frontend expertise, a strong understanding of UX, battle-tested experience, and instant culture to the company. He has taken many products from conception, to launch, to exit; all while never breaking a sweat.

When I founded Barbershop Labs, I knew that when I was ready to grow, I’d be knocking on his door. I’m glad that he answered.

Upgrading to Rails4: Encrypted Cookies, Observers, Et Al.

| Comments

The ‘activerecord-session_store’ gem provides support for those storing sessions in the database. My app stores sessions in cookies, so I removed that gem from my Gemfile. That said, Rails 4 introduces an encrypted cookie store, which I want to use. The first thing I had to do was edit config/initializers/session_store.rb and change this:

1
2
3
4
Screencast::Application.config.session_store
  :cookie_store,
  key: '_screencast_session',
  domain: :all

To this:

1
2
3
4
Screencast::Application.config.session_store
  :upgrade_signature_to_encryption_cookie_store,
  key: '_screencast_session',
  domain: :all

Then I ran:

1
2
$ rake secret
xxxyyy1234...

Finally, I edited config/intializers/secret_token.rb and added a config for secret_key_base, using the hash that I had just generated. Note, leave the secret_token config alone.

1
2
Screencast::Application.config.secret_token = '111222333leavemealone'
Screencast::Application.config.secret_key_base = 'xxxyyy1234...'

Another one down! We’re in the home stretch as the rest of the backward compatability gems you’re probably not using.

The ‘rails-observers’ gem allows Rails 4 apps to still use Observers, which have been phased out. I don’t use observers in this app, so I can safely remove this gem. For those using observers, the recommendation is to move your observed code into model callbacks (i.e., after_save)

Next up was the ‘actionview-encoded_mail_to’ gem. This provides backward compatability for those who used the “encode” option when using the “mail_to” helper. I never did that, and I bet you didn’t either, which is why the functionality was removed from Rails 4 and extracted to this gem. Remove the gem — another one down!

Next up was the ‘rails-perftest’ gem. Performance tests were removed from Rails 4 and extracted to this gem. If you don’t need them, you don’t need this gem. Remove it and move on. Another one down!

Finally, the ‘actionpack-xml_parser’ gem simply extracted XML parsing into its own gem. Again, my app didn’t need this, so I removed the gem.

Woo! All backward compatability gems have been addressed. I’ve converted all of my code and have removed all of them, including the ‘rails4_upgrade’ gem.

That’s it. I’m officially running a Rail4 upgraded app!

Upgrading to Rails4: Strong Parameters

| Comments

This is another post in a long series of Rails4 upgrades. At this point, I have a working Rails4 app that’s using all of the backward comptability gems. This post will be on removing the ‘protected_attributes’ gem and updating my code to use Rails 4’s strong parameters logic

As you likely know, Rails 4 changed how we protect our apps from mass assignment vulnerabilities. Previously we used “attr_accessor” in the model, but with Rails 4, we should use strong_parameters in the controller. The protected_attributes gem provides backward compatability, allowing attr_accessor to still work, but who wants to live in the past?

I went through all of my models and made changes like this:

1
2
3
4
5
6
7
8
9
10
# Plan.rb

# We no longer need the following line. Delete it
# attr_accessible :name, :description

# The next line is temporary. It allows us to change
# one model at a time before removing the protected_attributes
# gem completely. Remove it when all models/controllers
# have been updated.
include ActiveModel::ForbiddenAttributesProtection

Then I tweaked the corresponding controller

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# PlansController

def create
  ...
  # @plan = Plan.new(params[:plan]) ## remove this line
  @plan = Plan.new(plan_params)
  ...
end

def update
  ...
  # if @plan.update_attributes(params[:plan]) ## remove this line
  if @plan.update_attributes(plan_params)
  ...
end

def plan_params
  params.require(:plan).permit(:name, :description)
end

Once I had tested all of my changes, I removed the protected_attributes gem and removed the “include ActiveModel::ForbiddenAttributesProtection” from all of my models.

One down!

Upgrading to Rails4: Active_resource & Cache_digests

| Comments

Working my way down the list of backward-compatability gems, I came to to:

1
gem 'activddderesource', github: 'rails/activeresource'

This one was easy. I don’t use it, and it’s likely that you don’t either. I simply removed this gem from my Gemfile. Two down!

The next two gems were caching-related:

1
2
gem 'actionpack-action_caching', github: 'rails/actionpack-action_caching'
gem 'actionpack-page_caching', github: 'rails/actionpack-page_caching'

Rails 4 deprecated both page & action caching. Fragment caching is the preferred caching technique now, and fragment caching has even been improved with “Russion Doll” caching.

Luckily, my app wasn’t using any action or page caching, so I removed the action-caching and page-caching gems, but I did have some fragment caching that needed updating. This was easy enough as it’s just a matter of removing those pesky version numbers from cache blocks. Cache busting happens automagically now. So, I updated code like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# app/views/shows/show.html.erb
# app/views/shows/show.html.erb

<% cache ["v2", @show] do %>;
  <h1>;<%= @show.title %>;</h1>;
  ...
  <ul>;
    <%= render @show.episodes %>;
  </ul>;
  ...
<% end %>;

# app/views/shows/_episode.html.erb
<% cache ["v2", @episode] do %>;
  ...
  <li class="episode">;
    <%= @episode.title %>;
    ...
  </li>;
  ...
<% end %>;

To something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# app/views/shows/show.html.erb
<% cache @show do %>;
  <h1>;<%= @show.title %>;</h1>;
  ...
  <ul>;
    <%= render @show.episodes %>;
  </ul>;
  ...
<% end %>;

# app/views/shows/_episode.html.erb
<% cache @episode do %>;
  <li class="episode">;
    ...
    <%= @episode.title %>;
    ...
  </li>;
<% end %>;

Next up, tackling the remaining gems:

1
2
3
4
5
gem 'activerecord-session_store'
gem 'rails-observers'
gem 'actionview-encoded_mail_to'
gem 'rails-perftest'
gem 'actionpack-xml_parser', github: 'rails/actionpack-xml_parser'

Upgrading to Rails 4.0: Binstubs

| Comments

This will be one of many followup posts to my Upgrading Rails 3.2 to Rails 4.0 post. This post assumes a working Rails 4 app, albeit one that’s still using the backward compatibility gems.

But before I get to those gems, I wanted to add binstubs so that the rails, rake, bundle, and rspec binaries would be part of my app. Binstubs are not new to Rails 4, but using them is a strong Rails 4, or at least a strong DHH, opinion.

1
2
bundle exec rake rails:update:bin
bundle binstubs rspec-core

Now I had a bin/ directory, populated with the binaries I just mentioned. Instead of running “bundle exec rake”, I can now use “bin/rake”. I then added the binstubs to my git ignore:

git add .gitignore bin/ “`

Easy enough. Now, on to removing the backward compatability gems. I’ll start with strong parameters and work my way down the list.

Upgrading From Rails 3.2 to Rails 4.0

| Comments

Tonight, I upgraded my first Rails 3.2 app to Rails 4.0. It wasn’t too hard, but I did hit some bumps along the way. Hopefully this post will help others better navigate those bump.

The first thing I did was upgrade to Ruby 2.0. If you’re going to throw, throw heat.

1
2
3
rvm get stable
rvm install 2.0.0-p0
rvm use 2.0.0-p0

Then it was time to install Rails 4

1
gem install rails -v 4.0.0.beta1

I fully plan on creating a new gemset for this project moving forward, but I did this within the existing gemset.

Before updating Rails in my Gemfile, I added the rails4_upgrade gem so that I could do a dependency check. More on that in a second

1
2
# Gemfile
gem 'rails4_upgrade'

After running bundle, I was then able to run a new rake task

1
bundle exec rake rails4:check_gems

In my case, this was a relatively new app that wasn’t using a ton of gems. I only had a handful that conflicted with Rails 4.

1
2
3
4
5
6
7
8
9
+--------------------+--------------------+
| Dependency Path    | Rails Requirement  |
+--------------------+--------------------+
| coffee-rails 3.2.2 | railties ~&gt; 3.2.0  |
| devise 2.2.3       | railties ~&gt; 3.1    |
| sass-rails 3.2.6   | railties ~&gt; 3.2.0  |
| simple_form 2.0.4  | actionpack ~&gt; 3.0  |
| simple_form 2.0.4  | activemodel ~&gt; 3.0 |
+--------------------+--------------------+

Coffee & Sass were easy enough, I just updated my Gemfile to point to the Rails 4 versions:

1
2
gem 'sass-rails',   '~&gt; 4.0.0.beta1'
gem 'coffee-rails', '~&gt; 4.0.0.beta1'

I then swapped out my version of devise with the rails4 branch:

1
gem 'devise', :git =&gt; 'git://github.com/plataformatec/devise.git', :branch =&gt; "rails4"

I did the same thing with simple_form:

1
gem 'simple_form', :git =&gt; 'git://github.com/plataformatec/simple_form.git'

Because of the upgrades, I also had to update re-run the rake installs

1
2
rails generate simple_form:install --bootstrap
rails generate devise:install

At that point, I was ready to change my Gemfile to use Rails 4. I knew that I wanted to start with a more forgiving environment, so I also added gems for all of the things that Rails 4 removed.

1
2
3
4
5
6
7
8
9
10
11
12
# Gemfile
gem 'rails', '~&gt;4.0.0.beta1'

gem 'protected_attributes'
gem 'activeresource', github: 'rails/activeresource'
gem 'actionpack-action_caching', github: 'rails/actionpack-action_caching'
gem 'actionpack-page_caching', github: 'rails/actionpack-page_caching'
gem 'activerecord-session_store'
gem 'rails-observers'
gem 'actionview-encoded_mail_to'
gem 'rails-perftest'
gem 'actionpack-xml_parser', github: 'rails/actionpack-xml_parser'

I ran a bundle install, and everything bundled up properly.

For whatever reason, I then thought I’d try running the check_gems rake task again, but rake crapped out on me:

1
2
3
4
5
6
7
8
9
10
bundle exec rake rails4:check_gems
rake aborted!
cannot load such file -- active_resource/railtie
/Users/adam/src/sandbox/screencast/config/application.rb:7:in `require'
/Users/adam/src/sandbox/screencast/config/application.rb:7:in `'
/Users/adam/src/sandbox/screencast/Rakefile:5:in `require'
/Users/adam/src/sandbox/screencast/Rakefile:5:in `'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/bin/ruby_noexec_wrapper:14:in `eval'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/bin/ruby_noexec_wrapper:14:in `'
(See full trace by running task with --trace)

Honestly, this is where things get a little blurry, but I believe what solved this first problem was a manual patch to engine.rb in the railties-4.0.0.beta1 gem. For RVM users you can do the following to find the source location for that gem:

1
gem contents railties

The manual change I made was based on this gist, which I found from this issue. Long story short, I made that change, ran my rake command again, and got a new error. I wasn’t out of the woods completely, but I’d made progress. The new error looked like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
bundle exec rake rails4:check_gems
rake aborted!
uninitialized constant Rails::SubTestTask
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/rails-perftest-0.0.1/lib/rails/perftest/railties/testing.tasks:2:in `block in '
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/rails-perftest-0.0.1/lib/rails/perftest/railties/testing.tasks:1:in `'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/rails-perftest-0.0.1/lib/rails/perftest/railtie.rb:8:in `block in '
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/railtie.rb:201:in `instance_exec'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/railtie.rb:201:in `block in run_tasks_blocks'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/railtie.rb:201:in `each'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/railtie.rb:201:in `run_tasks_blocks'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/application.rb:241:in `block in run_tasks_blocks'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/engine/railties.rb:17:in `each'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/engine/railties.rb:17:in `each'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/application.rb:241:in `run_tasks_blocks'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/engine.rb:445:in `load_tasks'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/gems/railties-4.0.0.beta1/lib/rails/railtie/configurable.rb:30:in `method_missing'
/Users/adam/src/sandbox/screencast/Rakefile:7:in `'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/bin/ruby_noexec_wrapper:14:in `eval'
/Users/adam/.rvm/gems/ruby-2.0.0-p0@rails4/bin/ruby_noexec_wrapper:14:in `'
(See full trace by running task with --trace)

At that point, I decided to hold off on fixing rake, and move on to seeing if I could run rails s. The result was better than expected, but still not clean.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
rails s
=&gt; Booting WEBrick
=&gt; Rails 4.0.0.beta1 application starting in development
=&gt; Call with -d to detach
=&gt; Ctrl-C to shutdown server
DEPRECATION WARNING: config.whiny_nils option is deprecated and no longer works. (called from block in  at /Users/adam/src/sandbox/screencast/config/environments/development.rb:10)
config.eager_load is set to nil. Please update your config/environments/*.rb files accordingly:

  * development - set it to false
  * test - set it to false (unless you use a tool that preloads your test environment)
  * production - set it to true

DEPRECATION WARNING: The Active Record auto explain feature has been removed.

To disable this message remove the `active_record.auto_explain_threshold_in_seconds`
option from the `config/environments/*.rb` config file.

I did some research, then ultimately made a handful of changes to development.rb, production.rb, test.rb, and application.rb. I found that creating a fresh Rails 4 app, then comparing each of those files from the new app to those on my existing app, was really helpful. I’m not going to include the exact changes I made to those files because I don’t want to lead anyone down the wrong path, but the most import changes were: removing whiny_nils configs, adding eager_loading configs, and removing the active_record.auto_explain_threshold_in_seconds config. I also started with a new application.rb based on the fresh rails4 app, then added back my non-standard configs to that.

I then tried the rake command again, and sure enough, it worked. The change to application.rb had fixed that problem.

At this point, ‘rails s’ and ‘rake’ were both working and the app was running fine. This is great progress, but I still hadn’t fully converted my app to Rails 4 code. I’ll cover that progress in a series of followup posts. The first step was using binstubs.

Reopen Closed Tabs in Sublime Text2

| Comments

I frequently use cmd+shift+t in Chrome to reopen closed tabs, and I’ve often wished that Sublime Text 2 had the same functionality. Several times per day, I’ll accidentally cmd+w a tab and will then have to use cmd+t/p to search for it and reopen it. It’s one of those things that I’ve often thought about, but never took the time to research — until today. It turns out that it’s pretty damn easy to setup.

Assuming you have Package Control installed:

Go to Sublime Text 2 menu -> Preferences -> Package Control.
Select “Package Control: Install Package.”
Type in “Open Recent Files” and select that package.

Unfortunately, Open Recent Files uses Control+Shit+T, and I’m used to Comand+Shift+T, so I tweaked the key binding by doing the following:

Go to Sublime Text 2 menu -> Preferences -> Key Bindings - User
Add the following:

1
2
3
[
    { "keys": ["super+shift+t"], "command": "open_recent_files" }
]

Restart ST2, open a tab, close it, then Cmd+Shift+T to reopen it. Woo!

Cheap Whiteboards

| Comments

I went three weeks in the new office without a whiteboard before I completely lost my mind. The thing is, large whiteboards are expensive (for a 2 month old startup). $50 for 4x3 feet. $150 for 6x4 feet! I looked at http://www.ideapaint.com/ which has paint on whiteboards, but that required lots of prep work, multiple coats, and fumigating the office for a few days. I also looked at http://whiteyboard.com’s stick-on whiteboards, but after waiting two weeks for a sample and not hearing a word from them, I moved on. I ended up googling “cheap whiteboard” out of frustration and found a DIY project. The important part of that project was that it described a cheap whiteboard material that could be found at a Lowes or Home Depot. So, I went to my local Lowes and picked up four of these. They fit easily in my sedan and were light enough to carry by myself. I also picked up four packs of Scotch Heavy Duty Fasteners. I knew that I’d be resting the whiteboards on the ledge in my office, so I figured those would be suitable. They’d also give me the chance to reposition or replace panels easily.  I spent $50 total and now have a wall of whiteboards and my sanity back.

Speeding Up CSV Imports With Rails

| Comments

I was recently working on a project that required frequent importing of S3-hosted CSV files containing hundreds of thousands of users. My first pass at the import was fairly standard:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
require 'rubygems'
require 'fog'
require 'csv'

start_time = Time.now
counter = 0
tenant = Tenant.first
connection = Fog::Storage.new({
  :provider                 =&gt; 'AWS',
  :aws_access_key_id        =&gt; 'xxx',
  :aws_secret_access_key    =&gt; 'xxx'
})

directory = connection.directories.get("xxx")
file = directory.files.get('imports/test-import-medium.txt')
body = file.body

CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash
  user = User.new(
    first_name: row_hash["FirstName"],
    last_name: row_hash["LastName"],
    address: row_hash["Address1"],
    address2: row_hash["Address2"],
    city: row_hash["City"],
    state: row_hash["State"],
    zip: row_hash["ZipCode"],
    email: row_hash["Email"],
    gender: row_hash["Gender"],
  )
  user.set_random_password
  user.memberships.build(tenant: tenant, status: Membership::STATUSES[:created])
  user.save!

  counter += 1
end

end_time = Time.now
puts "#{counter} users imported #{((end_time - start_time) / 60).round(2)} minutes (#{( counter / (end_time - start_time)).round(2)} users/second)"

I ran that version against a test file containing 25,000 records. The result was — well, I’m not sure what the result was because I stopped it over an hour in when it had only imported 5,000 records.

After scratching my head for a minute and doing a little debugging, I discovered that the slowest part of the script was the “user.save!.” Sure, connecting to s3 and streaming down the file wasn’t lightning-fast, but the real culprit was saving a user. It was extraordinarily slow for some reason.

Then it hit me, the reason I was setting a password via my custom “set_random_password” method was because the User model used has_secure_password. My workflow didn’t even require the user to have a password at this point, but has_secure_password requires a password on creation, so I was passing in a dummy password to pass the validation. The generating the password was fast, but encrypting the passwords was where things were slowing down. I was taking a significant hit for something that I didn’t even need.

So, my first step was to rip out has_secure_password and write my own encryption/authentication methods. That way, I had control over the validation which meant I no longer needed to pass in a password during this import process. That alone was a huge gain. After that change, I re-ran the import and the result was:

1
25000 users imported 12.25 minutes (34.01 users/second)

12 minutes was a significant improvement from the first pass, but at that rate, it would still take 3-5 hours to process a file containing 200k rows (assuming the rate slowed with larger files). So, I began looking for big optimization gains.

After some quick googling, I found https://github.com/zdennis/activerecord-import. This looked extremely promising, so I cranked out some code to test it out

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
start_time = Time.now
counter = 0
tenant = Tenant.first
users = []
...

CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash
  user = User.new(
    first_name: row_hash["FirstName"],
    last_name: row_hash["LastName"],
    address: row_hash["Address1"],
    address2: row_hash["Address2"],
    city: row_hash["City"],
    state: row_hash["State"],
    zip: row_hash["ZipCode"],
    email: row_hash["Email"],
    gender: row_hash["Gender"]
  )
  user.memberships.build(tenant: tenant, status: Membership::STATUSES[:created])
  users &lt;&lt; user

  counter += 1
end

User.import users

end_time = Time.now
puts "#{counter} users imported #{((end_time - start_time) / 60).round(2)} minutes (#{( counter / (end_time - start_time)).round(2)} users/second)"

The good news: it was FAST

1
25000 users imported 1.57 minutes (265.99 users/second)

The bad news, it completely ignored the build() association and didn’t save the membership records. This was because activerecord-import can’t handle associations. When it inserts records, nothing is returned, so building an associated model just wouldn’t work. As frustrating as that was though, I wasn’t willing to abandon the activerecord-import path — 265 records/second was a huge impremovent over 34.

My next pass is where things got a little less attractive, but I was willing to sacrafice some elegance for the sake of speed. I decided to import the users, then loop through the CSV again, get the User.id for each freshly inserted record, manually build a Membership model, then import an array of Memberships. My theory was that doing two loops and a lookup of every user to get their id would still be faster than a non activerecord-import solution.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
require 'rubygems'
require 'fog'
require 'csv'

start_time = Time.now
counter = 0
tenant = Tenant.first
users = []
memberships = []
...
CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash
  user = User.new(
    first_name: row_hash["FirstName"],
    last_name: row_hash["LastName"],
    address: row_hash["Address1"],
    address2: row_hash["Address2"],
    city: row_hash["City"],
    state: row_hash["State"],
    zip: row_hash["ZipCode"],
    email: row_hash["Email"],
    gender: row_hash["Gender"]
  )
  users &lt;&lt; user
  counter += 1
end
User.import users

counter = 0
CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash

  user = User.where("email = ?",row_hash["Email"]).first
  membership = Membership.new(
    tenant: tenant,
    status: Membership::STATUSES[:created],
    user: user
  )
  memberships &lt;&lt; membership
  counter += 1
end
Membership.import memberships

I ran it again and the results were:

1
25000 users imported 2.72 minutes (153.3 users/second)

OK, so not as fast as before, but still plenty fast. Even with 200k records, this would certainly finish within 30-45 minutes. I felt good about the path I was on, but began looking for other improvements. First, I was getting an entire user record in the second loop, when I really only needed the User.id:

1
  user = User.where("email = ?",row_hash["Email"]).first

So, I replaced that line with the following:

1
user_id = User.where("email = ?",email).select(:id).first.id

I was also making unecessary calls within the loop to get the value of Membership::STATUSES[:created], so I moved that out of the loop and set a “status” variable once. I was also passing in an entire Tenant object, when again, all I needed was the Tenant.id, so I fixed that too.

My final version looked like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
...
start_time = Time.now
users = []
memberships = []
counter = 0
tenant_id = Tenant.first.id
status = Membership::STATUSES[:created]
...

CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash
  user = User.new(
    first_name: row_hash["FirstName"],
    last_name: row_hash["LastName"],
    address: row_hash["Address1"],
    address2: row_hash["Address2"],
    city: row_hash["City"],
    state: row_hash["State"],
    zip: row_hash["ZipCode"],
    email: row_hash["Email"],
    gender: row_hash["Gender"]
  )
  users &lt;&lt; user
  counter += 1
end
User.import users

counter = 0
CSV.parse(body, col_sep: "|", headers: true) do |row|
  row_hash = row.to_hash

  user_id = User.where("email = ?",row_hash["Email"]).select(:id).first.id
  membership = Membership.new(
    tenant_id: tenant_id,
    status: status,
    user_id: user_id
  )
  memberships &lt;&lt; membership
  counter += 1
end
Membership.import memberships

end_time = Time.now
puts "#{counter} users imported in #{((end_time - start_time) / 60).round(2)}  minutes."

And when I ran it, I got the following

1
25000 users imported 2.4 minutes (173.71 users/second)

Those last little optimizations had a 20 records per second improvement. From where I started, the import process went from 5-10 hours to under thirty minutes for 200k records. That’s a huge improvement and the time savings will yield huge rewards for my client. Not a bad day’s work.