At BackerKit, we occasionally see high volumes of traffic from malicious clients. (Kickstarter has faced a similar problem.) These DDoS attacks result in degraded performance and frustrate our customers. Not cool!

We implemented Kickstarter’s Rack::Attack and configured constraints on the number of requests allowed in a time period based on IP address on our troublesome endpoints. Yay, problem solved!

Like most tools, Rack::Attack requires tuning; our initial stab at configuration led to customers being blocked. We needed a way to clear out blocked IPs that were friendly.

Unblocking Friendlies

We started with tooling developers could use to manually clear a key.  Keys of blocked IPs look like rack::attack:allow2ban:ban:104.62.144.205. Given a target IP address we could go find that key in our cache (we use Redis) and delete it.

Our first manual step was to filter the Rails’ cache’s keys and find ones used by Rack::Attack that includes our target IP to unblock.

Rails.cache.data.keys
.find_all{ |k| k.include?(':ban:') }
.find_all{ |x| x.include?('104.62.144.205') }

We could then delete the found Redis key. Note this approach will work on only cache backends supporting the data method. Notably, ActiveSupport::Cache::FileStore (frequently used in development) does not support it. In addition, this operation is O(N) so beware of performance issues on large caches.

We wanted to type less and make fewer mistakes, so we wrapped some of that logic into the following helper:

# config/initializers/rack_attack.rb
module BannedIpGetters
def banned_ips
with do |conn|
return conn.keys.find_all { |x| x.include?(':ban:') }
end
end
end

ActiveSupport::Cache::RedisStore.include(BannedIpGetters)

the above helper lets us write:

ip_key_to_unblock = Rails.cache.banned_ips.find_all{|x| x.include?('104.62.144.205')}
Rails.cache.delete(ip_key_to_unblock)

Empowering our Coworkers

Fixing these issues became time-consuming for our dev team, so we created a simple dashboard for our coworkers to use. The dashboard lets them see which IPs have been blacklisted and provides a button to unblock a given IP.

The controller presents an index page with a list of banned IPs and an endpoint that allows for deletion of a blocked key:

class Staff::RackAttacksController < Staff::BaseController
def index
ips = Rails.cache.data.keys.find_all { |x| x.include?('rack::attack') }.sort
@banned_ips, @other_ips = ips.partition { |x| x.include?(':ban:') }
end

def destroy
if params[:ip].starts_with?('rack::attack')
Rails.cache.delete(params[:ip])
end
redirect_to :back, notice: "#{params[:ip]} unblocked"
end
end

Our view looks like:

<% @banned_ips.each do |ip| %>
<%= ip %>
<%= link_to 'Lookup', "http://ip-lookup.net/?ip=#{ip.split(':').last}", target:'_blank' %>
<%= link_to 'Unblock', staff_rack_attack_path(id: 1, ip: ip), method: :delete %>
<% end %>
Rack::Attack Dashboard

Rack::Attack Dashboard

Now our non-technical teammates can go to this admin page and quickly find out what Rack::Attack is doing.

Next Steps

  • We’re looking to extract this dashboard into a reusable gem. Email us at [email protected] if you’d like to work with us
  • If you’re in need of cargo boxes, check out rackattack.com