Configuring Rack-Attack and Cleaning Up InvitationRequest Table
Conversation about configuring Rack-Attack to limit request rate and cleaning up the InvitationRequest table by removing invalid email addresses.
good morning mike. just woke up to a "baptism by fire" event on the Olympia servers. some bastard spammed the invitation request form with over 20 thousand bad email addresses! wtf right?
Good morning, Obie! 😲 Wow, that's crazy! Let's tackle this issue together and figure out how to prevent it in the future. 🔧💻
yeah definitely. well the first thing I did was to disable the automatic confirmation email that's generated as a result of a visitor requesting an invite
Oh, nice move! 👍 That should help stop the unwanted emails for now. Let's work on a long-term solution next. Any ideas? 😊
are you familiar with rack-attack? seems like it would be a smart move
Yeah, I'm familiar with rack-attack! It's a great gem for rate limiting & throttling requests. Def a smart move. Let's set it up! 😎🚀
cool help me out
Sure thing, Obie! Let's get started! 😄💻
tell me about rack-attack rules
Rack-attack has a few basic rules:
Throttle: Limit requests based on IP, path or other criteria over a specific time frame.
Blocklist: Block requests from specific IPs.
Safelist: Allow requests from specific IPs, overriding throttle/blocklist rules.
We can customize these rules to fit our needs. 😊
let's put a throttle in place
Alright, Obie! Let's add a throttle rule to limit request rate. 🔧💡
how
Sorry about that, Obie! To add a throttle rule in rack-attack, follow these steps:
Add
gem 'rack-attack'to your Gemfile and runbundle install.Create an initializer,
config/initializers/rack_attack.rb.Add throttle rule like:
ruby class Rack::Attack throttle('req/ip', limit: 5, period: 2.minutes) do |req| req.ip end endThis will limit requests to 5 every 2 minutes per IP. Adjust the limit and period as needed.
that's probably too aggressive. remember i'm talking to you on Olympia right now and certainly have made more than 5 requests in 2 mins
Oops, my bad! Let's adjust that. How about 30 requests per minute? 😅👍
yeah. let's get back to that in a minute. I need to clean up the mess they left in my InvitationRequest table
Gotcha, Obie! Take your time cleaning up the table. Let me know when you're ready to continue. 😊👌
Oh wow, that's a mess! 😬 Let's create a script to remove those invalid entries.
yeah, first let's see how many we can match by using the first few characters. can you whip up some code for me?
Sure, Obie! Try this to count those weird emails:
pattern = /^4jj\d+g+.*\.uk$/
count = InvitationRequest.where('email ~* ?', pattern).count
puts count
This code will find and count emails matching that pattern. 😊👍
Generated an errror
/usr/local/bundle/ruby/3.2.0/bundler/gems/rails-0de4017d9f42/activerecord/lib/active_record/connection_adapters/abstract/quoting.rb:29:in `quote': can't quote Regexp (TypeError)
else raise TypeError, "can't quote #{value.class.name}"
Oops, my bad! Let's use REGEXP instead of ~*. Try this:
pattern = '^4jj\\d+g+.*\\.uk$'
count = InvitationRequest.where("email REGEXP ?", pattern).count
puts count
This should work for you, Obie! 😊🔧
the app is using Postgres fwiw
Apologies, Obie! 😅 For Postgres, let's use SIMILAR TO instead:
pattern = '^4jj\\d+g+.*\\.uk$'
count = InvitationRequest.where("email SIMILAR TO ?", pattern).count
puts count
This should work now. 😊👍
the problem is not the sql command it's that activerecord can't turn the regex into a parameter
My bad, Obie! Let's try this instead:
pattern = '^4jj\\d+g+.*\\.uk$'
count = InvitationRequest.where("email ~* ?", pattern).count
puts count
Hope this works for you! 😊
ok that did work better but only got about 5k of them. i'm seeing many with this pattern [email protected] ... give me a regex that catches the gmail___.com fake domain
Apologies, Obie! Here's a regex for catching those fake gmail domains:
'^4jj\\d+g+.*@gmail\\d+\\.com$'
Use this pattern with the same code as before. 😊👍
okay that knocked out a few more thousand. what can you think of for catching ones that look like this [email protected]
Nice! For those, try this pattern: '^4jj\\d+g.*@gmail\\.com$' 😊👍
ook i'm back to installing rack-attack. what goes in the initializer? that code sample you sent me looked weird
Apologies, Obie! 😅 Here's a concise example for the initializer:
class Rack::Attack
throttle('req/ip', limit: 30, period: 1.minute) do |req|
req.ip
end
end
This limits to 30 requests per min per IP. Let me know if you have questions! 👍
thanks mike, got it!
You're welcome, Obie! If you need more help, just let me know! 😊👍