Implementing URL detection and content fetching in Rails

June 12, 2023

The conversation discusses how to detect URLs in text and fetch their content in Rails. The assistant suggests using the `auto_link` method to detect URLs and the `open-uri` library to fetch their content. They also discuss using the `ensure` block in Ruby to ensure code is executed no matter what, and replacing a class with `OpenStruct` or `Struct` for a more concise solution.

Mike Nichols
Pair Programmer

I want to detect whether there's a URL in some text, so that I can pull its content

Obie Fernandez 9 months ago

Hey Obie! Sure thing. Let's create a method in Ruby that detects URLs in a given text and then fetches its content. Let me know if you have a specific place in the codebase where you'd like to add this functionality or if you need help deciding where to put it.

Mike Nichols 9 months ago

i don't need help placing it but I do want help writing it. I seem to recall that Rails has or had something called auto-link that might serve the purpose

Obie Fernandez 9 months ago

You're right, Obie! Rails used to have a helper called auto_link that detected URLs and created anchor tags for them. However, it was removed from Rails in version 3.1. You can still use it by adding the rails_autolink gem to your Gemfile and running bundle install. Then, you can use the auto_link method to find URLs in the text.

If you want to fetch the content of the URLs found in the text, you can use the open-uri library that comes with Ruby. Here's a quick example:

require 'open-uri'
require 'rails_autolink'

text = "Check out this cool website: https://example.com"
linked_text = auto_link(text)

linked_text.scan(URI.regexp) do |url|
  content = open(url).read
  puts "URL: #{url}"
  puts "Content: #{content}"
end

This example uses the auto_link method to find URLs in the text and the open-uri library to fetch the content of the URLs. Let me know if this works for you or if you need any further assistance!

Mike Nichols 9 months ago

is URI.regexp built-in?

Obie Fernandez 9 months ago

You're right, Obie. URI.regexp is built-in to Ruby. It's part of the URI module in the Ruby standard library and useful for detecting and extracting URLs from a string.

Mike Nichols 9 months ago

does autolink require the url to have https:// prepended? or will it pick up informally typed links such as cnn.com

Obie Fernandez 9 months ago

Autolink will pick up informally typed links like cnn.com, as well as those with http:// or https://. It's designed to handle various URL formats in the text.

Mike Nichols 9 months ago

what's the block you can put at the end of a ruby method that absolutely has to get executed no matter what

Obie Fernandez 9 months ago

In Ruby, you can use the ensure block to make sure code is executed no matter what, even if an exception occurs. It's placed inside a begin...rescue...ensure...end structure. Here's an example:

def example_method
  begin
    # Your code here
  rescue => error
    # Handle exceptions here
  ensure
    # Code in this block will always execute
  end
end
Mike Nichols 9 months ago

I can replace this simple little class with an OpenStruct or Struct right? what do you think?

class Memory
    attr_reader :question, :answer

    def initialize(question, answer)
      @question = question
      @answer = answer
    end
  end
Obie Fernandez 9 months ago

Sure, Obie! You can replace the Memory class with an OpenStruct or Struct for a more concise solution. Here's how you can do that using Struct:

Memory = Struct.new(:question, :answer)

Or with OpenStruct:

require 'ostruct'

memory = OpenStruct.new(question: "What's your favorite color?", answer: "Blue")

Both options will work, but Struct is generally faster and more memory-efficient than OpenStruct. Let me know if you have any questions or need more help!

Mike Nichols 9 months ago

thanks

Obie Fernandez 9 months ago

You're welcome, Obie! If you have any more questions or need further assistance, just let me know. Happy to help!

Mike Nichols 9 months ago