• Make a request to a remote resource
• Handle the response
• Parse JSON

Simple Ruby applications are capable of powerful feats. We can use them to handle all sorts of routine tasks on our computer. We can use them to represent real-world relationships and systems. It's like we've been given an infinite multi-tool that can help us work all kinds of data and make it useful.

The limitation at this point, it seems, is really that we don't have enough data to play with. Luckily, we have a massive source of data available to us: the Internet!

We typically experience the Internet through webpages in a browser, but a lot of information is accessible as pure data. If we know the tools to retrieve and organize that data, we'll have access to a wealth of knowledge. In this lesson, we're going to introduce some of the basic ways in which we can get remote data using Ruby.

When we go to a website in the browser, we type in a URL and press enter. When enter is pressed, the browser sends a request to that URL. The request travels to a server somewhere in the world that is hosting the website we want. The server sends back a response. This response includes the HTML code structure of the website, which your browser uses to build the page for you in front of your eyes.

This type of request is known as a HTTP GET request. We are simply retrieving information at a specified location. There are other types of requests for sending, updating and deleting information that we explore in later lessons, but for now, we will focus on the GET request. Don't worry though, the GET request can... get us quite far on its own!

In Ruby code, we don't have the luxurious graphical user interface of a browser, but we can still send GET requests in a similar fashion using built-in Ruby modules and classes.

To show how to make requests in Ruby, we'll first make an HTTP GET request in IRB. To start, once IRB is open, we need to require open-uri:

require 'open-uri'

If loaded correctly, this should return true. Next, we will set a URL that we want to get data from. For this example, we'll get data from a GitHub repository that is hosting a webpage: https://learn-co-curriculum.github.io/json-site-example/. We can store this URL in a variable:

url = "https://learn-co-curriculum.github.io/json-site-example/"
#=> "https://learn-co-curriculum.github.io/json-site-example/"

The first thing we do is pass this url variable into a method called parse that is part of the URI module we loaded with require 'open-uri':

uri = URI.parse(url)
#=> #<URI::HTTPS https://learn-co-curriculum.github.io/json-site-example/> 

Aside: URI stands for Universal Resource Identifier. URIs, in short, are a standard way to name a resource. URL stands for Universal Resource Locator. They are a standard way to locate something. URLs also happen to act as a standard name (a web address acts as both the name of the website and the text you need to enter to visit the website). Therefore, generally speaking, all URLs are considered a subset of URIs (but not all URIs are URLs).

By parsing our URL with URI.parse, we've stored the URL in a class object and gained access to some powerful methods. The main one we want to use is open:

uri.open
#=> #<StringIO:0x00007fa84b13de90 @base_uri=#<URI::HTTPS https://learn-co-curriculum.github.io/json-site-example/>, @meta={"server"=>"GitHub.com", "content-type"=>"text/html; charset=utf-8", "last-modified"=>"Thu, 15 Aug 2019 23:05:32 GMT", "etag"=>"W/\"5d55e53c-284\"", "access-control-allow-origin"=>"*", "expires"=>"Thu, 22 Aug 2019 19:42:27 GMT", "cache-control"=>"max-age=600", "x-proxy-cache"=>"MISS", "x-github-request-id"=>"D49E:101C:257158:30F953:5D5EEDC6", "content-length"=>"341", "accept-ranges"=>"bytes", "date"=>"Thu, 22 Aug 2019 19:38:38 GMT", "via"=>"1.1 varnish", "age"=>"214", "connection"=>"keep-alive", "x-served-by"=>"cache-ewr18140-EWR", "x-cache"=>"HIT", "x-cache-hits"=>"1", "x-timer"=>"S1566502719.880905,VS0,VE1", "vary"=>"Accept-Encoding", "x-fastly-request-id"=>"bf2550e15c1927f0d65dc31f98747ad56e51d4db"}, @metas={"server"=>["GitHub.com"], "content-type"=>["text/html; charset=utf-8"], "last-modified"=>["Thu, 15 Aug 2019 23:05:32 GMT"], "etag"=>["W/\"5d55e53c-284\""], "access-control-allow-origin"=>["*"], "expires"=>["Thu, 22 Aug 2019 19:42:27 GMT"], "cache-control"=>["max-age=600"], "x-proxy-cache"=>["MISS"], "x-github-request-id"=>["D49E:101C:257158:30F953:5D5EEDC6"], "content-length"=>["341"], "accept-ranges"=>["bytes"], "date"=>["Thu, 22 Aug 2019 19:38:38 GMT"], "via"=>["1.1 varnish"], "age"=>["214"], "connection"=>["keep-alive"], "x-served-by"=>["cache-ewr18140-EWR"], "x-cache"=>["HIT"], "x-cache-hits"=>["1"], "x-timer"=>["S1566502719.880905,VS0,VE1"], "vary"=>["Accept-Encoding"], "x-fastly-request-id"=>["bf2550e15c1927f0d65dc31f98747ad56e51d4db"]}, @status=["200", "OK"]> 

When we type uri.open in IRB, we get back an object full of data! As it turns out, we've actually just made a GET request! The returned result, in this case, is a StringIO object, which is not a String, but can be converted into one using a built-in string method. So we could write the following:

request_result = uri.open
request_result.string

Or just chain the methods:

uri.open.string

The result? HTML as a String!

"<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n  <meta charset=\"UTF-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n  <meta http-equiv=\"X-UA-Compatible\" content=\"ie=edge\">\n  <title>JSON Example Site</title>\n</head>\n<body>\n  <h1>JSON Example Site</h1>\n\n  <h3>Endpoints available to visit:</h3>\n  <ul>\n    <li><a href=\"https://learn-co-curriculum.github.io/json-site-example/endpoints/locations.json\" alt=\"locations JSON\">endpoints/locations.json</a></li>\n    <li><a href=\"https://learn-co-curriculum.github.io/json-site-example/endpoints/people.json\" alt=\"people JSON\">endpoints/people.json</a></li>\n  </ul>\n</body>\n</html>"

We requested and received a webpage. While it isn't rendered nicely in the way a browser would display this, it is a great start! And just to review, we did this using only four lines of Ruby:

require 'open-uri'
url = "https://learn-co-curriculum.github.io/json-site-example/"
uri = URI.parse(url)
uri.open.string

This is super cool, but working with StringIO can be a little limiting. To stick with conventions, we're going to look at a slightly different approach next.

Rather than use the open method, we're going to bring in another built-in Ruby class, NET::HTTP. URI (also known as OpenURI) is actually built using NET::HTTP. NET::HTTP will allow us to get back an object closer to the structure of the actual HTTP response being sent.

Starting from a new IRB session, we'll first require open-uri as before, but we will also need to add a second require line:

require 'open-uri'
require 'net/http'

Then, we'll keep the URL definition and URI parsing the same:

url = "https://learn-co-curriculum.github.io/json-site-example/"
uri = URI.parse(url)

But instead of uri.open at the end, we'll enter the following:

response = Net::HTTP.get_response(uri)
#=> #<Net::HTTPOK 200 OK readbody=true>

Here again we've sent a GET request, only this time, the return value isn't a StringIO object, but a Net::HTTPOK object. This object has a method, body, that should produce a familiar sight when used:

response.body
#=> "<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n  <meta charset=\"UTF-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n  <meta http-equiv=\"X-UA-Compatible\" content=\"ie=edge\">\n  <title>JSON Example Site</title>\n</head>\n<body>\n  <h1>JSON Example Site</h1>\n\n  <h3>Endpoints available to visit:</h3>\n  <ul>\n    <li><a href=\"https://learn-co-curriculum.github.io/json-site-example/endpoints/locations.json\" alt=\"locations JSON\">endpoints/locations.json</a></li>\n    <li><a href=\"https://learn-co-curriculum.github.io/json-site-example/endpoints/people.json\" alt=\"people JSON\">endpoints/people.json</a></li>\n  </ul>\n</body>\n</html>\n"

So, we've seen two ways to get the HTML code from a webpage. Although this is cool, in its current state, this isn't very useful information. There are actually tools designed specifically to take this raw HTML and turn it into organized data for us in a process known as scraping, but we don't need to learn that just yet. Plenty of data is already organized and made available to us separate from HTML code and we can retrieve it the same way we send a GET request to a website.

JSON stands for JavaScript Object Notation and is a standard way store and transfer nested data over the internet. The keyword here is Notation. Data stored in JSON format is actually just data stored as a string, but structured in a way that is easily converted into usable nested data. Ruby has a built-in JSON module that includes a parse method to take JSON formatted data and turn it into an array or hash.

Just as with the website, we write in the Universal Resource Locator, convert it to a URI object, then send a GET request. Previously, we sent requests to https://learn-co-curriculum.github.io/json-site-example/. The server, in this case, had an HTML file to serve in response. If we request a slightly different URL, we'll get a different response. This site has some example data stored as JSON on it. By changing out the request to https://learn-co-curriculum.github.io/json-site-example/endpoints/locations.json, instead of responding with HTML, this time, the server will send the JSON data.

require 'open-uri'
require 'net/http'
url = "https://learn-co-curriculum.github.io/json-site-example/endpoints/locations.json"
uri = URI.parse(url)
response = Net::HTTP.get_response(uri)
response.body

Running the above in in a fresh IRB session, you should see that the response.body now returns a string of data:

"[\n  {\n    \"name\": \"Flatiron School Manhattan\",\n    \"address\": \"11 Broadway, New York, NY 10004\",\n    \"coordinates\": {\n      \"latitude\": \"40.704521\",\n      \"longitude\": \"-74.012833\"\n    }\n  },\n  {\n    \"name\": \"Flatiron School Austin\",\n    \"address\": \"316 W 12th St, Austin, TX 78701\",\n    \"coordinates\": {\n      \"latitude\": \"30.275080\",\n      \"longitude\": \"-97.743700\"\n    }\n  },\n  {\n    \"name\": \"Flatiron School Denver\",\n    \"address\": \"3601 Walnut St 5th Floor, Denver, CO 80205\",\n    \"coordinates\": {\n      \"latitude\": \"39.743510\",\n      \"longitude\": \"-105.011360\"\n    }\n  },\n  {\n    \"name\": \"Flatiron School Seattle\",\n    \"address\": \"1411 4th Ave 13th Floor, Seattle, WA 98101\",\n    \"coordinates\": {\n      \"latitude\": \"47.684879\",\n      \"longitude\": \"-122.201363\"\n    }\n  },\n  {\n    \"name\": \"Flatiron School London\",\n    \"address\": \"131 Finsbury Pavement, Finsbury, London EC2A 1NT, UK\",\n    \"coordinates\": {\n      \"latitude\": \"51.520480\",\n      \"longitude\": \"-0.087190\"\n    }\n  }\n]"

This is JSON! It isn't very friendly to use yet, so we'll want to convert it. First, we'll need to require the json module:

require 'json'

Then we pass in response.body to the JSON parser:

JSON.parse(response.body)

The result is an array containing five hashes, each with some data about a Flatiron School campus:

[{"name"=>"Flatiron School Manhattan", "address"=>"11 Broadway, New York, NY 10004", "coordinates"=>{"latitude"=>"40.704521", "longitude"=>"-74.012833"}}, {"name"=>"Flatiron School Austin", "address"=>"316 W 12th St, Austin, TX 78701", "coordinates"=>{"latitude"=>"30.275080", "longitude"=>"-97.743700"}}, {"name"=>"Flatiron School Denver", "address"=>"3601 Walnut St 5th Floor, Denver, CO 80205", "coordinates"=>{"latitude"=>"39.743510", "longitude"=>"-105.011360"}}, {"name"=>"Flatiron School Seattle", "address"=>"1411 4th Ave 13th Floor, Seattle, WA 98101", "coordinates"=>{"latitude"=>"47.684879", "longitude"=>"-122.201363"}}, {"name"=>"Flatiron School London", "address"=>"131 Finsbury Pavement, Finsbury, London EC2A 1NT, UK", "coordinates"=>{"latitude"=>"51.520480", "longitude"=>"-0.087190"}}]

Aside: If you'd like a better view of this data, require awesome_print and then try typing ap JSON.parse(response.body) to see a better output.

If you've been following along, you've made three HTTP GET requests! We can make these requests using a URL and some Ruby tools. Some data is readily available as JSON, which we can retrieve and convert into Ruby data structures.

Having these tools unlocks access to a lot of data. Being able to communicate with remote resources is also the cornerstone of web development!