Class: Net::HTTP

Class Net::HTTP provides a rich library that implements the client in a client-server model that uses the HTTP request-response protocol. For information about HTTP, see:

• Hypertext Transfer Protocol.

• Technical overview.

About the Examples

Examples here assume that net/http has been required (which also requires uri):

require 'net/http'

Many code examples here use these example websites:

• jsonplaceholder.typicode.com.

• example.com.

Some examples also assume these variables:

uri = URI('https://jsonplaceholder.typicode.com/')
uri.freeze # Examples may not modify.
hostname = uri.hostname # => "jsonplaceholder.typicode.com"
path = uri.path         # => "/"
port = uri.port         # => 443

So that example requests may be written as:

Net::HTTP.get(uri)
Net::HTTP.get(hostname, '/index.html')
Net::HTTP.start(hostname) do |http|

http.get(‘/todos/1’) http.get(‘/todos/2’)

end

An example that needs a modified URI first duplicates uri, then modifies the duplicate:

_uri = uri.dup
_uri.path = '/todos/1'

Strategies

• If you will make only a few GET requests, consider using OpenURI.

• If you will make only a few requests of all kinds, consider using the various singleton convenience methods in this class. Each of the following methods automatically starts and finishes a session that sends a single request:

  # Return string response body.
  Net::HTTP.get(hostname, path)
  Net::HTTP.get(uri)
  
  # Write string response body to $stdout.
  Net::HTTP.get_print(hostname, path)
  Net::HTTP.get_print(uri)
  
  # Return response as Net::HTTPResponse object.
  Net::HTTP.get_response(hostname, path)
  Net::HTTP.get_response(uri)
  data = '{"title": "foo", "body": "bar", "userId": 1}'
  Net::HTTP.post(uri, data)
  params = {title: 'foo', body: 'bar', userId: 1}
  Net::HTTP.post_form(uri, params)
  data = '{"title": "foo", "body": "bar", "userId": 1}'
  Net::HTTP.put(uri, data)

• If performance is important, consider using sessions, which lower request overhead. This session has multiple requests for HTTP methods and WebDAV methods:

  Net::HTTP.start(hostname) do |http|
    # Session started automatically before block execution.

http.get(path) http.head(path)

body = 'Some text'

http.post(path, body) # Can also have a block. http.put(path, body) http.delete(path) http.options(path) http.trace(path) http.patch(path, body) # Can also have a block. http.copy(path) http.lock(path, body) http.mkcol(path, body) http.move(path) http.propfind(path, body) http.proppatch(path, body) http.unlock(path, body)

  # Session finished automatically at block exit.
end

The methods cited above are convenience methods that, via their few arguments, allow minimal control over the requests. For greater control, consider using request objects.

URIs

On the internet, a URI (Universal Resource Identifier) is a string that identifies a particular resource. It consists of some or all of: scheme, hostname, path, query, and fragment; see URI syntax.

A Ruby URI::Generic object represents an internet URI. It provides, among others, methods scheme, hostname, path, query, and fragment.

Schemes

An internet URI has a scheme.

The two schemes supported in Net::HTTP are 'https' and 'http':

uri.scheme                       # => "https"
URI('http://example.com').scheme # => "http"

Hostnames

A hostname identifies a server (host) to which requests may be sent:

hostname = uri.hostname # => "jsonplaceholder.typicode.com"
Net::HTTP.start(hostname) do |http|
  # Some HTTP stuff.
end

Paths

A host-specific path identifies a resource on the host:

_uri = uri.dup
_uri.path = '/todos/1'
hostname = _uri.hostname
path = _uri.path
Net::HTTP.get(hostname, path)

Queries

A host-specific query adds name/value pairs to the URI:

_uri = uri.dup
params = {userId: 1, completed: false}
_uri.query = URI.encode_www_form(params)
_uri # => #<URI::HTTPS https://jsonplaceholder.typicode.com?userId=1&completed=false>
Net::HTTP.get(_uri)

Fragments

A URI fragment has no effect in Net::HTTP; the same data is returned, regardless of whether a fragment is included.

Request Headers

Request headers may be used to pass additional information to the host, similar to arguments passed in a method call; each header is a name/value pair.

Each of the Net::HTTP methods that sends a request to the host has optional argument headers, where the headers are expressed as a hash of field-name/value pairs:

headers = {Accept: 'application/json', Connection: 'Keep-Alive'}
Net::HTTP.get(uri, headers)

See lists of both standard request fields and common request fields at Request Fields. A host may also accept other custom fields.

HTTP Sessions

A session is a connection between a server (host) and a client that:

• Is begun by instance method #start.

• May contain any number of requests.

• Is ended by instance method #finish.

See example sessions at Strategies.

Session Using Net::HTTP.start

If you have many requests to make to a single host (and port), consider using singleton method .start with a block; the method handles the session automatically by:

• Calling #start before block execution.

• Executing the block.

• Calling #finish after block execution.

In the block, you can use these instance methods, each of which that sends a single request:

• HTTP methods:

  • #get, #request_get: GET.

  • #head, #request_head: HEAD.

  • #post, #request_post: POST.

  • #delete: DELETE.

  • #options: OPTIONS.

  • #trace: TRACE.

  • #patch: PATCH.

• WebDAV methods:

  • #copy: COPY.

  • #lock: LOCK.

  • #mkcol: MKCOL.

  • #move: MOVE.

  • #propfind: PROPFIND.

  • #proppatch: PROPPATCH.

  • #unlock: UNLOCK.

Session Using Net::HTTP.start and Net::HTTP.finish

You can manage a session manually using methods #start and #finish:

http = HTTP.new(hostname) http.start http.get(‘/todos/1’) http.get(‘/todos/2’) http.delete(‘/posts/1’) http.finish # Needed to free resources.

Single-Request Session

Certain convenience methods automatically handle a session by:

• Creating an HTTP object

• Starting a session.

• Sending a single request.

• Finishing the session.

• Destroying the object.

Such methods that send GET requests:

• .get: Returns the string response body.

• .get_print: Writes the string response body to $stdout.

• .get_response: Returns a HTTPResponse object.

Such methods that send POST requests:

• .post: Posts data to the host.

• .post_form: Posts form data to the host.

HTTP Requests and Responses

Many of the methods above are convenience methods, each of which sends a request and returns a string without directly using Net::HTTPRequest and Net::HTTPResponse objects.

You can, however, directly create a request object, send the request, and retrieve the response object; see:

• HTTPRequest.

• HTTPResponse.

Following Redirection

Each returned response is an instance of a subclass of HTTPResponse. See the response class hierarchy.

In particular, class HTTPRedirection is the parent of all redirection classes. This allows you to craft a case statement to handle redirections properly:

def fetch(uri, limit = 10)
  # You should choose a better exception.
  raise ArgumentError, 'Too many HTTP redirects' if limit == 0

  res = Net::HTTP.get_response(URI(uri))
  case res
  when Net::HTTPSuccess     # Any success class.
    res
  when Net::HTTPRedirection # Any redirection class.
    location = res['Location']
    warn "Redirected to #{location}"
    fetch(location, limit - 1)
  else                      # Any other class.
    res.value
  end
end

fetch(uri)

Basic Authentication

Basic authentication is performed according to RFC2617:

req = Net::HTTP::Get.new(uri)
req.basic_auth('user', 'pass')
res = Net::HTTP.start(hostname) do |http|

http.request(req)

end

Streaming Response Bodies

By default Net::HTTP reads an entire response into memory. If you are handling large files or wish to implement a progress bar you can instead stream the body directly to an IO.

Net::HTTP.start(hostname) do |http|
  req = Net::HTTP::Get.new(uri)

http.request(req) do |res|

    open('t.tmp', 'w') do |f|
      res.read_body do |chunk|
        f.write chunk
      end
    end
  end
end

HTTPS

HTTPS is enabled for an HTTP connection by #use_ssl=:

Net::HTTP.start(hostname, :use_ssl => true) do |http|
  req = Net::HTTP::Get.new(uri)
  res = http.request(req)
end

Or if you simply want to make a GET request, you may pass in a URI object that has an HTTPS URL. Net::HTTP automatically turns on TLS verification if the URI object has a ‘https’ URI scheme:

uri # => #<URI::HTTPS https://jsonplaceholder.typicode.com/>
Net::HTTP.get(uri)

Proxy Server

An HTTP object can have a proxy server.

You can create an HTTP object with a proxy server using method .new or method .start.

The proxy may be defined either by argument p_addr or by environment variable 'http_proxy'.

Proxy Using Argument p_addr as a String

When argument p_addr is a string hostname, the returned http has the given host as its proxy:

http = HTTP.new(hostname, nil, ‘proxy.example’) http.proxy? # => true http.proxy_from_env? # => false http.proxy_address # => “proxy.example”

# These use default values.

http.proxy_port # => 80 http.proxy_user # => nil http.proxy_pass # => nil

The port, username, and password for the proxy may also be given:

http = HTTP.new(hostname, nil, ‘proxy.example’, 8000, ‘pname’, ‘ppass’)

# => #<Net::HTTP jsonplaceholder.typicode.com:80 open=false>

http.proxy? # => true http.proxy_from_env? # => false http.proxy_address # => “proxy.example” http.proxy_port # => 8000 http.proxy_user # => “pname” http.proxy_pass # => “ppass”

Proxy Using ‘ENV['http_proxy']’

When environment variable 'http_proxy' is set to a URI string, the returned http will have the server at that URI as its proxy; note that the URI string must have a protocol such as 'http' or 'https':

ENV['http_proxy'] = 'http://example.com'

http = HTTP.new(hostname) http.proxy? # => true http.proxy_from_env? # => true http.proxy_address # => “example.com”

# These use default values.

http.proxy_port # => 80 http.proxy_user # => nil http.proxy_pass # => nil

The URI string may include proxy username, password, and port number:

ENV['http_proxy'] = 'http://pname:ppass@example.com:8000'

http = HTTP.new(hostname) http.proxy? # => true http.proxy_from_env? # => true http.proxy_address # => “example.com” http.proxy_port # => 8000 http.proxy_user # => “pname” http.proxy_pass # => “ppass”

Filtering Proxies

With method .new (but not .start), you can use argument p_no_proxy to filter proxies:

• Reject a certain address:

http = HTTP.new(‘example.com’, nil, ‘proxy.example’, 8000, ‘pname’, ‘ppass’, ‘proxy.example’) http.proxy_address # => nil

• Reject certain domains or subdomains:

http = HTTP.new(‘example.com’, nil, ‘my.proxy.example’, 8000, ‘pname’, ‘ppass’, ‘proxy.example’) http.proxy_address # => nil

• Reject certain addresses and port combinations:

http = HTTP.new(‘example.com’, nil, ‘proxy.example’, 8000, ‘pname’, ‘ppass’, ‘proxy.example:1234’) http.proxy_address # => “proxy.example”

http = HTTP.new(‘example.com’, nil, ‘proxy.example’, 8000, ‘pname’, ‘ppass’, ‘proxy.example:8000’) http.proxy_address # => nil

• Reject a list of the types above delimited using a comma:

http = HTTP.new(‘example.com’, nil, ‘proxy.example’, 8000, ‘pname’, ‘ppass’, ‘my.proxy,proxy.example:8000’) http.proxy_address # => nil

http = HTTP.new(‘example.com’, nil, ‘my.proxy’, 8000, ‘pname’, ‘ppass’, ‘my.proxy,proxy.example:8000’) http.proxy_address # => nil

Compression and Decompression

Net::HTTP does not compress the body of a request before sending.

By default, Net::HTTP adds header 'Accept-Encoding' to a new request object:

Net::HTTP::Get.new(uri)['Accept-Encoding']
# => "gzip;q=1.0,deflate;q=0.6,identity;q=0.3"

This requests the server to zip-encode the response body if there is one; the server is not required to do so.

Net::HTTP does not automatically decompress a response body if the response has header 'Content-Range'.

Otherwise decompression (or not) depends on the value of header Content-Encoding:

• 'deflate', 'gzip', or 'x-gzip': decompresses the body and deletes the header.

• 'none' or 'identity': does not decompress the body, but deletes the header.

• Any other value: leaves the body and header unchanged.

What’s Here

First, what’s elsewhere. Class HTTP:

• Inherits from class Object.

This is a categorized summary of methods and attributes.

Net::HTTP Objects

• ::new: Creates a new instance.

• #inspect: Returns a string representation of self.

Sessions

• ::start: Begins a new session in a new Net::HTTP object.

• #started? (aliased as #active?): Returns whether in a session.

• #finish: Ends an active session.

• #start: Begins a new session in an existing Net::HTTP object (self).

Connections

• :continue_timeout: Returns the continue timeout.

• #continue_timeout=: Sets the continue timeout seconds.

• :keep_alive_timeout: Returns the keep-alive timeout.

• :keep_alive_timeout=: Sets the keep-alive timeout.

• :max_retries: Returns the maximum retries.

• #max_retries=: Sets the maximum retries.

• :open_timeout: Returns the open timeout.

• :open_timeout=: Sets the open timeout.

• :read_timeout: Returns the open timeout.

• :read_timeout=: Sets the read timeout.

• :ssl_timeout: Returns the ssl timeout.

• :ssl_timeout=: Sets the ssl timeout.

• :write_timeout: Returns the write timeout.

• write_timeout=: Sets the write timeout.

Requests

• ::get: Sends a GET request and returns the string response body.

• ::get_print: Sends a GET request and write the string response body to $stdout.

• ::get_response: Sends a GET request and returns a response object.

• ::post_form: Sends a POST request with form data and returns a response object.

• ::post: Sends a POST request with data and returns a response object.

• ::put: Sends a PUT request with data and returns a response object.

• #copy: Sends a COPY request and returns a response object.

• #delete: Sends a DELETE request and returns a response object.

• #get: Sends a GET request and returns a response object.

• #head: Sends a HEAD request and returns a response object.

• #lock: Sends a LOCK request and returns a response object.

• #mkcol: Sends a MKCOL request and returns a response object.

• #move: Sends a MOVE request and returns a response object.

• #options: Sends a OPTIONS request and returns a response object.

• #patch: Sends a PATCH request and returns a response object.

• #post: Sends a POST request and returns a response object.

• #propfind: Sends a PROPFIND request and returns a response object.

• #proppatch: Sends a PROPPATCH request and returns a response object.

• #put: Sends a PUT request and returns a response object.

• #request: Sends a request and returns a response object.

• #request_get (aliased as #get2): Sends a GET request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

• #request_head (aliased as #head2): Sends a HEAD request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

• #request_post (aliased as #post2): Sends a POST request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

• #send_request: Sends a request and returns a response object.

• #trace: Sends a TRACE request and returns a response object.

• #unlock: Sends an UNLOCK request and returns a response object.

Responses

• :close_on_empty_response: Returns whether to close connection on empty response.

• :close_on_empty_response=: Sets whether to close connection on empty response.

• :ignore_eof: Returns whether to ignore end-of-file when reading a response body with Content-Length headers.

• :ignore_eof=: Sets whether to ignore end-of-file when reading a response body with Content-Length headers.

• :response_body_encoding: Returns the encoding to use for the response body.

• #response_body_encoding=: Sets the response body encoding.

Proxies

• :proxy_address: Returns the proxy address.

• :proxy_address=: Sets the proxy address.

• ::proxy_class?: Returns whether self is a proxy class.

• #proxy?: Returns whether self has a proxy.

• #proxy_address (aliased as #proxyaddr): Returns the proxy address.

• #proxy_from_env?: Returns whether the proxy is taken from an environment variable.

• :proxy_from_env=: Sets whether the proxy is to be taken from an environment variable.

• :proxy_pass: Returns the proxy password.

• :proxy_pass=: Sets the proxy password.

• :proxy_port: Returns the proxy port.

• :proxy_port=: Sets the proxy port.

• #proxy_user: Returns the proxy user name.

• :proxy_user=: Sets the proxy user.

Security

• :ca_file: Returns the path to a CA certification file.

• :ca_file=: Sets the path to a CA certification file.

• :ca_path: Returns the path of to CA directory containing certification files.

• :ca_path=: Sets the path of to CA directory containing certification files.

• :cert: Returns the OpenSSL::X509::Certificate object to be used for client certification.

• :cert=: Sets the OpenSSL::X509::Certificate object to be used for client certification.

• :cert_store: Returns the X509::Store to be used for verifying peer certificate.

• :cert_store=: Sets the X509::Store to be used for verifying peer certificate.

• :ciphers: Returns the available SSL ciphers.

• :ciphers=: Sets the available SSL ciphers.

• :extra_chain_cert: Returns the extra X509 certificates to be added to the certificate chain.

• :extra_chain_cert=: Sets the extra X509 certificates to be added to the certificate chain.

• :key: Returns the OpenSSL::PKey::RSA or OpenSSL::PKey::DSA object.

• :key=: Sets the OpenSSL::PKey::RSA or OpenSSL::PKey::DSA object.

• :max_version: Returns the maximum SSL version.

• :max_version=: Sets the maximum SSL version.

• :min_version: Returns the minimum SSL version.

• :min_version=: Sets the minimum SSL version.

• #peer_cert: Returns the X509 certificate chain for the session’s socket peer.

• :ssl_version: Returns the SSL version.

• :ssl_version=: Sets the SSL version.

• #use_ssl=: Sets whether a new session is to use Transport Layer Security.

• #use_ssl?: Returns whether self uses SSL.

• :verify_callback: Returns the callback for the server certification verification.

• :verify_callback=: Sets the callback for the server certification verification.

• :verify_depth: Returns the maximum depth for the certificate chain verification.

• :verify_depth=: Sets the maximum depth for the certificate chain verification.

• :verify_hostname: Returns the flags for server the certification verification at the beginning of the SSL/TLS session.

• :verify_hostname=: Sets he flags for server the certification verification at the beginning of the SSL/TLS session.

• :verify_mode: Returns the flags for server the certification verification at the beginning of the SSL/TLS session.

• :verify_mode=: Sets the flags for server the certification verification at the beginning of the SSL/TLS session.

Addresses and Ports

• :address: Returns the string host name or host IP.

• ::default_port: Returns integer 80, the default port to use for HTTP requests.

• ::http_default_port: Returns integer 80, the default port to use for HTTP requests.

• ::https_default_port: Returns integer 443, the default port to use for HTTPS requests.

• #ipaddr: Returns the IP address for the connection.

• #ipaddr=: Sets the IP address for the connection.

• :local_host: Returns the string local host used to establish the connection.

• :local_host=: Sets the string local host used to establish the connection.

• :local_port: Returns the integer local port used to establish the connection.

• :local_port=: Sets the integer local port used to establish the connection.

• :port: Returns the integer port number.

HTTP Version

• ::version_1_2? (aliased as ::is_version_1_2? and ::version_1_2): Returns true; retained for compatibility.

Debugging

• #set_debug_output: Sets the output stream for debugging.