Class: WEBrick::HTTPProxyServer

Relationships & Source Files
Super Chains via Extension / Inclusion / Inheritance
Class Chain: self, `HTTPServer`, `GenericServer`
Instance Chain: self, `HTTPServer`, `GenericServer`
Inherits:	WEBrick::HTTPServer Object WEBrick::GenericServer WEBrick::HTTPServer WEBrick::HTTPProxyServer
Defined in:	lib/webrick/httpproxy.rb

Overview

An HTTP Proxy server which proxies GET, HEAD and POST requests.

To create a simple proxy server:

require 'webrick'
require 'webrick/httpproxy'

proxy = WEBrick::HTTPProxyServer.new Port: 8000

trap 'INT'  do proxy.shutdown end
trap 'TERM' do proxy.shutdown end

proxy.start

See .new for proxy-specific configuration items.

Modifying proxied responses

To modify content the proxy server returns use the :ProxyContentHandler option:

handler = proc do |req, res|
  if res['content-type'] == 'text/plain' then
    res.body << "\nThis content was proxied!\n"
  end
end

proxy =
  WEBrick::HTTPProxyServer.new Port: 8000, ProxyContentHandler: handler

Constant Summary

HopByHop = Internal use only

Some header fields should not be transferred.

# File 'lib/webrick/httpproxy.rb', line 232
```
%w( connection keep-alive proxy-authenticate upgrade
proxy-authorization te trailers transfer-encoding )
```
ShouldNotTransfer = Internal use only
# File 'lib/webrick/httpproxy.rb', line 234
```
%w( set-cookie proxy-connection )
```

Class Method Summary

.new(config = {}, default = Config::HTTP) ⇒ HTTPProxyServer constructor

Proxy server configurations.

`HTTPServer` - Inherited

.new	Creates a new HTTP server according to `config`

`GenericServer` - Inherited

.new	Creates a new generic server from `config`.

Instance Attribute Summary

`GenericServer` - Inherited

#config	The server configuration.
#listeners	Sockets listening for connections.
#logger	The server logger.
#status	The server status.
#tokens	Tokens control the number of outstanding clients.

Instance Method Summary

#do_CONNECT(req, res) Internal use only
#do_GET(req, res) Internal use only
#do_HEAD(req, res) Internal use only
#do_OPTIONS(req, res) Internal use only
#do_POST(req, res) Internal use only
#proxy_auth(req, res) Internal use only
#proxy_service(req, res) Internal use only
#proxy_uri(req, res) Internal use only
#service(req, res) Internal use only
#choose_header(src, dst) private Internal use only
#perform_proxy_request(req, res, req_class, body_stream = nil) private Internal use only
#set_cookie(src, dst) private Internal use only

Net::HTTP is stupid about the multiple header fields.
#set_via(h) private Internal use only
#setup_proxy_header(req, res) private Internal use only
#setup_upstream_proxy_authentication(req, res, header) private Internal use only
#split_field(f) private Internal use only

`HTTPServer` - Inherited

#access_log	Logs `req` and `res` in the access logs.
#create_request	Creates the `HTTPRequest` used when handling the HTTP request.
#create_response	Creates the `HTTPResponse` used when handling the HTTP request.
#do_OPTIONS	The default OPTIONS request handler says GET, HEAD, POST and OPTIONS requests are allowed.
#lookup_server	Finds the appropriate virtual host to handle `req`
#mount	Mounts `servlet` on `dir` passing `options` to the servlet at creation time.
#mount_proc	Mounts `proc` or `block` on `dir` and calls it with a `HTTPRequest` and `HTTPResponse`
#run	Processes requests on `sock`
#search_servlet	Finds a servlet for `path`
#service	Services `req` and fills in `res`
#ssl_servername_callback	ServerNameIndication callback.
#umount	Alias for HTTPServer#unmount.
#unmount	Unmounts `dir`
#orig_virtual_host	Check whether `server` is also SSL server.
#virtual_host	Adds `server` as a virtual host.

`GenericServer` - Inherited

#[]	Retrieves `key` from the configuration.
#run	You must subclass `GenericServer` and implement #run which accepts a TCP client socket.
#shutdown	Shuts down the server and all listening sockets.
#ssl_servername_callback	ServerNameIndication callback.
#start	Starts the server and runs the `block` for each connection.
#stop	Stops the server from accepting new connections.
#listen	Updates `listen` to enable SSL when the SSL configuration is active.
#setup_ssl_context	Sets up an SSL context for `config`
#ssl_context	SSL context for the server when run in SSL mode.
#accept_client	Accepts a TCP client socket from the TCP server socket `svr` and returns the client socket.
#alarm_shutdown_pipe,
#call_callback	Calls the callback `callback_name` from the configuration with `args`
#cleanup_listener, #cleanup_shutdown_pipe, #setup_shutdown_pipe,
#start_thread	Starts a server thread for the client socket `sock` that runs the given `block`.

Constructor Details

.new(config = {}, default = Config::HTTP) ⇒ `HTTPProxyServer`

Proxy server configurations. The proxy server handles the following configuration items in addition to those supported by HTTPServer:

:ProxyAuthProc: Called with a request and response to authorize a request
:ProxyVia: Appended to the via header
:ProxyURI: The proxy server’s URI
:ProxyContentHandler: Called with a request and response and allows modification of the response
:ProxyTimeout: Sets the proxy timeouts to 30 seconds for open and 60 seconds for read operations

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 84


def initialize(config={}, default=Config::HTTP)
  super(config, default)
  c = @config
  @via = "#{c[:HTTPVersion]} #{c[:ServerName]}:#{c[:Port]}"
end

Instance Method Details

#choose_header(src, dst) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 237


def choose_header(src, dst)
  connections = split_field(src['connection'])
  src.each{|key, value|
    key = key.downcase
    if HopByHop.member?(key)          || # RFC2616: 13.5.1
       connections.member?(key)       || # RFC2616: 14.10
       ShouldNotTransfer.member?(key)    # pragmatics
      @logger.debug("choose_header: `#{key}: #{value}'")
      next
    end
    dst[key] = value
  }
end

#do_CONNECT(req, res)

This method is for internal use only.

Raises:

(HTTPStatus::InternalServerError)

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 133


def do_CONNECT(req, res)
  # Proxy Authentication
  proxy_auth(req, res)

  ua = Thread.current[:WEBrickSocket]  # User-Agent
  raise HTTPStatus::InternalServerError,
    "[BUG] cannot get socket" unless ua

  host, port = req.unparsed_uri.split(":", 2)
  # Proxy authentication for upstream proxy server
  if proxy = proxy_uri(req, res)
    proxy_request_line = "CONNECT #{host}:#{port} HTTP/1.0"
    if proxy.userinfo
      credentials = "Basic " + [proxy.userinfo].pack("m0")
    end
    host, port = proxy.host, proxy.port
  end

  begin
    @logger.debug("CONNECT: upstream proxy is `#{host}:#{port}'.")
    os = TCPSocket.new(host, port)     # origin server

    if proxy
      @logger.debug("CONNECT: sending a Request-Line")
      os << proxy_request_line << CRLF
      @logger.debug("CONNECT: > #{proxy_request_line}")
      if credentials
        @logger.debug("CONNECT: sending credentials")
        os << "Proxy-Authorization: " << credentials << CRLF
      end
      os << CRLF
      proxy_status_line = os.gets(LF)
      @logger.debug("CONNECT: read Status-Line from the upstream server")
      @logger.debug("CONNECT: < #{proxy_status_line}")
      if %r{^HTTP/\d\.\d\s+200\s*} =~ proxy_status_line
        while line = os.gets(LF)
          break if /\A(#{CRLF}|#{LF})\z/om =~ line
        end
      else
        raise HTTPStatus::BadGateway
      end
    end
    @logger.debug("CONNECT #{host}:#{port}: succeeded")
    res.status = HTTPStatus::RC_OK
  rescue => ex
    @logger.debug("CONNECT #{host}:#{port}: failed `#{ex.message}'")
    res.set_error(ex)
    raise HTTPStatus::EOFError
  ensure
    if handler = @config[:ProxyContentHandler]
      handler.call(req, res)
    end
    res.send_response(ua)
    access_log(@config, req, res)

    # Should clear request-line not to send the response twice.
    # see: HTTPServer#run
    req.parse(NullReader) rescue nil
  end

  begin
    while fds = IO::select([ua, os])
      if fds[0].member?(ua)
        buf = ua.readpartial(1024);
        @logger.debug("CONNECT: #{buf.bytesize} byte from User-Agent")
        os.write(buf)
      elsif fds[0].member?(os)
        buf = os.readpartial(1024);
        @logger.debug("CONNECT: #{buf.bytesize} byte from #{host}:#{port}")
        ua.write(buf)
      end
    end
  rescue
    os.close
    @logger.debug("CONNECT #{host}:#{port}: closed")
  end

  raise HTTPStatus::EOFError
end

#do_GET(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 213


def do_GET(req, res)
  perform_proxy_request(req, res, Net::HTTP::Get)
end

#do_HEAD(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 217


def do_HEAD(req, res)
  perform_proxy_request(req, res, Net::HTTP::Head)
end

#do_OPTIONS(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 225


def do_OPTIONS(req, res)
  res['allow'] = "GET,HEAD,POST,OPTIONS,CONNECT"
end

#do_POST(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 221


def do_POST(req, res)
  perform_proxy_request(req, res, Net::HTTP::Post, req.body_reader)
end

#perform_proxy_request(req, res, req_class, body_stream = nil) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 298


def perform_proxy_request(req, res, req_class, body_stream = nil)
  uri = req.request_uri
  path = uri.path.dup
  path << "?" << uri.query if uri.query
  header = setup_proxy_header(req, res)
  upstream = setup_upstream_proxy_authentication(req, res, header)

  body_tmp = []
  http = Net::HTTP.new(uri.host, uri.port, upstream.host, upstream.port)
  req_fib = Fiber.new do
    http.start do
      if @config[:ProxyTimeout]
        ##################################   these issues are
        http.open_timeout = 30   # secs  #   necessary (maybe because
        http.read_timeout = 60   # secs  #   Ruby's bug, but why?)
        ##################################
      end
      if body_stream && req['transfer-encoding'] =~ /\bchunked\b/i
        header['Transfer-Encoding'] = 'chunked'
      end
      http_req = req_class.new(path, header)
      http_req.body_stream = body_stream if body_stream
      http.request(http_req) do |response|
        # Persistent connection requirements are mysterious for me.
        # So I will close the connection in every response.
        res['proxy-connection'] = "close"
        res['connection'] = "close"

        # stream Net::HTTP::HTTPResponse to WEBrick::HTTPResponse
        res.status = response.code.to_i
        res.chunked = response.chunked?
        choose_header(response, res)
        set_cookie(response, res)
        set_via(res)
        response.read_body do |buf|
          body_tmp << buf
          Fiber.yield # wait for res.body Proc#call
        end
      end # http.request
    end
  end
  req_fib.resume # read HTTP response headers and first chunk of the body
  res.body = ->(socket) do
    while buf = body_tmp.shift
      socket.write(buf)
      buf.clear
      req_fib.resume # continue response.read_body
    end
  end
end

#proxy_auth(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 101


def proxy_auth(req, res)
  if proc = @config[:ProxyAuthProc]
    proc.call(req, res)
  end
  req.header.delete("proxy-authorization")
end

#proxy_service(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 113


def proxy_service(req, res)
  # Proxy Authentication
  proxy_auth(req, res)

  begin
    self.send("do_#{req.request_method}", req, res)
  rescue NoMethodError
    raise HTTPStatus::MethodNotAllowed,
      "unsupported method `#{req.request_method}'."
  rescue => err
    logger.debug("#{err.class}: #{err.message}")
    raise HTTPStatus::ServiceUnavailable, err.message
  end

  # Process contents
  if handler = @config[:ProxyContentHandler]
    handler.call(req, res)
  end
end

#proxy_uri(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 108


def proxy_uri(req, res)
  # should return upstream proxy server's URI
  return @config[:ProxyURI]
end

#service(req, res)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 91


def service(req, res)
  if req.request_method == "CONNECT"
    do_CONNECT(req, res)
  elsif req.unparsed_uri =~ %r!^http://!
    proxy_service(req, res)
  else
    super(req, res)
  end
end

#set_cookie(src, dst) (private)

This method is for internal use only.

Net::HTTP is stupid about the multiple header fields. Here is workaround:

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 253


def set_cookie(src, dst)
  if str = src['set-cookie']
    cookies = []
    str.split(/,\s*/).each{|token|
      if /^[^=]+;/o =~ token
        cookies[-1] << ", " << token
      elsif /=/o =~ token
        cookies << token
      else
        cookies[-1] << ", " << token
      end
    }
    dst.cookies.replace(cookies)
  end
end

#set_via(h) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 269


def set_via(h)
  if @config[:ProxyVia]
    if  h['via']
      h['via'] << ", " << @via
    else
      h['via'] = @via
    end
  end
end

#setup_proxy_header(req, res) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 279


def setup_proxy_header(req, res)
  # Choose header fields to transfer
  header = Hash.new
  choose_header(req, header)
  set_via(header)
  return header
end

#setup_upstream_proxy_authentication(req, res, header) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 287


def setup_upstream_proxy_authentication(req, res, header)
  if upstream = proxy_uri(req, res)
    if upstream.userinfo
      header['proxy-authorization'] =
        "Basic " + [upstream.userinfo].pack("m0")
    end
    return upstream
  end
  return FakeProxyURI
end

#split_field(f) (private)

This method is for internal use only.

[ GitHub ]

# File 'lib/webrick/httpproxy.rb', line 235


def split_field(f) f ? f.split(/,\s+/).collect{|i| i.downcase } : [] end

Class: WEBrick::HTTPProxyServer

Overview

Modifying proxied responses

Constant Summary

Class Method Summary

HTTPServer - Inherited

GenericServer - Inherited

Instance Attribute Summary

GenericServer - Inherited

Instance Method Summary

HTTPServer - Inherited

GenericServer - Inherited

Constructor Details

.new(config = {}, default = Config::HTTP) ⇒ HTTPProxyServer

Instance Method Details

#choose_header(src, dst) (private)

#do_CONNECT(req, res)

#do_GET(req, res)

#do_HEAD(req, res)

#do_OPTIONS(req, res)

#do_POST(req, res)

#perform_proxy_request(req, res, req_class, body_stream = nil) (private)

#proxy_auth(req, res)

#proxy_service(req, res)

#proxy_uri(req, res)

#service(req, res)

#set_cookie(src, dst) (private)

#set_via(h) (private)

#setup_proxy_header(req, res) (private)

#setup_upstream_proxy_authentication(req, res, header) (private)

#split_field(f) (private)

`HTTPServer` - Inherited

`GenericServer` - Inherited

`GenericServer` - Inherited

`HTTPServer` - Inherited

`GenericServer` - Inherited

.new(config = {}, default = Config::HTTP) ⇒ `HTTPProxyServer`