Class: ActiveRecord::ConnectionAdapters::ConnectionPool

Relationships & Source Files
Namespace Children
Classes: `Queue`, `Reaper`
Super Chains via Extension / Inclusion / Inheritance
Instance Chain: self, `QueryCache::ConnectionPoolConfiguration`, MonitorMixin
Inherits:	Object
Defined in:	activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb, activerecord/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb, activerecord/lib/active_record/connection_adapters/abstract/connection_pool/reaper.rb

Overview

Active Record Connection Pool

Connection pool base class for managing Active Record database connections.

Introduction

A connection pool synchronizes thread access to a limited number of database connections. The basic idea is that each thread checks out a database connection from the pool, uses that connection, and checks the connection back in. ConnectionPool is completely thread-safe, and will ensure that a connection cannot be used by two threads at the same time, as long as ConnectionPool’s contract is correctly followed. It will also handle cases in which there are more threads than connections: if all connections have been checked out, and a thread tries to checkout a connection anyway, then ConnectionPool will wait until some other thread has checked in a connection, or the #checkout_timeout has expired.

Obtaining (checking out) a connection

Connections can be obtained and used from a connection pool in several ways:

Simply use {ActiveRecord::Base.connection}. When you’re done with the connection(s) and wish it to be returned to the pool, you call ActiveRecord::Base.connection_handler.clear_active_connections!. This is the default behavior for Active Record when used in conjunction with Action Pack’s request handling cycle.
Manually check out a connection from the pool with ActiveRecord::Base.connection_pool.checkout. You are responsible for returning this connection to the pool when finished by calling ActiveRecord::Base.connection_pool.checkin(connection).
Use {ActiveRecord::Base.connection_pool.with_connection(&block)}, which obtains a connection, yields it as the sole argument to the block, and returns it to the pool after the block completes.

Connections in the pool are actually AbstractAdapter objects (or objects compatible with AbstractAdapter’s interface).

While a thread has a connection checked out from the pool using one of the above three methods, that connection will automatically be the one used by ::ActiveRecord queries executing on that thread. It is not required to explicitly pass the checked out connection to Rails models or queries, for example.

Options

There are several connection-pooling-related options that you can add to your database connection configuration:

pool: maximum number of connections the pool may manage (default 5).
idle_timeout: number of seconds that a connection will be kept unused in the pool before it is automatically disconnected (default 300 seconds). Set this to zero to keep connections forever.
#checkout_timeout: number of seconds to wait for a connection to become available before giving up and raising a timeout error (default 5 seconds).

Class Method Summary

.new(pool_config) ⇒ ConnectionPool constructor

Creates a new ConnectionPool object.

Instance Attribute Summary

#active_connection? ⇒ Boolean readonly

Returns true if there is an open connection being used for the current thread.
#async_executor readonly
#automatic_reconnect rw
#checkout_timeout rw
#connected? ⇒ Boolean readonly

Returns true if a connection has already been opened.
#db_config readonly
#lock_thread=(lock_thread) writeonly
#pool_config readonly
#reaper readonly
#role readonly
#schema_reflection rw
#shard readonly
#size readonly

Instance Method Summary

#checkin(conn)

Check-in a database connection back into the pool, indicating that you no longer need this connection.
#checkout(checkout_timeout = @checkout_timeout)

Check-out a database connection from the pool, indicating that you want to use it.
#clear_reloadable_connections(raise_on_acquisition_timeout = true)

Clears the cache which maps classes and re-connects connections that require reloading.
#clear_reloadable_connections!

Clears the cache which maps classes and re-connects connections that require reloading.
#connection

Retrieve the connection associated with the current thread, or call #checkout to obtain one if necessary.
#connection_klass

Alias for #connection_class.
#connections

Returns an array containing the connections currently in the pool.
#disconnect(raise_on_acquisition_timeout = true)

Disconnects all connections in the pool, and clears the pool.
#disconnect!

Disconnects all connections in the pool, and clears the pool.
#flush(minimum_idle = @idle_timeout)

Disconnect all connections that have been idle for at least minimum_idle seconds.
#flush!

Disconnect all currently idle connections.
#reap

Recover lost connections for the pool.
#release_connection(owner_thread = ActiveSupport::IsolatedExecutionState.context)

Signal that the thread is finished with the current connection.
#remove(conn)

Remove a connection from the connection pool.
#stat

Returns the connection pool’s usage statistic.
#with_connection

Yields a connection from the connection pool to the block.

`QueryCache::ConnectionPoolConfiguration` - Included

#disable_query_cache!, #enable_query_cache!, #initialize, #query_cache_enabled

Constructor Details

.new(pool_config) ⇒ `ConnectionPool`

Creates a new ConnectionPool object. #pool_config is a PoolConfig object which describes database connection information (e.g. adapter, host name, username, password, etc), as well as the maximum size for this ConnectionPool.

The default ConnectionPool maximum size is 5.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 120


def initialize(pool_config)
  super()

  @pool_config = pool_config
  @db_config = pool_config.db_config
  @role = pool_config.role
  @shard = pool_config.shard

  @checkout_timeout = db_config.checkout_timeout
  @idle_timeout = db_config.idle_timeout
  @size = db_config.pool

  # This variable tracks the cache of threads mapped to reserved connections, with the
  # sole purpose of speeding up the connection method. It is not the authoritative
  # registry of which thread owns which connection. Connection ownership is tracked by
  # the connection.owner attr on each connection instance.
  # The invariant works like this: if there is mapping of <tt>thread => conn</tt>,
  # then that thread does indeed own that conn. However, an absence of such
  # mapping does not mean that the thread doesn't own the said connection. In
  # that case conn.owner attr should be consulted.
  # Access and modification of <tt>@thread_cached_conns</tt> does not require
  # synchronization.
  @thread_cached_conns = Concurrent::Map.new(initial_capacity: @size)

  @connections         = []
  @automatic_reconnect = true

  # Connection pool allows for concurrent (outside the main synchronize section)
  # establishment of new connections. This variable tracks the number of threads
  # currently in the process of independently establishing connections to the DB.
  @now_connecting = 0

  @threads_blocking_new_connections = 0

  @available = ConnectionLeasingQueue.new self

  @lock_thread = false

  @async_executor = build_async_executor

  @reaper = Reaper.new(self, db_config.reaping_frequency)
  @reaper.run
end

Instance Attribute Details

#active_connection? ⇒ `Boolean` (readonly)

Returns true if there is an open connection being used for the current thread.

This method only works for connections that have been obtained through #connection or #with_connection methods. Connections obtained through #checkout will not be detected by #active_connection?

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 196


def active_connection?
  @thread_cached_conns[connection_cache_key(current_thread)]
end

#async_executor (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#automatic_reconnect (rw)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 109


attr_accessor :automatic_reconnect, :checkout_timeout

#checkout_timeout (rw)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 109


attr_accessor :automatic_reconnect, :checkout_timeout

#connected? ⇒ `Boolean` (readonly)

Returns true if a connection has already been opened.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 233


def connected?
  synchronize { @connections.any? }
end

#db_config (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#lock_thread=(lock_thread) (writeonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 164


def lock_thread=(lock_thread)
  if lock_thread
    @lock_thread = ActiveSupport::IsolatedExecutionState.context
  else
    @lock_thread = nil
  end

  if (active_connection = @thread_cached_conns[connection_cache_key(current_thread)])
    active_connection.lock_thread = @lock_thread
  end
end

#pool_config (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#reaper (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#role (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#schema_reflection (rw)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 112


delegate :schema_reflection, :schema_reflection=, to: :pool_config

#shard (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

#size (readonly)

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 110


attr_reader :db_config, :size, :reaper, :pool_config, :async_executor, :role, :shard

Instance Method Details

#checkin(conn)

Check-in a database connection back into the pool, indicating that you no longer need this connection.

conn: an AbstractAdapter object, which was obtained by earlier by calling #checkout on this pool.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 363


def checkin(conn)
  conn.lock.synchronize do
    synchronize do
      remove_connection_from_thread_cache conn

      conn._run_checkin_callbacks do
        conn.expire
      end

      conn.lock_thread = nil
      @available.add conn
    end
  end
end

#checkout(checkout_timeout = @checkout_timeout)

Check-out a database connection from the pool, indicating that you want to use it. You should call #checkin when you no longer need this.

This is done by either returning and leasing existing connection, or by creating a new connection and leasing it.

If all connections are leased and the pool is at capacity (meaning the number of currently leased connections is greater than or equal to the size limit set), an ::ActiveRecord::ConnectionTimeoutError exception will be raised.

Returns: an AbstractAdapter object.

Raises:

::ActiveRecord::ConnectionTimeoutError no connection can be obtained from the pool.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 352


def checkout(checkout_timeout = @checkout_timeout)
  connection = checkout_and_verify(acquire_connection(checkout_timeout))
  connection.lock_thread = @lock_thread
  connection
end

#clear_reloadable_connections(raise_on_acquisition_timeout = true)

Clears the cache which maps classes and re-connects connections that require reloading.

Raises:

::ActiveRecord::ExclusiveConnectionTimeoutError if unable to gain ownership of all connections in the pool within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds).

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 310


def clear_reloadable_connections(raise_on_acquisition_timeout = true)
  with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do
    synchronize do
      @connections.each do |conn|
        if conn.in_use?
          conn.steal!
          checkin conn
        end
        conn.disconnect! if conn.requires_reloading?
      end
      @connections.delete_if(&:requires_reloading?)
      @available.clear
    end
  end
end

#clear_reloadable_connections!

Clears the cache which maps classes and re-connects connections that require reloading.

The pool first tries to gain ownership of all connections. If unable to do so within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds), then the pool forcefully clears the cache and reloads connections without any regard for other connection owning threads.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 334


def clear_reloadable_connections!
  clear_reloadable_connections(false)
end

#connection

Retrieve the connection associated with the current thread, or call #checkout to obtain one if necessary.

#connection can be called any number of times; the connection is held in a cache keyed by a thread.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 181


def connection
  @thread_cached_conns[connection_cache_key(current_thread)] ||= checkout
end

#connection_klass

Alias for #connection_class.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 188


alias :connection_klass :connection_class

#connections

Returns an array containing the connections currently in the pool. Access to the array does not require synchronization on the pool because the array is newly created and not retained by the pool.

However; this method bypasses the ConnectionPool’s thread-safe connection access pattern. A returned connection may be owned by another thread, unowned, or by happen-stance owned by the calling thread.

Calling methods on a connection without ownership is subject to the thread-safety guarantees of the underlying method. Many of the methods on connection adapter classes are inherently multi-thread unsafe.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 248


def connections
  synchronize { @connections.dup }
end

#disconnect(raise_on_acquisition_timeout = true)

Disconnects all connections in the pool, and clears the pool.

Raises:

::ActiveRecord::ExclusiveConnectionTimeoutError if unable to gain ownership of all connections in the pool within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds).

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 258


def disconnect(raise_on_acquisition_timeout = true)
  with_exclusively_acquired_all_connections(raise_on_acquisition_timeout) do
    synchronize do
      @connections.each do |conn|
        if conn.in_use?
          conn.steal!
          checkin conn
        end
        conn.disconnect!
      end
      @connections = []
      @available.clear
    end
  end
end

#disconnect!

Disconnects all connections in the pool, and clears the pool.

The pool first tries to gain ownership of all connections. If unable to do so within a timeout interval (default duration is spec.db_config.checkout_timeout * 2 seconds), then the pool is forcefully disconnected without any regard for other connection owning threads.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 280


def disconnect!
  disconnect(false)
end

#flush(minimum_idle = @idle_timeout)

Disconnect all connections that have been idle for at least minimum_idle seconds. Connections currently checked out, or that were checked in less than minimum_idle seconds ago, are unaffected.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 435


def flush(minimum_idle = @idle_timeout)
  return if minimum_idle.nil?

  idle_connections = synchronize do
    return if self.discarded?
    @connections.select do |conn|
      !conn.in_use? && conn.seconds_idle >= minimum_idle
    end.each do |conn|
      conn.lease

      @available.delete conn
      @connections.delete conn
    end
  end

  idle_connections.each do |conn|
    conn.disconnect!
  end
end

#flush!

Disconnect all currently idle connections. Connections currently checked out are unaffected.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 457


def flush!
  reap
  flush(-1)
end

#reap

Recover lost connections for the pool. A lost connection can occur if a programmer forgets to checkin a connection at the end of a thread or a thread dies unexpectedly.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 412


def reap
  stale_connections = synchronize do
    return if self.discarded?
    @connections.select do |conn|
      conn.in_use? && !conn.owner.alive?
    end.each do |conn|
      conn.steal!
    end
  end

  stale_connections.each do |conn|
    if conn.active?
      conn.reset!
      checkin conn
    else
      remove conn
    end
  end
end

#release_connection(owner_thread = ActiveSupport::IsolatedExecutionState.context)

Signal that the thread is finished with the current connection. #release_connection releases the connection-thread association and returns the connection to the pool.

This method only works for connections that have been obtained through #connection or #with_connection methods, connections obtained through #checkout will not be automatically released.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 207


def release_connection(owner_thread = ActiveSupport::IsolatedExecutionState.context)
  if conn = @thread_cached_conns.delete(connection_cache_key(owner_thread))
    checkin conn
  end
end

#remove(conn)

Remove a connection from the connection pool. The connection will remain open and active but will no longer be managed by this pool.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 380


def remove(conn)
  needs_new_connection = false

  synchronize do
    remove_connection_from_thread_cache conn

    @connections.delete conn
    @available.delete conn

    # @available.any_waiting? => true means that prior to removing this
    # conn, the pool was at its max size (@connections.size == @size).
    # This would mean that any threads stuck waiting in the queue wouldn't
    # know they could checkout_new_connection, so let's do it for them.
    # Because condition-wait loop is encapsulated in the Queue class
    # (that in turn is oblivious to ConnectionPool implementation), threads
    # that are "stuck" there are helpless. They have no way of creating
    # new connections and are completely reliant on us feeding available
    # connections into the Queue.
    needs_new_connection = @available.any_waiting?
  end

  # This is intentionally done outside of the synchronized section as we
  # would like not to hold the main mutex while checking out new connections.
  # Thus there is some chance that needs_new_connection information is now
  # stale, we can live with that (bulk_make_new_connections will make
  # sure not to exceed the pool's @size limit).
  bulk_make_new_connections(1) if needs_new_connection
end

#stat

Returns the connection pool’s usage statistic.

ActiveRecord::Base.connection_pool.stat # => { size: 15, connections: 1, busy: 1, dead: 0, idle: 0, waiting: 0, checkout_timeout: 5 }

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 469


def stat
  synchronize do
    {
      size: size,
      connections: @connections.size,
      busy: @connections.count { |c| c.in_use? && c.owner.alive? },
      dead: @connections.count { |c| c.in_use? && !c.owner.alive? },
      idle: @connections.count { |c| !c.in_use? },
      waiting: num_waiting_in_queue,
      checkout_timeout: checkout_timeout
    }
  end
end

#with_connection

Yields a connection from the connection pool to the block. If no connection is already checked out by the current thread, a connection will be checked out from the pool, yielded to the block, and then returned to the pool when the block is finished. If a connection has already been checked out on the current thread, such as via #connection or #with_connection, that existing connection will be the one yielded and it will not be returned to the pool automatically at the end of the block; it is expected that such an existing connection will be properly returned to the pool by the code that checked it out.

[ GitHub ]

# File 'activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb', line 222


def with_connection
  unless conn = @thread_cached_conns[connection_cache_key(ActiveSupport::IsolatedExecutionState.context)]
    conn = connection
    fresh_connection = true
  end
  yield conn
ensure
  release_connection if fresh_connection
end

Class: ActiveRecord::ConnectionAdapters::ConnectionPool

Overview

Active Record Connection Pool

Introduction

Obtaining (checking out) a connection

Options

Class Method Summary

Instance Attribute Summary

Instance Method Summary

QueryCache::ConnectionPoolConfiguration - Included

Constructor Details

.new(pool_config) ⇒ ConnectionPool

Instance Attribute Details

#active_connection? ⇒ Boolean (readonly)

#async_executor (readonly)

#automatic_reconnect (rw)

#checkout_timeout (rw)

#connected? ⇒ Boolean (readonly)

#db_config (readonly)

#lock_thread=(lock_thread) (writeonly)

#pool_config (readonly)

#reaper (readonly)

#role (readonly)

#schema_reflection (rw)

#shard (readonly)

#size (readonly)

Instance Method Details

#checkin(conn)

#checkout(checkout_timeout = @checkout_timeout)

#clear_reloadable_connections(raise_on_acquisition_timeout = true)

#clear_reloadable_connections!

#connection

#connection_klass

#connections

#disconnect(raise_on_acquisition_timeout = true)

#disconnect!

#flush(minimum_idle = @idle_timeout)

#flush!

#reap

#release_connection(owner_thread = ActiveSupport::IsolatedExecutionState.context)

#remove(conn)

#stat

#with_connection

`QueryCache::ConnectionPoolConfiguration` - Included

.new(pool_config) ⇒ `ConnectionPool`

#active_connection? ⇒ `Boolean` (readonly)

#connected? ⇒ `Boolean` (readonly)