Class: PStore

Relationships & Source Files
Namespace Children
Exceptions: Error
Inherits:	Object
Defined in:	lib/pstore.rb

Overview

PStore implements a file based persistence mechanism based on a Hash. User code can store hierarchies of Ruby objects (values) into the data store file by name (keys). An object hierarchy may be just a single object. User code may later read values back from the data store or even update data, as needed.

The transactional behavior ensures that any changes succeed or fail together. This can be used to ensure that the data store is not left in a transitory state, where some values were updated but others were not.

Behind the scenes, Ruby objects are stored to the data store file with Marshal. That carries the usual limitations. Proc objects cannot be marshalled, for example.

Usage example:

require "pstore"

# a mock wiki object...
class WikiPage
  def initialize( page_name, author, contents )
    @page_name = page_name
    @revisions = Array.new

    add_revision(author, contents)
  end

  attr_reader :page_name

  def add_revision( author, contents )
    @revisions << { :created  => Time.now,
                    :author   => author,
                    :contents => contents }
  end

  def wiki_page_references
    [@page_name] + @revisions.last[:contents].scan(/\b(?:[A-Z][a-z]){2,}/)
  end

  # ...
end

# create a new page...
home_page = WikiPage.new( "HomePage", "James Edward Gray II",
                          "A page about the JoysOfDocumentation..." )

# then we want to update page data and the index together, or not at all...
wiki = PStore.new("wiki_pages.pstore")
wiki.transaction do  # begin transaction; do all of this or none of it
  # store page...
  wiki[home_page.page_name] = home_page
  # ensure that an index has been created...
  wiki[:wiki_index] ||= Array.new
  # update wiki index...
  wiki[:wiki_index].push(*home_page.wiki_page_references)
end                   # commit changes to wiki data store file

### Some time later... ###

# read wiki data...
wiki.transaction(true) do  # begin read-only transaction, no changes allowed
  wiki.roots.each do |data_root_name|
    p data_root_name
    p wiki[data_root_name]
  end
end

Transaction modes

By default, file integrity is only ensured as long as the operating system (and the underlying hardware) doesn't raise any unexpected I/O errors. If an I/O error occurs while PStore is writing to its file, then the file will become corrupted.

You can prevent this by setting pstore.ultra_safe = true. However, this results in a minor performance loss, and only works on platforms that support atomic file renames. Please consult the documentation for #ultra_safe for details.

Needless to say, if you're storing valuable data with PStore, then you should backup the PStore files from time to time.

Constant Summary

EMPTY_MARSHAL_CHECKSUM =
# File 'lib/pstore.rb', line 356
```
Digest::MD5.digest(EMPTY_MARSHAL_DATA)
```
EMPTY_MARSHAL_DATA =
# File 'lib/pstore.rb', line 355
```
Marshal.dump({})
```
EMPTY_STRING =

Constant for relieving Ruby's garbage collector.

# File 'lib/pstore.rb', line 354
```
""
```

RDWR_ACCESS =
# File 'lib/pstore.rb', line 94

{mode: IO::RDWR | IO::CREAT | IO::BINARY, encoding: Encoding::ASCII_8BIT}.freeze

RD_ACCESS =
# File 'lib/pstore.rb', line 95

{mode: IO::RDONLY | IO::BINARY, encoding: Encoding::ASCII_8BIT}.freeze

WR_ACCESS =
# File 'lib/pstore.rb', line 96

{mode: IO::WRONLY | IO::CREAT | IO::TRUNC | IO::BINARY, encoding: Encoding::ASCII_8BIT}.freeze

Class Method Summary

.new(file, thread_safe = false) ⇒ PStore constructor

To construct a PStore object, pass in the file path where you would like the data to be stored.

Instance Attribute Summary

#ultra_safe rw

Whether PStore should do its best to prevent file corruptions, even when under unlikely-to-occur error conditions such as out-of-space conditions and other unusual OS filesystem errors.
#on_windows? ⇒ Boolean readonly private

Instance Method Summary

#[](name)

Retrieves a value from the PStore file data, by name.
#[]=(name, value)

Stores an individual Ruby object or a hierarchy of Ruby objects in the data store file under the root name.
#abort

Ends the current #transaction, discarding any changes to the data store.
#commit

Ends the current #transaction, committing any changes to the data store immediately.
#delete(name)

Removes an object hierarchy from the data store, by name.
#fetch(name, default = PStore::Error)

This method is just like #[], save that you may also provide a default value for the object.
#path

Returns the path to the data store file.
#root?(name) ⇒ Boolean

Returns true if the supplied name is currently in the data store.
#roots

Returns the names of all object hierarchies currently in the store.
#transaction(read_only = false)

Opens a new transaction for the data store.
#empty_marshal_checksum private
#empty_marshal_data private
#in_transaction private

Raises Error if the calling code is not in a #transaction.
#in_transaction_wr private

Raises Error if the calling code is not in a #transaction or if the code is in a read-only #transaction.
#load_data(file, read_only) private

Load the given PStore file.
#open_and_lock_file(filename, read_only) private

Open the specified filename (either in read-only mode or in read-write mode) and lock it for reading or writing.
#save_data(original_checksum, original_file_size, file) private
#save_data_with_atomic_file_rename_strategy(data, file) private
#save_data_with_fast_strategy(data, file) private

Constructor Details

.new(file, thread_safe = false) ⇒ `PStore`

To construct a PStore object, pass in the file path where you would like the data to be stored.

PStore objects are always reentrant. But if thread_safe is set to true, then it will become thread-safe at the cost of a minor performance hit.

[ GitHub ]

# File 'lib/pstore.rb', line 118


def initialize(file, thread_safe = false)
  dir = File::dirname(file)
  unless File::directory? dir
    raise PStore::Error, format("directory %s does not exist", dir)
  end
  if File::exist? file and not File::readable? file
    raise PStore::Error, format("file %s not readable", file)
  end
  @filename = file
  @abort = false
  @ultra_safe = false
  @thread_safe = thread_safe
  @lock = Mutex.new
end

Instance Attribute Details

#on_windows? ⇒ `Boolean` (readonly, private)

[ GitHub ]

# File 'lib/pstore.rb', line 420


def on_windows?
  is_windows = RUBY_PLATFORM =~ /mswin|mingw|bccwin|wince/
  self.class.__send__(:define_method, :on_windows?) do
    is_windows
  end
  is_windows
end

#ultra_safe (rw)

Whether PStore should do its best to prevent file corruptions, even when under unlikely-to-occur error conditions such as out-of-space conditions and other unusual OS filesystem errors. Setting this flag comes at the price in the form of a performance loss.

This flag only has effect on platforms on which file renames are atomic (e.g. all POSIX platforms: Linux, MacOS X, FreeBSD, etc). The default value is false.

[ GitHub ]

# File 'lib/pstore.rb', line 109


attr_accessor :ultra_safe

Instance Method Details

#[](name)

Retrieves a value from the PStore file data, by name. The hierarchy of Ruby objects stored under that root name will be returned.

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 154


def [](name)
  in_transaction
  @table[name]
end

#[]=(name, value)

Stores an individual Ruby object or a hierarchy of Ruby objects in the data store file under the root name. Assigning to a name already in the data store clobbers the old data.

Example:

require "pstore"

store = PStore.new("data_file.pstore")
store.transaction do  # begin transaction
  # load some data into the store...
  store[:single_object] = "My data..."
  store[:obj_heirarchy] = { "Kev Jackson" => ["rational.rb", "pstore.rb"],
                            "James Gray"  => ["erb.rb", "pstore.rb"] }
end                   # commit changes to data store file

WARNING: This method is only valid in a #transaction and it cannot be read-only. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 199


def []=(name, value)
  in_transaction_wr
  @table[name] = value
end

#abort

Ends the current #transaction, discarding any changes to the data store.

Example:

require "pstore"

store = PStore.new("data_file.pstore")
store.transaction do  # begin transaction
  store[:one] = 1     # this change is not applied, see below...
  store[:two] = 2     # this change is not applied, see below...

  store.abort         # end transaction here, discard all changes

  store[:three] = 3   # this change is never reached
end

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 287


def abort
  in_transaction
  @abort = true
  throw :pstore_abort_transaction
end

#commit

Ends the current #transaction, committing any changes to the data store immediately.

Example:

require "pstore"

store = PStore.new("data_file.pstore")
store.transaction do  # begin transaction
  # load some data into the store...
  store[:one] = 1
  store[:two] = 2

  store.commit        # end transaction here, committing changes

  store[:three] = 3   # this change is never reached
end

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 261


def commit
  in_transaction
  @abort = false
  throw :pstore_abort_transaction
end

#delete(name)

Removes an object hierarchy from the data store, by name.

WARNING: This method is only valid in a #transaction and it cannot be read-only. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 209


def delete(name)
  in_transaction_wr
  @table.delete name
end

#empty_marshal_checksum (private)

[ GitHub ]

# File 'lib/pstore.rb', line 481


def empty_marshal_checksum
  EMPTY_MARSHAL_CHECKSUM
end

#empty_marshal_data (private)

[ GitHub ]

# File 'lib/pstore.rb', line 478


def empty_marshal_data
  EMPTY_MARSHAL_DATA
end

#fetch(name, default = PStore::Error)

This method is just like #[], save that you may also provide a default value for the object. In the event the specified name is not found in the data store, your default will be returned instead. If you do not specify a default, ::PStore::Error will be raised if the object is not found.

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 168


def fetch(name, default=PStore::Error)
  in_transaction
  unless @table.key? name
    if default == PStore::Error
      raise PStore::Error, format("undefined root name `%s'", name)
    else
      return default
    end
  end
  @table[name]
end

#in_transaction (private)

Raises ::PStore::Error if the calling code is not in a #transaction.

Raises:

(PStore::Error)

[ GitHub ]

# File 'lib/pstore.rb', line 134


def in_transaction
  raise PStore::Error, "not in transaction" unless @lock.locked?
end

#in_transaction_wr (private)

Raises ::PStore::Error if the calling code is not in a #transaction or if the code is in a read-only #transaction.

Raises:

(PStore::Error)

[ GitHub ]

# File 'lib/pstore.rb', line 141


def in_transaction_wr
  in_transaction
  raise PStore::Error, "in read-only transaction" if @rdonly
end

#load_data(file, read_only) (private)

Load the given PStore file. If read_only is true, the unmarshalled Hash will be returned. If read_only is false, a 3-tuple will be returned: the unmarshalled Hash, an MD5 checksum of the data, and the size of the data.

[ GitHub ]

# File 'lib/pstore.rb', line 392


def load_data(file, read_only)
  if read_only
    begin
      table = load(file)
      raise Error, "PStore file seems to be corrupted." unless table.is_a?(Hash)
    rescue EOFError
      # This seems to be a newly-created file.
      table = {}
    end
    table
  else
    data = file.read
    if data.empty?
      # This seems to be a newly-created file.
      table = {}
      checksum = empty_marshal_checksum
      size = empty_marshal_data.bytesize
    else
      table = load(data)
      checksum = Digest::MD5.digest(data)
      size = data.bytesize
      raise Error, "PStore file seems to be corrupted." unless table.is_a?(Hash)
    end
    data.replace(EMPTY_STRING)
    [table, checksum, size]
  end
end

#open_and_lock_file(filename, read_only) (private)

Open the specified filename (either in read-only mode or in read-write mode) and lock it for reading or writing.

The opened File object will be returned. If read_only is true, and the file does not exist, then nil will be returned.

All exceptions are propagated.

[ GitHub ]

# File 'lib/pstore.rb', line 367


def open_and_lock_file(filename, read_only)
  if read_only
    begin
      file = File.new(filename, RD_ACCESS)
      begin
        file.flock(File::LOCK_SH)
        return file
      rescue
        file.close
        raise
      end
    rescue Errno::ENOENT
      return nil
    end
  else
    file = File.new(filename, RDWR_ACCESS)
    file.flock(File::LOCK_EX)
    return file
  end
end

#path

Returns the path to the data store file.

[ GitHub ]

# File 'lib/pstore.rb', line 235


def path
  @filename
end

#root?(name) ⇒ `Boolean`

Returns true if the supplied name is currently in the data store.

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 230


def root?(name)
  in_transaction
  @table.key? name
end

#roots

Returns the names of all object hierarchies currently in the store.

WARNING: This method is only valid in a #transaction. It will raise ::PStore::Error if called at any other time.

[ GitHub ]

# File 'lib/pstore.rb', line 220


def roots
  in_transaction
  @table.keys
end

#save_data(original_checksum, original_file_size, file) (private)

[ GitHub ]

# File 'lib/pstore.rb', line 428


def save_data(original_checksum, original_file_size, file)
  new_data = dump(@table)

  if new_data.bytesize != original_file_size || Digest::MD5.digest(new_data) != original_checksum
    if @ultra_safe && !on_windows?
      # Windows doesn't support atomic file renames.
      save_data_with_atomic_file_rename_strategy(new_data, file)
    else
      save_data_with_fast_strategy(new_data, file)
    end
  end

  new_data.replace(EMPTY_STRING)
end

#save_data_with_atomic_file_rename_strategy(data, file) (private)

[ GitHub ]

# File 'lib/pstore.rb', line 443


def save_data_with_atomic_file_rename_strategy(data, file)
  temp_filename = "#{@filename}.tmp.#{Process.pid}.#{rand 1000000}"
  temp_file = File.new(temp_filename, WR_ACCESS)
  begin
    temp_file.flock(File::LOCK_EX)
    temp_file.write(data)
    temp_file.flush
    File.rename(temp_filename, @filename)
  rescue
    File.unlink(temp_file) rescue nil
    raise
  ensure
    temp_file.close
  end
end

#save_data_with_fast_strategy(data, file) (private)

[ GitHub ]

# File 'lib/pstore.rb', line 459


def save_data_with_fast_strategy(data, file)
  file.rewind
  file.write(data)
  file.truncate(data.bytesize)
end

#transaction(read_only = false)

Opens a new transaction for the data store. Code executed inside a block passed to this method may read and write data to and from the data store file.

At the end of the block, changes are committed to the data store automatically. You may exit the transaction early with a call to either #commit or #abort. See those methods for details about how changes are handled. Raising an uncaught Exception in the block is equivalent to calling #abort.

If read_only is set to true, you will only be allowed to read from the data store during the transaction and any attempts to change the data will raise a ::PStore::Error.

Note that PStore does not support nested transactions.

[ GitHub ]

# File 'lib/pstore.rb', line 310


def transaction(read_only = false)  # :yields:  pstore
  value = nil
  if !@thread_safe
    raise PStore::Error, "nested transaction" unless @lock.try_lock
  else
    begin
      @lock.lock
    rescue ThreadError
      raise PStore::Error, "nested transaction"
    end
  end
  begin
    @rdonly = read_only
    @abort = false
    file = open_and_lock_file(@filename, read_only)
    if file
      begin
        @table, checksum, original_data_size = load_data(file, read_only)

        catch(:pstore_abort_transaction) do
          value = yield(self)
        end

        if !@abort && !read_only
          save_data(checksum, original_data_size, file)
        end
      ensure
        file.close if !file.closed?
      end
    else
      # This can only occur if read_only == true.
      @table = {}
      catch(:pstore_abort_transaction) do
        value = yield(self)
      end
    end
  ensure
    @lock.unlock
  end
  value
end

Class: PStore

Overview

Usage example:

Transaction modes

Constant Summary

Class Method Summary

Instance Attribute Summary

Instance Method Summary

Constructor Details

.new(file, thread_safe = false) ⇒ PStore

Instance Attribute Details

#on_windows? ⇒ Boolean (readonly, private)

#ultra_safe (rw)

Instance Method Details

#[](name)

#[]=(name, value)

Example:

#abort

Example:

#commit

Example:

#delete(name)

#empty_marshal_checksum (private)

#empty_marshal_data (private)

#fetch(name, default = PStore::Error)

#in_transaction (private)

#in_transaction_wr (private)

#load_data(file, read_only) (private)

#open_and_lock_file(filename, read_only) (private)

#path

#root?(name) ⇒ Boolean

#roots

#save_data(original_checksum, original_file_size, file) (private)

#save_data_with_atomic_file_rename_strategy(data, file) (private)

#save_data_with_fast_strategy(data, file) (private)

#transaction(read_only = false)

.new(file, thread_safe = false) ⇒ `PStore`

#on_windows? ⇒ `Boolean` (readonly, private)

#root?(name) ⇒ `Boolean`