Class: Gem::Indexer
Relationships & Source Files | |
Super Chains via Extension / Inclusion / Inheritance | |
Instance Chain:
|
|
Inherits: | Object |
Defined in: | lib/rubygems/indexer.rb |
Overview
Top level class for building the gem repository index.
Class Method Summary
-
.new(directory, options = {}) ⇒ Indexer
constructor
Create an indexer that will index the gems in #directory.
Instance Attribute Summary
-
#build_modern
rw
Build indexes for RubyGems 1.2.0 and newer when true.
-
#dest_directory
readonly
Index install location.
-
#dest_latest_specs_index
readonly
Latest specs index install location.
-
#dest_prerelease_specs_index
readonly
Prerelease specs index install location.
-
#dest_specs_index
readonly
Specs index install location.
-
#directory
readonly
Index build directory.
DefaultUserInteraction - Included
Instance Method Summary
-
#build_indices
Build various indices.
-
#build_marshal_gemspecs(specs)
Builds Marshal quick index gemspecs.
-
#build_modern_index(index, file, name)
Build a single index for RubyGems 1.2 and newer.
-
#build_modern_indices(specs)
Builds indices for RubyGems 1.2 and newer.
-
#compact_specs(specs)
Compacts Marshal output for the specs index data source by using identical objects as much as possible.
-
#compress(filename, extension)
Compress
filename
withextension
. -
#compress_indices
Compresses indices on disk.
-
#gem_file_list
List of gem file names to index.
-
#generate_index
Builds and installs indices.
-
#gzip(filename)
Zlib::GzipWriter
wrapper that gzipsfilename
on disk. -
#install_indices
Install generated indices into the destination directory.
-
#make_temp_directories
Make directories for index generation.
- #map_gems_to_specs(gems)
-
#paranoid(path, extension)
Ensure path and path with
extension
are identical. -
#update_index
Perform an in-place update of the repository from newly added gems.
-
#update_specs_index(index, source, dest)
Combines specs in
index
andsource
then writes out a new copy todest
.
UserInteraction - Included
#alert | Displays an alert |
#alert_error | Displays an error |
#alert_warning | Displays a warning |
#ask | Asks a |
#ask_for_password | Asks for a password with a |
#ask_yes_no | Asks a yes or no |
#choose_from_list | Asks the user to answer |
#say | Displays the given |
#terminate_interaction | Terminates the RubyGems process with the given |
#verbose | Calls |
DefaultUserInteraction - Included
Constructor Details
.new(directory, options = {}) ⇒ Indexer
Create an indexer that will index the gems in #directory.
# File 'lib/rubygems/indexer.rb', line 52
def initialize(directory, = {}) require 'fileutils' require 'tmpdir' require 'zlib' unless defined?(Builder::XChar) then raise "Gem::Indexer requires that the XML Builder library be installed:" + "\n\tgem install builder" end = { :build_modern => true }.merge @build_modern = [:build_modern] @dest_directory = directory @directory = File.join(Dir.tmpdir, "gem_generate_index_#{$$}") marshal_name = "Marshal.#{Gem.marshal_version}" @master_index = File.join @directory, 'yaml' @marshal_index = File.join @directory, marshal_name @quick_dir = File.join @directory, 'quick' @quick_marshal_dir = File.join @quick_dir, marshal_name @quick_marshal_dir_base = File.join "quick", marshal_name # FIX: UGH @quick_index = File.join @quick_dir, 'index' @latest_index = File.join @quick_dir, 'latest_index' @specs_index = File.join @directory, "specs.#{Gem.marshal_version}" @latest_specs_index = File.join(@directory, "latest_specs.#{Gem.marshal_version}") @prerelease_specs_index = File.join(@directory, "prerelease_specs.#{Gem.marshal_version}") @dest_specs_index = File.join(@dest_directory, "specs.#{Gem.marshal_version}") @dest_latest_specs_index = File.join(@dest_directory, "latest_specs.#{Gem.marshal_version}") @dest_prerelease_specs_index = File.join(@dest_directory, "prerelease_specs.#{Gem.marshal_version}") @files = [] end
Instance Attribute Details
#build_modern (rw)
Build indexes for RubyGems 1.2.0 and newer when true
# File 'lib/rubygems/indexer.rb', line 22
attr_accessor :build_modern
#dest_directory (readonly)
Index install location
# File 'lib/rubygems/indexer.rb', line 27
attr_reader :dest_directory
#dest_latest_specs_index (readonly)
Latest specs index install location
# File 'lib/rubygems/indexer.rb', line 37
attr_reader :dest_latest_specs_index
#dest_prerelease_specs_index (readonly)
Prerelease specs index install location
# File 'lib/rubygems/indexer.rb', line 42
attr_reader :dest_prerelease_specs_index
#dest_specs_index (readonly)
Specs index install location
# File 'lib/rubygems/indexer.rb', line 32
attr_reader :dest_specs_index
#directory (readonly)
Index build directory
# File 'lib/rubygems/indexer.rb', line 47
attr_reader :directory
Instance Method Details
#build_indices
Build various indices
# File 'lib/rubygems/indexer.rb', line 99
def build_indices specs = map_gems_to_specs gem_file_list Gem::Specification._resort! specs build_marshal_gemspecs specs build_modern_indices specs if @build_modern compress_indices end
#build_marshal_gemspecs(specs)
Builds Marshal quick index gemspecs.
# File 'lib/rubygems/indexer.rb', line 111
def build_marshal_gemspecs specs count = specs.count progress = ui.progress_reporter count, "Generating Marshal quick index gemspecs for #{count} gems", "Complete" files = [] Gem.time 'Generated Marshal quick index gemspecs' do specs.each do |spec| next if spec.default_gem? spec_file_name = "#{spec.original_name}.gemspec.rz" marshal_name = File.join @quick_marshal_dir, spec_file_name marshal_zipped = Gem.deflate Marshal.dump(spec) open marshal_name, 'wb' do |io| io.write marshal_zipped end files << marshal_name progress.updated spec.original_name end progress.done end @files << @quick_marshal_dir files end
#build_modern_index(index, file, name)
Build a single index for RubyGems 1.2 and newer
# File 'lib/rubygems/indexer.rb', line 144
def build_modern_index(index, file, name) say "Generating #{name} index" Gem.time "Generated #{name} index" do open(file, 'wb') do |io| specs = index.map do |*spec| # We have to splat here because latest_specs is an array, while the # others are hashes. spec = spec.flatten.last platform = spec.original_platform # win32-api-1.0.4-x86-mswin32-60 unless String === platform then alert_warning "Skipping invalid platform in gem: #{spec.full_name}" next end platform = Gem::Platform::RUBY if platform.nil? or platform.empty? [spec.name, spec.version, platform] end specs = compact_specs(specs) Marshal.dump(specs, io) end end end
#build_modern_indices(specs)
Builds indices for RubyGems 1.2 and newer. Handles full, latest, prerelease
# File 'lib/rubygems/indexer.rb', line 174
def build_modern_indices specs prerelease, released = specs.partition { |s| s.version.prerelease? } latest_specs = Gem::Specification._latest_specs specs build_modern_index(released.sort, @specs_index, 'specs') build_modern_index(latest_specs.sort, @latest_specs_index, 'latest specs') build_modern_index(prerelease.sort, @prerelease_specs_index, 'prerelease specs') @files += [@specs_index, "#{@specs_index}.gz", @latest_specs_index, "#{@latest_specs_index}.gz", @prerelease_specs_index, "#{@prerelease_specs_index}.gz"] end
#compact_specs(specs)
Compacts Marshal output for the specs index data source by using identical objects as much as possible.
# File 'lib/rubygems/indexer.rb', line 242
def compact_specs(specs) names = {} versions = {} platforms = {} specs.map do |(name, version, platform)| names[name] = name unless names.include? name versions[version] = version unless versions.include? version platforms[platform] = platform unless platforms.include? platform [names[name], versions[version], platforms[platform]] end end
#compress(filename, extension)
Compress filename
with extension
.
# File 'lib/rubygems/indexer.rb', line 259
def compress(filename, extension) data = Gem.read_binary filename zipped = Gem.deflate data open "#{filename}.#{extension}", 'wb' do |io| io.write zipped end end
#compress_indices
Compresses indices on disk
#gem_file_list
List of gem file names to index.
# File 'lib/rubygems/indexer.rb', line 272
def gem_file_list Dir[File.join(@dest_directory, "gems", '*.gem')] end
#generate_index
Builds and installs indices.
# File 'lib/rubygems/indexer.rb', line 279
def generate_index make_temp_directories build_indices install_indices rescue SignalException ensure FileUtils.rm_rf @directory end
#gzip(filename)
Zlib::GzipWriter
wrapper that gzips filename
on disk.
# File 'lib/rubygems/indexer.rb', line 291
def gzip(filename) Zlib::GzipWriter.open "#{filename}.gz" do |io| io.write Gem.read_binary(filename) end end
#install_indices
Install generated indices into the destination directory.
# File 'lib/rubygems/indexer.rb', line 300
def install_indices verbose = Gem.configuration.really_verbose say "Moving index into production dir #{@dest_directory}" if verbose files = @files files.delete @quick_marshal_dir if files.include? @quick_dir if files.include? @quick_marshal_dir and not files.include? @quick_dir then files.delete @quick_marshal_dir dst_name = File.join(@dest_directory, @quick_marshal_dir_base) FileUtils.mkdir_p File.dirname(dst_name), :verbose => verbose FileUtils.rm_rf dst_name, :verbose => verbose FileUtils.mv(@quick_marshal_dir, dst_name, :verbose => verbose, :force => true) end files = files.map do |path| path.sub(/^#{Regexp.escape @directory}\/?/, '') # HACK? end files.each do |file| src_name = File.join @directory, file dst_name = File.join @dest_directory, file FileUtils.rm_rf dst_name, :verbose => verbose FileUtils.mv(src_name, @dest_directory, :verbose => verbose, :force => true) end end
#make_temp_directories
Make directories for index generation
# File 'lib/rubygems/indexer.rb', line 336
def make_temp_directories FileUtils.rm_rf @directory FileUtils.mkdir_p @directory, :mode => 0700 FileUtils.mkdir_p @quick_marshal_dir end
#map_gems_to_specs(gems)
[ GitHub ]# File 'lib/rubygems/indexer.rb', line 194
def map_gems_to_specs gems gems.map { |gemfile| if File.size(gemfile) == 0 then alert_warning "Skipping zero-length gem: #{gemfile}" next end begin spec = Gem::Package.new(gemfile).spec spec.loaded_from = gemfile spec.abbreviate spec.sanitize spec rescue SignalException alert_error "Received signal, exiting" raise rescue Exception => e msg = ["Unable to process #{gemfile}", "#{e.} (#{e.class})", "\t#{e.backtrace.join "\n\t"}"].join("\n") alert_error msg end }.compact end
#paranoid(path, extension)
Ensure Gem.path and path with extension
are identical.
# File 'lib/rubygems/indexer.rb', line 345
def paranoid(path, extension) data = Gem.read_binary path compressed_data = Gem.read_binary "#{path}.#{extension}" unless data == Gem.inflate(compressed_data) then raise "Compressed file #{compressed_path} does not match uncompressed file #{path}" end end
#update_index
Perform an in-place update of the repository from newly added gems.
# File 'lib/rubygems/indexer.rb', line 357
def update_index make_temp_directories specs_mtime = File.stat(@dest_specs_index).mtime newest_mtime = Time.at 0 updated_gems = gem_file_list.select do |gem| gem_mtime = File.stat(gem).mtime newest_mtime = gem_mtime if gem_mtime > newest_mtime gem_mtime >= specs_mtime end if updated_gems.empty? then say 'No new gems' terminate_interaction 0 end specs = map_gems_to_specs updated_gems prerelease, released = specs.partition { |s| s.version.prerelease? } files = build_marshal_gemspecs specs Gem.time 'Updated indexes' do update_specs_index released, @dest_specs_index, @specs_index update_specs_index released, @dest_latest_specs_index, @latest_specs_index update_specs_index(prerelease, @dest_prerelease_specs_index, @prerelease_specs_index) end compress_indices verbose = Gem.configuration.really_verbose say "Updating production dir #{@dest_directory}" if verbose files << @specs_index files << "#{@specs_index}.gz" files << @latest_specs_index files << "#{@latest_specs_index}.gz" files << @prerelease_specs_index files << "#{@prerelease_specs_index}.gz" files = files.map do |path| path.sub(/^#{Regexp.escape @directory}\/?/, '') # HACK? end files.each do |file| src_name = File.join @directory, file dst_name = File.join @dest_directory, file # REFACTOR: duped above FileUtils.mv src_name, dst_name, :verbose => verbose, :force => true File.utime newest_mtime, newest_mtime, dst_name end end
#update_specs_index(index, source, dest)
Combines specs in index
and source
then writes out a new copy to dest
. For a latest index, does not ensure the new file is minimal.
# File 'lib/rubygems/indexer.rb', line 419
def update_specs_index(index, source, dest) specs_index = Marshal.load Gem.read_binary(source) index.each do |spec| platform = spec.original_platform platform = Gem::Platform::RUBY if platform.nil? or platform.empty? specs_index << [spec.name, spec.version, platform] end specs_index = compact_specs specs_index.uniq.sort open dest, 'wb' do |io| Marshal.dump specs_index, io end end