123456789_123456789_123456789_123456789_123456789_

Class: Enumerator

Relationships & Source Files
Namespace Children
Classes:
Extension / Inclusion / Inheritance Descendants
Subclasses:
Super Chains via Extension / Inclusion / Inheritance
Instance Chain:
self, ::Enumerable
Inherits: Object
Defined in: enumerator.c,
enumerator.c

Overview

A class which allows both internal and external iteration.

An Enumerator can be created by the following methods.

Most methods have two forms: a block form where the contents are evaluated for each item in the enumeration, and a non-block form which returns a new Enumerator wrapping the iteration.

enumerator = %w(one two three).each
puts enumerator.class # => Enumerator

enumerator.each_with_object("foo") do |item, obj|
  puts "#{obj}: #{item}"
end

# foo: one
# foo: two
# foo: three

enum_with_obj = enumerator.each_with_object("foo")
puts enum_with_obj.class # => Enumerator

enum_with_obj.each do |item, obj|
  puts "#{obj}: #{item}"
end

# foo: one
# foo: two
# foo: three

This allows you to chain Enumerators together. For example, you can map a list’s elements to strings containing the index and the element as a string via:

puts %w[foo bar baz].map.with_index { |w, i| "#{i}:#{w}" }
# => ["0:foo", "1:bar", "2:baz"]

External Iteration

An Enumerator can also be used as an external iterator. For example, #next returns the next value of the iterator or raises ::StopIteration if the Enumerator is at the end.

e = [1,2,3].each   # returns an enumerator object.
puts e.next   # => 1
puts e.next   # => 2
puts e.next   # => 3
puts e.next   # raises StopIteration

#next, #next_values, #peek, and #peek_values are the only methods which use external iteration (and Array#zip(Enumerable-not-Array) which uses #next internally).

These methods do not affect other internal enumeration methods, unless the underlying iteration method itself has side-effect, e.g. IO#each_line.

::FrozenError will be raised if these methods are called against a frozen enumerator. Since #rewind and #feed also change state for external iteration, these methods may raise ::FrozenError too.

External iteration differs significantly from internal iteration due to using a ::Fiber:

  • The Fiber adds some overhead compared to internal enumeration.

  • The stacktrace will only include the stack from the Enumerator, not above.

  • Fiber-local variables are not inherited inside the Enumerator Fiber, which instead starts with no Fiber-local variables.

  • ::Fiber storage variables are inherited and are designed to handle Enumerator Fibers. Assigning to a Fiber storage variable only affects the current Fiber, so if you want to change state in the caller Fiber of the Enumerator Fiber, you need to use an extra indirection (e.g., use some object in the Fiber storage variable and mutate some ivar of it).

Concretely:

Thread.current[:fiber_local] = 1
Fiber[:storage_var] = 1
e = Enumerator.new do |y|
  p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1
  p Fiber[:storage_var] # => 1, inherited
  Fiber[:storage_var] += 1
  y << 42
end

p e.next # => 42
p Fiber[:storage_var] # => 1 (it ran in a different Fiber)

e.each { p _1 }
p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)

Convert External Iteration to Internal Iteration

You can use an external iterator to implement an internal iterator as follows:

def ext_each(e)
  while true
    begin
      vs = e.next_values
    rescue StopIteration
      return $!.result
    end
    y = yield(*vs)
    e.feed y
  end
end

o = Object.new

def o.each
  puts yield
  puts yield(1)
  puts yield(1, 2)
  3
end

# use o.each as an internal iterator directly.
puts o.each {|*x| puts x; [:b, *x] }
# => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

# convert o.each to an external iterator for
# implementing an internal iterator.
puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] }
# => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

Class Method Summary

Instance Method Summary

::Enumerable - Included

#all?

Returns whether every element meets a given criterion.

#any?

Returns whether any element meets a given criterion.

#chain

Returns an enumerator object generated from this enumerator and given enumerables.

#chunk

Each element in the returned enumerator is a 2-element array consisting of:

#chunk_while

Creates an enumerator for each chunked elements.

#collect

Alias for Enumerable#map.

#collect_concat
#compact

Returns an array of all non-nil elements:

#count

Returns the count of elements, based on an argument or block criterion, if given.

#cycle

When called with positive integer argument n and a block, calls the block with each element, then does so again, until it has done so n times; returns nil:

#detect

Alias for Enumerable#find.

#drop

For positive integer n, returns an array containing all but the first n elements:

#drop_while

Calls the block with successive elements as long as the block returns a truthy value; returns an array of all elements after that point:

#each_cons

Calls the block with each successive overlapped n-tuple of elements; returns self:

#each_entry

Calls the given block with each element, converting multiple values from yield to an array; returns self:

#each_slice

Calls the block with each successive disjoint n-tuple of elements; returns self:

#each_with_index

With a block given, calls the block with each element and its index; returns self:

#each_with_object

Calls the block once for each element, passing both the element and the given object:

#entries

Alias for Enumerable#to_a.

#filter

Returns an array containing elements selected by the block.

#filter_map

Returns an array containing truthy elements returned by the block.

#find

Returns the first element for which the block returns a truthy value.

#find_all
#find_index

Returns the index of the first element that meets a specified criterion, or nil if no such element is found.

#first

Returns the first element or elements.

#flat_map

Returns an array of flattened objects returned by the block.

#grep

Returns an array of objects based elements of self that match the given pattern.

#grep_v

Returns an array of objects based on elements of self that don’t match the given pattern.

#group_by

With a block given returns a hash:

#include?
#inject

Returns an object formed from operands via either:

#lazy

Returns an Lazy, which redefines most ::Enumerable methods to postpone enumeration and enumerate values only on an as-needed basis.

#map

Returns an array of objects returned by the block.

#max

Returns the element with the maximum element according to a given criterion.

#max_by

Returns the elements for which the block returns the maximum values.

#member?

Returns whether for any element object == element:

#min

Returns the element with the minimum element according to a given criterion.

#min_by

Returns the elements for which the block returns the minimum values.

#minmax

Returns a 2-element array containing the minimum and maximum elements according to a given criterion.

#minmax_by

Returns a 2-element array containing the elements for which the block returns minimum and maximum values:

#none?

Returns whether no element meets a given criterion.

#one?

Returns whether exactly one element meets a given criterion.

#partition

With a block given, returns an array of two arrays:

#reduce
#reject

Returns an array of objects rejected by the block.

#reverse_each

With a block given, calls the block with each element, but in reverse order; returns self:

#select
#slice_after

Creates an enumerator for each chunked elements.

#slice_before

With argument pattern, returns an enumerator that uses the pattern to partition elements into arrays (“slices”).

#slice_when

Creates an enumerator for each chunked elements.

#sort

Returns an array containing the sorted elements of self.

#sort_by

With a block given, returns an array of elements of self, sorted according to the value returned by the block for each element.

#sum

With no block given, returns the sum of initial_value and the elements:

#take

For non-negative integer n, returns the first n elements:

#take_while

Calls the block with successive elements as long as the block returns a truthy value; returns an array of all elements up to that point:

#tally

Returns a hash containing the counts of equal elements:

#to_a

Returns an array containing the items in self:

#to_h

When self consists of 2-element arrays, returns a hash each of whose entries is the key-value pair formed from one of those arrays:

#to_set

Makes a set from the enumerable object with given arguments.

#uniq

With no block, returns a new array containing only unique elements; the array has no two elements e0 and e1 such that e0.eql?(e1):

#zip

With no block given, returns a new array new_array of size self.size whose elements are arrays.

Constructor Details

.new(size = nil) {|yielder| ... }

Creates a new Enumerator object, which can be used as an ::Enumerable.

Iteration is defined by the given block, in which a “yielder” object, given as block parameter, can be used to yield a value by calling the yield method (aliased as <<):

fib = Enumerator.new do |y|
  a = b = 1
  loop do
    y << a
    a, b = b, a + b
  end
end

fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The optional parameter can be used to specify how to calculate the size in a lazy fashion (see #size). It can either be a value or a callable object.

[ GitHub ]

  
# File 'enumerator.c', line 478

static VALUE
enumerator_initialize(int argc, VALUE *argv, VALUE obj)
{
    VALUE iter = rb_block_proc();
    VALUE recv = generator_init(generator_allocate(rb_cGenerator), iter);
    VALUE arg0 = rb_check_arity(argc, 0, 1) ? argv[0] : Qnil;
    VALUE size = convert_to_feasible_size_value(arg0);

    return enumerator_init(obj, recv, sym_each, 0, 0, 0, size, false);
}

Class Method Details

.produce(initial = nil) {|prev| ... } ⇒ Enumerator

Creates an infinite enumerator from any block, just called over and over. The result of the previous iteration is passed to the next one. If initial is provided, it is passed to the first iteration, and becomes the first element of the enumerator; if it is not provided, the first iteration receives nil, and its result becomes the first element of the iterator.

Raising StopIteration from the block stops an iteration.

Enumerator.produce(1, &:succ)   # => enumerator of 1, 2, 3, 4, ....

Enumerator.produce { rand(10) } # => infinite random number sequence

ancestors = Enumerator.produce(node) { |prev| node = prev.parent or raise StopIteration }
enclosing_section = ancestors.find { |n| n.type == :section }

Using .produce together with ::Enumerable methods like Enumerable#detect, Enumerable#slice_after, Enumerable#take_while can provide Enumerator-based alternatives for while and until cycles:

# Find next Tuesday
require "date"
Enumerator.produce(Date.today, &:succ).detect(&:tuesday?)

# Simple lexer:
require "strscan"
scanner = StringScanner.new("7+38/6")
PATTERN = %r{\d|[-/*]}
Enumerator.produce { scanner.scan(PATTERN) }.slice_after { scanner.eos? }.first
# => ["7", "+", "38", "/", "6"]
[ GitHub ]

  
# File 'enumerator.c', line 3078

static VALUE
enumerator_s_produce(int argc, VALUE *argv, VALUE klass)
{
    VALUE init, producer;

    if (!rb_block_given_p()) rb_raise(rb_eArgError, "no block given");

    if (rb_scan_args(argc, argv, "01", &init) == 0) {
        init = Qundef;
    }

    producer = producer_init(producer_allocate(rb_cEnumProducer), init, rb_block_proc());

    return rb_enumeratorize_with_size_kw(producer, sym_each, 0, 0, producer_size, RB_NO_KEYWORDS);
}

.product(*enums) ⇒ Enumerator .product(*enums) {|elts| ... } ⇒ Enumerator

Generates a new enumerator object that generates a Cartesian product of given enumerable objects. This is equivalent to Product.new.

e = Enumerator.product(1..3, [4, 5])
e.to_a #=> [[1, 4], [1, 5], [2, 4], [2, 5], [3, 4], [3, 5]]
e.size #=> 6

When a block is given, calls the block with each N-element array generated and returns nil.

[ GitHub ]

  
# File 'enumerator.c', line 3731

static VALUE
enumerator_s_product(int argc, VALUE *argv, VALUE klass)
{
    VALUE enums = Qnil, options = Qnil, block = Qnil;

    rb_scan_args(argc, argv, "*:&", &enums, &options, &block);

    if (!NIL_P(options) && !RHASH_EMPTY_P(options)) {
        rb_exc_raise(rb_keyword_error_new("unknown", rb_hash_keys(options)));
    }

    VALUE obj = enum_product_initialize(argc, argv, enum_product_allocate(rb_cEnumProduct));

    if (!NIL_P(block)) {
        enum_product_run(obj, block);
        return Qnil;
    }

    return obj;
}

Instance Method Details

#+(enum) ⇒ Enumerator

Returns an enumerator object generated from this enumerator and a given enumerable.

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]
[ GitHub ]

  
# File 'enumerator.c', line 3387

static VALUE
enumerator_plus(VALUE obj, VALUE eobj)
{
    return new_enum_chain(rb_ary_new_from_args(2, obj, eobj));
}

#each {|elm| ... } ⇒ Object #eachEnumerator #each(*appending_args) {|elm| ... } ⇒ Object #each(*appending_args) ⇒ Enumerator

Iterates over the block according to how this Enumerator was constructed. If no block and no arguments are given, returns self.

Examples

"Hello, world!".scan(/\w+/)                     #=> ["Hello", "world"]
"Hello, world!".to_enum(:scan, /\w+/).to_a      #=> ["Hello", "world"]
"Hello, world!".to_enum(:scan).each(/\w+/).to_a #=> ["Hello", "world"]

obj = Object.new

def obj.each_arg(a, b=:b, *rest)
  yield a
  yield b
  yield rest
  :method_returned
end

enum = obj.to_enum :each_arg, :a, :x

enum.each.to_a                  #=> [:a, :x, []]
enum.each.equal?(enum)          #=> true
enum.each { |elm| elm }         #=> :method_returned

enum.each(:y, :z).to_a          #=> [:a, :x, [:y, :z]]
enum.each(:y, :z).equal?(enum)  #=> false
enum.each(:y, :z) { |elm| elm } #=> :method_returned
[ GitHub ]

  
# File 'enumerator.c', line 611

static VALUE
enumerator_each(int argc, VALUE *argv, VALUE obj)
{
    struct enumerator *e = enumerator_ptr(obj);

    if (argc > 0) {
        VALUE args = (e = enumerator_ptr(obj = rb_obj_dup(obj)))->args;
        if (args) {
#if SIZEOF_INT < SIZEOF_LONG
            /* check int range overflow */
            rb_long2int(RARRAY_LEN(args) + argc);
#endif
            args = rb_ary_dup(args);
            rb_ary_cat(args, argv, argc);
        }
        else {
            args = rb_ary_new4(argc, argv);
        }
        RB_OBJ_WRITE(obj, &e->args, args);
        e->size = Qnil;
        e->size_fn = 0;
    }
    if (!rb_block_given_p()) return obj;

    if (!lazy_precheck(e->procs)) return Qnil;

    return enumerator_block_call(obj, 0, obj);
}

#each_with_index {|(*args), idx| ... } #each_with_index

Same as #with_index(0), i.e. there is no starting offset.

If no block is given, a new Enumerator is returned that includes the index.

[ GitHub ]

  
# File 'enumerator.c', line 695

static VALUE
enumerator_each_with_index(VALUE obj)
{
    return enumerator_with_index(0, NULL, obj);
}

#each_with_object(obj) {|(*args), obj| ... } #each_with_object(obj) #with_object(obj) {|(*args), obj| ... } #with_object(obj)

Alias for #with_object.

#feed(obj) ⇒ nil

Sets the value to be returned by the next yield inside e.

If the value is not set, the yield returns nil.

This value is cleared after being yielded.

# Array#map passes the array's elements to "yield" and collects the
# results of "yield" as an array.
# Following example shows that "next" returns the passed elements and
# values passed to "feed" are collected as an array which can be
# obtained by StopIteration#result.
e = [1,2,3].map
p e.next           #=> 1
e.feed "a"
p e.next           #=> 2
e.feed "b"
p e.next           #=> 3
e.feed "c"
begin
  e.next
rescue StopIteration
  p $!.result      #=> ["a", "b", "c"]
end

o = Object.new
def o.each
  x = yield         # (2) blocks
  p x               # (5) => "foo"
  x = yield         # (6) blocks
  p x               # (8) => nil
  x = yield         # (9) blocks
  p x               # not reached w/o another e.next
end

e = o.to_enum
e.next              # (1)
e.feed "foo"        # (3)
e.next              # (4)
e.next              # (7)
                    # (10)
[ GitHub ]

  
# File 'enumerator.c', line 1053

static VALUE
enumerator_feed(VALUE obj, VALUE v)
{
    struct enumerator *e = enumerator_ptr(obj);

    rb_check_frozen(obj);

    if (!UNDEF_P(e->feedvalue)) {
        rb_raise(rb_eTypeError, "feed value already set");
    }
    RB_OBJ_WRITE(obj, &e->feedvalue, v);

    return Qnil;
}

#initialize_copy(orig)

This method is for internal use only.
[ GitHub ]

  
# File 'enumerator.c', line 490

static VALUE
enumerator_init_copy(VALUE obj, VALUE orig)
{
    struct enumerator *ptr0, *ptr1;

    if (!OBJ_INIT_COPY(obj, orig)) return obj;
    ptr0 = enumerator_ptr(orig);
    if (ptr0->fib) {
        /* Fibers cannot be copied */
        rb_raise(rb_eTypeError, "can't copy execution context");
    }

    TypedData_Get_Struct(obj, struct enumerator, &enumerator_data_type, ptr1);

    if (!ptr1) {
        rb_raise(rb_eArgError, "unallocated enumerator");
    }

    RB_OBJ_WRITE(obj, &ptr1->obj, ptr0->obj);
    RB_OBJ_WRITE(obj, &ptr1->meth, ptr0->meth);
    RB_OBJ_WRITE(obj, &ptr1->args, ptr0->args);
    ptr1->fib  = 0;
    ptr1->lookahead  = Qundef;
    ptr1->feedvalue  = Qundef;
    RB_OBJ_WRITE(obj, &ptr1->size, ptr0->size);
    ptr1->size_fn  = ptr0->size_fn;

    return obj;
}

#inspectString

Creates a printable version of e.

[ GitHub ]

  
# File 'enumerator.c', line 1226

static VALUE
enumerator_inspect(VALUE obj)
{
    return rb_exec_recursive(inspect_enumerator, obj, 0);
}

#nextObject

Returns the next object in the enumerator, and move the internal position forward. When the position reached at the end, ::StopIteration is raised.

Example

a = [1,2,3]
e = a.to_enum
p e.next   #=> 1
p e.next   #=> 2
p e.next   #=> 3
p e.next   #raises StopIteration

See class-level notes about external iterators.

[ GitHub ]

  
# File 'enumerator.c', line 919

static VALUE
enumerator_next(VALUE obj)
{
    VALUE vs = enumerator_next_values(obj);
    return ary2sv(vs, 0);
}

#next_valuesArray

Returns the next object as an array in the enumerator, and move the internal position forward. When the position reached at the end, ::StopIteration is raised.

See class-level notes about external iterators.

This method can be used to distinguish yield and yield nil.

Example

o = Object.new
def o.each
  yield
  yield 1
  yield 1, 2
  yield nil
  yield [1, 2]
end
e = o.to_enum
p e.next_values
p e.next_values
p e.next_values
p e.next_values
p e.next_values
e = o.to_enum
p e.next
p e.next
p e.next
p e.next
p e.next

## yield args       next_values      next
#  yield            []               nil
#  yield 1          [1]              1
#  yield 1, 2       [1, 2]           [1, 2]
#  yield nil        [nil]            nil
#  yield [1, 2]     [[1, 2]]         [1, 2]
[ GitHub ]

  
# File 'enumerator.c', line 862

static VALUE
enumerator_next_values(VALUE obj)
{
    struct enumerator *e = enumerator_ptr(obj);
    VALUE vs;

    rb_check_frozen(obj);

    if (!UNDEF_P(e->lookahead)) {
        vs = e->lookahead;
        e->lookahead = Qundef;
        return vs;
    }

    return get_next_values(obj, e);
}

#peekObject

Returns the next object in the enumerator, but doesn’t move the internal position forward. If the position is already at the end, ::StopIteration is raised.

See class-level notes about external iterators.

Example

a = [1,2,3]
e = a.to_enum
p e.next   #=> 1
p e.peek   #=> 2
p e.peek   #=> 2
p e.peek   #=> 2
p e.next   #=> 2
p e.next   #=> 3
p e.peek   #raises StopIteration
[ GitHub ]

  
# File 'enumerator.c', line 1000

static VALUE
enumerator_peek(VALUE obj)
{
    VALUE vs = enumerator_peek_values(obj);
    return ary2sv(vs, 1);
}

#peek_valuesArray

Returns the next object as an array, similar to #next_values, but doesn’t move the internal position forward. If the position is already at the end, ::StopIteration is raised.

See class-level notes about external iterators.

Example

o = Object.new
def o.each
  yield
  yield 1
  yield 1, 2
end
e = o.to_enum
p e.peek_values    #=> []
e.next
p e.peek_values    #=> [1]
p e.peek_values    #=> [1]
e.next
p e.peek_values    #=> [1, 2]
e.next
p e.peek_values    # raises StopIteration
[ GitHub ]

  
# File 'enumerator.c', line 970

static VALUE
enumerator_peek_values_m(VALUE obj)
{
    return rb_ary_dup(enumerator_peek_values(obj));
}

#rewinde

Rewinds the enumeration sequence to the beginning.

If the enclosed object responds to a “rewind” method, it is called.

[ GitHub ]

  
# File 'enumerator.c', line 1077

static VALUE
enumerator_rewind(VALUE obj)
{
    struct enumerator *e = enumerator_ptr(obj);

    rb_check_frozen(obj);

    rb_check_funcall(e->obj, id_rewind, 0, 0);

    e->fib = 0;
    e->dst = Qnil;
    e->lookahead = Qundef;
    e->feedvalue = Qundef;
    e->stop_exc = Qfalse;
    return obj;
}

#sizeInteger, ...

Returns the size of the enumerator, or nil if it can’t be calculated lazily.

(1..100).to_a.permutation(4).size # => 94109400
loop.size # => Float::INFINITY
(1..100).drop_while.size # => nil
[ GitHub ]

  
# File 'enumerator.c', line 1243

static VALUE
enumerator_size(VALUE obj)
{
    struct enumerator *e = enumerator_ptr(obj);
    int argc = 0;
    const VALUE *argv = NULL;
    VALUE size;

    if (e->procs) {
        struct generator *g = generator_ptr(e->obj);
        VALUE receiver = rb_check_funcall(g->obj, id_size, 0, 0);
        long i = 0;

        for (i = 0; i < RARRAY_LEN(e->procs); i++) {
            VALUE proc = RARRAY_AREF(e->procs, i);
            struct proc_entry *entry = proc_entry_ptr(proc);
            lazyenum_size_func *size_fn = entry->fn->size;
            if (!size_fn) {
                return Qnil;
            }
            receiver = (*size_fn)(proc, receiver);
        }
        return receiver;
    }

    if (e->size_fn) {
        return (*e->size_fn)(e->obj, e->args, obj);
    }
    if (e->args) {
        argc = (int)RARRAY_LEN(e->args);
        argv = RARRAY_CONST_PTR(e->args);
    }
    size = rb_check_funcall_kw(e->size, id_call, argc, argv, e->kw_splat);
    if (!UNDEF_P(size)) return size;
    return e->size;
}

#with_index(offset = 0) {|(*args), idx| ... } #with_index(offset = 0)

Iterates the given block for each element with an index, which starts from offset. If no block is given, returns a new Enumerator that includes the index, starting from offset

offset

the starting index to use

[ GitHub ]

  
# File 'enumerator.c', line 674

static VALUE
enumerator_with_index(int argc, VALUE *argv, VALUE obj)
{
    VALUE memo;

    rb_check_arity(argc, 0, 1);
    RETURN_SIZED_ENUMERATOR(obj, argc, argv, enumerator_enum_size);
    memo = (!argc || NIL_P(memo = argv[0])) ? INT2FIX(0) : rb_to_int(memo);
    return enumerator_block_call(obj, enumerator_with_index_i, (VALUE)MEMO_NEW(memo, 0, 0));
}

#each_with_object(obj) {|(*args), obj| ... } #each_with_object(obj) #with_object(obj) {|(*args), obj| ... } #with_object(obj)
Also known as: #each_with_object

Iterates the given block for each element with an arbitrary object, obj, and returns obj

If no block is given, returns a new Enumerator.

Example

to_three = Enumerator.new do |y|
  3.times do |x|
    y << x
  end
end

to_three_with_string = to_three.with_object("foo")
to_three_with_string.each do |x,string|
  puts "#{string}: #{x}"
end

# => foo: 0
# => foo: 1
# => foo: 2
[ GitHub ]

  
# File 'enumerator.c', line 739

static VALUE
enumerator_with_object(VALUE obj, VALUE memo)
{
    RETURN_SIZED_ENUMERATOR(obj, 1, &memo, enumerator_enum_size);
    enumerator_block_call(obj, enumerator_with_object_i, memo);

    return memo;
}