Making Ruby's Enumerator lazier
Problem:
Enumerable#map
and its brothers return array. Ruby 1.9's Enumerator
mixes in Enumerable
. That is, calling #map
, #find_all
, #reject
etc on an Enumerator
will try to pull all items from it to gather them into array. Good, industrious Enumerator. But not lazy.
Why it is bad:
Consider an continual enumerator, having an infinite loop inside. For example, such a way to continually prompt user for something from console (and yield the results):
prompter = Enumerator.new{|y| loop{ y << gets } }
It's easy now to make #each
on this enumerator, but if we try to prepend it with some transformations...
prompter.map(&:strip).reject(&:empty?).each{|s| ... }
...we will fail to build command processor on it. Execution wil NOT reach the block of each
until the prompter
runs out, that is NEVER. That is the direct result of the Problem stated above.
How to fix:
If standard #map
, #reject
methods do not satisfy β let's write our own! A good idea how to do it is given here. I just give it here with some further extensions.
By adding a very general defer
method we achieve full control over enumerator:
class Enumerator
def defer(&blk)
self.class.new do |y|
each do |*input|
blk.call(y, *input)
end
end
end
end
The defer
method returns a new Enumerator
, passing its yielder and items into block.
So now we can work with it like this:
prompter.defer {|out, in| out << in.strip } # "map"
prompter.defer {|out, in| out << in if ... } # "reject"
Looks verbose... Yep, why not make shortcuts for it?
class Enumerator
def lazy_map
defer { |out, in| out << yield(in) }
end
def lazy_reject
defer { |out, in| out << inp unless yield(inp) }
end
end
Good! But in the fight against verbosity, we must be ultimate. So let's get back to the first listing and wrap it into a method, too, to make it look just like Scala's Iterator.continually
:
class Enumerator
def self.continually(&blk)
self.new do |y|
loop { y << blk.call }
end
end
end
Now we can finally write the prompter
example in a short and readable way:
Enumerator.continually { gets }.
lazy_map(&:strip).
lazy_reject(&:empty?).
each{|input| work_hard(input) }
Woohoo!
How to make it even better:
To carry the work through it would be great to:
- find all the methods of Enumerable that can be made lazy;
- instead of patching Enumerator with lazy... shit, create a subclass, in which override all these methods, and patch Enumerator only with tolazy method returning that subclass;
- create a gem for this all.
Maybe I will try to do this when I have more spare time.
Thanks for reading! Hope this helps you to work with Enumerator
s with joy.
Written by Ivan Novikov
Related protips
2 Responses
FYI: In Ruby 2.0+ there's Enumerable#lazy. http://patshaughnessy.net/2013/4/3/ruby-2-0-works-hard-so-you-can-be-lazy
(Yes, I do realize this post talks about 1.9 π)
@ravicious Thanks, that's important to know that the desired functionality has finally come official :)