Why on earth do fibers exist?
First, let’s answer the question right away. Fibers were created so one could implement generator pattern which in ruby is incorporated in
Enumerator. That’s it.
To quote one book:
Fibers are an advanced and relatively obscure control structure; the majority of Ruby programmers will never need to use the Fiber class directly
Now that you know the truth. Let’s start from the beginning.
Back in ruby 1.8 generator was implemented through continuations, little-known control structure inspired by Lisp. Continuations main use is again to make generators and programs that needed backtracking: ambiguous operator, for example. It has bugs, unpredictable behavior and implemented only in CRuby. Very few understand what it is for and even fewer actually tried to use it.
Then, ruby 1.9 with YARV came along. Continuations became to be harmful and were moved to the standard library.
Enumerator was rewritten in C using fibers.
In other languages, fiber is the name for a lightweight thread. In ruby, it is a coroutine. Why it’s not named coroutine then, you might ask? Well, because fiber sounds better apparently(Page 20).
There are two types of coroutines: semicouroutune and coroutine. They only differ in the way they transfer control.
Semicouroutune a.k.a Asymmetric Coroutines
These coroutines called asmymetric, because there is fundamental asymmetry between caller and coroutine. Resumed by the caller, coroutine can’t transfer control to any other coroutine, only to suspend itself and yield control back to the caller.
This is the default mode for ruby fibers.
require 'fiber', though.
Fiber becomes symmetric coroutine, which basically means that now, fibers can transfer control between one another (there are limitations in ruby, though. But here is the basic idea)
Okay, now when we become familiar with fibers and coroutines. Let’s look at the thing you can build with them.
Why would I need generators, again?
I’ll be completely honest. You probably don’t.
Having said that, they are pretty cool. They are generally useful for partial computations or laziness. In ruby, there are two options to make something lazy: through
Enumerator#next(which uses fibers) and through
Here is the basic generator’s algorithm:
- Compute a partial result
- Return the result back to the caller
- Save the state
- If needed, resume the generator to get the next result
Let’s see an example with enumerator:
enumerator = Enumerator.new do |yielder| i = 0 loop do i += 1 yielder << i end end # => #<Enumerator: #<Enumerator::Generator:0x00007fe6ac9d41e0>:each> enumerator.next # => 1 enumerator.next # => 2 enumerator.next # => 3 enumerator.rewind enumerator.next # => 1
Yeah, I know. That example probably doesn’t raise your eyebrows, except for the syntax, of course. I mean use
<< to return control, really?
By the way, there is an even more confusing alias for that –
yield. It corresponds to
Fiber.yield, but has nothing to do with ruby keyword
Anyway, let’s see how it works.
- You pass a block, where you do computation
#nextthe block is called from the beginning or where it left of
- You return from the function with a result using
- Go back to step .2
Enumerator allows you to do two types of iterations: internal and external.
In daily life, we usually see enumerators as internal iterators. It is returned everytime you call
Enumerable methods without any arguments and it allows you to chain those methods together:
Additionally, you can use enumerator as an external iterator as shown in the example above. This construct might be useful for heavy computations, where saving the previous state would save a lot of time and doing so with
Enumerator would also be more readable than saving the previous state yourself. Unfortunately, it’s not a very popular method (as laziness in ruby in general) and I couldn’t find any good examples of it in the wild 😞
The Fun Part
You’ve read a lot of theory, not it’s time for the fun part. The best way to learn something is to build it, right? Also, it’s probably the only time you’ll ever actually use fibers, so let’s get to it.
Ok, let’s look at the
Enumerator example again. What we can say about it?
- Enumerator accepts block
- The block is being called with an argument
- The argument has a method
<<that returns computation result
That’s pretty much all we need to be able to construct a simple abstraction over fiber. We’ll call it
Generator because that’s what it is.
class Generator class Yielder def <<(x) # Argument that accepts << Fiber.yield(x) # And returns computation result end end def initialize(&block) # Accepts block @block = block @fiber = create_fiber end def next @fiber.resume end def rewind @fiber = create_fiber end private def create_fiber Fiber.new do @block.call(Yielder.new) # Block being call with an argument end end end
That’s it. Here is the tiniest possible version of a generator.
Let’s test that it works as expected. We’ll use
Enumerable this time.
generator = Generator.new do |yielder| (1..Float::INFINITY).each do |i| yielder << i end end generator.next # => 1 generator.next # => 2 generator.next # => 3 generator.rewind generator.next # => 1
For a long time, I couldn’t understand why fibers exist. I thought there was something special about them, something that I just couldn’t grasp. Turned out there wasn’t. They are very specific things, created for a very specific task.
I hope that this article gave you a better understanding of what
Fiber is, what it’s not and how it’s used.
Update about concurrency
Remember that fibers in ruby have two mods? I haven’t really talked about fibers as symmetric coroutines and one redditor pointed that out.
Given an ability to transfer control between one another, fibers can be used to write asynchronous concurrent code. To do this you’ll need a library that implements event loop such as async or eventmachine or you can even build your own simple reactor with ruby
I tried to give a brief explanation here, so nothing was left out and you had a complete picture. While it’s not in depth look at fibers as means for concurrency, I hope you now better understand how they can be utilized.