riot's dead, long live riot

thoughts on the evolution of riot, and its future.

if you write any OCaml you probably have heard of Riot: an actor-model multi-core scheduler for OCaml 5.

the gist is that it lets you spin up many actors that can run concurrently or in parallel across many cores, and they communicate via message passing.

if this sounds like Erlang or Elixir its because that’s where I came from before begin a typehead. I spent time honing my Erlang/Elixir skills at companies like Klarna and Erlang Solutions.

there was a transitioning period where I was trying to build a compiler that’d let you run OCaml on the Erlang VM (Caramel). and a few talks out there about this too. but i digress.

Riot was a fun experiment, and on stream on twitch we built it from scratch, as well as building a small ecosystem of libraries around it: MintTea for writing terminal applications, Suri for a web framework, etc.

here i want to explore a bit what are the foundations necessary for building something like Riot, what are some of the problems I found trying to realize this vision with OCaml, and finally what steps I’ll be taking next.

Foundations

To make something like Riot work, you need just a few things:

1. some form of resumable fibers or continuations — so you can create actors and suspend them at certain points (like receive expressions).

2. control over how/where fibers are resumed — this lets you schedule stuff across cores at different points in time

Pretty much everything else can be built atop this:

• Message passing is just a mailbox attached to the fiber

• Supervision trees build on links between actors, which are also just metadata on fibers!

• Multi-core scheduler is balancing the fibers across the threads of the program (in its most basic form this is just a queue every scheduler-core pulls from)

This is enough to build a version of Riot that lets you build with actors that send messages to each other and supervision trees to coordinate. It’s not that much work, and the implementation mostly depends on what language you’re building it on.

In OCaml 5, this is achieved with Domains for building the multi-threaded scheduler (1 domain ~= 1 thread ~= 1 scheduler), and with Effects for suspending the program continuation. That continuation becomes an actor.

When an effect is performed in OCaml 5, an effect handler can catch it and do something, including resuming the continuation that performed the effect. This is why you’ll see OCaml 5 effectful programs normally run within a big “run” function — like `Eio_main.run`, or `Miou.run`— as this functions install the effect handlers for you.

In practice effect handling can be slow, since there’s a lot of “accounting” to be done, so Riot’s implementation only used effects for the bits that absolutely needed to suspend the actor:

• receive expressions, so if you didn’t have any message in your mailbox, it’d suspend the actor until a message arrived

• async i/o, so if any syscall like readv needed to block, we could suspend the actor until the syscall was ready to continue, and then resume the actor — this leads to code that looks like its blocking, but isn’t!

• explicitly yielding control back to the scheduler

Spawning new actors, sending messages, linking actors, etc, are all operations that originally were effects but that didn’t need to be. Matter of fact, spawning a new actor just allocates a new data structure for the actor, which will initialize its internal fiber/continuation when its first picked up by a scheduler.

Anyway, we can dig a lot into the details of how Riot is implemented, but what I find more interesting are the fundamental problems an implementation of Riot on a language like OCaml will have.

Problem #1: starving schedulers

If the actors are only suspendable at specific user-defined points, then an actor that doesn’t cooperate may never be suspended. For actor programs where you’d have thousands or even millions of actors, this is Bad™️

An uncooperative actor starves a scheduler thread, meaning that if you had 4 cores, now 1 core is only running that actor. If your web server spawns one actor per request, then for the duration of that unyielding request, you have a whole core handling only that request.

So real actor languages like Erlang and Elixir have, at the runtime level, this idea of reductions. This is borrowed from Erlang’s original implementation in Prolog, but in modern day Erlang it more or less means “how many operations can this actor do” or a “run-budget”.

Think of it like this: you spawn an actor, it starts with 1000 reductions. Every function call happening inside that actor, every math operation, message received or sent, any one thing it does consumes a bit of the budget. When the actor’s reductions reaches 0, it is suspended, and another actor takes the stage.

-module(hog).
-export([run/0]).

% this function can be ran in an actor, and it'll infinitely call itself
% and the the whole system will be unaffected by it
run() -> run().

This allows us to spin up millions of actors, and be guaranteed that no single actor will take over all compute and leave us with a dead app. Pretty neat.

Riot implements this idea as best it can: with effect budgets. An actor gets N effects that it can run, and when it runs N effects it gets suspended, then another actor takes the stage. Of course it may be suspended before N effects, for ex. if it calls receive and it has no messages on its mailbox.

However, this breaks if any code the actor runs calls an infinitely recursive function, uses a while loop or a for loop.

(* if we spawn any of these as an actor,
   they will render a scheduler thread unusable
*)

let rec run () = run ()

let rec loop () = 
  while true do
    ()
  done

So to combat this we introduced the Yield effect that can be used inside unbounded recursive functions or long-running loops, and while they’d slow down the actor a bit it’d let the scheduler take a breath and do its job.

The ideal solution here, however, would be to have a version of the OCaml compiler that automatically injects these reduction counts at clear stages (imperative loops, function calls, etc), so all programs could cooperate nicely. But of course achieving this means forking the compiler.

But even if you did that and all code compiled with Riot’s OCaml fork was scheduler friendly, we’d still have a big problem ahead: ecosystem-wide I/O.

Problem #2: how OCaml does I/O

Before OCaml 5, there were 3 big ways to do I/O in OCaml: there was direct blocking code, there was async code with Lwt, and async code with Jane Street’s Async.

There’s a lot of OCaml code out there built with Lwt and Async, but you normally only see direct blocking code in examples, tutorials, etc. I don’t see anyone actively advocating for it for non-trivial, long running applications.

This has a big problem: code written with Lwt isn’t directly usable with Async, and so libraries learned to split themselves into smaller packages like mylib, mylib-lwt, mylib-async, mylib-unix. That way you could support the entire ecosystem by letting them choose what I/O monad they wanted to use!

Now with the introduction of Effects in OCaml 5, it became possible to do direct-style I/O that looked blocking, but was actually async! And so a few people in the community got their hands dirty and built schedulers with different properties like miou, fuseau, eio, riot, and eventually meta-approaches like picos.

So OCaml 5 may have gotten rid of I/O monads for good (big win imo) but now you had to choose and commit to an I/O ecosystem, and if you’re publishing libraries for others to use, now you gotta maintain (or ask others to maintain) N adapters: mylib, mylib-lwt, mylib-eio, mylib-riot, mylib-unix, etc…

I don’t think that’s a scalable place to be in, and I think it hurts adoption and brings friction to becoming an active contributor of the ecosystem.

There’s also the issue that there isn’t a shared low-level I/O infrastructure in the ecosystem with proper bindings to modern syscalls, but we’re not gonna get into that. Lwt, Async, Eio, Riot, and pretty much everyone else ship their share of C code to work around this.

Why is this a problem for Riot?

Riot decided from the beginning that it would not care about any other I/O, and instead the libraries we built would only work on Riot. This also meant we just can’t use existing libraries that run on Lwt, Eio, Miou, etc. We can only use libraries that are pure OCaml, and that’s a rather small subset. Interesting stuff tends to do I/O.

In fact every library pre-effects that binds to C-code needs to be carefully audited to make sure that it isn’t blocking. I haven’t ran the numbers, but I suspect most of opam was not prepared for a Riot-style scheduler.

So its a problem because Riot would have to rewrite its own ecosystem that built on top of the actor-model, and in doing so it would often not be able to contribute back to the larger OCaml ecosystem.

Problem #3: adoption friction in OCaml

So far we’ve discussed some of the work that Riot had to (or would’ve had to) work with to be somewhat viable. This is already a lot of work and it has ton of caveats.

Suppose all of that was there, and Riot flourished as a framework that’s stable, performant, and covers the use-cases you need: type safe multicore programs.

Who’d use it?

I’d say very few people — and not because it wouldn’t be valuable, or cool. But its the same reasoning behind why very few people use OCaml, with vanilla OCaml tools and libraries.

OCaml is simply harder to adopt than alternatives like Go, or Rust.

OCaml has made many decisions that in isolation (or I should say, when considering only OCaml) look great, but that when compared to other modern platforms with widespread adoption just put friction on newcomers adopting it.

Multiple I/O libraries, multiple standard libraries, a build system with an arcane configuration format, lack of several commonplace facilities like built-in conditional compilation, a package ecosystem that moves slowly, lack of standard tools like linters or integration with common IDE’s for step debugging, etc.

Plenty of things to work on.

And for Riot to succeed, OCaml must succeed too. That is a much larger task than I’m ready to take on again, and that has very little to do with changing the software that is OCaml and a lot more with changing the culture that makes OCaml.

Next Steps

All things considered, OCaml 5 was an excellent platform to build Riot on. OCaml is expressive, it taught me so much about how to build something like Riot, and to this day still remains my favorite typed-driven-development experience.

If you haven’t tried OCaml, you should! It’ll show you a way of working with types that will change how you think about and with types.

Anyway, I think the only way to realize the vision I have for Riot going for ward is to go lower than a library for another language, and make Riot a programming language on its own: a RiotML if you will — a statically typed, natively compiled, functional, actor-model language.

This is the next step.