The OCaml 5 update brought much-anticipated support for programming on multiple cores. It also introduced support for concurrency via effect handlers – one of the first mainstream languages to do so. This significant update has had profound performance and UX implications, propelling OCaml into new areas of software development. At the core of this leap forward is the ambition to craft a modern, direct-style I/O stack that seamlessly interfaces with the latest kernel I/O advancements, such as io_uring. This is where Eio comes in.

When we started work on the I/O stack there was no previous ecosystem to build upon, and we had to break new ground for everything (threading, scheduling, and so on). As users started to port large applications to Eio - such as Routine, ocaml-wayland or Irmin - we learnt a lot. We received many suggestions for how to improve various aspects of the I/O project, especially on how to write large-scale applications combining effect-handlers. This feedback loop accelerated the ecosystem's evolution and led to new research and community insights, such as those discussed in "Composing Schedulers using Effect Handlers", and experiments with lightweight effect-based concurrency models like Picos. Today, Eio has matured to a point where it is the first "feature complete" direct-style effects library, and we are calling this version 1.0.

But this is just the start of our shared journey to figure out what the effects story is! There will be plenty of iterations as we go through future releases, and we look forward to continuing the community-driven exploration. If you're looking to get stuck in with Eio immediately, we recommend exploring its documentation on OCaml.org, including installation instructions. If, however, you want to know more about the context and features of the I/O stack, this post has you covered. We will guide you through the motivations and history behind Eio as well as some of its most prominent features.

Why Eio?

Eio provides an effects-based direct-style I/O stack for OCaml 5. You can use Eio to read and write files, make network connections, or perform CPU-intensive calculations running multiple operations simultaneously. After much hard work and optimisations, the first full release of Eio 1.0 is now publicly available. This release focuses on two main areas:

• Performance:

Eio 1.0 capitalises on newer kernel I/O interfaces for enhanced parallelism efficiency. It features Eio_posix for broad platform compatibility, whilst also targeting optimal support for various modern (and often incompatible) kernel interfaces. Notably, Eio 1.0 introduces an io_uring backend for Linux and a specialised Eio_js scheduler for JavaScript platforms. Moreover, we developed prototype backends for extensive support across systems: aiming to accommodate Grand Central Dispatch on macOS, IOCP on Windows, kqueue on BSD*, and Solo5 for MirageOS, demonstrating our confidence in the API's versatility for future adaptations.

• Security:

First, to improve safety, some parts of Eio have been formally verified. Second, the landscape of containment approaches across macOS, Windows, Linux, BSD, as well as within hypervisors, containers, and WebAssembly (Wasm), has diversified significantly necessitating new strategies that ensure portability. Eio addresses this with distinct interfaces: a "low-level" interface tailored to each supported system, which offers the best platform-specific control but isn't portable, and a "high-level" portable interface designed for applications, which prioritises security by not exposing ambient resources. Instead, it provides an interface that grants capabilities, facilitating a secure and controlled environment for application development (further details below). These capabilites can also be enforced at runtime: for instance, Eio 1.0 comes with capsicum support on systems that provide the cap_enter system call (FreeBSD). This is similar to Rust's cap-std (and to a lesser extent to Scala 3's checked effects).

As an important side note, the compiler itself remains unopinionated about the user's choice of scheduling policies. While you may want to use Eio to benefit from its features, you do not have to use Eio with OCaml 5.

The Improvements Coming With Eio

What makes Eio different from its predecessors? The Unix library, which uses blocking I/O operations, provided the previous I/O stack for OCaml. Blocking operations are not well suited for concurrent programming and hence with OCaml 4.*, two libraries provide this support instead: Async and Lwt, which both have a monadic interface. These libraries let the developer write code as if there are multiple threads of execution running, each with their own stack, where the stacks are simulated using the heap. However, these libraries require the developer to use non-concurrent and concurrent code in different ways, modifying the style in which they write their code depending on the context. Doing so causes extra work while reading and writing code. Eio uses effect handlers instead which enable users to write their code in a natural direct style (as opposed to the callback-oriented style), while still benefiting from performant asynchronous I/O.

Effects

OCaml 5 added support for effects, removing the need for monadic code in the I/O stack and making an effects-based I/O stack like Eio possible. The OCaml Manual describes effect handlers as follows: "Effect handlers are a mechanism for modular programming with user-defined effects. Effect handlers allow the programmers to describe computations that perform effectful operations, whose meaning is described by handlers that enclose the computations."

Using effect handlers has several advantages:

1. Speed: Using effects speeds up the code since no heap allocations are needed to simulate a stack.
2. Ease-of-use: Developers can write concurrent code in the same style as plain non-concurrent code.
3. Language Features: Developers can now use OCaml language features like try ... with in their concurrent code.

In addition to these benefits from effects, having an effects-based I/O stack lets users take advantage of some additional features of modern operating systems. Many modern operating systems provide high-performance alternatives to the traditional Unix select call. For example, Linux's alternative io_uring has applications that write the operations they want to perform to a ring buffer, which can then handle those operations asynchronously, something Eio can take advantage of.

Eio 1.0 Features

Let's take a look at some of Eio's main features. If you want to discover more, including some code examples, I recommend you check out the readme in the repo.

• Tracing

Eio 1.0 can take advantage of the tracing tools available with OCaml 5.1. When switched on, Eio can write events about various actions, such as creating fibres or resolving promises. Eio-trace and third-party tools like olly can capture and display the traces of these events in a window (for example using Perfetto, giving users a visual representation of their data. Having an I/O stack compatible with the tracing capabilities of OCaml 5.1 gives developers the tools to visualise what their code is doing and monitor it for changes.

• Multicore Support

OCaml 5 now lets programs create multiple domains to run code, meaning that programs can use multiple CPUs simultaneously. This significantly speeds up processing time, especially for CPU-intensive tasks.

Eio 1.0 provides Eio.Executor_pool, which distributes jobs (functions to execute) among a pool of domain workers. Domains are reused and can execute multiple jobs concurrently, and jobs are queued up if they cannot be started immediately due to all workers being busy. Having an I/O stack that can run on multiple domains lets developers create more efficient code, reducing the time it takes to perform a task. Eio.Executor_pool is the recommended module for leveraging OCaml 5's multicore capabilities. It is built on top of the low-level Eio.Domain.manager, which lets developers use multiple domains by having the fibres of the calling domain run in parallel with another, new, domain.

• Integrations

It was essential to the team designing Eio that it was compatible with other I/O libraries and that you could use Eio in the same domain as Async and Lwt. For example, you can use Lwt_eio and Async_eio to run Lwt, Async, and Eio fibres together in a single domain and to convert promises from Lwt and Async to Eio. This is useful when porting existing code to Eio.

You can use Eio with OCaml's Unix module by using Eio_unix, and Domainslib and kcas can also interact with Eio. It can be helpful to send work to a pool of Domainslib worker domains when managing compute-intensive tasks. Helpfully, resolving an Eio promise from a non-Eio domain is possible, making the results easy to retrieve. Eio is compatible with kcas which provides blocking in lock-free software transactional memory (STM) implementations.

Integration is important because it gives users the flexibility to use multiple tools and the tools they prefer for the job. Rather than forcing everyone to use one workflow, the team behind Eio wanted to open up possibilities for the user whilst giving them an upgraded I/O that leverages effects and multicore programming.

History of Eio

Originally, OCaml 5 was not supposed to include support for effects! Therefore, Eio started out as a prototype by the same team working on OCaml Multicore at Tarides. To explore and test the experimental I/O design, we ported it to several large libraries and applications with the help of other open-source maintainers. A special thanks go to Tezos and MirageOS maintainers for early reviews and discussions!

During the porting and testing, we discovered and fixed several bugs in the OCaml 5 runtime. The Tarides team also created and upstreamed many different testing and monitoring tools that make concurrent programming with effects easier. Thanks to this rigorous process, Eio is now a very efficient, portable (it's the only OCaml scheduler compatible with Linux, MacOS, Windows, JavaScript and MirageOS!), and flexible stack. These efforts also laid the groundwork for other scheduler libraries to take advantage of new tools and past learnings to explore new design options (while - hopefully - keeping the ecosystems compatible). See for instance RIOT that brings Erlang-style concurrency to OCaml using a multicore actor-model, or Miou, Fuseau,or Affect, and so on.

We can't just focus on the positives – as with any new workflow, Eio has had its share of growing pains. Early in its development, the community gave constructive criticism about the API design. Some users didn't like the explicit capability model -- other disliked the use objects in the API as it was yet another feature of the language that newcomers would have to learn before writing a relatively 'simple' direct-style I/O application. As a response we tried to keep the explicit capability model, but without using objects. While some users appreciated the change, the resulting API is more complex. We need more community input to decide what to do for Eio 2.0, so please continue to voice your opinions on the relevant GitHub issue.

Finally, some users found the API surface too big and making it difficult to compose with other schedulers. We explored several solutions for better composability and our latest effort is Picos. We plan to integrate Picos with Eio once the project is more mature. We've also received feedback that the capability API is too intrusive for some users. Whilst its model works well with modern sandboxing tools like Wasm and Capsicum and is compatible with typed effects, we understand it's a divisive design choice. To address these comments, which we have already begun to do with Picos, we are open to collaborate further with the community on a design that works the best for everyone.

Stay in Touch

The focus for the upcoming months is to gather information about how the I/O stack is performing and whether there are any pain points or improvement opportunities. The team appreciates your feedback, so if you are using Eio in your projects or are curious to test it out, please share your experience directly in the repo or on OCaml Discuss. There are also regular developer meetings and forum discussions that are open to everyone. We also recommend our tutorial on porting Lwt applications to Eio if you want to get started with porting some of your on applications to the new I/O stack.

Want to stay in touch? You can follow us on X (formerly known as Twitter) and LinkedIN to keep up-to-date with our projects and events.

Tarides invites potential sponsors interested in utilising Eio commercially to reach out. Your support is crucial for the ongoing development of our open-source projects!

Acknowledgements

We would like to thank all the Eio contributors, including but not limited to: Bikal Gurung, Patrick Ferris, Simon Grondin, Christiano Haesbaert, Lucas Pluvinage, and Vesa Karvonen and the Tarides Multicore Application team, notably Sudha Parimala, for their support. The initial development of Eio has been partly sponsored by Jane Street and the Tezos Foundation.