Porting Charrua-Unix and Rawlink to Eio

This article describes the porting of the DHCP daemon charrua-unix and its companion library rawlink to Eio for the upcoming OCaml 5 release. Before we get started, it makes sense to briefly describe what DHCP is and how we use it in production.

What is DHCP?

DHCP stands for Dynamic Host Configuration Protocol, and it's described in RFC2131, RFC2132, and others. It was first published in 1993, so it's considerably old, yet very much alive in virtually every network these days—from your home, to your office, to your ISP Wide Area Network.

When your computer, laptop, phone, or any IP-connected device boots up or changes network, it requests network parameters via broadcast. These parameters are requested and answered via the DHCP protocol. The common/minimum parameters a client requests are:

• An IPv4 address
• An IPv4 gateway
• The address of a DNS resolver

This is enough to get connectivity in most networks. DHCP can also provide many extra parameters, but they are outside of the scope of this document.

What is charrua-dhcp?

charrua-dhcp is a DHCP library suite written in pure OCaml. You might not know it, but if you have ever used Docker Desktop, be it on Windows or macOS, you're a user of charrua-dhcp already !

In Docker Desktop, a complete Linux VM is run in the background in order to be able to run Docker containers. This VM needs to acquire network parameters from the host operating system, and this is done via charrua-dhcp. You can check more details on how OCaml and charrua are used to power Docker Desktop in this article.

charrua-dhcp is also the standard DHCP implementation used in Mirage OS, both when used as a server or a client, and perhaps more importantly, it is used on high profile, critical cases, like the home network of yours truly. It is a stable and tested library that has been in use for years, and it has also been put to the challenge against Crowbar. See more details in this article by Mindy Preston.

charrua-dhcp is split into charrua-core and charrua-unix:

charrua-core implements the DHCP server and client logic in pure OCaml, as well as providing serialisers and deserialisers for the protocol wire format. It also provides a textual configuration interface, like ISC-DHCP does.

When we say pure OCaml, we mean it! charrua-core is purely functional and doesn't produce anything via side-effects; therefore, it also does not perform any kind of I/O.

charrua-unix implements the effect-full bits, and it does I/O, feeding incoming packets to charrua-core and sending out replies given by charrua-core.

The idea is that charrua-core has the complex DHCP logic, while charrua-unix does the basic things: logging, sending/receiving packets, making sure the environment is secure, and so on.

The name charrua is a reference to the seminomadic tribe Charrúa from what is today Uruguay, Argentina, and southern Brazil. The rationale is that DHCP serves parameters to roaming (nomadic) clients.

What is rawlink?

DHCP is not an IP protocol. It sits above the Ethernet layer, which means a DHCP application must be able to craft and receive the full Ethernet packet, not just the layers above IP.

Each operating system provides a slightly different mechanism on how to accomplish this. Linux provides a special socket family called AF_SOCKET, whereas BSDs (OpenBSD, FreeBSD, macOS...) and most other Unix systems provide the same via BPF.

rawlink is an OCaml library with C stubs that abstracts these differences away. You get a link on a network interface, which you use to craft and receive full Ethernet packets, bypassing most of the operating system network stack. In other words, rawlink allows you to work with raw packets on an Ethernet link.

What Changes in OCaml 5?

OCaml 5 provides two main new features:

• Parallelism
• Effect handlers

Parallelism makes little sense on a slow, control protocol like DHCP, so we don't use it and it's not the focus of this article.

Effect handlers allow OCaml programs to write non-blocking code as if they were blocking.

Until OCaml 5 and effect handlers, the common way to write non-blocking code was through Lwt, a concurrent programming library for OCaml. Lwt provides a concurrent scheduler and a monadic style of writing programs through promises. With it, the program becomes a long string of binding promises.

One issue with Lwt is that it's very "infectious," and as soon as you add the first Lwt promise (called "thread" in Lwt lingo), the whole code must now behave as a promise as well. Another issue is that the monadic programming is somewhat syntax heavy, so it can clutter the code. Since the promises are allocations themselves, it can also negatively affect performance. Lwt is a great library, but with OCaml 5 and effects we can do better.

With OCaml 5 and effect handlers we can have the best of both worlds. We can write non-blocking code in a blocking style without the monadic clutter imposed by Lwt. The library we are proposing to replace Lwt in OCaml 5 is Eio, which takes full advantage of the effect system, as well as providing a framework to express parallelism.

Lwt vs. Eio

This code snippet is the main function of charrua-unix, using Eio (left) and Lwt (right). We can summarize what is happening as follows:

1 - We read a packet from the network. 2 - We feed the packet to charrua-core, which then gives us a possible Reply (reply, db), the packet to be sent out and the new DHCP database state, respectively. 3 - We send the reply out and loop for more packets.

It's a fairly simple code, but it shows how much less cluttered the Eio version can be by removing all Lwt decorators. Another nice advantage is that if we were to write a blocking version of the same code, we would only need to change Eio_rawlink.{read,write}_packet to Rawlink.{read,write}_packet as their signatures remain the same, something impossible with Lwt.

rawlink and Eio Switches

rawlink uses a file descriptor that Eio knows nothing about, so in order for us to use Eio with it, we want to attach an Eio.Flow.t to the file descriptor. An Eio.Flow.t is an Eio abstraction of a bidirectional socket, even though it was designed mostly for a STREAM-like socket in mind, the semantics fit rawlink case. We do this in Rawlink_eio.opensock:

let open_link ?filter ?(promisc=false) ifname ~sw =
  let fd = Rawlink_lowlevel.opensock ?filter:filter ~promisc ifname in
  let flow = Eio_unix.FD.as_socket ~sw ~close_unix:true fd in
  { flow; fd; packets = ref []; buffer = (Cstruct.create 65536) }

Rawlink_lowlevel.opensock is a call into the actual C stub that returns a BPF or AF_PACKET descriptor, we then create the Eio.Flow.t with Eio_unix.FD.as_socket.

Two things appear out of the ordinary in the flow creation call: The sw (Eio.Switch.t) and close_unix arguments, in order to make sense of them we have to understand what an Eio.Switch.t is.

A long standing issue with Lwt was "how to make sure my file descriptors are not leaked if something goes wrong." Eio attempts to solve this by forcing each Eio.Flow.t to belong to a Eio.Switch.t. You can't create a Eio.Flow.t without giving it a Eio.Switch.t, so this is what the Eio_unix.FD.as_socket does. Since Flows are also attached to normal file descriptors, Eio.Switch.t also takes care of them.

An Eio program creates one or more Eio.Switch.t in order to attach a Eio.Flow.t to it. An Eio.Switch.t can also be nested, creating a tree-like structure, as every new Eio.Switch.t becomes a child of its parent Eio.Switch.t. When an Eio.Switch.t terminates, either succesfully or by some exception, all of its children Eio.Flow.t are also terminated, automatically closing the file descriptor and guaranteeing we don't have a descriptor leak.

close_unix tells Eio to call close(2) when the Eio.Switch.t terminates.

Imagine a TCP server where each client has at least one dedicated Eio.Switch.t, and some of these clients create additional Eio.Switch.t to handle a specific unit of work:

Conclusion

Both Lwt and Eio provide means to achieve concurrency, but they only provide parallelism with Domains. Lwt uses monadic-style promises to achieve concurrency, which pollutes the code and makes it harder to reason about it. Eio makes full use of the new effect handlers and Domains of OCaml 5, providing concurrency and parallelism while maintaining the same programming style of synchronous blocking programs.

Eio is a library that aims to replace Lwt, but with a more modern style and feature set. It provides abstractions for sockets, fibers, streams, flows, and more.

To review,charrua-unix is a feature-packed, yet simple DHCP server implementation for Unix systems based on the OCaml library charrua-core.rawlink makes it possible to read and craft Ethernet packets on most Unix-like systems through an easy-to-use library.

It's relatively easy to port rawlink to Eio by attaching an Eio abstraction of a bidirectional socket, namely Eio.Flow.t, to the file descriptor.

We hope you enjoyed this article and found it helpful. As always, if there are any questions or concerns, feel free to reach out.