Reflections on the MirageOS Retreat in Morocco

Introduction

Since we are a hybrid remote and distributed company, everyone at Tarides knows first-hand how important in-person retreats are for collaborating on software development. They give us a chance to focus more deeply on our work, collaborate closely, and learn from one another. We are particularly enthusiastic about the MirageOS retreats, which are organised by @hannesm from Robur and happen once to twice a year. These retreats bring together OCaml programmers and MirageOS enthusiasts from all over the world to share ideas and work on projects.

For those unacquainted with it, MirageOS is a library operating system that lets users create 'unikernels' – light-weight, single-purpose machine images designed for secure, efficient, high-performance applications. MirageOS unikernels are written in OCaml, which is a functional, semantically-rich, and type-safe programming language.

This blog post offers a glimpse into our journey to the recent MirageOS retreat, which took us to Morocco. We will share our most memorable experiences from the retreat - the personal stories, the community bonding, the projects we worked on, and the things we learned. So, buckle up and join us as we reminisce on a journey of technical exploration and personal growth under the Moroccan sky.

The Journey to Morocco

Our experience of the MirageOS retreat was as much about the journey as it was about the destination. Some of us started our trip with a train ride to La Feria in Seville. It's a big spring festival in Spain that we were excited to experience. From there, we headed to Cadiz, another city in Spain known for its history and food.

Our final destination was Marrakech, Morocco. We stayed at a traditional Moroccan house called a riad, named the Queen of the Medina. The owner of the riad was very welcoming, and the house was comfortable and filled with local art. During our stay, we shared rooms and meals, growing closer to the rest of the community.

Right outside our riad was the famous Jemaa el-Fnaa square. It's a busy marketplace and a UNESCO World Heritage site, filled with music, food stalls, and plenty of action – especially in the evenings.

The journey to Morocco and the experiences we had along the way helped set the stage for a productive and enjoyable retreat.

The MirageOS Retreat Experiences

At the heart of the retreat was the daily 'circle'. Each day, we gathered together to share our experiences and discuss what we had been working on. These discussions provided insights into the different projects, and it was inspiring to hear about the progress that each person was making, often with the help of other participants at the retreat.

One highlight of the retreat was the night-time presentations. These covered various subjects, and not all were directly related to OCaml or MirageOS. The diversity of topics always sparked interesting conversations and created opportunities for us to learn from each other.

Throughout the retreat, a topic that came up often was how to increase the adoption of MirageOS. This spurred a lot of creative thinking, as we brainstormed new ways to promote the wider use of MirageOS.

And of course, we also had the opportunity to work on personal projects. Two of us, for instance, added the Git server commands to the ocaml-git library. Another project played music from a bare-metal Raspberry Pi 4!

But the retreat wasn't all work. We also found time for fun and relaxation. One memorable activity was the contact improvisation dance accompanied by live music, in which several retreat participants took part. After the week-long retreat ended, some of us stayed in Morocco to visit the Atlas Mountains, climb Jbel Toubkal, and go surfing in Imsouane.

Projects and Collaborations

During the retreat, we split into small groups of one or two engineers to work on different projects. The projects let us explore different aspects of Mirage that we found interesting and test the boundaries of what Mirage can do. Some of our projects included:

MIDI over Bare-Metal Raspberry Pi

Contributors: @pitag-ha, @Engil

We set out to explore the capabilities of a Raspberry Pi (RPi) in handling MIDI signals. We had a host of adapters at our disposal, including a GPIO board with MIDI DIN plugs and an adapter that could transform MIDI DIN to USB. Our colleague @Engil, who was one of the two main people working on this project, brought a synthesiser, enabling us to establish a direct connection to the GPIO board of the RPi. Additionally, the DIN-USB cable allowed us to connect our computers to the RPi for debugging purposes, using a program called Midisnoop.

Our primary objective was to send MIDI output from the RPi, which proved to be a straightforward task. We made use of @Dinosaure's bare-metal RPi toolchain, Gibraltar, which already had UART write support for logging. Since MIDI operates on a serial protocol, we decided to send it over UART as well. We adjusted the baudrate, converted some music into MIDI bytes, and sent it to the UART. Thanks to the functionalities already present in Gibraltar, we managed to play the intro of "Mr Sandman"!

We also attempted to receive MIDI signals. After some troubleshooting and experiments, we concluded that the GPIO board's MIDI Out DIN connector was faulty. We confirmed our theory by installing Linux on the RPi and running a Python program for MIDI output provided by the GPIO board's provider, but to no avail. It's amusing to note that it took us a whole day to install Linux and run a single program, compared to the ease and speed of booting a bare-metal MirageOS unikernel 🤓.

We had envisioned implementing MIDI In support to create a "bare-metal OCaml drum machine." The idea was to convert incoming MIDI signals into drum samples, similar to how synthesisers operate. The intention was to load drum samples into the unikernel's memory and generate the corresponding drum sounds upon receiving a MIDI event.

In a bid to broaden our experimentation, we also wanted to explore how we could send audio from the unikernel to the host system. The solution involved writing music to the unikernel's stdout and piping the unikernel into an ALSA function, which then played the received music. Although this wasn't typical usage of a unikernel, it proved to be a really fun experiment.

This project served as a testament to the flexibility and creative applications of unikernels, and we're excited about the further possibilities that this experiment will inspire.

Adding Git Server Commands to the ocaml-git

Contributors: @panglesd, @Julow

The primary goal of our project was to create a unikernel that could act as a Git server. We needed several components in order to accomplish this, including an OCaml implementation of the SSH protocol and an OCaml implementation of the Git protocol. For the SSH protocol, everything we needed was implemented in awa-ssh, but for ocaml-git things were a bit more complicated.

ocaml-git implements the Git format and part of the protocol. It is also used as a backend for Irmin and for fetching data in a unikernel. However, the server side part of the protocol that we needed was missing. It had not been needed in ocaml-git use cases before.

Our main challenge was that programming in ocaml-git can be really hard! There were a lot of monads at the same time, as well as Higher Kinded Types. Abstractions were necessary to support all the use cases we wanted, which were to use it in a Unix program or in a unikernel, as an Irmin backend, and as a library by a unikernel. We also needed to decipher some slightly vague documentation for the Git protocol, so there was some trial and error and reverse engineering of Git going on.

We were lucky enough to get some great help from several people. @Dinosaure walked us through the code of ocaml-git and answered many of our questions about Git's protocol, whereas @reynir helped us write a unikernel and answered our questions about SSH.

We implemented the project in a series of steps, starting by writing a 'cat' SSH unikernel as a basis for our server. We then implemented a server-side fetch protocol called upload-pack. We needed to do a lot of iterations and experimentation before we got it right, as the protocol was full of hidden details. We were finally able to create a git-clone that was answered by our server. It was just in time as well, as it was the last night of the retreat!

The next steps would be to implement the missing features like shallow clones for upload-pack, implement the server-side of push which is called receive-pack, and integrate all of this into a unikernel! If you'd like to help or just check out our project, you can look at the PR on GitHub.

Exploring Solo5 and Multicore

Contributors: @haesbaert, @fabbing

We were experimenting with Solo5 and Multicore. @haesbaert was trying to figure out how Unikraft booted and what parts of SMP it already had, as well as understanding what people expect from something like Unikraft. This involved checking the reservations and so on. Together with @kit-ty-kate, we tried to fix halting on Google Compute for Mirage, which involved diving into OpenBSD to see how they managed halting. After a lot of investigation, it seemed that in order to fix halting, we would need a proper ACPI implementation and some minor table parsing to achieve proper shutdown. @haesbaert also collaborated with Hannesm to fix an Eio bug in FreeBSD.

@fabbing was also working with @Dinosaure to learn about Solo5 and how to get multiple CPUs running. While they were working on Solo5 together, @Dinosaure went through and updated the Solo5 documentation.

OCaml Splashscreen

Contributors: @MisterDA

We started by wanting to explore DNS over HTTPS with MirageOS, and we managed to deploy a DNS server locally! However, with all of the exciting things going on all around us, we got distracted and started to work on smaller projects. We learned about generating OCaml bindings to C libraries with ctypes and started to expand the coverage of ocaml-posix. We set an informal goal to write a binding for FUSE with ctypes, which is still a work in progress. We also explored the steps involved with building MirageOS and OCaml on macOS, and found and fixed a couple of bugs.

We then started on a really fun project! It was all about splashscreens, or the windows that are shown when a program starts. We based it on the first chronophotography of a camel walking. @MisterDA extracted the camel using GIMP and turned it into a computer animation, displayed with the OCaml binding to SDL2. We then added a futuristic soundtrack, composed by @Engil, before being ready to present the OCamlWalk project.

It also acts as a wrapper around ocamlrun that you can use to launch OCaml bytecode executables. The camel walks, and OCaml runs!

Conclusion

The retreat was a great opportunity to meet other developers who are enthusiastic about MirageOS. We had some great discussions and brainstorming sessions, sharing ideas and insights with each other. Morocco provided an amazing setting for the retreat, with beautiful nature and historic cultural landmarks.

We're happy to be part of a vibrant community with a lot of passionate people, and we're already looking forward to the next opportunity to get together!