OCaml Weekly News

Previous Week Up Next Week


Here is the latest OCaml Weekly News, for the week of June 22 to 29, 2021.

Table of Contents

wasicaml - a code emitter for OCaml targeting WebAssembly

Gerd Stolpmann announced

I'd like to announce a new project to develop a code generator that emits WebAssembly:


With the support of RemixLabs I could already create a very first version that takes the OCaml bytecode as input and translates it to WebAssembly. While this approach probably doesn't lead to the fastest code, it is easy to accomplish, and it demonstrates the challenge (and already shows how to solve many of the part problems along the road).

To be precisely, the target of the translator is wasm32-unknown-wasi, i.e. the WASI ABI. This ABI is still in early development, but provides already the syscalls (or better, host calls) to access files, to get the current time, and to read the environment. This is almost enough to run a compiler - I only had to add system() so that ocamlc can start external preprocessors. Also, due to the fact that the current wasm implementations still lack exception handling, I had to assume the presence of a host emulation of exceptions (which is easy to provide if the host environment is Javascript, but not necessarily for other environments).

The translator takes the OCaml bytecode as input, i.e. you first create an excecutable

$ ocamlc -o myexec ...

and then make wasm out of it:

$ wasicaml -o myexec.wasm myexec

If you omit the .wasm suffix, wasicaml will put a preamble in front of the wasm code that starts the execution:

$ wasicaml -o myexec_wasm myexec
$ ./myexec_wasm

Because of this trick, many problems of cross-compiling can be avoided.

You may ask what the benefits of yet another "Web" language are. We already have two emitters targeting Javascript - isn't that enough? Well, two answers here.

First, WASI is a proper LLVM target. Because of this, you can link code from other languages with your executable (e.g. C or Rust). So you are not limited to OCaml but can use any language that also targets the WASI ABI. E.g. you can do

$ wasicaml -o myexec.wasm myexec -ccopt -lfoo

to also link in libfoo.a (which must also be compiled to wasm). So it is multi-lingual from the beginning.

Second, WebAssembly can be used outside the web, too. WASI targets more the command-line, and server plugins, and generally any OS-independent environments. For example, imagine you have an Electron app with a great UI, but for some special functionality you need to include some OCaml code, too. You don't want to give up the OS-independence, and WASI gives you now a natural option to add the OCaml code. And you still have access to the filesystem without hassle. - Another example is edge computing, i.e. when the cloud is extended by computers outside the data center, and the code should be in a form so that it can be run on as many platforms as possible. - All in all, WASI plays well when you need to combine OS-independence with a classic way of organizing the code as command or as server function, and you also need predictable performance.

The challenge of translating OCaml to wasm is mainly the garbage collector. Wasm doesn't permit many of the tricks ocamlopt is using to know in which memory (or register) locations OCaml values are stored. In wasm, there are no registers but the closest vehicle are local variables. Now, it is not possible to scan these variables from the GC function, making it practically impossible to put OCaml values there while a function is called that might trigger a GC. There is also no really cheap way of obtaining a stack descriptor.

Wasicaml inherits the stack from the bytecode interpreter and uses it as its own shadow stack for OCaml values. As wasicaml bases on the bytecode representation of the code, the bytecode instructions already ensure that values always live in this stack when the GC might run. Wasicaml additionally tries to identify values that don't need this special treatment (like ints and bools) and that are preferably stored in local variables, giving the wasm executor freedom to put these into registers or other high-speed locations. (Unfortunately, most of the type information is already erased in the bytecode, and this is definitely one of the deficiencies of the bytecode approach.)

In order to maximize the performance, it is probably best to avoid the stack whenever possible. The current approach of transforming the bytecode hasn't brought to an end yet with respect to such optimizations. For example, there could be more analyses that figure out when GC runs are actually possible and when it is safe to use local variables.

Another problem of the bytecode basis is that all function calls are indirect, preventing the wasm executor from inlining functions.

As a project, I'd like to see wasicaml progressing in two directions. First, make the current approach as good as possible - although basing it on the bytecode representation has its downsides, it is easy to understand and it is possible to figure out what the necessary ingredients for fast code are. Second, get an idea where a possible real wasm backend would fit into the OCaml compiler (maybe it is c– but maybe this doesn't give us much and you start better with lambda).

Anyway, welcome to the new world of WebAssembly!


PS. If you are interested in WebAssembly and like to work with me on another Wasm port for some time, there is a position: https://www.mixtional.de/recruiting/2021-01/index.html

PPS. Wasicaml is a project of Figly, Inc., commonly known as RemixLabs, developing a reactive low-code and code collaboration platform. https://remixlabs.com/

opam 2.1.0~rc2

David Allsopp announced

The opam team has great pleasure in announcing opam 2.1.0~rc2!

The focus since beta4 has been preparing for a world with more than one released version of opam (i.e. 2.0.x and 2.1.x). The release candidate extends CLI versioning further and, under the hood, includes a big change to the opam root format which allows new versions of opam to indicate that the root may still be read by older versions of the opam libraries. A plugin compiled against the 2.0.9 opam libraries will therefore be able to read information about an opam 2.1 root (plugins and tools compiled against 2.0.8 are unable to load opam 2.1.0 roots).

Please do take this release candidate for a spin! It is available in the Docker images at ocaml/opam on Docker Hub as the opam-2.1 command (or you can sudo ln -f /usr/bin/opam-2.1 /usr/bin/opam in your Dockerfile to switch to it permanently). The release candidate can also be tested via our installation script (see the wiki for more information).

Thank you to anyone who noticed the unannounced first release candidate and tried it out. Between tagging and what would have been announcing it, we discovered an issue with upgrading local switches from earlier alpha/beta releases, and so fixed that for this second release candidate.

Assuming no showstoppers, we plan to release opam 2.1.0 next week. The improvements made in 2.1.0 will allow for a much faster release cycle, and we look forward to posting about the 2.2.0 plans soon!

Set up OCaml 2.0.0-beta2

Sora Morimoto announced

This release includes changes to address a corner case primarily related to multicore OCaml.


Any OCaml bindings to Apache Arrow?

UnixJunkie asked and Laurent Mazare announced

Looks interesting:



I've put together some simple ocaml-arrow library. It works reasonably well and is quite battle tested but definitely needs a bit of cleanup as the bits under src/ are deprecated in favor of the ones under c_api/. There is also a ppx to automatically convert ocaml records to/from arrow. Some examples using this can be seen in the tests directory.

If there is some interest, I can certainly push up on cleaning this and make an actual opam package.

Compiler engineer for OCaml and WebAssembly, Germany

Gerd Stolpmann announced

We are developing a compiler for a no-code platform that translates our DSL to bytecode and/or WebAssembly. The language is largely of functional type but is also able to manage state with a spreadsheet model, allowing reactive programming without having to resort to libraries. The language is statically typed using a Hindley-Milner type checker. The compiler is primarily written in OCaml. Other languages of our platform are Go, C, Elm, and Javascript.

We are looking for a compiler engineer with skills in code generation for WebAssembly:

  • Translation of an intermediate representation to WebAssembly
  • Writing runtimes and SDKs targeting WebAssembly
  • Code optimization

See the full ad here: https://www.mixtional.de/recruiting/2021-01/index.html

v3.0.0 release of reparse, reparse-lwt, reparse-lwt-unix

Bikal Lem announced

I am happy to announce v3.0.0 of reparse - an OCaml library for constructing various types of parsers in OCaml.

The release follows a complete overhaul of the internal working of the library to achieve the following goals:

  1. Allow construction of efficient, zero-copy parsers. See String parser for example. The library provides a Make functor parametrised over a Promise and a Input type allowing you control over both parser memory allocation and copying.
  2. Support usage of async libraries - lwt and async. Following the first point the library can now be used together with lwt and/or async. A lwt parse - for example - can now be used seamlessly with your other lwt code. The integration is seamless.
  3. Provide Make_buffered functor to produce parsers where the input type natively doesn't allow random read, for example sockets, lwt streams and channels. There is now two new supporting packages reparse-lwt which provides parsing from char Lwt_stream.t input type and reparse-lwt-unix which provides parsing from Lwt_unix.file_descr and ~Lwt_unix.input_channel' respectively.
  4. Provide Make_unbuffered functor to produce parsers where the input type natively supports random read, for example strings, bigstrings, bytes.
  5. Introduce function unsafe_any_char to allow efficient (zero-copy?) parsing.
  6. Prune dependencies by removing base.

P.S. The documentation is bit behind in this release so please bear with me while work through the issues in the coming days.

Reparse repo

Progress 0.2.0

Craig Ferguson announced

I'm pleased to announce the 0.2.0 release of Progress, now available via Opam.


Progress is an OCaml library for defining and using progress bars. It has the following features:

  • allows user-defined progress bar layouts;
  • supports rendering multiple progress bars simultaneously;
  • dynamically responds to changes in terminal size;
  • supports interleaving logging with progress bar rendering.

This second release contains a much-improved DSL for specifying progress bars, alongside improvements and extensions to the rendering logic. The bars in the screenshot above are defined as follows:

let bar ~color ~total =
  let open Progress.Line in
    [ spinner ~color:(Progress.Color.ansi ~green) ()
    ; brackets (elapsed ())
    ; bar ~color total
    ; bytes
    ; parens (const "eta: " ++ eta total)

It also comes with more complete documentation and many more examples, including:

  • a Cargo-like progress bar w/ logging of intermediate results:


  • a Yarn-like stack of spinners:


  • a showcase of various progress bar styles:


The changelog is here and the API documentation is here. The library is not yet feature-complete, but should still be reasonably useful :-) Happy hacking!

http-multipart-formdata v2.0.0

Bikal Lem announced

I am pleased to announce v2.0.0 release of http-multpart-formdata. This release departs from previous in-memory representation of http multipart forms to a streaming, memory efficient representation. The new streaming mechanism should help when processing larg file uploads in your OCaml web applications.

  1. httpaf sample web app
  2. http-multipart-formdata repo


If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.