OCaml Weekly News

Previous Week Up Next Week


Here is the latest OCaml Weekly News, for the week of May 30 to June 06, 2023.

Table of Contents

Meetup group in New York City

Ashish Agarwal announced

I’m pleased to announce that the OCaml NYC Meetup is back! We have scheduled our first new event for June 20th, where we will discuss the use of OCaml in Tezos. Please join the meetup group to stay informed about future events. We normally will not post to this forum.

We are always looking for speakers. Please reach out to me if you are in the New York City area and are interested in giving a talk.

Creating a tutorial on sequences

Cuihtlauac Alvarado announced

This tutorial is now online: https://ocaml.org/docs/sequences

Thanks to Miod Vallat, Sayo Bamigdade (@SaySayo), Christine Rose (@professor.rose), Sabine Schmaltz (@sabine), Guillaume Petiot (@gpetiot), Xavier Van de Woestyne (@xvw) and Simon Cruanes (@c-cube ) for their feedback

new release: Merlin 4.9

vds announced

I am pleased to announce a new release of Merlin!

Merlin is an editor service that provides modern IDE features for OCaml.

This new release brings a number of improvements and bug fixes. In particular we identified and patched an important memory consumption issue that could greatly affect Merlin’s performance in heavily functorized projects.

Full changelog: https://github.com/ocaml/merlin/blob/master/CHANGES.md#merlin-49

Taking Inventory of the OCaml Ecosystem on OCaml.org

Sabine Schmaltz announced

we have an open PR on ocaml/ocaml.org (https://github.com/ocaml/ocaml.org/pull/1226) to show an approximation of what the state of the OCaml ecosystem is with respect to different topics / use cases.

This addition is inspired by Rust’s excellent “Are we X yet?” pages which

  1. highlight libraries that are production-ready, well-documented, and have a nice API in the different categories. This is a showcase where we want to proudly point newcomers to.
  2. offer a high-level overview of the usability of the language for certain applications. This makes visible where contributions to the ecosystem would be particularly valuable to the OCaml community. They provide a starting point for (prospective) package authors to see where gaps in the ecosystem are, so they can create successful open source projects that meet community demand! (I know at least one company which is interested in contributing to funding such projects. :wink:)

This is where you come in:

  1. Please help us give these pages an iconic name / title. Got an idea? Reply to this post! Make it memorable.
  2. Are there any important categories missing?
  3. Contributions are always welcome - none of use here is an expert on the ever-growing OCaml ecosystem!

New release: DocuLib 1.3.5

nguermond announced

I’m happy to announce a new release of DocuLib on OPAM, a lightweight and easy to use GUI for locally managing metadata for books, textbooks, and articles (kind of like Zotero). This release is mostly for minor fixes but I want to take the opportunity to advertise DocuLib again.

Its core features are

  • facilitating looking up metadata with an interface to openlibrary.org, semanticscholar.org, and bibtex references through crossref.org
  • automatically detecting duplicates (by md5 hash), file renamings, and files moved between libraries without losing metadata
  • keeping track of authors, title, tags, personal notes, date, DOI/ISBN
  • error permissive search

For a full list of changes see CHANGES

Feel free to make suggestions in the comments!

Kiran Gopinathan asked and nguermond replied

Could you comment a bit more on the comparison to zotero? Are there certain features in doculib that you can’t easily achieve in zotero?

The biggest difference is the way data is stored:

  • Zotero puts a priority on metadata entries where a physical file is a child of that entry if it exists, whereas DocuLib entries are one-to-one with physical files on your computer
  • files added to Zotero are either stored in a Zotero data directory (over which you have no control) or a link to a file on your computer (which you have to keep track of manually, so clearly not the preferred method), whereas DocuLib files are stored in libraries, of which you can have multiple. A library is a directory containing files you want DocuLib to index, but metadata for that library is also stored in that library.This means libraries are portable, so you can share them or sync them independently of DocuLib.
  • Zotero stores metadata in a database, whereas DocuLib stores metadata as json files one-to-one with the corresponding document

So in short, use DocuLib if you are a document hoarder and want local control over your files. I’d say creating bibliographic references is not the primary focus of DocuLib, whereas it is for Zotero.

opam 2.1.5 release!

R. Boujbel announced

We are pleased to announce the patch release of opam 2.1.5.

This opam release consists of backported bug fixes & a security fix.

You’ll find more information in the release blog post, and the local cache corruption issue in the security post.

Thanks to robur for the security review!

To upgrade simply run:

bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh) --version 2.1.5"

dune 3.8.0

Etienne Millon announced

The dune team is pleased to announce the release of Dune 3.8.0.

It is now available in opam-repository. As usual, it should always be safe to upgrade your dune package: new features and deprecations are only availble if you upgrade the language version in your dune-project files.


  • Introduce mdx stanza 0.4 requiring mdx >= 2.3.0 which updates the default list of files to include *.mld files (#7582, @Leonidas-from-XIV)
  • Allow (stdlib ...) to be used with (wrapped false) in library stanzas (#7139, @anmonteiro).
  • Allow the main module of a library with (stdlib ...) to depend on other libraries (#7154, @anmonteiro).
  • Support (link_flags ...) in (cinaps ...) stanza. (#7423, fixes #7416, @nojb)
  • Allow (package ...) in any position within (rule ...) stanza (#7445, @Leonidas-from-XIV)
  • Added a new user action (concurrent ) which is like (progn ) but runs the actions concurrently. (#6933, @Alizter)
  • Accept the Ordered Set Language for the modes field in library stanzas (#6611, @anmonteiro).
  • Allow parallel execution of inline tests partitions (#7012, @hhugo)
  • Add the --display-separate-messages flag to separate the error messages produced by commands with a blank line. (#6823, fixes #6158, @esope)
  • Add --watch-exclusions to Dune build options (#7216, @jonahbeckford)
  • Adds support for loading plugins in toplevels (#6082, fixes #6081, @ivg, @richardlford)
  • Introduce a public_headers field on libraries. This field is like install_c_headers, but it allows to choose the extension and choose the paths for the installed headers. (#7512, @rgrinberg)
  • Dune can now detect Coq theories from outside the workspace. This allows for composition with installed theories (not necessarily installed with Dune). (#7047, @Alizter, @ejgallego)
  • Added a --no-build option to dune coq top for avoiding rebuilds (#7380, fixes #7355, @Alizter)
  • Add a coqdoc_flags field to the coq.theory stanza allowing the user to pass extra arguments to coqdoc. (#7676, fixes #7954 @Alizter)
  • Preliminary support for Coq compiled intefaces (.vos files) enabled via (mode vos) in coq.theory stanzas. This can be used in combination with dune coq top to obtain fast re-building of dependencies (with no checking of proofs) prior to stepping into a file. (#7406, @rlepigre)
  • Read pkg-config arguments from the PKG_CONFIG_ARGN environment variable (#1492, #7734, @anmonteiro)
  • Use $PKG_CONFIG, when set, to find the pkg-config binary (#7469, fixes #2572, @anmonteiro)


  • Bootstrap: remove reliance on shell. Previously, we’d use the shell to get the number of processors. (#7274, @rgrinberg)
  • Non-user proccesses such as version control or config checking are now run silently. (#6994, fixes #4066, @Alizter)
  • Bytecode executables built for JSOO are linked with -noautolink and no longer depend on the shared stubs of their dependent libraries (#7156, @nojb)
  • Always include opam files in the generated .install file. Previously, it would not be included whenever (generate_opam_files true) was set and the .install file wasn’t yet generated. (#7547, @rgrinberg)


  • Modules that were declared in (modules_without_implementation), (private_modules) or (virtual_modules) but not declared in (modules) will cause Dune to emit a warning which will become an error in 3.9. (#7608, fixes #7026, @Alizter)
  • Coq language versions less 0.8 are deprecated, and will be removed in an upcoming Dune version. All users are required to migrate to (coq lang 0.8) which provides the right semantics for theories that have been globally installed, such as those coming from opam (@ejgallego, @Alizter)


  • Find pps dependencies in the host context when cross-compiling, (#7415, fixes #4156, @anmonteiro)
  • Fix plugin loading with findlib. The functionality was broken in 3.7.0. (#7556, @anmonteiro)
  • Load the host context findlib.conf when cross-compiling (#7428, fixes #1701, @rgrinberg, @anmonteiro)
  • Allow overriding the ocaml binary with findlib configuration (#7648, @rgrinberg)
  • Resolve ppx_runtime_libraries in the target context when cross compiling (#7450, fixes #2794, @anmonteiro)
  • Fix dune install when cross compiling (#7410, fixes #6191, @anmonteiro, @rizo)
  • Fix string quoting in the json file written by --trace-file (#7773, @rleshchinskiy)
  • Correctly set MANPATH in dune exec. Previously, we would use the bin/ directory of the context. (#7655, @rgrinberg)
  • merlin: ignore instrumentation settings for preprocessing. (#7606, fixes #7465, @Alizter)
  • When a rule’s action is interrupted, delete any leftover directory targets. This is consistent with how we treat file targets. (#7564, @rgrinberg)
  • Fix dune crashing on MacOS in watch mode whenever $PATH contains $PWD (#7441, fixes #6907, @rgrinberg)
  • Dune in watch mode no longer builds concurrent rules in serial (#7395 @rgrinberg, @jchavarri)
  • dune coq top now correctly respects the project root when called from a subdirectory. However, absolute filenames passed to dune coq top are no longer supported (due to being buggy) (#7357, fixes #7344, @rlepigre and @Alizter)
  • RPC: Ignore SIGPIPE when clients suddenly disconnect (#7299, #7319, fixes #6879, @rgrinberg)
  • Always clean up the UI on exit. (#7271, fixes #7142 @rgrinberg)
  • Bootstrap: correctly detect the number of processors by allowing nproc to be looked up in $PATH (#7272, @Alizter)
  • Speed up file copying on macos by using clonefile when available (@rgrinberg, #7210)
  • Support commands that output 8-bit and 24-bit colors in the terminal (#7188, @Alizter)
  • Speed up rule generation for libraries and executables with many modules (#7187, @jchavarri)
  • Do not re-render UI on every frame if the UI doesn’t change (#7186, fix #7184, @rgrinberg)
  • Make coq_db creation in scope lazy (@ejgallego, #7133)
  • dune install now respects –display quiet mode (#7116, fixes #4573, fixes #7106, @Alizter)
  • Stub shared libraries (dllXXX_stubs.so) in Dune-installed libraries could not be used as dependencies of libraries in the workspace (eg when compiling to bytecode and/or Javascript). This is now fixed. (#7151, @nojb)
  • Fix regression where Merlin was unable to handle filenames with uppercase letters under Windows. (#7577, @nojb)
  • On nix+macos, pass -f to the codesign hook to avoid errors when the binary is already signed (#7183, fixes #6265, @greedy)
  • Fix bug where RPC clients built with dune-rpc-lwt would crash when closing their connection to the server (#7581, @gridbugs)
  • Fix RPC server on Windows (used for OCaml-LSP). (#7666, @nojb)

Second alpha release of OCaml 5.1.0

octachron announced

With the progress of the ongoing stabilisation effort for OCaml 5.1.0, I am happy to announce a second alpha release for OCaml 5.1.0.

This second alpha release contains many noteworthy fixes:

  • a long-awaited GC fix
  • a Windows ABI fix

as announced in the first alpha but also

  • a compiler-libs (parsetree) fix
  • a type system compatibility enhancement change
  • a restored backed for s390x/IBM Z

The full list of changes since the first alpha is available below.

Once most major OCaml tools are updated to the last compiler-libs changes, we will switch to beta releases. Hopefully, this will happen in the upcoming weeks. The progress on stabilising the ecosystem is tracked on the opam readiness for 5.1.0 meta-issue.

Currently, the release is still planned for around July.

If you find any bugs, please report them on OCaml’s issue tracker.

If you are interested in the ongoing list of new features and bug fixes, the updated change log for OCaml 5.1.0 is available on GitHub.

Installation Instructions

The base compiler can be installed as an opam switch with the following commands on opam 2.1:

opam update
opam switch create 5.1.0~alpha2

The source code for the alpha is also available at these addresses:

  • Fine-Tuned Compiler Configuration

    If you want to tweak the configuration of the compiler, you can switch to the option variant with:

    opam update
    opam switch create <switch_name> ocaml-variants.5.1.0~alpha2+options <option_list>

    where option_list is a space-separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch:

    opam switch create 5.1.0~alpha2+flambda+nffa ocaml-variants.5.1.0~alpha2+options ocaml-option-flambda ocaml-option-no-flat-float-array

    All available options can be listed with opam search ocaml-option.

Changes Compared To The First Alpha Release

  • Runtime System:
    • #11589, #11903: Modify the GC pacing code to make sure the GC keeps up with allocations in the presence of idle domains. (Damien Doligez and Stephen Dolan, report by Florian Angeletti, review by KC Sivaramakrishnan and Sadiq Jaffer)
    • (breaking change) #11865, #11868, #11876: Clarify that the operations of a custom block must never access the OCaml runtime. The previous documentation only mentioned the main illicit usages. In particular, since OCaml 5.0, it is no longer safe to call caml_remove_global_root or caml_remove_generational_global_root from within the C finalizer of a custom block, or within the finalization function passed to caml_alloc_final. As a workaround, such a finalization operation can be registered with Gc.finalize instead, which guarantees to run the finalizer at a safe point. (Report by Timothy Bourke, discussion by Yotam Barnoy, Timothy Bourke, Sadiq Jaffer, Xavier Leroy, Guillaume Munch-Maccagnoni, and Gabriel Scherer)
    • #11827, +#12249: Restore prefetching for GC marking (Fabrice Buoro and Stephen Dolan, review by Gabriel Scherer and Sadiq Jaffer)
    • #12131: Simplify implementation of weak hash sets, fixing a performance regression. (Nick Barnes, review by François Bobot, Alain Frisch and Damien Doligez).
    • #12231: Support MinGW-w64 11.0 winpthreads library, where the macro to set up to get flexdll working changed (David Allsopp and Samuel Hym, light review by Xavier Leroy)
  • Type System:
    • (breaking change) #12189, #12211: anonymous row variables in explicitly polymorphic type annotation, e.g. 'a. [< X of 'a ] -> 'a, are now implicitly universally quantified (in other words, the example above is now read as 'a 'r. ([< X of 'a ] as 'r) -> 'a). (Florian Angeletti and Gabriel Scherer, review by Jacques Garrigue)
  • Code Generation And Optimizations:
    • #11712, #12258, #12261: s390x / IBM Z multicore support: OCaml & C stack separation; dynamic stack size checks; fiber and effects support. (Aleksei Nikiforov, with help from Vincent Laviron and Xavier Leroy, additional suggestions by Luc Maranget, review by the same and KC Sivaramakrishnan)
  • Internal/compiler-libs Changes:
    • #12119, #12188, #12191: mirror type constraints on value binding in the parsetree: the constraint typ in let pat : typ = exp is now directly stored in the value binding node in the parsetree. (Florian Angeletti, review by Richard Eisenberg)
  • Bug Fixes
    • #11846: Mark rbx as destroyed at C call for Win64 (mingw-w64 and Cygwin64). Reserve the shadow store for the ABI in the c_stack_link struct instead of explictly when calling C functions. This simultaneously reduces the number of stack pointer manipulations and also fixes a bug when calling noalloc functions where the shadow store was not being reserved. (David Allsopp, report by Vesa Karvonen, review by Xavier Leroy and KC Sivaramakrishnan)
    • #12170: fix pthread_geaffinity_np configure check for android (David Allsopp, review by Sébastien Hinderer)
    • #12252: Fix shared library build error on RISC-V. (Edwin Török, review by Nicolás Ojeda Bär and Xavier Leroy)
    • #12255, #12256: Handle large signal numbers correctly (Nick Barnes, review by David Allsopp).
    • #12277: ARM64, fix a potential assembler error for very large functions by emitting stack reallocation code before the body of the function. (Xavier Leroy, review by KC Sivaramakrishnan)

ML’23: ACM SIGPLAN ML Family Workshop — Call for presentations

Guillaume Munch-Maccagnoni announced

I’m letting you know that the deadline has been extended to June 8th (AoE).

qcheck-lin and qcheck-stm 0.2

Jan Midtgaard announced

I’m happy to share release 0.2 of qcheck-lin and qcheck-stm for black-box property-based testing.

  • qcheck-lin requires little more than an interface description. It allows to test a library for sequential consistency, that is, whether results obtained from using it in parallel agree with some linear, single domain execution.
  • qcheck-stm is a model-based, state-machine framework for both sequential and parallel testing. It allows to test an imperative interface against a pure model description, and thereby allows to express and test intended behaviour beyond a signature description.

For example, here’s a minimal qcheck-lin test of a selection of the Stdlib Hashtbl interface:

module HashtblSig =
  type t = (char, int) Hashtbl.t
  let init () = Hashtbl.create ~random:false 42
  let cleanup _ = ()

  open Lin
  let a,b = char_printable,nat_small
  let api =
    [ val_ "Hashtbl.add"    Hashtbl.add    (t @-> a @-> b @-> returning unit);
      val_ "Hashtbl.remove" Hashtbl.remove (t @-> a @-> returning unit);
      val_ "Hashtbl.find"   Hashtbl.find   (t @-> a @-> returning_or_exc b);
      val_ "Hashtbl.mem"    Hashtbl.mem    (t @-> a @-> returning bool);
      val_ "Hashtbl.length" Hashtbl.length (t @-> returning int); ]

module HT = Lin_domain.Make(HashtblSig)
QCheck_base_runner.run_tests_main [
  HT.lin_test ~count:1000 ~name:"Hashtbl DSL test";

Running this test quickly finds a minimal counterexample to illustrate that Hashtbl is not safe to use in parallel:

Messages for test Hashtbl DSL test:

  Results incompatible with sequential execution

                        Hashtbl.add t '<' 0 : ()
                 |                                    |
     Hashtbl.add t 'a' 0 : ()             Hashtbl.remove t '<' : ()
       Hashtbl.length t : 0

We presented preliminary work on both these libraries at the OCaml Workshop 2022. The libraries furthermore underlie our continuing effort to test the multicore runtime of OCaml 5.x, and have helped identify several issues.

The 0.2 release adds a range of features and bugfixes, including support for OCaml 4.12.x, 4.13.x and 4.14.x without the Domain and Effect modes.

Detailed release notes and more information is available from the GitHub repository: https://github.com/ocaml-multicore/multicoretests

Happy testing!

Melange 1.0 – compile OCaml / ReasonML to JavaScript

Antonio Nuno Monteiro announced

The Melange team and I are thrilled to announce the release of Melange 1.0 today, marking a major milestone in the life of the project. This release represents the culmination of many months of hard work and incredible collaboration.

Melange, which started as a fork of BuckleScript, was created with the vision of maintaining compatibility with OCaml and providing the best OCaml experience within the modern JavaScript ecosystem. Today, we are proud to present Melange 1.0, a mature and reliable tool for compiling OCaml to efficient and readable JavaScript that teams rely on to deliver complex OCaml / ReasonML applications.

Get it now:

$ opam install melange.1.0.0


Melange 1.0 radically improves user experience. This release focuses on robustness, OCaml compatibility and developer experience: Melange is fully embracing the OCaml Platform to make it easy and reliable for OCaml users to target JavaScript.

  • Dune Integration

    Integrating with Dune was our biggest priority. Dune 3.8, released very recently, adds Melange support by understanding the following types of stanzas:

      (modes melange) ;; <- new Melange mode
    ;; emit JS to ~js-output~ folder in this
    ;; directory
     (target js-output))

    In Melange 1.0, the Dune integration is the officially supported workflow to build Melange projects. It provides robust rule generation, static assets support (your HTML / CSS / SVG / images), seamless editor integration (e.g. with OCaml LSP or Merlin).

  • Documentation

    With Melange 1.0, we’re also launching a new documentation effort, melange.re. This website should currently be considered a work in progress, and we’re looking for feedback on how to best explain the Melange workflow and its available features. Feel free to get in touch in the OSS repository.

    Additionally, the Dune documentation includes reference materials specific to using Melange with Dune.

  • Everything else
    • Wider OCaml version support

      Melange was previously only available on OCaml 4.14. In this release, we’ve widened that range to versions of OCaml starting from version 4.13. This includes the OCaml 5 release line and allows Melange projects to share the same OCaml compiler switch as e.g. server-side projects.

      Editor integration is the only caveat: it only works on OCaml 4.14, since Melange emits .cmt artifacts (used by e.g. LSP) targeting the OCaml 4.14 binary format.

    • Multiple syntaxes

      Dune supports ReasonML out of the box via dialects, keeping ReasonML support in Melange unchanged from a user perspective. Internally, however, Melange 1.0 has dropped any knowledge of ReasonML, relying on the existing, battle-tested Dune support for dialects instead.

      A rescript-syntax package is part of the Melange release too. It enables support for ReScript syntax in Melange, which Dune also supports. Keep in mind, however, that newer ReScript features are unlikely to be supported by this best-effort compatibility package.

    • Separate PPX

      A big benefit of deep integration with the OCaml platform is having the freedom to assume that a native toolchain is present. That made it possible to unbundle the Melange distribution into a few separate components.

      Melange now ships with a melange.ppx preprocessor based on Ppxlib that can be added to (preprocess (pps melange.ppx)), as per Dune’s preprocessing specification.

      The React JS PPX (for Reason + JSX) has also been extracted and is now distributed separately as reactjs-jsx-ppx.

    • Enabling modern JS workflows

      The Melange design in Dune was designed from day one with the goal of embracing the JavaScript platform:

      • The Dune integration generates JavaScript files in a predictable way
      • To exercise modern workflows, Melange has been tested in a Next.js application using React Server Components, and the available constructs were deemed sufficient to enable similar use cases.
    • Full list of changes:

      The full list of changes can be consulted here.

Support & Sponsorship

The effort that went into this release would not have been possible without the support of many.

We’d like to thank everyone who made it possible:

We’d also like to thank the following notable contributors to this release:

  • Rudi Grinberg, for his indispensable guidance and direction on the design and implementation of the Dune integration.
  • Javier Chávarri, for migrating a huge production codebase at Ahrefs to Melange, working on the Dune integration, the Melange documentation effort and providing vital feedback to the project.
  • David Sancho, for trying out our most bleeding edge ideas and providing early feedback on how to move forward with ways that encompass the whole ecosystem.

Looking Forward

We are enthusiastic about the progress we have made and the positive feedback we have received from the community. We remain committed to continuously improving Melange, ensuring it remains a robust and efficient tool for OCaml developers targeting the JavaScript platform.

Our Q2 2023 roadmap includes most of the goals that we set out to achieve over the past few months, and some of what we’re thinking about working in the months ahead. Melange 1.0 only marks the beginning of our journey towards the best OCaml experience on the JS platform.

Finally, we would like to extend our deepest thanks to everyone who has supported the project, whether through code contributions, testing, or providing feedback. This is your achievement as much as it is ours, and we look forward to continuing this journey together.

Debugging Native Code in “Second OCaml” YouTube Video

jbeckford announced

In response to an earlier post (https://discuss.ocaml.org/t/enhancing-ocaml-debugging-experience-in-visual-studio-code/12236/4?u=jbeckford) I’ve uploaded a video on YouTube.

Direct Link: https://youtu.be/OV19_FqAUCw

Quick Summary: Pre-requisite skill is the ability to compile your own OCaml compiler. Only macOS and Linux. Breakpoints and single-stepping; no display of OCaml values.

Hopefully it will be the first of several if a few people subscribe or comment. The video, and others that I may make for that new channel, are not for OCaml beginners.

Aside: Personally, I don’t become a beginner in a new subject without first having a glimpse of what I can accomplish in that subject. My expectation is that the people curious about OCaml may land on a few videos and then become beginners.

Since this is my very first YouTube video, I’d appreciate feedback!

Sandmark nightly now supports latency profiling

Puneeth Chaganti announced

Sandmark nightly now monitors tail latency of sequential and parallel applications enabled by new features in OCaml 5.


Click to see the Sequential latency benchmark run


Click here to see the Parallel latency benchmark run

Instrumented runtime of the past

In the past, Sandmark used to support monitoring GC latencies using the instrumented runtime that was present in OCaml 4. But this GC latency feature was disabled due to breaking changes in Sandmark when moving from OCaml 4 to OCaml 5. It is also useful to note that the instrumented runtime wrote to a file, and had a noticeable impact on the program speed. As a result, this instrumentation had to be enabled with a compile-time flag that linked the instrumented runtime with the application rather than the default runtime. The instrumented runtime was used to generate the graphs that were used in the ICFP paper, Retrofitting Parallelism onto OCaml (Fig 10 and Fig 12). However, given its cost, the instrumented runtime was seen as only to be used by GC hackers for performance debugging.

Latency profiling through olly

OCaml 5 supports Runtime Events — a new feature that enables continuous monitoring of production applications. The key differences to the earlier instrumented runtime approach are

  1. Instead of a file, the events are now written to a shared in-memory ring. The events may be read out by an external process from this ring.
  2. Some of the frequent (expensive) probes associated are eschewed to keep the costs low. The expensive probes are still available using the instrumented runtime.

Due to this design, every OCaml 5 program may be continuously monitored for performance, not just the ones compiled with the instrumented runtime. On top of this runtime events feature, we have built olly, an observability tool for OCaml programs. Olly can extract traces of GC events that can be viewed by Perfetto and also produce a short report on GC behaviour including tail latency profiles.

The Sandmark team has now replaced the old latency profiling feature developed over OCaml 4 instrumented runtimes to using olly to generate the profiles. (See Sandmark PR here). Now, the OCaml compiler is continuously monitored not only for speed and memory usage, but also for latency.

Call for action

If you are interested in profiling and analysing the performance of the development branch of the OCaml compiler, please submit your branch through Sandmark Nightly Config.

Update on Eio (effects-based direct-style IO for OCaml 5)

Thomas Leonard announced

With Eio 0.10 just released, it’s time for another update! Since the above post (which was for Eio 0.5), some of the bigger changes are:

  • A new eio_posix backed for Unix-type systems provides much better performance than the old libuv one. Removing libuv has also made it safe to share file-descriptors between domains, so you can now accept a connection with one domain and handle it with another, for example.
  • There is now an API for spawning sub-processes.
  • Networking changes include better support for datagram sockets and the new Eio.Net.run_server convenience function.
  • Many of the data-structures (promises, conditions, semaphores and synchronous streams) are now lock-free, making them faster to use across multiple domains.
  • It is safe to handle signals in Eio now that Eio.Condition.broadcast is lock-free (signal handlers can’t take locks, since they may have interrupted the thread holding the lock). Though note that reliable signal handling on OCaml 5 requires OCaml#12253 to be fixed too.
  • Fiber.fork_seq provides an easy way to create generator functions.
  • Eio now supports domain-local-await, which allows sharing e.g. kcas data-structures across Eio and Domainslib domains.
  • Error handling has been improved. You can now add extra context information to errors (e.g. an error opening a missing file will now include the path of the file). You can also choose how specific to be: e.g. you can catch all IO errors, all network errors, or all connection reset errors.
  • There are also some experimental backends under development:
    • eio_solo5 is for MirageOS unikernels.
    • eio_js is for browsers.
    • eio_windows is for Windows - see #125 if you’d like to help out.

A more detailed list of changes can be found in the release notes.

Eio’s README.md provides an introduction to most of the features.

If you’d like to get involved, the new HACKING.md document explains the structure of the code for people who want to contribute to Eio, and there are regular Eio developer meetings for anyone who wants to get involved.

Initial Emissions Monitoring of the OCaml.org Infrastructure

Patrick Ferris announced

I’m happy to announce that some initial emissions monitoring has been added to the OCaml.org infrastructure. A more detailed write up can be found at https://infra.ocaml.org/2023/05/30/emissions-monitoring.html. This is a first step in accurately measuring the amount of emissions we are generating. There was a discuss thread touching on some of this a while ago https://discuss.ocaml.org/t/ocaml-carbon-footprint/8580.

I think there are two important next tasks: getting full coverage of all of the infrastructure machines and making the data publicly available. I hope to work on this (in an open-source way) in the future, if anyone else is interested do let me know!

Thanks to Tarides (who funded the initial work on this) and to @lambda_foo, Mark Elvers and @avsm for helping with the deployment and ideas for measuring emissions. There is an issue on the ocaml/infrastructure repository for the next steps https://github.com/ocaml/infrastructure/issues/47 :seedling:

Other OCaml News


If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.