OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of October 11 to 18, 2022.

Table of Contents

Routes v2.0.0 released

Anurag Soni announced

I’d like to announce release of version 2.0.0 of routes to the ocaml package registry.

Routes provides a DSL for typed bi-directional URI dispatch. It allows writing route definitions that can be used for both matching, and printing URI paths. The internal representation of the router uses a trie to perform route matching.

Changes since the last opam release:

  • New updates
    • use ppx_expect for tests.
    • A new route function is available and is an alias for @--> which is used to connect a route pattern to a “handler”.
  • Breaking changes
    • Drop support for OCaml 4.05-4.07
    • Switch to a new model for trailing slash handling. In routes 1.0.0 users needed to be careful about using /? and //? as the former would only match routes without a trailing slash, and the latter would enforce a trailing slash.
      • Users only need to use /? to end routes, and it will cover both routes ending with trailing slashes and without
      • The type used for representing match results has more information about whether it was an exact match, or if it was a match but the input target had a trailing slash at the end.
      • MatchWithTrailingSlash informs the user that the current target was considered a match, but that the target has an additional trailing slash

    Examples of how to use the library are available in the tests and a small demo

    Documentation can be found here

Domainslib 0.5.0 released

Sudha Parimala announced

I’m happy to announce the release of Domainslib 0.5.0.

Domainslib is a parallel programming library for the upcoming OCaml 5. Domainslib provides support for parallel operations such as parallel_for, async~/~await tasks etc. with an efficient work-stealing mechanism in its core. Tasks are expressed with effect handlers, enabling efficient nested parallelism.

You can give it a spin with the 5.0.0~alpha1 compiler. For examples of Domainslib programs, here is a tutorial on parallel programming with domainslib, KC’s hands on tutorial at the Tarides retreat, and another one Marek and I did at ICFP 2022.

Submissions of interesting Domainslib benchmarks appreciated at sandmark. Results of existing ones can be viewed at https://sandmark.tarides.com/ under the parallel benchamrks tab. Please feel free to add your multicore/effects libraries and experiments to awesome-multicore-ocaml.

Changes from the previous release:

Videos of ML 2022 workshop talks

Benoit Montagu announced

Dear community, the videos of the talks given at the ML workshop in September 2022 in Ljubljana are now available. Enjoy!

OCaml 5.0.0, first beta release

octachron announced

The release of OCaml 5.0.0 is drawing near.

After two alpha releases, we have released a first beta version to help you update your softwares and libraries ahead of the release (see below for the installation instructions). The standard library has been stabilized and many opam packages already work with this release. If you find any bugs, please report them here:

https://github.com/ocaml/ocaml/issues

Compared to the last alpha release, this beta contains many small internal runtime fixes (in particular in the systhreads library). At the user level, the interfaces of the Domain and Effect modules have been tweaked to be more forward-compatible:

  • Exceptions related to effects are now defined in the Effect module.
  • The value Domain.recommended_domain_count is no longer a constant and the function Domain.at_each_spawn has been removed.

With those changes, the standard library should be stable now. The final release of OCaml 5.0.0 is currently expected to be in December.

If you are interested by the ongoing list of bug fixes, the updated change log for OCaml 5.0.0 is available at:

https://github.com/ocaml/ocaml/blob/5.0/Changes

You can also follow the state of the opam ecosystem on

https://github.com/ocaml/opam-repository/issues/21526

and

http://check.ocamllabs.io/

A short summary of the changes since the last alpha release is also available below.

Installation instructions

The base compiler can be installed as an opam switch with the following commands on opam 2.1:

opam update opam switch create 5.0.0~beta1

For previous version of opam, the switch creation command line is slightly more verbose:

opam update
opam switch create 5.0.0~beta1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

It might be also interesting to check the new support for parallelism by installing the domainslib library with

opam install domainslib

The source code for the beta release is available at these addresses:

  • Fine-tuned compiler configuration

    If you want to tweak the configuration of the compiler, you can switch to the option variant with:

    opam update
    opam switch create <switch_name> ocaml-variants.5.0.0~beta1+options <option_list>
    

    where option_list is a comma separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch:

    opam switch create 5.0.0~beta1+flambda+nffa ocaml-variants.5.0.0~beta1+options ocaml-option-flambda
    ocaml-option-no-flat-float-array
    

    The command line above is slightly more complicated for opam versions anterior to 2.1:

    opam update
    opam switch create <switch_name> --packages=ocaml-variants.5.0.0~beta1+options,<option_list>
    --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
    

    In both cases, all available options can be listed with “opam search ocaml-option”.

  • Optional opam alpha repository

    During the beta release, if your dependencies are not yet compatible with OCaml 5.0.0, you might want to check the alpha opam repository:

    https://github.com/kit-ty-kate/opam-alpha-repository

    Which can be installed with

    opam repo add alpha git+https://github.com/kit-ty-kate/opam-alpha-repository.git
    

    You can check that the alpha repository has been correctly installed with

    $ opam repo
    <><> Repository configuration for switch 5.0.0~beta1 <><><><><><><><><><><><><>
     1 alpha   git+https://github.com/kit-ty-kate/opam-alpha-repository.git
     2 default https://opam.ocaml.org
    

    This alpha repository contains various fixes that are in the process of being upstreamed, but it should be less and less required with the progress of the beta release.

Changes since the last alpha release

  • Stdlib changes
    • #11309, #11424, #11427, +#11545: Add Domain.recommended_domain_count. (Christiano Haesbaert, Konstantin Belousov, review by David Allsopp, KC Sivaramakrishnan, Gabriel Scherer, Nicolas Ojeda Bar)
    • #11423: Move the effect exceptions to the Effect module (KC Sivaramakrishnan, Xavier Leroy, and Florian Angeletti, review by Florian Angeletti, Xavier Leroy, and KC Sivaramakrishnan)
    • #11593: Remove Domain.at_each_spawn (Florian Angeletti, review by Guillaume Munch-Maccagnoni and KC Sivaramakrishnan)
  • Bug fixes
    • #11303: Ensure that GC is not invoked from bounds check failures (Stephen Dolan, review by Sadiq Jaffer and Xavier Leroy)
    • #5299, #4787, #11138, #11272, #11506: To help debugging, Caml_state now dynamically checks that the domain lock is held, and fails otherwise (with a fatal error at most entry points of the C API, or systematically in debug mode). A new variable Caml_state_opt is introduced, and is NULL when the domain lock is not held. This allows to test from C code if the current thread holds the lock of its domain. (Guillaume Munch-Maccagnoni, review by Florian Angeletti, Damien Doligez, Sadiq Jaffer, Xavier Leroy, and Gabriel Scherer)
    • #11223: The serialization format of custom blocks changed in 4.08, but the deserializer would still support the pre-4.08 format. OCaml 5.x removed support for this old format; provide a clear error message in this case. (Hugo Heuzard, review by Gabriel Scherer)
    • #11504, #11522: Use static allocation in caml_make_float_vect in no-flat-float-array mode, it’s more efficient and avoids a a race condition (Xavier Leroy, report by Guillaume Munch-Maccagnoni, review by David Allsopp)
    • #11461, #11466: Fix gethostbyaddr for IPv6 arguments and make it domain-safe (Olivier Nicole, Nicolás Ojeda Bär, David Allsopp and Xavier Leroy, review by the same)
    • #11479: Make Unix.symlink domain-safe on Windows (Olivier Nicole, review by Xavier Leroy and David Allsopp)
    • #11294: Switch minimum required autoconf to 2.71. (David Allsopp, review by Xavier Leroy)
    • #11370, #11373: Don’t pass CFLAGS to flexlink during configure. (David Allsopp, report by William Hu, review by Xavier Leroy and Sébastien Hinderer)
    • #11487: Thwart FMA test optimization during configure (William Hu, review by David Allsopp and Sébastien Hinderer)
    • #11468: Fix regression from #10186 (OCaml 4.13) detecting IPv6 on Windows for mingw-w64 i686 port. (David Allsopp, review by Xavier Leroy and Sébastien Hinderer)
    • #11482, #11542: Fix random crash in large closure allocation (Damien Doligez, report by Thierry Martinez and Vincent Laviron, review by Xavier Leroy)
    • #11508, #11509: make Bytes.escaped domain-safe (Christiano Haesbaert and Gabriel Scherer, review by Xavier Leroy, report by Jan Midtgaard and Tom Kelly)
    • #11516, #11524: Fix the deprecated_mutable attribute. (Chris Casinghino, review by Nicolás Ojeda Bär and Florian Angeletti)
    • #11576: Fix bug in Bigarray.Genarray.init in the the case of zero-dimensional arrays. (Nicolás Ojeda Bär, Jeremy Yallop, report by Masayuki Takeda, review by Jeremy Yallop and Florian Angeletti)
    • #11587: Prevent integer comparison from being used on pointers (Vincent Laviron, review by Gabriel Scherer)
  • Documentation changes
    • #11093: Add tutorials on parallelism features and the relaxed memory model (KC Sivaramakrishnan, review by Damien Doligez, Anil Madhavapeddy, Gabriel Scherer, Thomas Leonard, Tom Ridge, Xavier Leroy, Luc Maranget, Fabrice Buoro, Olivier Nicole, Guillaume Munch-Maccagnoni, Jacques-Henri Jourdan)

ppx_deriving_yaml 0.1.0

Patrick Ferris announced

A new 0.2.0 version has just been released. Thanks to all the contributors (including Outreachy applicants!), there’s a lot of nice additions including:

  • to_yaml and of_yaml attributes allowing you to add custom encoders and decoders
  • a skip_unknown flag for ignoring yaml keys so you can partially decode yaml values
  • a default attribute
  • [@@deriving yaml] is now an alias to [@@deriving to_yaml] and [@@ deriving of_yaml] so you can get decoders, encoders or both.

See the documentation in the README: https://github.com/patricoferris/ppx_deriving_yaml

A sandbox for proposing new features in odoc

jbeckford announced

For people interested in good documentation for their projects, but need a bit more from odoc.

I wanted to make a place where the broader OCaml community can experiment and propose odoc features: https://diskuv.github.io/odoc-sandbox/

I had two goals for the sandbox:

  • Let you visually see any proposals for new features to odoc. Hopefully this will make scoping the proposal easier.
  • No one should be blocked. Assuming the proposal is ok, it may take months (or even years) to implement. Any early adopter could use the custom Dune rules from the experiment to adopt in their own projects. Early adoption would be complex (custom Dune rules) but since the syntax shouldn’t change they won’t have to throw out their documentation.

I have one proposal in the sandbox and more will come later; there are also a few experiments using Sphinx and Markdown tools.

The first proposal would translate an odoc verbatim block:

{v
::code-block:: LANGUAGE

source code
v}

into syntax highlighted code. The visual results and the original .mli are available in experiment 400.

As I mentioned earlier, it has Dune rules which can be copied if you need it in your own projects. I don’t precisely know how it would be implemented for real in odoc (probably it would be a .mli transformer) but that is not the point. Instead if you have an implementation idea or simply love/hate the proposal, you can just go to GitHub and file an issue at https://github.com/diskuv/odoc-sandbox/issues.

OCaml Platform Installer alpha release

Thibaut Mattio announced

In anticipation of the forthcoming OCaml 5 release—and hot on the heels of its beta release—we are thrilled to announce the alpha release of the OCaml Platform Installer.

As a reminder, the OCaml Platform is the recommended toolchain for developers to work with OCaml.

The Platform Installer allows the user to easily setup OCaml’s development environment, both for a first-time installation and for any new opam switches.

You can try it now by following the [installation instructions](#platform-installer-2), but TL;DR, you can install it with

$ bash < <(curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)

And run it to install the Platform in your opam switch with

$ ocaml-platform

Don’t hesitate to open an issue if you encounter any problem!

Update of the Platform State

As part of the work on the Installer, we’ve updated the state of the Platform to make it up to date and clarify the requirements to include a project in the Platform. Here are some of the notable changes:

  • odoc has been promoted from Incubate to Active
  • OCamlformat has been promoted from Incubate to Active
  • ppxlib and other metaprogramming frameworks have been removed from the Platform (and ppxlib is now documented as the official way to do metaprogramming, as part of the official OCaml documentation)

You can see the complete changes on the corresponding PR.

In parallel, we also want to make the Platform more open and transparent. To do this, we want to provide a clear governance model that can be driven by the community. The governance model should answer questions like:

  • How can I incubate my project in the Platform?
  • Who decides when a project can be promoted?
  • How to discuss changes that would impact multiple Platform projects?

We’re currently following the OCaml.org’s governance, but are exploring new ways to govern the OCaml Platform.

We are leaning toward an RFC process for this, but we are still discussing the alternatives and how this could take shape. We will continue to experiment on the best governance model for the Platform with the project maintainers in the coming months, and we will publish the resulting governance on OCaml.org.

Platform Installer

The Platform Installer provides the best way to install OCaml and the recommended development tools for both newcomers and existing users.

Simplicity. It aims at replacing the existing installation steps with a much simpler workflow.

To install the Platform Installer ocaml-platform, you can run:

$ bash < <(curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)

This script will install opam, if not already present in the system, and the latest version of ocaml-platform.

Then, to install the Platform tools:

$ ocaml-platform

If opam is not initialised, this command will initialise it. Then it will proceed to installing the Platform tools in the current opam switch.

For first-time users, the above two lines will set them up with a working environment that’s complete enough to hack comfortably with OCaml.

Speed. In order to speed up the process of installing development tools, the Installer caches the binaries to avoid for redundant compilation. For instance, you will only need to compile ocamlformat once per version of the tool.

However, some tools such as Merlin depend on the OCaml version. For those, the Installer’s cache distinguishes the binaries, depending on the version of the tool as well the OCaml version it was compiled with.

Integration. The Installer integrates the development tools it installs as opam packages to make it fully aware of what has been installed. The binary provided by the Installer for the dune tool will allow the installation of any package with a Dune dependency without reinstalling it.

However, some development tools include libraries in their opam package that are not provided by the Installer. In this case, installing the original opam package for the tool will replace the one provided by ocaml-platform.

The opam packages installed by ocaml-platform are dependency-free. This means that installing specific versions of your development tools will never mess with the actual dependencies of your project!

  • The Tools Installed

    The list of tools installed by ocaml-platform will ultimately be the platform tools listed in the Platform Docs as either Active and Incubate. Currently, this list is still incomplete for different reasons, such as keeping installation time short for new users, the number of dependencies, or other technical constraints.

    The set of installed tools already provides a complete working environment, with:

    • A build system: Dune
    • A documentation generator: odoc
    • A code formatter: OCamlformat
    • A release helper: dune-release
    • Editor integration: ocaml-lsp and Merlin.

Next Steps

There’s still a lot to do!

Governance. As mentioned above, we should have a clear governance model for how incubation and promotions happen in the Platform, who decides, and which criterias are applied. This should obviously be an open process driven by the community; however, we want to make sure that we propose a governance model that will work well, so we’re still discussing alternatives and experimenting on some options. We’d love your input on this, so expect a Discuss post with a proposed governance model for the Platform soon. Don’t hesitate to [reach out](mailto:thibaut@tarides.com) to us before then if you want to get involved!

Editor Integration. While the Installer is a step forward towards a simple way to install OCaml, the UX for newcomers can still be improved. We plan to integrate the Installer to the official VSCode plugin. This will provide a way to get a complete development environment in only a few clicks directly in the editor. We will also explore how to make the setup of Emacs and Vim more straighforward.

Remote Cache. In order to further reduce the time taken to start hacking on a project, we are working on a remote cache for the Installer. The cache will be populated by a CI and would remove the need for compiling the tools locally. The local cache and compilation mechanism would still be used as a fallback if the remote repo is down or incomplete (such as for a pinned compiler).

Acknowledgments

Thank you to the developers and alpha-testers who contributed to the Plaform Installer project, particularly the Tarides engineers who have been driving the development:

We’d also like to thank our major funder Jane Street for supporting our work to improve OCaml’s installation experience!

dkml-dune-dsl 0.x.x - Parameterizable Dune files embedded in OCaml

jbeckford announced

I am pleased to announce dkml-dune-dsl, an embedding of Dune inside OCaml (aka. an eDSL) for developers that need to simplify complex Dune logic. An excerpt from the README:

Once installed you will be able to write DSL expressions like:

open DkmlDuneDsl

module Build (I : Dune.SYM) = struct
  open I

  let res =
    [
      rule
        [
          target "{{{ name }}}.txt";
          action
            (with_stdout_to "%{target}"
              (echo [ "{{{ age }}}" ]));
        ];
    ]
end

that are run over the parameters in a JSON file:

{
  "param-sets": [
    {"name": "batman", "age": 39},
    {"name": "robin", "age": 24}
  ]
}

You can do also do aggregation or, if you are really adventurous, define your own interpreter. Even if you don’t use parameterization you get things you take for granted with OCaml: type-safety, auto-complete and `let` constants.

The full documentation including installation instructions and examples are available in the README.

A few cautionary notes:

  • The current version (0.1.0 as of Oct 17 2022) does not have 100% coverage of all Dune expressions; it just has the parts of Dune I’ve needed in my own projects. That includes an encoding of the Ordered Set Language and virtual libraries but not (for example) select forms, plugins, ctypes and lex/yacc. But I strongly suspect the same people who need a tool like dkml-dune-dsl are the same people who can easily contribute a PR to add any parts of Dune they need.
  • The API is unstable; if someone adds more Dune expressions they may have to tweak the API. And I haven’t settled on whether the API needs some first-class features to track Dune’s (lang dune X.Y.Z) versioning.
  • This is not blessed in any way by the Dune team! Hopefully they don’t mind though.

Other OCaml News

From the ocaml.org blog

Old CWN

If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.