OCaml Weekly News
Hello
Here is the latest OCaml Weekly News, for the week of October 11 to 18, 2022.
Table of Contents
- Routes v2.0.0 released
- Domainslib 0.5.0 released
- Videos of ML 2022 workshop talks
- OCaml 5.0.0, first beta release
- ppx_deriving_yaml 0.1.0
- A sandbox for proposing new features in odoc
- OCaml Platform Installer alpha release
- dkml-dune-dsl 0.x.x - Parameterizable Dune files embedded in OCaml
- Other OCaml News
- Old CWN
Routes v2.0.0 released
Anurag Soni announced
I’d like to announce release of version 2.0.0 of routes to the ocaml package registry.
Routes provides a DSL for typed bi-directional URI dispatch. It allows writing route definitions that can be
used for both matching, and printing URI paths. The internal representation of the router uses a trie
to perform
route matching.
Changes since the last opam release:
- New updates
- use
ppx_expect
for tests. - A new
route
function is available and is an alias for@-->
which is used to connect a route pattern to a “handler”.
- use
- Breaking changes
- Drop support for OCaml 4.05-4.07
- Switch to a new model for trailing slash handling. In routes 1.0.0 users needed to be careful about using
/?
and//?
as the former would only match routes without a trailing slash, and the latter would enforce a trailing slash.- Users only need to use
/?
to end routes, and it will cover both routes ending with trailing slashes and without - The type used for representing match results has more information about whether it was an exact match, or if it was a match but the input target had a trailing slash at the end.
MatchWithTrailingSlash
informs the user that the current target was considered a match, but that the target has an additional trailing slash
- Users only need to use
Examples of how to use the library are available in the tests and a small demo
Documentation can be found here
Domainslib 0.5.0 released
Sudha Parimala announced
I’m happy to announce the release of Domainslib 0.5.0.
Domainslib is a parallel programming library for the upcoming OCaml 5. Domainslib provides support for parallel operations such as parallel_for
, async~/~await
tasks etc. with an
efficient work-stealing mechanism in its core. Tasks are expressed with effect handlers, enabling efficient nested
parallelism.
You can give it a spin with the 5.0.0~alpha1 compiler. For examples of Domainslib programs, here is a tutorial on parallel programming with domainslib, KC’s hands on tutorial at the Tarides retreat, and another one Marek and I did at ICFP 2022.
Submissions of interesting Domainslib benchmarks appreciated at sandmark. Results of existing ones can be viewed at https://sandmark.tarides.com/ under the parallel benchamrks tab. Please feel free to add your multicore/effects libraries and experiments to awesome-multicore-ocaml.
Changes from the previous release:
- Bug fix for
parallel_for_reduce
on empty loops. - Make Chan.t and Task.promise injective Make Chan.t and Task.promise injective ocaml-multicore/domainslib#69
- Add lockfree dependency remove Ws_deque module ocaml-multicore/domainslib#70
- CI fixes (Add alpha repo to GitHub Actions ocaml-multicore/domainslib#73, Run
5.0.0~alpha0
in the CI ocaml-multicore/domainslib#76) - Breaking change: Rename
num_additional_domains
tonum_domains
for setup_pool - Documentation updates (Use
invalid_arg
ocaml-multicore/domainslib#80, Document that non-commutative operators are okay ocaml-multicore/domainslib#81, Explain what will happen with an empty pool ocaml-multicore/domainslib#82)
Videos of ML 2022 workshop talks
Benoit Montagu announced
Dear community, the videos of the talks given at the ML workshop in September 2022 in Ljubljana are now available. Enjoy!
OCaml 5.0.0, first beta release
octachron announced
The release of OCaml 5.0.0 is drawing near.
After two alpha releases, we have released a first beta version to help you update your softwares and libraries ahead of the release (see below for the installation instructions). The standard library has been stabilized and many opam packages already work with this release. If you find any bugs, please report them here:
https://github.com/ocaml/ocaml/issues
Compared to the last alpha release, this beta contains many small internal runtime fixes (in particular in the systhreads library). At the user level, the interfaces of the Domain and Effect modules have been tweaked to be more forward-compatible:
- Exceptions related to effects are now defined in the Effect module.
- The value
Domain.recommended_domain_count
is no longer a constant and the functionDomain.at_each_spawn
has been removed.
With those changes, the standard library should be stable now. The final release of OCaml 5.0.0 is currently expected to be in December.
If you are interested by the ongoing list of bug fixes, the updated change log for OCaml 5.0.0 is available at:
https://github.com/ocaml/ocaml/blob/5.0/Changes
You can also follow the state of the opam ecosystem on
https://github.com/ocaml/opam-repository/issues/21526
and
A short summary of the changes since the last alpha release is also available below.
Installation instructions
The base compiler can be installed as an opam switch with the following commands on opam 2.1:
opam update opam switch create 5.0.0~beta1
For previous version of opam, the switch creation command line is slightly more verbose:
opam update
opam switch create 5.0.0~beta1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
It might be also interesting to check the new support for parallelism by installing the domainslib library with
opam install domainslib
The source code for the beta release is available at these addresses:
- Fine-tuned compiler configuration
If you want to tweak the configuration of the compiler, you can switch to the option variant with:
opam update opam switch create <switch_name> ocaml-variants.5.0.0~beta1+options <option_list>
where
option_list
is a comma separated list ofocaml-option-*
packages. For instance, for aflambda
andno-flat-float-array
switch:opam switch create 5.0.0~beta1+flambda+nffa ocaml-variants.5.0.0~beta1+options ocaml-option-flambda ocaml-option-no-flat-float-array
The command line above is slightly more complicated for opam versions anterior to 2.1:
opam update opam switch create <switch_name> --packages=ocaml-variants.5.0.0~beta1+options,<option_list> --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
In both cases, all available options can be listed with “opam search ocaml-option”.
- Optional opam alpha repository
During the beta release, if your dependencies are not yet compatible with OCaml 5.0.0, you might want to check the alpha opam repository:
https://github.com/kit-ty-kate/opam-alpha-repository
Which can be installed with
opam repo add alpha git+https://github.com/kit-ty-kate/opam-alpha-repository.git
You can check that the alpha repository has been correctly installed with
$ opam repo <><> Repository configuration for switch 5.0.0~beta1 <><><><><><><><><><><><><> 1 alpha git+https://github.com/kit-ty-kate/opam-alpha-repository.git 2 default https://opam.ocaml.org
This alpha repository contains various fixes that are in the process of being upstreamed, but it should be less and less required with the progress of the beta release.
Changes since the last alpha release
- Stdlib changes
- #11309, #11424, #11427, +#11545: Add Domain.recommended_domain_count. (Christiano Haesbaert, Konstantin Belousov, review by David Allsopp, KC Sivaramakrishnan, Gabriel Scherer, Nicolas Ojeda Bar)
- #11423: Move the effect exceptions to the Effect module (KC Sivaramakrishnan, Xavier Leroy, and Florian Angeletti, review by Florian Angeletti, Xavier Leroy, and KC Sivaramakrishnan)
- #11593: Remove
Domain.at_each_spawn
(Florian Angeletti, review by Guillaume Munch-Maccagnoni and KC Sivaramakrishnan)
- Bug fixes
- #11303: Ensure that GC is not invoked from bounds check failures (Stephen Dolan, review by Sadiq Jaffer and Xavier Leroy)
- #5299, #4787, #11138, #11272, #11506: To help debugging,
Caml_state
now dynamically checks that the domain lock is held, and fails otherwise (with a fatal error at most entry points of the C API, or systematically in debug mode). A new variableCaml_state_opt
is introduced, and isNULL
when the domain lock is not held. This allows to test from C code if the current thread holds the lock of its domain. (Guillaume Munch-Maccagnoni, review by Florian Angeletti, Damien Doligez, Sadiq Jaffer, Xavier Leroy, and Gabriel Scherer) - #11223: The serialization format of custom blocks changed in 4.08, but the deserializer would still support the pre-4.08 format. OCaml 5.x removed support for this old format; provide a clear error message in this case. (Hugo Heuzard, review by Gabriel Scherer)
- #11504, #11522: Use static allocation in
caml_make_float_vect
in no-flat-float-array mode, it’s more efficient and avoids a a race condition (Xavier Leroy, report by Guillaume Munch-Maccagnoni, review by David Allsopp) - #11461, #11466: Fix gethostbyaddr for IPv6 arguments and make it domain-safe (Olivier Nicole, Nicolás Ojeda Bär, David Allsopp and Xavier Leroy, review by the same)
- #11479: Make Unix.symlink domain-safe on Windows (Olivier Nicole, review by Xavier Leroy and David Allsopp)
- #11294: Switch minimum required autoconf to 2.71. (David Allsopp, review by Xavier Leroy)
- #11370, #11373: Don’t pass CFLAGS to flexlink during configure. (David Allsopp, report by William Hu, review by Xavier Leroy and Sébastien Hinderer)
- #11487: Thwart FMA test optimization during configure (William Hu, review by David Allsopp and Sébastien Hinderer)
- #11468: Fix regression from #10186 (OCaml 4.13) detecting IPv6 on Windows for mingw-w64 i686 port. (David Allsopp, review by Xavier Leroy and Sébastien Hinderer)
- #11482, #11542: Fix random crash in large closure allocation (Damien Doligez, report by Thierry Martinez and Vincent Laviron, review by Xavier Leroy)
- #11508, #11509: make Bytes.escaped domain-safe (Christiano Haesbaert and Gabriel Scherer, review by Xavier Leroy, report by Jan Midtgaard and Tom Kelly)
- #11516, #11524: Fix the
deprecated_mutable
attribute. (Chris Casinghino, review by Nicolás Ojeda Bär and Florian Angeletti) - #11576: Fix bug in Bigarray.Genarray.init in the the case of zero-dimensional arrays. (Nicolás Ojeda Bär, Jeremy Yallop, report by Masayuki Takeda, review by Jeremy Yallop and Florian Angeletti)
- #11587: Prevent integer comparison from being used on pointers (Vincent Laviron, review by Gabriel Scherer)
- Documentation changes
- #11093: Add tutorials on parallelism features and the relaxed memory model (KC Sivaramakrishnan, review by Damien Doligez, Anil Madhavapeddy, Gabriel Scherer, Thomas Leonard, Tom Ridge, Xavier Leroy, Luc Maranget, Fabrice Buoro, Olivier Nicole, Guillaume Munch-Maccagnoni, Jacques-Henri Jourdan)
ppx_deriving_yaml 0.1.0
Patrick Ferris announced
A new 0.2.0
version has just been released. Thanks to all the contributors (including Outreachy applicants!),
there’s a lot of nice additions including:
to_yaml
andof_yaml
attributes allowing you to add custom encoders and decoders- a
skip_unknown
flag for ignoring yaml keys so you can partially decode yaml values - a
default
attribute [@@deriving yaml]
is now an alias to[@@deriving to_yaml]
and[@@ deriving of_yaml]
so you can get decoders, encoders or both.
See the documentation in the README: https://github.com/patricoferris/ppx_deriving_yaml
A sandbox for proposing new features in odoc
jbeckford announced
For people interested in good documentation for their projects, but need a bit more from odoc.
I wanted to make a place where the broader OCaml community can experiment and propose odoc features: https://diskuv.github.io/odoc-sandbox/
I had two goals for the sandbox:
- Let you visually see any proposals for new features to odoc. Hopefully this will make scoping the proposal easier.
- No one should be blocked. Assuming the proposal is ok, it may take months (or even years) to implement. Any early adopter could use the custom Dune rules from the experiment to adopt in their own projects. Early adoption would be complex (custom Dune rules) but since the syntax shouldn’t change they won’t have to throw out their documentation.
I have one proposal in the sandbox and more will come later; there are also a few experiments using Sphinx and Markdown tools.
The first proposal would translate an odoc verbatim block:
{v ::code-block:: LANGUAGE source code v}
into syntax highlighted code. The visual results and the original .mli
are available in experiment
400.
As I mentioned earlier, it has Dune rules which can be copied if you need it in your own projects. I don’t precisely know how it would be implemented for real in odoc (probably it would be a .mli transformer) but that is not the point. Instead if you have an implementation idea or simply love/hate the proposal, you can just go to GitHub and file an issue at https://github.com/diskuv/odoc-sandbox/issues.
OCaml Platform Installer alpha release
Thibaut Mattio announced
In anticipation of the forthcoming OCaml 5 release—and hot on the heels of its beta release—we are thrilled to announce the alpha release of the OCaml Platform Installer.
As a reminder, the OCaml Platform is the recommended toolchain for developers to work with OCaml.
The Platform Installer allows the user to easily setup OCaml’s development environment, both for a first-time installation and for any new opam switches.
You can try it now by following the [installation instructions](#platform-installer-2), but TL;DR, you can install it with
$ bash < <(curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)
And run it to install the Platform in your opam switch with
$ ocaml-platform
Don’t hesitate to open an issue if you encounter any problem!
Update of the Platform State
As part of the work on the Installer, we’ve updated the state of the Platform to make it up to date and clarify the requirements to include a project in the Platform. Here are some of the notable changes:
odoc
has been promoted from Incubate to Active- OCamlformat has been promoted from Incubate to Active
ppxlib
and other metaprogramming frameworks have been removed from the Platform (andppxlib
is now documented as the official way to do metaprogramming, as part of the official OCaml documentation)
You can see the complete changes on the corresponding PR.
In parallel, we also want to make the Platform more open and transparent. To do this, we want to provide a clear governance model that can be driven by the community. The governance model should answer questions like:
- How can I incubate my project in the Platform?
- Who decides when a project can be promoted?
- How to discuss changes that would impact multiple Platform projects?
We’re currently following the OCaml.org’s governance, but are exploring new ways to govern the OCaml Platform.
We are leaning toward an RFC process for this, but we are still discussing the alternatives and how this could take shape. We will continue to experiment on the best governance model for the Platform with the project maintainers in the coming months, and we will publish the resulting governance on OCaml.org.
Platform Installer
The Platform Installer provides the best way to install OCaml and the recommended development tools for both newcomers and existing users.
Simplicity. It aims at replacing the existing installation steps with a much simpler workflow.
To install the Platform Installer ocaml-platform
, you can run:
$ bash < <(curl -sL https://github.com/tarides/ocaml-platform-installer/releases/latest/download/installer.sh)
This script will install opam, if not already present in the system, and the latest version of ocaml-platform
.
Then, to install the Platform tools:
$ ocaml-platform
If opam is not initialised, this command will initialise it. Then it will proceed to installing the Platform tools in the current opam switch.
For first-time users, the above two lines will set them up with a working environment that’s complete enough to hack comfortably with OCaml.
Speed. In order to speed up the process of installing development tools, the Installer caches the binaries to
avoid for redundant compilation. For instance, you will only need to compile ocamlformat
once per version of the
tool.
However, some tools such as Merlin depend on the OCaml version. For those, the Installer’s cache distinguishes the binaries, depending on the version of the tool as well the OCaml version it was compiled with.
Integration. The Installer integrates the development tools it installs as opam packages to make it fully aware
of what has been installed. The binary provided by the Installer for the dune
tool will allow the installation of
any package with a Dune dependency without reinstalling it.
However, some development tools include libraries in their opam package that are not provided by the Installer. In
this case, installing the original opam package for the tool will replace the one provided by ocaml-platform
.
The opam packages installed by ocaml-platform
are dependency-free. This means that installing specific versions of
your development tools will never mess with the actual dependencies of your project!
- The Tools Installed
The list of tools installed by
ocaml-platform
will ultimately be the platform tools listed in the Platform Docs as either Active and Incubate. Currently, this list is still incomplete for different reasons, such as keeping installation time short for new users, the number of dependencies, or other technical constraints.The set of installed tools already provides a complete working environment, with:
- A build system: Dune
- A documentation generator:
odoc
- A code formatter: OCamlformat
- A release helper:
dune-release
- Editor integration:
ocaml-lsp
and Merlin.
Next Steps
There’s still a lot to do!
Governance. As mentioned above, we should have a clear governance model for how incubation and promotions happen in the Platform, who decides, and which criterias are applied. This should obviously be an open process driven by the community; however, we want to make sure that we propose a governance model that will work well, so we’re still discussing alternatives and experimenting on some options. We’d love your input on this, so expect a Discuss post with a proposed governance model for the Platform soon. Don’t hesitate to [reach out](mailto:thibaut@tarides.com) to us before then if you want to get involved!
Editor Integration. While the Installer is a step forward towards a simple way to install OCaml, the UX for newcomers can still be improved. We plan to integrate the Installer to the official VSCode plugin. This will provide a way to get a complete development environment in only a few clicks directly in the editor. We will also explore how to make the setup of Emacs and Vim more straighforward.
Remote Cache. In order to further reduce the time taken to start hacking on a project, we are working on a remote cache for the Installer. The cache will be populated by a CI and would remove the need for compiling the tools locally. The local cache and compilation mechanism would still be used as a fallback if the remote repo is down or incomplete (such as for a pinned compiler).
Acknowledgments
Thank you to the developers and alpha-testers who contributed to the Plaform Installer project, particularly the Tarides engineers who have been driving the development:
We’d also like to thank our major funder Jane Street for supporting our work to improve OCaml’s installation experience!
dkml-dune-dsl 0.x.x - Parameterizable Dune files embedded in OCaml
jbeckford announced
I am pleased to announce dkml-dune-dsl, an embedding of Dune inside OCaml (aka. an eDSL) for developers that need to simplify complex Dune logic. An excerpt from the README:
Once installed you will be able to write DSL expressions like:
open DkmlDuneDsl module Build (I : Dune.SYM) = struct open I let res = [ rule [ target "{{{ name }}}.txt"; action (with_stdout_to "%{target}" (echo [ "{{{ age }}}" ])); ]; ] endthat are run over the parameters in a JSON file:
{ "param-sets": [ {"name": "batman", "age": 39}, {"name": "robin", "age": 24} ] }You can do also do aggregation or, if you are really adventurous, define your own interpreter. Even if you don’t use parameterization you get things you take for granted with OCaml: type-safety, auto-complete and `let` constants.
The full documentation including installation instructions and examples are available in the README.
A few cautionary notes:
- The current version (0.1.0 as of Oct 17 2022) does not have 100% coverage of all Dune expressions; it just has the parts of Dune I’ve needed in my own projects. That includes an encoding of the Ordered Set Language and virtual libraries but not (for example)
select
forms, plugins, ctypes and lex/yacc. But I strongly suspect the same people who need a tool like dkml-dune-dsl are the same people who can easily contribute a PR to add any parts of Dune they need. - The API is unstable; if someone adds more Dune expressions they may have to tweak the API. And I haven’t settled on whether the API needs some first-class features to track Dune’s
(lang dune X.Y.Z)
versioning. - This is not blessed in any way by the Dune team! Hopefully they don’t mind though.
Other OCaml News
From the ocaml.org blog
Here are links from many OCaml blogs aggregated at the ocaml.org blog.
Old CWN
If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.