OCaml Weekly News
Hello
Here is the latest OCaml Weekly News, for the week of December 13 to 20, 2022.
Table of Contents
B·o·B, an universal and secure peer-to-peer file-transfer in OCaml
Calascibetta Romain announced
As we said, the announcement of Bob being experimental, we had various feedbacks to improve Bob but also bugs on its use (particularly on MacOS).
In this, we fixed the problems and decided to offer an Atom feed to follow the development of Bob but also the changes made on our relay (available to users).
The website has been improved by implementing a “dark” mode to make it easier to read. We therefore advise you not only to download the latest version of Bob if you are a user, but also to subscribe to our feed in order to keep up to date with our development.
OCaml 5.0.0 is out!
octachron announced
We have the pleasure of celebrating the birthdays of Jane Austen and Arthur C. Clarke by announcing the release of OCaml version 5.0.0.
The highlight of this new major version of OCaml is the long-awaited runtime support for shared memory parallelism and effect handlers.
This multicore support is the culmination of more than 8 years of effort, and required a full rewrite of the OCaml runtime environment. Consequently, OCaml 5.0.0 is expected to be a more experimental version of OCaml than the usual OCaml releases.
In this release, the native compiler only supports the x86-64 and arm64 architectures. In terms of operating systems, Linux, the BSDs, macOS and mingw64 on Windows are supported.
We are planning to restore the support for most previously supported architectures and operating systems, and address remaining known issues over the course of the next year.
Don’t let those words of caution hinder your experimentation with OCaml 5: OCaml 5 as a language is fully compatible with OCaml 4 down to the performance characteristics of your programs. In other words, any code that works with OCaml 4 should work the same with OCaml 5.
The currently known exceptions to this rule are:
- the removal of many long-deprecated functions and modules
- changes to the internal runtime API
- the performance of ephemerons is currently (and temporarily) strongly degraded.
Please report any unexpected behaviours on the [OCaml issue tracker] (https://github.com/ocaml/ocaml/issues).
During the maturing period for OCaml 5, the OCaml 4.14 branch will be maintained by backporting bug fixes from the OCaml 5 branch at regular intervals. The next release on the OCaml 4.14 branch, OCaml 4.14.1, should soon follow this release.
The full list of changes can be found in the changelog below.
Installation Instructions
The base compiler can be installed as an opam switch with the following commands:
opam update opam switch create 5.0.0
It might be also interesting to check the new support for parallelism by installing the domainslib library with
opam install domainslib
The source code for the release candidate is also directly available on:
- Fine-Tuned Compiler Configuration
If you want to tweak the configuration of the compiler, you can switch to the option variant with:
opam update opam switch create <switch_name> ocaml-variants.5.0.0+options <option_list>
where
<option_list>is a comma-separated list ofocaml-option-*packages. For instance, for aflambdaandno-flat-float-arrayswitch:opam switch create 5.0.0+flambda+nffa ocaml-variants.5.0.0+options ocaml-option-flambda ocaml-option-no-flat-float-array
The command line above is slightly more complicated for opam versions before 2.1:
opam update opam switch create <switch_name> --packages=ocaml-variants.5.0.0+options,<option_list>
In both cases, all available options can be listed with
opam search ocaml-option.
Changelog for OCaml 5.0.0 (16 December 2022)
Editor’s note: as the Changelog is huge, please follow the archive link above to read it.
ThreadSanitizer support for OCaml 5.0.0 – first public release
Archive: https://discuss.ocaml.org/t/ann-threadsanitizer-support-for-ocaml-5-0-0-first-public-release/10975/1
Olivier Nicole announced
We are happy to announce the first release of ThreadSanitizer support for OCaml 5.
Motivation in a nutshell
With OCaml 5 comes parallel programming, and, with it, the possibility of introducing data races. A data race is when two or more threads access the same memory location concurrently, and at least one of the accesses is a write. Data races can lead to particularly hard-to-debug problems.
ThreadSanitizer (TSan for short) is an approach developed by Google to locate data races in code bases. It consists in instrumenting your executables to keep a history of previous memory accesses (at a certain performance cost), in order to detect a potential data race, even when this race has no visible effect on the execution. ThreadSanitizer has proved very effective in detecting hundreds of concurrency bugs in large projects.
There is now a new opam switch, 5.0.0+tsan, on which all your libraries and executables will be compiled with ThreadSanitizer
instrumentation. On that switch, the ThreadSanitizer runtime will run as part of your executables and emit reports when it detects a
potential data race:
==================
WARNING: ThreadSanitizer: data race (pid=4170290)
Read of size 8 at 0x7f072bbfe498 by thread T4 (mutexes: write M0):
#0 camlSimpleRace__fun_524 /tmp/simpleRace.ml:7 (simpleRace.exe+0x43dc9d)
#1 camlStdlib__Domain__body_696 /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/stdlib/domain.ml:202
(simpleRace.exe+0x47b5dc)
#2 caml_start_program ??:? (simpleRace.exe+0x4f51c3)
#3 caml_callback_exn /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/callback.c:168
(simpleRace.exe+0x4c2b93)
#4 caml_callback /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/callback.c:256
(simpleRace.exe+0x4c36e3)
#5 domain_thread_func /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/domain.c:1093
(simpleRace.exe+0x4c6ad1)
Previous write of size 8 at 0x7f072bbfe498 by thread T1 (mutexes: write M1):
#0 camlSimpleRace__fun_520 /tmp/simpleRace.ml:6 (simpleRace.exe+0x43dc45)
#1 camlStdlib__Domain__body_696 /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/stdlib/domain.ml:202
(simpleRace.exe+0x47b5dc)
#2 caml_start_program ??:? (simpleRace.exe+0x4f51c3)
#3 caml_callback_exn /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/callback.c:168
(simpleRace.exe+0x4c2b93)
#4 caml_callback /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/callback.c:256
(simpleRace.exe+0x4c36e3)
#5 domain_thread_func /home/olivier/.opam/5.0.0+tsan/.opam-switch/build/ocaml-variants.5.0.0+tsan/runtime/domain.c:1093
(simpleRace.exe+0x4c6ad1)
Mutex M0 (0x000000567ad8) created at:
#0 pthread_mutex_init /home/olivier/other_projects/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1316
(libtsan.so.0+0x3cafb)
[...]
SUMMARY: ThreadSanitizer: data race /tmp/simpleRace.ml:7 in camlSimpleRace__fun_524
==================
ThreadSanitizer: reported 1 warnings
OCaml executables are instrumented so that ThreadSanitizer will show precise backtraces, including when using exceptions, effect handlers, or C stubs.
ThreadSanitizer support for OCaml is a joint work with Fabrice Buoro, based on the work of Anmol Sahoo.
Usage instructions
At this stage, ThreadSanitizer for OCaml is supported only on the x86_64 architecture, and is not available on Windows.
Internally, the OCaml instrumentation uses the libunwind library, so if you’re on Linux you will need to install it using your system’s package manager. If you’re a macOS user, you have nothing to do: libunwind is installed by default.
Then you simply need to build and run your programs in the 5.0.0+tsan switch:
opam update opam switch create 5.0.0+tsan
Usage examples and further information can be found in the README of the project.
We are eager to get your feedback! We believe that running your Multicore programs or test suites with TSan can be a huge time-saver in debugging.
Guillaume Munch-Maccagnoni asked and Olivier Nicole replied
This is using the C memory model to analyze OCaml programs, right?
It works by instrumenting the executables with calls to the TSan runtime whose API follows the C model, yes. That’s why we use the mapping between the OCaml and C11 memory models described in https://github.com/ocaml/ocaml/pull/10995. As a consequence, every data race reported by TSan should be a data race in the OCaml sense, and conversely all OCaml races (on code paths that are actually executed at runtime) should be reported.
For instance, from the point of view of TSan, all OCaml non-atomic stores are release stores (more precisely a sequence consisting of an acquire fence and a release store).
Parany for OCaml-5
UnixJunkie announced
Parany: generalized map reduce for parallel / mluti-core computers.
Version v13.0.0 of Parany is made for OCaml >= 5.0.0. https://github.com/UnixJunkie/parany/tree/domains If you observe some performance improvement of your parallel programs, don’t hesitate to share!
PS: apparently github doesn’t show the latest version of parany’s git repository; but the tagged v13.0.0 is the right version of the code for ocaml>=5
WebAssembly/Wasm and OCaml, by OCamlPro
OCamlPro announced
We just published a blog post “WebAssembly/Wasm and OCaml” introducing our contributions to the GC-related Working Group on Wasm:
https://ocamlpro.com/blog/2022_12_14_wasm_and_ocaml This post will be followed in January by more technical posts.
We are also looking for sponsors to fund our work on Wasm. Contact us if you are interested in the topic !
opam 2.1.4, opam-publish 2.2.0, opam-file-format 2.1.5
Archive: https://discuss.ocaml.org/t/ann-opam-2-1-4-opam-publish-2-2-0-opam-file-format-2-1-5/10999/1
Kate announced
We are pleased to announce the minor release of opam 2.1.4, opam-publish 2.2.0 and opam-file-format 2.1.5.
This opam release consists of backported fixes:
- Make opam able to fully build with OCaml 5.0. `dose3 >= 6.1` and `base64 >= 3.1.0` are now required (#5357)
- Fix a bug where opam would not output any explanation when a conflict arises when attempting to install a package (#5378)
The opam-publish release consists of the following breaking change:
- Shorten the generated branch name. This is breaking change for people who want to force-push changes on an already opened PR using a previous version of opam-publish as the branch name might not be the same (#141)
and the following fixes:
- Fix the github markdown syntax generated by opam-publish (#131)
- Disambiguate GitHub archive URL computed from repository and tag (#135)
- Upgrade to cmdliner >= 1.1.0 (#134)
- Support git remotes with multiple push targets (#137)
- Fix broken state if the initial opam-repository clone fails (#140)
The opam-file-format release consists of the following fixes:
- Fix extra lines on non empty headers (#49)
- Make the tests flambda2-ready (#52)
- Fix build failure on OCaml 5.0 when using the non-dune build mode (#50)
To upgrade opam simply run:
bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh) --version 2.1.4"
For the rest, simply run:
opam update opam upgrade opam-publish opam-file-format
Old CWN
If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.