OCaml Weekly News

Sandmark Nightly is a service for the OCaml compiler developers that helps benchmark the development branches of the compiler on the large set of Sandmark benchmarks on tuned machines and reports the results in an interactive UI. Currently, Sandmark nightly reported many metrics including running time. But running time is a notoriously noisy metric on modern architectures due to the effects of modern OS, arch and micro-arch optimisations. There could be swings of 50% in either directions when the directory in which the program is run changes.

While we run Sandmark benchmarks on tuned machines, we still see impact of noise on the real time measurements. To this end, we introduce a new metric into Sandmark nightly that in addition to real time, would help interpret the results accounting for the noise. We now report “instructions retired” for Sandmark runs. Instructions retired report the number of instructions executed by the program during its run and hence is shielded from the noise that affects real time measurements. Of course, the same number of instructions may be discharged at different rates by the processor due to instruction-level parallelism and hence, the instructions discharged metric should be used in conjunction with real time measurements. For example, if a new compiler feature adds 2 instructions to the prolog of the function, then the instructions retired metric should inform you how many new instructions are actually executed on top of the baseline.

The instructions retired metric is collected from perf command. We also have other useful metrics from perf such as page faults, branches, branch misses, cache misses at various levels of the hierarchy, etc. We will add graphs to report these going forward. Enjoy the new feature, and as ever, report issues and bugs to Sandmark Issues.

The web service is available at https://sandmark.tarides.com and you can select the Perfstat Output radio button on the left panel as shown below.

After selecting the variants and a baseline for comparison, you can view the normalised instructions per cycle change as illustrated below:

You can also request for your development branches to be added to the Sandmark Nightly Service at the sandmark-nightly-config repository for the nightly runs.

Just a reminder the deadline for mentor signup is in 9 days, the same day as https://discuss.ocaml.org/t/outreachy-summer-22-closing-commemoration-session-on-23rd-sept/10450 :camel:

Unicode 15.0.0 was released on the 13th of september. It adds 4489 new characters to the standard.

Given the increasing contributions from the South Asian subcontinent to OCaml we are glad this includes support for the 42 (sic) characters of the Nag Mundari script. For information about the other additions, see the announcement page.

Accordingly the libraries mentioned at the end of this message had to be updated. Consult the individual release notes for details. Both Uucd and Uucp are incompatible releases sinces new script and block enumerants had to be added.

Note that starting from Unicode 16.0.0 – that is in a year – these libraries will be changed to use the UTF decoders of the standard library rather than rely on Uutf. They will thus only be available for OCaml >= 4.14.

The OCaml tips of the minimal Unicode introduction, which you should read if Unicode still puzzles you, have been updated to mention the new standard library UTF decoders.

Also, the ucharinfo tool distributed with uucp[^1] can now also lookup characters by matching substrings in their Unicode name or name aliases.

Best,

Daniel

A big thanks for funding from the OCaml Software Foundation and from my faithful donators.

• Uucd 15.0.0 Unicode character database decoder for OCaml. http://erratique.ch/software/uucd
• Uucp 15.0.0 Unicode character properties for OCaml. http://erratique.ch/software/uucp
• Uunf 15.0.0 Unicode text normalization for OCaml. http://erratique.ch/software/uunf
• Uuseg 15.0.0 Unicode text segmentation for OCaml. http://erratique.ch/software/uuseg

[^1]: It’s a depopt you’ll need opam install cmdliner uutf uunf uucp to install it.

(this is in french only as the talks will be in french it’s probably not relevant for english speakers)

Le prochain OUPS aura lieu le jeudi 29 septembre 2022. Le rendez-vous est fixé à 19h au 4 place Jussieu (salle à préciser), 75005 Paris.

L’inscription est obligatoire pour pouvoir accéder au meetup !

Les exposés seront également retransmis en ligne sur le galène du OUPS.

Toutes les informations sont disponibles sur le site du OUPS.

Programme :

COBOL 101 – Émilien Lemaire

COBOL est un langage très ancien et est assez éloigné de ceux que nous manipulons tous les jours. Malgré cela il reste l’un des plus utilisés dans le monde.

Durant cette présentation je vais donc vous introduire au langage, voir comment sont écrit les programmes, comment les variables sont-elles déclarées et comment les manipuler. Je vais aussi vous présenter quelques features “intéressantes” du langage, dont certaines sont inattendues.

OCaml Multicore – Florian Angeletti

Opam-bin: Opam et paquets binaires – Fabrice Le Fessant

L’utilisation d’un gestionnaire de paquets sources comme Opam n’est pas toujours optimale en temps, car l’outil passe beaucoup de temps à recompiler des paquets, dèjà compilés dans le passé ou par d’autres utilisateurs. Le plugin Opam-bin répond à ce problème en permettant de créer à la volée des paquets binaires, qui seront réutilisés à l’avenir et peuvent être partagés avec d’autres utilisateurs. L’exposé montre son utilisation et comment il fonctionne.

Les présentations seront suivies par des discussions libres. Les pizzas seront offertes par la fondation OCaml ! :pizza:

As has just been announced at the OCAML 2022 workshop, the new, re-written version of strymonas library is now available at

https://strymonas.github.io

Strymonas is the stream processing library that achieves the highest performance of existing OCaml streaming libraries, attaining the speed and memory efficiency of hand-written state machines. It supports finite and infinite streams with the familiar declarative interface, of any combination of map, filter, take(while), drop(while), zip, flatmap combinators and tupling. Experienced users may use the lower-level interface of stateful streams and implement accumulating maps, compression and windowing. The library is based on assured code generation (at present, of OCaml and C) and guarantees in all cases complete fusion.

Compared with the original strymonas (POPL 2017), the new version is completely re-written and:

• Generates not only OCaml but also C (which needs no OCaml run-time and vectorizable)
• Has Core + code-generation Backends architecture: MetaOCaml is needed only for the OCaml backend and benchmarks; the Core and the C generation backend are pure OCaml. More backends can be easily added.
• The complete fusion is now achieved in all cases
• Supports both user-friendly and familiar declarative combinators, and low-level core of stafeful streams (which can be used together)
• Core streams support streams over tuples, records and even abstract data types
• Fusion is now performed as normalization-by-evaluation

The paper https://strymonas.github.io/docs/ocaml-22.pdf and the OCAML 2022 talk (soon to be available on YouTube’s SIGPLAN channel, among all other talks of the ICFP 2022 event) give more details. The github repo contains the complete code of the library, examples and all benchmarks.

Hello all! On behalf of the OCaml CoC committee, I’d like to present the proposed Code of Conduct for the OCaml community. We hope this is a step towards ensuring a friendly and inclusive community for everyone.

The CoC text, based on Contributor Covenant can be found here.

For the Neovim-specific use case, you may want to take a look at vcaml, which lets you write OCaml programs that interact with Neovim over msgpack RPC. Do note though that while the library as-is should provide you with the functionality you need, it is under active development so the API may change (improve) in significant ways between releases.

Doesn’t necessarily help much, but a while ago I wrote a proof-of-concept ML-style language (using OCaml, of course) that transpiled to Lua - https://github.com/merle-lang/luml (I was mostly thinking about targeting game engines… I did enough to implement Tetris and then gave up on it).

The module that emits Lua source code was pretty simple: https://github.com/merle-lang/luml/blob/master/lib/compile.ml - I did thinking about trying to target byte code but it seemed tricky due to different Lua versions, I think.

Any chance we might see a conf-bazel package added to OPAM so a package can depend on a compatible version of Bazel being installed on the host?

I’ve put a lot of work into seamless OPAM integration, but only in one direction: make it easy to use OPAM resources in a Bazel build program. I have not put much thought into integrating Bazel itself into the OPAM ecosystem. For example publishing a Bazel-enabled package to OPAM. It looks like writing such a conf-bazel package would be pretty easy, but I’m not sure it would do us much good at the moment. What specific use cases do you have in mind?

There are two ways to integrate Bazel and OPAM. One is to automatically generate BUILD.bazel files for OPAM packages. Then Bazel would build everything, eliminating the need for the OPAM engine. This is the strategy followed by rust (tool: cargo_raze, evidently now supplanted by crate_universe) and go (tool: gazelle). Unfortunately a complete solution along these lines is not feasible for OCaml, since source files do not carry enough information to support inference to a build program, and OPAM packages may use a variety of build languages (Dune, Makefiles, OMake, etc.). On the other hand, Dune seems to be the most widely used build engine by a considerable margin, and the Dune language is easy to parse (if not so easy to interpret), so I’m working on a conversion tool that automatically converts Dune files to BUILD.bazel files.

The other strategy is to rely on OPAM to build dependencies and then “import” the built artifacts into Bazel. OBazl defines an opam_import rule for this purpose, and a tool that bazelizes OPAM switches, generating an OBazl ’coswitch’. The mapping from OPAM package name to Bazel label is straightforward: ’yojson’ to @yojson//lib/yojson, ’lwt.unix’ to @lwt//lib/unix, etc.

So in practice OBazl supports a hybrid approach. Use Bazel to build your code, but import pre-built OPAM dependencies. To do that you run the opam conversion tool to generate a ’coswitch’ which defines a local Bazel repo for each OPAM package, and configure your WORKSPACE.bazel to import those repos. Write your BUILD.bazel files using opam labels as above. If your project already uses dune, you can run the dune conversion tool to generate your BUILD.bazel files, which in some cases will need some tweaking, since some Dune stanzas lack sufficient information for conversion, and in others the conversion code needs complicated logic that I haven’t gotten around to writing, or that does not seem worth the bother.

The OPAM “import” conversion tool is fairly stable. It converts the META files in OPAM into BUILD.bazel files, which include dependency information. So when you depend on an opam_import target you get its entire dependency graph.

The Dune migration tool is another matter. Reverse-engineering the Dune language is a non-trivial task, lemme tell ya. The good news is that after what seems like eons of work the end is in sight. I’ve been running it against a semi-random set of projects (js_of_ocaml, ocaml-protoc, some ppx libs, etc.) and working through the quirks inch-by-inch. Rule stanzas are a real PITA, can I just say that? In any case, it looks like I should have an alpha release with documentation and some case studies within a week or so. I hope. At the very least I’ll convert my dev configuration into something usable by others so you can follow along if you want.