OCaml Weekly News
Hello
Here is the latest OCaml Weekly News, for the week of August 30 to September 06, 2022.
Table of Contents
- Day of the Camel 2022: OCaml in Academia and Industry (online, 9 September 2022)
- Advice for combining multiple monads
- New OCaml meetup group in Toulouse (in French)
- GADTs for tracking dimensions of multidimensional arrays?
- ocaml-eris 1.0.0 - Encoding for Robust Immutable Storage (ERIS) (+ Using Zig code compiled to WebAssembly from OCaml)
- OCamlverse updates
- Diskuv OCaml 1.x.x; Windows OCaml installer no longer in preview
- Does OCaml infer types between files?
- Other OCaml News
- Old CWN
Day of the Camel 2022: OCaml in Academia and Industry (online, 9 September 2022)
Roberto Blanco announced
The Day of the Camel 2022 is a one-day hybrid workshop dedicated to the OCaml programming language and its industrial users.
It comprises talks and discussions with members of the OCaml development team and companies using the language to solve complex and interesting problems.
We will present a broad picture of the OCaml ecosystem and, more widely, of functional programming as a viable and powerful choice for building correct and reliable computer systems.
The event is organized as part of an OCaml summer school at the University of Zaragoza, and sponsored by the OCaml Software Foundation.
Participation is free and open to everyone. For more details about the schedule, updates and links to the online seminar, see the website:
https://webdiis.unizar.es/evpf/event.html
9 September 2022, all times CEST (UTC+2)
Morning - Language session
09:15-10:00: | Welcome and reception |
Darío Suárez (CS Department Secretary, University of | |
Zaragoza) | |
10:00-11:00: | Gabriel Scherer (Inria) - The OCaml project and ecosystem |
https://ocaml.org/ | |
11:00-11:30: | Break |
11:30-12:30: | OCaml developers - Round table and Q&A |
Afternoon - Industry session
14:30-14:55: | Ulysse Gérard (Tarides) |
https://tarides.com/ - Building functional systems | |
14:55-15:20: | Nicolás Ojeda Bär (LexiFi) |
https://www.lexifi.com/ - Modeling language for finance | |
15:20-15:45: | Javier Chávarri (Ahrefs) |
https://ahrefs.com/ - Petabyte-scale web crawler | |
15:45-16:10: | Grant Passmore (Imandra) |
https://www.imandra.ai/ - Automated reasoning as a service | |
16:10-16:30: | Break |
16:30-17:30: | Industry speakers - Round table and Q&A |
We look forward to seeing you there!
Roberto Blanco later added
Quick correction: Paul-Elliot Anglès d’Auriac from Tarides will be presenting instead of Ulysse. :slight_smile:
Advice for combining multiple monads
Continuing this thread, Raphaël Proust said
In the Octez project we have face a similar issue: we use both Lwt
and Result
pervasively.
TL;DR:
- We use dedicated syntax module for each monad (Lwt, result, Lwt+result) which export binding operators (
let*
,let+
,return
, etc.). - We have a Stdlib supplement exposing monadic variants of all the Stdlib’s traversal functions (like
Lwt_list
but much more extensive)
Some time ago
Until not so long ago we used infix operators for bind. And we would mix different operators depending on what
sub-monad a specific expression would be in. So we would have >>=?
for Lwt+result, >>?
for result-only, and
>>=
for Lwt-only. Plus we had a dedicated operator for when you use a result-only expression in an Lwt+result
context: >>?=
. We don’t need the other specialised binder because Lwt-only and Lwt+result mix quite well: (_, _)
result Lwt.t
is just a specific case of _ Lwt.t
so >>=
just works.
We also had a very flat namespace where all the operators as well as some helper functions (e.g., we had error :
'err -> ('a, 'err) result
and fail : 'err -> ('a, 'err) result Lwt.t
) were exported by an Error_monad
module
which was opened everywhere (using -open
as a build flag).
Now
We have changed a few things.
- We use binding operators (
let*
and such) which are exported by dedicated syntax modules.Lwt_syntax
is essentially our local version ofLwt.Syntax
Result_syntax
is for the result-monadLwt_result_syntax
is for the combination monad- (We also have
Option_syntax
andLwt_option_syntax
, but we don’t use those as often.)
- Our Lwt+result monad syntax includes dedicated binding operators for Lwt-only and result-only expressions
- Internally, we recommend to only open those locally. So you start your function by
let open Lwt_result_syntax
(or whichever monad you are actually using there). - We have an extensive Stdlib supplement with many of the monadic variants of the provided traversors baked in. E.g.,
List.map_s
is the Lwt-only equivalent ofList.map
,List.map_e
is the result-only, andList.map_es
is the Lwt+result.- We also have concurrent variants (e.g.,
List.map_p
andList.map_ep
) for the traversals where it makes sense. - We have a consistent semantic regarding error management across all those traversors.
- We have several modules:
List
,Map
,Set
,Seq
, etc.
- We also have concurrent variants (e.g.,
One thing I didn’t mention is that we actually have a specialised result
called tzresult
: 'a tzresult = ('a,
tzrerror trace) result
where tzerror
is a custom error type and trace
is a data-strucutre holding several
errors. Traces (of errors) allows us to combine errors in different ways. The first way is you can add an error to a
trace to add higher-level context about the lower-level error. The second way is for errors that happen concurrently
(e.g., you evaluate concurrently several Lwt+result expressions and more than one fails). This ability to combine
concurrent errors gives us a semantic for
and*
in the Lwt+result syntax module.
There are several downsides to our current approach, but all in all it works well enough.
- Even for functions of modest size, it’s not always immediately visible which monad you are located in. This is even worse when you are viewing just a chunk of a diff.
- Some parts of the Stdlib don’t lend themselves to our monadification (e.g.,
Seq
) or require a bit of boilerplate because they don’t expose internal representations (e.g.,Map
’s monadic traversors are largely implemented in terms ofSeq
). - The Stdlib supplement is a lot of code with a lot of tests and a lot of comments. It’s just a large volume of low-complexity code to deal with.
There are also some very pleasant upsides:
- The separate monad syntax modules encourages you to write each function with the smallest monad that it needs. This in turns
- encourages you to split your function into smaller components which are more easily testable,
- makes the type of functions informative: an
Lwt.t
function is very likely to actually be doing I/O at some point down the line, and aresult
function is very likely to actually returnError
in some situations.
- The Stdlib supplement makes it quite easy to adapt code for one monad to another.
For more complete information, you can check the Error-monad tutorial.
Later
There are a few things we want to change. Albeit we have no urgent need to do so.
- Remove some legacy helper functions that are still hanging around.
- Organise the namespace better.
- Generate a lot of the code and doc of Lwtreslib automatically.
- Improve traces (currently, for mostly historical reasons, we use
list
).
Daniel Bünzli replied
We use dedicated syntax module for each monad (Lwt, result, Lwt+result) which export binding operators (
let*
,let+
,return
, etc.).
Brr uses the same approach for its futures which are used to FFI with JavaScript promises (these are not called promises because they don’t directly map on JavaScript promises as those have a non-monadic semantics which would break OCaml type safety).
Basically you have futures, future results and two syntaxes to choose from depending on your context.
Rahul asked and Raphaël Proust replied
I quite like the trace idea - I wish I’d thought of this for our codebase, but it would require too many changes in the calling code to adopt now. We just use
string
for the error type in our codebase and I’m not entirely satisfied with the “just pick the first error / fail fast” semantics I proposed forand*
inLwt_result.Syntax
. Another choice might even have been to parametrize theSyntax
module over desired behavior of how to combine concurrent errors, but we decided against it to avoid having to invoke a functor repeatedly all over the code (we’re already quite verbose - we don’t do global opens, and try to be explicit, e.g:let open Lwt_result.Syntax in
- to know exactly which monad we’re talking about).
Implementation details on our side:
- We do have a functor for Tracing which is parametrised by an implementation of trace.
- We only initialise it once in the production part of our code-base. We also initialise another time for some tests.
- We provide examples for trace implementations including a version that gives you the left-most error.
I’d say that string
is quite manageable for error management. If you want to structure it a bit more you can mash
strings together a bit. Something like
val trace : string -> ('a, string) result -> ('a, string) result let trace high_msg = function | Ok v -> Ok v | Error low_msg -> (* indent all of the local errors *) let low_msg = "\t" ^ String.concat "\n\t" (String.split '\n' low_msg) in (* layout high error and low error *) Error (Printf.sprintf "%s\n%s" high_msg low_msg) val parmash : ('a, string) result list -> ('a list, string) result let parmash rs = let rec loop acc = function | (Ok v) :: xs -> loop (v :: acc) xs (* accumulate Oks *) | (Error msg) :: xs -> loop_err [msg] xs (* Switch to err loop on first Error *) | [] -> Ok (List.rev acc) (* just return accumulated Oks *) and loop_err acc = function | Ok _ :: xs -> loop_err acc xs (* Discard Oks *) | Error msg :: xs -> loop_err (msg :: acc) xs (* Accumulate Errors *) | [] -> match acc with | [] -> assert false | [msg] -> Error msg (* single error: return as-is *) | _ :: _ :: _ as msgs -> (*multiple errors: pretty-print with separators *) Error ("|" ^ String.concat "\n|" (List.rev msgs)) ) in loop [] rs
Or something along those lines. And then and*
calls parmash
on the resolved promises to make line-separated
|
-prefixed error messages.
Basically you end-up just doing the printing of the trace in advance. And it’s heavier than to carry a type trace =
Nest of string * string trace | Par of string trace list | Leaf of string
and do the printing only once. Plus you
can’t have fine control over the printing like hbox and vbox. But it might be acceptable if the errors are truly the
exceptional case and you don’t have too many levels of them.
New OCaml meetup group in Toulouse (in French)
David Declerck announced
[this post is about a new French-speaking OCaml meetup group in Toulouse, thus the remainder of the text will be in French]
Bonjour à tous les cameleux toulousains !
Ma collègue @rjbou et moi-même mettons en place un meetup group OCaml à Toulouse.
S’inspirant des meetups “OUPS” à Paris, nous avons souhaité créer un événement similaire à Toulouse. L’objectif de ces meetups est de rassembler des personnes aux profils variés, qu’ils soient utilisateurs débutants ou chevronnées d’OCaml (développeurs, chercheurs…), ou qui souhaitent simplement découvrir ce langage. Les orateurs sont invités à présenter un sujet qui leur plaît, qu’il s’agisse d’une nouvelle librairie ou un nouvel outil, les nouvelles fonctionnalités du langages, des travaux de recherche, un bout de code amusant (liste non exhaustive)… Ces meetups se termineront par des discussions autour d’une collation et/ou un apéritif (sponsorisés par OCamlPro).
Le premier meetup devrait se tenir fin septembre / début octobre. La date exacte et le programme seront communiqués sous peu. N’hésitez pas à vous inscrire dès maintenant dans le groupe pour ne manquer aucune information !
Les meetups se dérouleront à l’ENSEEIHT (2 Rue Charles Camichel, 31000 Toulouse), que nous remercions chaleureusement pour la mise à disposition de ses locaux.
Nous projetons d’ores et déjà d’organiser ces événements approximativement tous les deux mois. N’hésitez pas à venir présenter un sujet qui vous tient à coeur !
GADTs for tracking dimensions of multidimensional arrays?
Archive: https://discuss.ocaml.org/t/gadts-for-tracking-dimensions-of-multidimensional-arrays/10407/12
Deep in this thread, Markus Mottl said
Lennart Augustsson has recently developed a Haskell library called Orthotope for dealing with multidimensional arrays. It can reflect shape and dimension information in types. You can also watch a video presentation about Orthotope.
There is likely no way to translate this to OCaml without significant type system extensions, but some of the concepts and ideas might inspire similar work.
ocaml-eris 1.0.0 - Encoding for Robust Immutable Storage (ERIS) (+ Using Zig code compiled to WebAssembly from OCaml)
pukkamustard announced
I’m pleased to announce an initial release of ocaml-eris - an OCaml implementation of the Encoding for Robust Immutable Storage (ERIS): https://codeberg.org/eris/ocaml-eris/releases/tag/v1.0.0 (also available on OPAM).
ERIS defines an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier that can be encoded as an URN. The content can be reassembled from the blocks only with this identifier. ERIS allows a form of content-addressing that is optimized for transport over networks and allows content to be made available robustly.
ERIS is application and transport agnostic. A very similar encoding (ECRS) is used in GNUNet for file-sharing. Applications include transporting small pieces of RDF data or distributing Guix substitutes. Possible transports include HTTP, IPFS, CoAP and Sneakernets.
ERIS is formally specified and implementations for other language exist (and are being developed).
The OCaml ERIS implementation allows:
- Streaming encoding: Allows large content to be encoded while only keeping a minimal amount in memory.
- Random-access decoding: Decode small pieces of large content while de-referencing a minimal amount of blocks.
- Cross-platform support: See section below.
Cross-platform support
ERIS requires two cryptographic functions:
- Blake2b (with keying support)
- ChaCha20 (IETF variant as defined by RFC 8439)
Multiple implementations are provided for various platforms. Users can choose the implementation by using one of
following packages (same trick as used by digestif
):
eris.crypto-monocypher
: Uses the Monocypher cryptographic library via the ocaml-monocypher bindings. This implementation works well on the Unix platform and is selected by default.eris.crypto-zig
: Uses the cryptographic primitives provided in the Zig standard library. This implementation works on the Unix platform and requires the Zig compiler installed.eris.crypto-wasm
: Uses the cryptographic primitives provided by the Zig standard library compiled to WebAssembly. This implementation can be used with js_of_ocaml and requires the Zig compiler installed. The WASM code is bundled using Crunch so users do not need to worry about provisioning or loading WASM.
eris.crypto-zig
is an example of how Zig code can be used from OCaml. eris.crypto-wasm
is an example of how Zig code compiled to WebAssembly can be used from OCaml.
One large appeal of using Zig is that the entire build infrastructure is much smaller than compared to using something like Emscripten. it is also possible to guarantee reproducible and boot-strappable builds (using Guix).
OCamlverse updates
Deep in this thread, Yotam Barnoy announced
Note: ocamverse has now moved to [ocamlverse.net](ocamlverse.net).
Diskuv OCaml 1.x.x; Windows OCaml installer no longer in preview
jbeckford announced
Diskuv OCaml 1.0.1 Release
This is mostly a bug fix. It is available at https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.1/setup-diskuv-ocaml-windows_x86_64-1.0.1.exe and the full release notes are at https://github.com/diskuv/dkml-installer-ocaml/releases/tag/v1.0.1
Summary of changes:
- The installer now checks whether files are in use when overwriting a previous installation just like the uninstaller already did.
- Fix Dune shim so
dune build
works consistently on Windows. https://github.com/diskuv/dkml-installer-ocaml/issues/6 - Fix detection of Jane Street package versions so
ppx_jane
dependencies likefieldslib
, and other JS packages, are pinned to versions likev0.14.0
(etc.). Also pin transitive dependencies ofppx_jane
. https://github.com/diskuv/dkml-installer-ocaml/issues/8 - MSYS2 variables are available as Opam global variables. The list is here. For example,
opam var --global msystem
isCLANG64
andopam var --global mingw-package-prefix
ismingw-w64-clang-x86_64
. They will become useful when Opam depext functionality is added for MSYS2. - Fix version in Add/Remove Programs that was
dev
instead of1.0.1
(etc.)
Does OCaml infer types between files?
Deep in this thread, Yawar Amin said
Yes, OCaml decouples compilation of modules from each other. I collected some thoughts on that here: https://dev.to/yawaramin/ocaml-interface-files-hero-or-menace-2cib
Other OCaml News
From the ocaml.org blog
Here are links from many OCaml blogs aggregated at the ocaml.org blog.
Old CWN
If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.