OCaml Weekly News

I would greatly appreciate some help with testing the upcoming Logarion 0.6.

Logarion is a free and open-source a blog-wiki hybrid text system.

The easy way to install; binary tarballs: https://logarion.orbitalfox.eu/downloads/

I'm happy to announce the release of Yojson version 1.6.0.

In an effort to modernize it a little, each submodule now comes with a type t and equal, pp and show functions.

The json aliases for t are being deprecated, they'll be removed for the next major release.

Special thanks to @Leonidas since he basically did all the work!

You can find Yojson on github here.

Dynlinking is a quite complex topic in OCaml and still [unsound][1] (as of OCaml 4.07 and below). And if you will dig further you will even find out that OCaml has 3 dynlinking facilities that differ in their behavior, the toplevel linker is different from the bytecode lifter and both are different from the native linker.

The good news is it is possible to implement a sound and robust linking facility in OCaml (even in the presence of the inherited unsoundness). We have implemented it in BAP, and other projects have their own implementations with varying flexibility and safety.

In BAP the support for dynamic linking is split into three parts,

1. we have the bapbuild tool which is an ocamlbuild plugin that knows how to properly build plugins. It is shipped as a tool, but could be also used as a normal plugin, since it is packed into [a library][2]. Our plugins are zip-files that contain cmxs and cma so that they could be used in both three loaders, as well as meta information (the reason why meta is needed will be given below) and optional dependencies, which makes a plugin independent off the environment. The bapbuild tool is also clever in that it knows how to handle broken packages, that forget to ship cmxs or cma or even both.
2. The loading part is implemented by the [bap-plugins][3] library. This part tracks carefully which compilation units are already linked into the process image. This is why we need the meta information in plugins, that describe which compilation units are already linked into the shared object and which are required.
3. Finally a [myocamlbuild.ml plugin][4] is required for the host program (i.e., the program that loads plugins), since usually the host program is by itself composed of some compilation units and we should be sure that the loaded plugin doesn't try to link into the program body any compilation unit that is already there. We don't want to parse the ELF data structure of the host program to figure out which units are already linked since it is not portable and fragile, so we need some support from the build system. In particular we use the ocamlfind.dynlink library that stores this information in a special data structure inside the binary, and extend it to support internal modules (since a plugin and a host program could be built from the same source tree, so that ocamlfind won't notice that they are linked).

So if you will follow carefully our implementation and ensure that you have all three ingredients in this of that form, you will be able to load your shared code using all three OCaml linkers. If stuck, don't hesitate to ask questions. This is all not to scare you away from dynlinking. BAP wouldn't be possible without it, for example.

[1]: https://github.com/BinaryAnalysisPlatform/bap/blob/master/lib/bap_plugins/bap_plugins.ml
[2]: https://github.com/BinaryAnalysisPlatform/bap/blob/master/lib/bap_build/bap_build.ml
[3]: https://github.com/BinaryAnalysisPlatform/bap/blob/master/lib/bap_plugins/bap_plugins.ml
[4]: https://github.com/BinaryAnalysisPlatform/bap/blob/master/myocamlbuild.ml.in

As planned, we are entering the second stage of the jbuilder deprecation. FTR, support for jbuilder will be discontinued in July 2019. The following post gives a bit more detail and explains how to upgrade to dune, in particular using the automatic upgrader:

https://dune.build/blog/second-step-deprecation/

I made a minor update to version 1.1.0 of some of the Caqti packages. This fixes an issue with recovery after connection loss for PostgreSQL discovered by Dave Aitken, and adds recovery for MariaDB when using a connection pool, so I recommend updating any continuously running services.

Here is the full release note:

• Add pretty printer for requests.
• Add variance to 'a future declarations.
• Add blocking instance of API.
• Generalize $. to $<var>. in queries.
• Infer the expansion of $(<var>.) from $(<var>) if not provided.
• Fix connection recovery for PostgreSQL (issue #19, Dave Aitken).
• Fix some unhandled exceptions for PostgreSQL.
• Fix connection validation for MariaDB.

Just a headsup about an incoming deprecation to the cstruct library.

In the old days (pre cstruct.3.0.0), the ocamlfind libraries were subpackages of the main cstruct library: cstruct.lwt, cstruct.async, cstruct.ppx and cstruct.unix. They were also built in the same directory in some cases, so it was possible to "accidentally" use one of the submodules and have linking work without accurate dependencies being specified.

In Cstruct 3.0.0, we split out all the subpackages into their own first-class opam packages, so that they became cstruct-async, cstruct-lwt, cstruct-unix and ppx_cstruct. However, to ease the transition for old projects, the META file still exposed the old names as a dependency redirect.

With Cstruct 3.4.0 about to go into opam, those compatibility shims will now be removed. The reverse dependency analysis of the opam build doesn't look too bad, and it's mainly mirage projects that were affected. However, if you mysteriously find that your project is failing due to finding a cstruct library, try just (e.g.) renaming cstruct.lwt to cstruct-lwt in the dune file, and adding cstruct-lwt to the opam dependency list.

The reason for removing the compatibility shim is that the old ocamlfind names do not work when embedding the dune libraries in a dune vendor workspace. So I'm removing the technical debt now to make those so-called "duniverse" deployments significantly easier to manage.

This has now been merged into opam-repository so please don't forget to update any dev versions of opam files you have in your own repositories with the new ocamlfind names, and an opam constraint to "cstruct" >= "3.0.0" to make them available.

Here are links from many OCaml blogs aggregated at OCaml Planet.

• Improving Tezos Storage : Gitlab branch for testers
• Elixir Developers & Functional Programmers at CareDox (Full-time)
• Improving Tezos Storage : update and beta-testing
• Tezos and OCamlPro
• L2 Regularization and Batch Norm
• opam 2.0.3 release