OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of November 13 to 20, 2018.

Table of Contents

New release of Menhir (20181113)

Fran├žois Pottier announced

I would like to announce a new release of Menhir, 20181113. It is available via opam (opam update && opam install menhir).

https://gitlab.inria.fr/fpottier/menhir/blob/master/CHANGES.md

The main change is an experimental new syntax for rules. The new syntax is slightly more pleasant and powerful than the old syntax. The old syntax remains available; the user chooses between the two syntaxes on a per-rule basis.

This blog post presents a couple demos that use the new syntax:

http://gallium.inria.fr/blog/parser-construction-menhir-appetizers/

For expert users, I have also written a brief a summary of the differences between the old and new syntaxes:

https://gitlab.inria.fr/fpottier/menhir/blob/master/doc/new-rule-syntax-summary.md

Modern Standard Library Documentation

seadynamic8 announced

I'm new to learning OCaml and also new to this forum.

I am really enjoying learning OCaml, but there's a lot of quirks and weird things since I feel like a lot of things are in transition right now, but one thing that really bugged me was how outdated the docs on the standard library looked.

So I decided to re-theme it. Here's my interpretation of it, based on Elixir docs: https://www.streamingspring.com/ocaml/docs/

It has a sidebar to quickly navigate between modules and functions. Also has a search bar.

Its a little heavy right now since all the docs are basically downloaded to front-end, but it works for now. In the future, I may change it to be served from the server-side.

How it works right now is this:

  • I use lambdasoup to parse the original standard library
  • I use atd-gen to convert it into json
  • I use bs-json to read it back in on the front-end. (with a hack for compatibility)
  • I then use bucklescript-tea to display it.

I feel that I could rearrange the content better, but I'm still a bit confused on certain things and I'm still a bit confused on Functors and more advanced concepts. I'll get to that a little later.

I think that the content itself could be improved, but as I'm a beginner, not really going to be working on that for now.

Anyways, I hope it helps some people.

2-year Postdoc Position on Frama-C/E-ACSL

Julien Signoles announced

The Software Reliability and Security Lab at CEA LIST (Paris Saclay, France) is hiring a 2-year postdoctoral researcher to improve the Frama-C runtime verification plug-in E-ACSL.

Frama-C is an opensource framework providing several analyzers for C code. The analyzed programs can be annotated by formal specifications written in the ACSL specification language. E-ACSL is one of the existing Frama-C analyzers. It converts ACSL annotations into C code in order to verify their validity at runtime, when the program is being executed.

The goal of this postdoctoral position is twofolds: on the one hand, the postdoctoral researcher shall propose new compilation schemes to support additional ACSL constructs; on the other hand (s)he shall adapt existing compilation techniques or define new ones in order to optimize the generated code for reducing the time overhead and the memory footprint of the generated program. The work will be guided by and evaluated on case studies providing by a few of our academic and industrial partners.

Knowledge in at least one of the following fields is required:

  • OCaml programming (at least, functional programming)
  • C programming
  • runtime verification
  • compilation
  • static analysis
  • semantics of programming languages
  • formal specification

A full description of the position is available online:

http://julien.signoles.free.fr/positions/postdoc-eacsl.pdf

Feel free to contact me for additional details, Julien Signoles

Opam-ci: new distributions available

Anil Madhavapeddy announced

I've pushed a update to the Dockerfile generation on the ocaml/opam2 Docker Hub images to account for newer distributions that have been released. We now build:

  • fedora-29 (alias fedora)
  • alpine-3.8 (alias alpine)
  • opensuse-15.0 (alias opensuse)
  • ubuntu-18.10 (alias ubuntu)

We now also publish the development version of the compiler to make for easier testing of trunk, via the 4.08 tag.

The full list can be found on the containers infrastructure wiki along with general instructions on how to use the Docker images.

The source code that generates these images is in ocaml-dockerfile for the library, obi for the binaries. If you are interested in adding a new Linux distribution, take a look at Dockerfile_distro module for how to get started.

A WebAssembly backend for OCaml

Sander announced

An article on the WebAssembly backend work done so far and what lies ahead:

https://medium.com/@sanderspies/a-webassembly-backend-for-ocaml-b78e7eeea9d5

Orsetto: structured data interchange languages (preview)

james woodyatt announced

I'm nearing completion of the ~preview.1 release to OPAM of Orsetto, my toolkit for structured data interchange. Here's a brief summary of things left to do before I can commit to collaborating with the community and to engaging with feedback.

  • Build pipeline for continuous integration on Bitbucket.Org.
  • Generate documentation with odoc using the Conjury build system.
  • Consolidate Cf_seq as extension of the Stdlib.Seq module.
  • Minor last-minute improvements (e.g. leap second database, et cetera).
  • Removal of some experimental modules (e.g. Cf_dyn and Cf_psys).

Not sure when I'll have done this, but I'm hoping by the end of the month. I'm writing this now because I feel like I've gotten to the point where there isn't any major new logic that needs to be done before an initial preview can be posted. Anybody interested in my work who wants to comment before I post a preview to OPAM is invited to check out the sources at Bitbucket (linked above) and file issues.

p.s. I would have just added this as an update to the previous topic I filed about it, but Discourse is now refusing to let me reply more than three times to myself.

Mirage 3.3.0 released

Hannes Mehnert announced

It is my pleasure to announce MirageOS 3.3.0, which was just merged into opam-repository! MirageOS 3.3.0 contains some breaking changes, see below for details.

What is MirageOS?
MirageOS is a library operating system that constructs unikernels for secure, high-performance network applications across a variety of cloud computing and mobile platforms. Code can be developed on a traditional OS such as Linux or MacOS X, and then compiled into a fully-standalone, specialised unikernel that runs under the Xen or KVM hypervisors as well as lightweight hypervisors like FreeBSD's BHyve, OpenBSD's VMM. Xen and KVM power many public clouds; MirageOS unikernels are currently running on Amazon's Elastic Compute Cloud and Google Compute Engine, and maybe others!

New target: (via solo5) Genode:
"Genode is a free and open-source operating system framework consisting of a microkernel abstraction layer and a collection of userspace components. The framework is notable as one of the few open-source operating systems not derived from a proprietary OS, such as Unix. The characteristic design philosophy is that a small trusted computing base is of primary concern in a security oriented OS." (from wikipedia, more at https://genode.org/ #942, by @ehmry)

User-visible changes

  • use mirage-bootvar-unix instead of OS.Env.argv (deprecated since mirage-{xen,unix,os-shim}.3.1.0, mirage-solo5.0.5.0) on unix (#931, by @hannesm)

    WARNING: this leads to a different semantics for argument passing on Unix: all arguments are concatenated (using a whitespace " " as separator), and split on the whitespace character again (by parse-argv). This is coherent with all other backends, but the whitespace in "–hello=foo bar" needs to be escaped now.

  • mirage now generates upper bounds for hard-coded packages that are used in generated code. When we now break the API, unikernels which are configured with an earlier version won't accept the new release of the dependency. This means API breakage is much smoother for us, apart from that we now track version numbers in the mirage utility. The following rules were applied for upper bounds:

    • if version < 1.0.0 then ~min:"a.b.c" ~max:"a.(b+1).0"
    • if version > 1.0.0 then ~min:"a.b.c" ~max:"(a+1).0.0"
    • exceptions: tcpip (~min:"3.5.0" ~max:"3.6.0"), mirage-block-ramdisk (unconstrained)

    WARNING: Please be careful when release any of the referenced libraries by taking care of appropriate version numbering. (initial version in #855 by @avsm, final #946 by @hannesm) See below for a full list of current constraints.

  • since functoria.2.2.2, the "package" function (used in unikernel configuration) is extended with the labeled argument ~pin that receives a string (e.g. ~pin:"git+https://github.com/mirage-random/mirage-random.git"), and is embedded into the generated opam file as pin-depends
  • mirage-random-stdlib is now used for default_random instead of mirage-random (which since 1.2.0 no longer bundles the stdlib Random module). mirage-random-stdlib is not cryptographically secure, but "a lagged-Fibonacci F(55, 24, +) with a modified addition function to enhance the mixing of bits.", which is now seeded using mirage-entropy. If you configure your unikernel with "mirage configure –prng fortuna" (since mirage 3.0.0), a cryptographically secure PRNG will be used (read more at https://mirage.io/blog/mirage-entropy)
  • mirage now revived its command-line "–no-depext", which removes the call to "opam depext" in the depend and depends target of the generated Makefile (#948, by @hannesm)
  • make depend no longer uses opam pin for opam install –deps-only (#948, by @hannesm)
  • remove unused io_page configuration (initial discussion in #855, #940, by @hannesm)
  • charrua-client requires a Mirage_random interface since 0.11.0 (#938, by @hannesm)
  • split implementations into separate modules (#933, by @emillon)
  • improved opam2 support (declare ocaml as dependency #926)
  • switch build system to dune (#927, by @emillon)
  • block device writes has been fixed in mirage-solo5.0.5.0 (see https://github.com/mirage/mirage-solo5/issues/37)

The following package versions are constrained (generated by "git grep package\ " in the mirage repository):

lib/mirage.ml:          package ~build:true ~min:"4.04.2" "ocaml";
lib/mirage.ml:          package "lwt";
lib/mirage.ml:          package ~min ~max "mirage-types-lwt";
lib/mirage.ml:          package ~min ~max "mirage-types";
lib/mirage.ml:          package ~min ~max "mirage-runtime" ;
lib/mirage.ml:          package ~build:true "ocamlfind" ;
lib/mirage.ml:          package ~build:true "ocamlbuild" ;
lib/mirage.ml:          package ~min:"3.1.0" ~max:"3.2.0" "mirage-unix"
lib/mirage.ml:          package ~min:"3.1.0" ~max:"3.2.0" "mirage-xen"
lib/mirage.ml:          package ~min:"0.4.0" ~max:"0.5.0" ~ocamlfind:[]
"solo5-bindings-*"
lib/mirage.ml:          package ~min:"0.5.0" ~max:"0.6.0" "mirage-solo5"
lib/mirage_impl_argv.ml:      Key.pure [ package ~min:"0.1.0"
~max:"0.2.0" "mirage-bootvar-unix" ]
lib/mirage_impl_argv.ml:      Key.pure [ package ~min:"0.4.0"
~max:"0.5.0" "mirage-bootvar-xen" ]
lib/mirage_impl_arpv4.ml:    Key.pure [ package ~min:"3.5.0"
~max:"3.6.0" ~sublibs:["arpv4"] "tcpip" ]
lib/mirage_impl_arpv4.ml:    Key.pure [ package ~min:"0.2.0"
~max:"0.3.0" ~sublibs:["mirage"] "arp" ]
lib/mirage_impl_block.ml:  [ package ~min:"1.5.0" ~max:"2.0.0"
~sublibs:["front"] "mirage-block-xen" ]
lib/mirage_impl_block.ml:        [ package ~min:"0.4.0" ~max:"0.5.0"
"mirage-block-solo5" ]
lib/mirage_impl_block.ml:        [ package ~min:"2.5.0" ~max:"3.0.0"
"mirage-block-unix" ]
lib/mirage_impl_block.ml:      Key.pure [ package "mirage-block-ramdisk" ]
lib/mirage_impl_block.ml:      Key.pure [ package ~min:"0.9.0"
~max:"0.10.0" "tar-mirage" ]
lib/mirage_impl_conduit_connector.ml:let pkg = package ~min:"3.0.1"
~max:"4.0.0" "mirage-conduit"
lib/mirage_impl_conduit_connector.ml:        package ~min:"0.9.2"
~max:"0.10.0" ~sublibs:["mirage"] "tls" ;
lib/mirage_impl_console.ml:      Key.pure [ package ~min:"2.2.0"
~max:"3.0.0" "mirage-console-unix" ]
lib/mirage_impl_console.ml:      Key.pure [ package ~min:"2.2.0"
~max:"3.0.0" "mirage-console-xen" ]
lib/mirage_impl_console.ml:      Key.pure [ package ~min:"0.3.0"
~max:"0.4.0" "mirage-console-solo5" ]
lib/mirage_impl_ethernet.ml:    Key.pure [ package ~min:"3.5.0"
~max:"3.6.0" ~sublibs:["ethif"] "tcpip" ]
lib/mirage_impl_fs.ml:let fat_pkg = package ~min:"0.12.0" ~max:"0.13.0"
"fat-filesystem"
lib/mirage_impl_fs.ml:      Key.pure [package ~min:"1.0.0" ~max:"2.0.0"
"mirage-fs-lwt"]
lib/mirage_impl_http.ml:      Mirage_key.pure [ package ~min:"1.0.0"
~max:"2.0.0" "cohttp-mirage" ]
lib/mirage_impl_ip.ml:  Key.pure [ package ~min:"0.11.0" ~max:"0.12.0"
"charrua-client-mirage" ]
lib/mirage_impl_ip.ml:      Key.pure [ package ~min:"0.6" ~max:"0.7"
"mirage-qubes-ipv4" ]
lib/mirage_impl_kv_ro.ml:        package ~min:"2.0.0" ~max:"3.0.0"
"io-page";
lib/mirage_impl_kv_ro.ml:        package ~min:"2.0.0" ~max:"3.0.0"
~build:true "crunch"
lib/mirage_impl_kv_ro.ml:      Key.pure [ package ~min:"1.5.0"
~max:"2.0.0" "mirage-fs-unix" ]
lib/mirage_impl_mclock.ml:      [ package ~min:"1.2.0" ~max:"2.0.0"
"mirage-clock-unix" ]
lib/mirage_impl_mclock.ml:      [ package ~min:"1.2.0" ~max:"2.0.0"
"mirage-clock-freestanding" ]
lib/mirage_impl_network.ml:      | `Unix -> [ package ~min:"2.3.0"
~max:"3.0.0" "mirage-net-unix" ]
lib/mirage_impl_network.ml:      | `MacOSX -> [ package ~min:"1.4.0"
~max:"2.0.0" "mirage-net-macosx" ]
lib/mirage_impl_network.ml:      | `Xen -> [ package ~min:"1.7.0"
~max:"2.0.0" "mirage-net-xen"]
lib/mirage_impl_network.ml:        [ package ~min:"1.7.0" ~max:"2.0.0"
"mirage-net-xen" ;
lib/mirage_impl_network.ml:        [ package ~min:"0.4.0" ~max:"0.5.0"
"mirage-net-solo5" ]
lib/mirage_impl_pclock.ml:      [ package ~min:"1.2.0" ~max:"2.0.0"
"mirage-clock-unix" ]
lib/mirage_impl_pclock.ml:      [ package ~min:"1.2.0" ~max:"2.0.0"
"mirage-clock-freestanding" ]
lib/mirage_impl_qubesdb.ml:let pkg = package ~min:"0.4" ~max:"0.7"
"mirage-qubes"
lib/mirage_impl_random.ml:    Mirage_key.pure [ package ~max:"0.1.0"
"mirage-random-stdlib" ]
lib/mirage_impl_random.ml:   the package array. *)
lib/mirage_impl_random.ml:        [ package ~min:"0.5.4" ~max:"0.6.0"
~sublibs:["mirage"] "nocrypto";
lib/mirage_impl_random.ml:          package ~max:"2.0" ~ocamlfind:[]
"zarith-xen" ]
lib/mirage_impl_random.ml:        [ package ~min:"0.5.4" ~max:"0.6.0"
~sublibs:["mirage"] "nocrypto";
lib/mirage_impl_random.ml:          package ~max:"2.0" ~ocamlfind:[]
"zarith-freestanding" ]
lib/mirage_impl_random.ml:        [ package ~min:"0.5.4" ~max:"0.6.0"
~sublibs:["lwt"] "nocrypto" ]
lib/mirage_impl_random.ml:    Mirage_key.pure [ package ~min:"0.5.4"
~max:"0.6.0" "nocrypto" ]
lib/mirage_impl_reporter.ml:      Key.pure [ package ~min:"0.3.0"
~max:"0.4.0" "mirage-logs" ]
lib/mirage_impl_resolver.ml:          package ~min:"1.0.0" ~max:"2.0.0"
"conduit-lwt-unix"; ]
lib/mirage_impl_syslog.ml:  Key.pure [ package ~min:"0.2.0" ~max:"0.3.0"
~sublibs "logs-syslog" ]
lib/mirage_impl_tracing.ml:        [ package ~max:"1.0.0" "mirage-profile";
lib/mirage_impl_tracing.ml:          package ~max:"1.0.0"
"mirage-profile-xen" ]
lib/mirage_impl_tracing.ml:        [ package ~max:"1.0.0" "mirage-profile";
lib/mirage_impl_tracing.ml:          package ~max:"1.0.0"
"mirage-profile-unix" ]

Experimenting with a new opam repository release strategy for large libraries

Anil Madhavapeddy announced

Dear opam repository contributors,

You will all be familiar with the difficulty of contributing a large new set of libraries into modern opam, due to the need to establish upper bounds on third party packages that use your libraries that have made breaking changes to interfaces. This difficulty exists due to how successful we have been at importing in multiple versions of the same package into the opam-repository, and leveraging the opam solver to decide on the freshest versions of packages to install.

In order to make the contribution process a little easier, we are experimenting with a new release strategy for the latest large library set to drop into opam-repository (in this case, the new JS Core release). It is as follows:

  1. add upper bounds to all opam-repository packages that use your library in one PR. This prevents the newer libraries from being selected by the solver for existing packages.
  2. import the new libraries, knowing that they will not be selected by the solver for existing packages.
  3. relax the upper bounds on third party packages that have been tested to work with the new release.

This three-step process effectively allows the new versions of a library set to be staged in the opam repository without causing a massive interlock with third party packages. It also (via step 3) gives maintainers time to get used to a new library set without users being affected.

The main downside to this approach is that it may break development pins, since that metadata exists separately in each upstream repository and needs to be updated with upper bounds (or preferably, the code fixed to support the newer library).

The more minor downside is that it involves the maintainer of a library you depend on adding version constraints to "your" packages. Since the only change here is to add an upper bound to an existing dependency, we judge this to be acceptable to the long-term health of the opam-repository database. However, maintainers do have ample chance to comment on the step 1) repository if you really do not want anyone else making changes to your packages. In this case however, we expect the maintainers who have objections to step up and do some testing with the newer releases and/or propose any changes in good time.

I'd like to emphasise that this is an experiment at the moment and not firm opam-repository policy, but one that we feel is worth trying. If it does work, then we can improve the tooling around the 3 steps to make it as easy as possible for maintainers. In the meanwhile, you can see the first step of it in action in ocaml/opam-repository#13010 which adds upper bounds in preparation for the new version of Core.

Other OCaml News

From the ocamlcore planet blog

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.