OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of April 21 to 28, 2020.

Table of Contents

opam 2.0.7 and 2.1.0 alpha

R. Boujbel announced

We are pleased to announce the release of opam 2.0.7 and an 2.1.0 alpha.

This 2.0.7 version contains backported fixes, you can find more information in this blog post.

The 2.1.0~alpha contains many new features (cf. blog post or release note). If you want to take a look, a few glitches or regressions are expected, please report them to the bug-tracker.

opam is a source-based package manager for OCaml. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.

Anil Madhavapeddy then added

Thanks for all the hard work that's gone into this release, @rjbou @AltGr and @dra27!

To set expectations, this alpha release is for our users to smoke test the new features and let us know if they work for your usecases.

In particular, the opam external dependency management and support for recursive pins are both commonly requested features. Please do take this alpha for a spin, and report feedback here on this thread.

After this alpha is cut, there will be a sequence of beta releases (the number of which depend on the reported bug tail), and then the final opam 2.1.0 release. Your testing now will greatly help us put a quality release out of the door.

OCaml 4.11, release plan

octachron announced

The new version of OCaml, OCaml 4.11.0, has started its bugfix period: the set of new features is now mostly frozen, and in the three upcoming months, we will focus mostly on fixing bugs.

For this release cycle, we will experiment releasing an alpha version of the compiler.

This new alpha version is expected to work as a synchronization point for people working on updating the opam ecosystem for the new release. Once the opam ecosystem is in shape for some wider audience testings, we will publish a beta version as usual. This should be happen around June.

One of the most notable change in this release is Statmemprof, a new statistical memory profiler directly integrated into the GC.

The provisional Changes list is here.

At this point of time, it is better to take this list with a grain of salt: there are a handful of new features that are still under integration, problematic features might be removed, and of course the list of bug fixes is incomplete.

But one of the most notable feature in this change log, Statmemprof which a new statistical memory profiler API, is most probably here to stay.

Guillaume Munch-Maccagnoni then added

It should be mentioned that Memprof is documented as “~EXPERIMENTAL~”, and at least one breaking change is being considered in 4.12. This also mean that suggestion for improvement will be welcome (AFAIU).

ocamlformat pre-commit hook

Brendan Long announced

This is kind of trivial but I figured it might be useful for other people. We created a hook config for using ocamlformat with pre-commit:

https://github.com/arenadotio/pre-commit-ocamlformat

pre-commit is a tool that makes it easier to run checks on changed files before commiting them, and this makes it so you can auto-run ocamlformat and ensure no unformatted code gets into your repo.

  1. Install pre-commit like pip install pre-commit
  2. In your repo, add a .pre-commit-config.yaml like:

    ---
    repos:
      - repo: https://github.com/arenadotio/pre-commit-ocamlformat
        rev: master # or pick a commit sha I guess
        hooks:
         - id: ocamlformat
    
  1. Run pre-commit install
  2. Now every time you run git commit (or pre-commit run), it will run every staged OCaml file through ocamlformat and complain if there are any changes:

    $ pre-commit run ocamlformat
    ocamlformat.....Failed
    - hook id: ocamlformat
    - files were modified by this hook
    $ git add .
    $ pre-commit run ocamlformat
    ocamlformat.....Passed
    

New release of naboris 0.1.2

Shawn McGinty announced

Simple http server for OCaml/ReasonML.

naboris has been updated to 0.1.2

This release comes with a few improvements to the API but most notably it has much better documentation at naboris.dev

ANN: Releases of ringo

Raphaël Proust announced

On behalf of Nomadic Labs, I am please to announce the first few releases of Ringo: a library for caches. Ringo offers two kinds of caches: Maps for caches of key-value pairs and Sets for caches of simple elements. In addition, each kind of cache can be tweaked to handle their bounds differently.

Ringo versions 0.1, 0.2 and 0.3 are available on opam. As the version number and the bundled announce suggests, this library is still in early phases of release: additional replacement policies will be added, the interface will probably change somewhat, etc. Suggestions welcome!

Even though the interface is still in early phases of release, the implementation is covered by a lot of tests and is already in use in the Tezos project.

The code is available at https://gitlab.com/nomadic-labs/ringo

resto 0.2 released

Raphaël Proust announced

Release of resto 0.5

On behalf of Nomadic Labs, I'm happy to announce the release of version 0.5 of resto.

The main change brought in this release are:

  • relaxing of dependency bounds,
  • documentation!

Retrofitting Parallelism onto OCaml (research paper)

Guillaume Munch-Maccagnoni announced

The following paper on the multicore GC design by @kayceesrk and his coauthors has been posted on arXiv today and might interest the community: https://arxiv.org/abs/2004.11663

Multicore Update: April 2020, with a preprint paper

Anil Madhavapeddy announced

Welcome to the April 2020 update from the Multicore OCaml team, across the UK, India, France and Switzerland! Although most of us are in lockdown, we continue to march forward. As with previous updates, thanks to @shakthimaan and @kayceesrk for help assembling it all.

  • Preprint: Retrofitting Parallelism onto OCaml

    We've put up a preprint of a paper titled "Retrofitting Parallelism onto OCaml" for which we would be grateful to receive feedback. The paper lays out the problem space for the multicore extension of OCaml and presents the design choices, implementation and evaluation of the concurrent garbage collector (GC).

    Note that this is not a final paper as it is currently under peer review, so any feedback given now can still be incorporated. Please use the e-mail contact details in the pdf paper for @kayceesrk and myself so we can aggregate (and acknowledge!) any such comments.

  • Rebasing Progress

    The Multicore OCaml rebase from 4.06.1 has gained momentum. We have successfully rebased the parallel-minor-GC all the way onto the 4.09 OCaml trees. We will publish updated opam packages when we get to the recently branched 4.11 in the next couple of weeks.

    Rebasing complex features like this is a "slow and steady" process due to the number of intermediate conflicts and bootstrapping, so we will not be publishing opam packages for every intermediate version – instead, the 4.11 trees will form the new "stable base" for any PRs.

  • Higher-level Domainslib API

    A thread from last month's update on building a parallel raytracer led to some useful advancements in the domainslib library to provide async/await-style task support. See the updates below for more details.

    There is also an interesting discussion on ocaml-multicore/ocaml-multicore#324 about how to go about profiling and optimising your own small programs. More experiments with parallel algorithms with different scheduling properties would be most useful at this time.

  • Upstreamed features in 4.11

    The 4.11 release has recently branched and has the following multicore-relevant changes in it:

    • A concurrency-safe marshalling implementation (originally in ocaml#9293, then implemented again in ocaml#9353). This will have a slight speed hit to marshalling-heavy programs, so feedback on trying this in your projects with 4.11 will be appreciated to the upstream OCaml issue tracker.
    • A runtime eventlog tracing system using the CTF format is on the verge of being merged in 4.11 over in ocaml#9082. This will also be of interest to those who need sequential program profiling, and is a generalisation of the infrastructure that was essential to our development of the multicore GC. If anyone is interested in helping with hacking on the OCaml side of CTF support to build clients, please get in touch with me or @kayceesrk.

    In addition to the above highlights, we have also been making continuous improvements and additions to the Sandmark benchmarking test infrastructure. The various ongoing and completed tasks are provided below for your reference.

Multicore OCaml

  • Ongoing
    • ocaml-multicore/ocaml-multicore Promote Multicore OCaml to trunk

      The rebasing of Multicore OCaml from 4.06 to 4.10 is being worked, and we are now at 4.09! In a few weeks, we expect to complete the rebase to the latest trunk release.

    • ocaml-multicore/eventlog-tools: OCaml Eventlog Tools

      A project that provides a set of tools for runtime tracing for OCaml 4.11.0 and higher has been created. This includes a simple OCaml decoder for eventlog's trace and a built-in chrome converter tool.

    • ocaml-multicore/domainslib#5 Add parallel_scan to domainslib

      A parallel_scan implementation that uses the Task API with prefix_sum and summed_area_table has now been added to the Domain-level Parallel Programming library for Multicore OCaml (domainslib) library.

  • Completed

    The following PRs have been merged into Multicore OCaml and its ecosystem projects:

    • ocaml-multicore/ocaml-multicore#328 Multicore compiler with Flambda

      Support for Flambda has been merged into the Multicore OCaml project repository. The translation is now performed at cmmgen instead of lambda for clambda conversion.

    • ocaml-multicore/ocaml-multicore#324 Optimizing a Multicore program

      The following documentation provides a detailed example on how to do performance debugging for a Multicore program to improve the runtime performance.

    • ocaml-multicore/ocaml-multicore#325 Added eventlog_to_latencies.py script

      A script to generate a latency report from an eventlog has now been included in the ocaml-multicore repository.

    • ocaml-multicore/domainslib#4 Add support for task_pools

      The domainslib library now has support for work-stealing task pools with async/await parallelism. You are encouraged to try the examples.

Benchmarking

A number of new benchmarks are being ported to the Sandmark performance benchmarking test suite.

  • ocaml-bench/sandmark#104 Added python pip3 dependency

    A check_dependency function has now been defined in the Makefile along with a list of dependencies and pip packages for Ubuntu. You can now run make depend prior to building the benchmark suite to ensure that you have the required software. The python3-pip package has been added to the list of dependencies.

  • ocaml-bench/sandmark#96 Sandmark Analyze notebooks

    The setup, builds and execution scripts for developer branches on bench2.ocamllabs.io have been migrated to winter.ocamllabs.io.

    A UI and automated script driven notebooks for analyzing sequential bench results is being worked upon.

  • ocaml-bench/sandmark#108 Porting mergesort and matrix multiplication using Task Pool API library

    This is an on-going PR to implement merge sort and matrix_multiplication using parallel_for.

  • cubicle

    Cubicle is a model checker and an automatic SMT theorem prover. At present, it is being ported to Multicore OCaml, and this is a work in progress.

  • raytracers

    Raytracers is a repository that contains ray tracer implementation for different parallel functional programming languages. The OCaml implementation has now been updated to use the new Domainslib.Task API.

    Also, a few experiments were performed on flambda parameters for the Multicore raytracer which gives around 25% speedup, but it does not yet remove the boxing of floats. The experiments are to be repeated with a merge against the wip flambda2 trees on 4.11, that removes float boxing.

OCaml

  • Ongoing
    • ocaml/ocaml#9082 Eventlog tracing system

      A substantial number of commits have gone into this PR based on reviews and feedback. These include updates to the configure script, handling warnings and exceptions, adding build support for Windows, removing unused code and coding style changes. This patch will be cherry-picked for the 4.11 release.

  • Completed
    • ocaml/ocaml#9353 Reimplement output_value using a hash table to detect sharing

      This PR which implements a hash table and bit vector as required for Multicore OCaml has been merged to 4.11.

    Our thanks as always go to all the OCaml developers and users in the community for their continued support, and contribution to the project!

Acronyms

  • API: Application Programming Interface
  • GC: Garbage Collector
  • PIP: Pip Installs Python
  • PR: Pull Request
  • SMT: Satisfiability Modulo Theories
  • UI: User Interface

Why did Core remove polymorphic comparison operators in OCaml 4.10.0?

Trung Ta asked

I'm using the Core library in a project, and recently when I upgraded my OCaml from 4.08.1 to 4.10.0, plenty of compilation errors suddenly appears for comparison expressions like:

if (xs = []) then ... or if (x = true) then ...

I saw that this change was discussed in this thread about monomorphic comparison operators in Base, but did not expect that Core would make it a default behavior.

So I'd like to ask since which version that Core removed such polymorphic comparison operators? (I couldn't find it in release notes of Core)

Also, if I defined a sum type like type ternary = True | False | Unkn, what will be a correct way to write if (x = True) then ... (which is allowed in the new Core)?

I can temporarily fix by writing if (x == True) then ..., but using == doesn't seem correct, since == is about comparing physical objects…

Thanks for spending your time to check my question.

Aaron L. Zeng replied

The change was announced in https://discuss.ocaml.org/t/ann-v0-13-release-of-jane-street-packages/4735, although unfortunately it doesn't look like the CHANGES.md file was updated in the repo. I would consider the thread to be the canonical announcement.

Also, if I defined a sum type like type ternary = True | False | Unkn , what will be a correct way to write if (x = True) then ... (which is allowed in the new Core)?

Here's a few suggestions:

  1. Define equality/compare functions using ppx_compare

    type ternary = True | False | Unkn [@@deriving equal]
    
    let f x = if (equal_ternary x True) then ...
    
  2. Define equality/compare functions manually

    let equal_ternary t1 t2 =
      match t1, t2 with
      | True, True | False, False | Unkn, Unkn -> true
      | _ -> false
    
  3. Explicitly request polymorphic comparison operators using the Poly module:

    let f x = if (Poly.(=) x True) then ...
    

Trung said and Aaron L. Zeng replied

btw,

type ternary = True | False | Unkn [@@deriving equal]

should be: [@@deriving eq]

That depends on which preprocessor you are using. [@@deriving equal] comes from ppx_compare, whereas [@@deriving eq] comes from ppx_deriving. Base/Core and the like have better support for the former, which is a Jane Street project, although you can feel free to use the latter—the naming conventions are different, so it may not be as convenient.

New release of js_of_ocaml 3.6.0

Hhugo announced

I'm pleased to announce the release Js_of_ocaml 3.6.0.

Js_of_ocaml is a compiler from OCaml bytecode to JavaScript. It makes it possible to run pure OCaml programs in JavaScript environment like browsers and Node.js.

Try it online.

Notable changes:

  • The js_of_ocaml compiler now accepts sub-commands (link, build-runtime, build-fs, ..). The plan for future versions is to remove other binary (e.g. jsoo_link) and consolidate everything inside the js_of_ocaml binary itself.
  • The standard JavaScript runtime is now embedded in the compiler (findlib is no longer needed to locate it)
  • Add support for the Str library (Regular expressions and high-level string processing) shipped with the OCaml compiler
  • Change memory representation of Int64.t (you might need to update your JavaScript stubs)
  • Many bug fixes (thanks to many more tests)

Kirill Alexander Khalitov asked and Hhugo replied

1 Does the project have roadmap?

There is no official roadmap, the project evolves based on issues, requests and contributions. You can take a look at some of the Github issues

2 Is the project generally exists only for Ocsigen needs?

js_of_ocaml is used by various projects, not only Ocsigen. See Bonsai, sketch-sh or jscoq for instance.

3 Will it be adopted for modern front-end development (commonjs/esmodules compatibility for working with existing building tools ex. webpack, etc).

Being more friendly with the JavaScript ecosystem as been discussed here and there in the past but little has been done, possibly by lack of interest or use cases.

4 Does the project competing with bucklescript?

I don't think so. The two projects have different goals and different audience. One of Js_of_ocaml main goal is to stay as close as possible to the official OCaml semantic, allowing to leverage existing OCaml libraries without any modification.

5 Why not to do ocaml to js compiler tools (based on js_of_ocaml and bucklescript experience) that combine possibility of using native ocaml and js libraries across back-end and front-end like implemented in Scala.js/Fable F#?

I don't understand this question. I would expect both js_of_ocaml and bucklescript to be like Scala.js/Fable F# in their own way.

Kirill Alexander Khalitov then said

I mean what Scala.js/Fable F# allows to use the most native libraries (not all) and JS ones (from npm registry or from custom JS module) in one project (ex. front-end). But in case of js_of_ocaml we limited to native OCaml libs and "HTML scripts" (not JS compatible modules). For bucklescript case we have whole JS ecosystem but have no access to useful native libs from opam registry.

Xavier Van de Woestyne replied

In Js_of_OCaml, you can deal with JavaScript's module (and npm/yarn), using for example:

(* val require : string -> 'a *)
let require module_name =
  let open Js.Unsafe in
  fun_call
    (js_expr "require")
    [|inject (Js.string module_name)|]

Other OCaml News

From the ocamlcore planet blog

Here are links from many OCaml blogs aggregated at OCaml Planet.

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.