OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of November 20 to 27, 2018.

Table of Contents

Published a post about what we are doing and going to do about BuckleScript development

Hongbo Zhang announced

In case you are interested, here is the link: https://bucklescript.github.io/blog/2018/11/19/next-half

Ocamlformat and the ocaml.org coding guidelines

Anil Madhavapeddy announced

I just wanted to draw the community's attention to [ocamlformat#501], which gathers together many of the coding guidelines for OCaml, with the goal of ensuring that ocamlformat can automate the formatting process.

As noted on the issue, the current coding guidelines are designed for humans, but we are increasingly getting tools such as ocamlformat and ocp-indent which mechanise the process. This is a good chance to get involved at this point in ocamlformat's development with this, especially if you have a large codebase which could benefit from standardised formatting in the near future. Please weigh in at ocamlformat#501, especially with pointers to counterexamples to your own codebases if applicable.

Release of OCaml-R

Philippe Veber announced

I'd like to announce a preview release of ocaml-r, a library that can be used to interact with the R language interpreter and write clean wrappers to R functions and packages. This library was developped a long time ago by Maxence Guesdon and Guillaume Yziquel, and I've maintained it since then for my own needs. The library is available on opam, under version 0.1.1, and on github [0].

The goal of this release is to see if other people could find it useful, in which case I can spend a little more time writing documentation on how to use it, and help write more wrappers. Please get in touch via the issue tracker if interested.

Cheers,
Philippe.

[0] https://github.com/pveber/ocaml-r

Resources for Advanced Module Usage

robkuz asked

does anybody have links/tutorials etc. for usage of modules? I have read the lightweight higher kinded types paper and I am searching for other resources in that direction.

kantian replied

You can have a look on the paper ML module mania which present a type safe, separetly compiled and extensible interpreter for the Lua language. In the same spirit, you have the work of Oleg Kiselyov on tagless-final embedded DSL. Or from the same author than the paper on ligthweight higher-kinded types, you have their proposal for modular implicit which go further.

About the last paper, imagine that you want to write a function show that works on mutliple data types but not uniformly. In a lot of languages, this is done with dynamic dispatching. Since OCaml doesn't have runtime type representation, we have to rely on something which is closed to a static dispatching, and we can do this with first-class modules.

module type Showable = sig
  type t
  val show : t -> string
end

let show (type a) (module M : Showable with type t = a) x = M.show x

Now you can define a module for each type that you want to convert to a `string`, and you have to explicitly pass it to your fonction `show`.

module Int_show = struct
  type t = int
  let show = string_of_int
end

module Float_show = struct
  type t = float
  let show = string_of_float
end

# show (module Int_show) 3;;
- : string = "3"

# show (module Float_show) 3.4;;
- : string = "3.4"

The idea behind modular implicit is that in this situation, the module to use to show your value is automatically infered by the compiler in such a way that you don't have to explicitly pass it to your function. This will be very useful, especially when the module in question is the result of a functor application. Indeed, imagine that you know how to show values of two types a and b, then you know how to show a pair of type a * b. To encode this you have to write a functor:

module Pair_show (A : Showable) (B : Showable):
Showable with type t = A.t * B.t = struct
  type t = A.t * B.t
  let show (x,y) = Printf.sprintf "(%s, %s)" (A.show x) (B.show y)
end

# show (module Pair_show (Int_show) (Float_show)) (1, 2.5);;
- : string = "(1, 2.5)"

Michael Thomas asked and Daniel Bünzli replied

> This looks really interesting - do you know if the complete source code is available anywhere?

It's here.

Ivan Gotovchits also replied

The [Modules][1] section is a prerequisite must read. All advance papers more or less are based on these papers. The [F-ing modules][2] paper is a very interesting read, albeit being a little bit fictional. I also found [this thesis][3] to be an interesting compilation, although it is focusing more on SML.

[1]: https://ocaml.org/docs/papers.html#Modules
[2]: https://people.mpi-sws.org/~rossberg/f-ing/f-ing-jfp.pdf
[3]: https://people.mpi-sws.org/~dreyer/thesis/old/thesis050405.pdf

Digestif 0.7.1

Charles Edouard Lecat announced

I’m happy to announce a new release of digestif 0.7.1, available for installation via OPAM.

Digestif is a library which contains some hashes function like:

  • MD5
  • SHA1
  • SHA224
  • SHA256
  • SHA384
  • SHA512
  • BLAKE2B
  • BLAKE2S
  • RIPEMD160

And the recently added

  • WHIRLPOOL

Particularities of digestif:

Linking trick: This project provides 2 implementations of the hash algorithms: one written in C (digestif.c) and the other in OCaml (digestif.ocaml), each one fulfilling the same interface. In order to choose your implementation, you can simply use the linking trick, by selecting at link time the implementation you wish to use. The C implementation will provide a smaller execution time, while the OCaml one will bring more portability to your project (ex: to JavaScript with js_of_ocaml).

Constant time: Some applications require that secret values are compared in constant time. This Functions like String.equal do not have this property, so we provide a small package — eqaf — providing a constant-time equal function. Digestif uses it to check equality of hashes — it also exposes unsafe_compare if you don’t care about timing attacks in your application.

Run-time lock: When calling C code from the OCaml environment, the garbage collector states needs to be saved as C code may need allocated values, which the GC could move or change. This is why we implemented the hash algorithms with bigarrays, which are contained in a specific area of the GC, and never will be moved. This way, we were able to remove the run-time lock when calling the others C functions, improving the speed of the computations. A behavior even more useful as the GC is global in a multi-threaded environment, locking it pauses every threads: With digestif you can compute the hash in parallel of others operations without affecting your performances.

Xavier Leroy (and others) - On Computer Science

Jp R announced

For those of you who understand French, I invite you to follow a serie of courses on computer science at the College de France.

Xavier Leroy a mis son beau costume, sa chemise blanche et sa cravate pour nous parler des sciences du logiciel.

A must see!

https://www.college-de-france.fr/site/xavier-leroy/inaugural-lecture-2018-11-15-18h00.htm

md2mld 0.3.0

Marcello Seri announced

I am happy to announce the first release of md2mld, available now from OPAM.

md2mld converts a Markdown-format file into the mld format used by odoc to render HTML documentation or OCaml libraries. You can use this script to automatically embed a README.md file into API documentation for an OCaml library.

I made it because multiple repositories have their most interesting short examples in their README, and I wanted an easy way to integrate them in the generated documentation without duplicating too much the efforts.

You can see an example of use in the documentation of the tool itself and in the new documentation of ocaml-rpc.

Wanted: new maintainer for cppo

Martin Jambon announced

Cppo is a C-style preprocessor for OCaml. I started the project in 2009, when I was working on yojson and didn't want to be tied to camlp4 for simple things like #include and conditional compilation. Since then, other projects have used it, typically as a last resort to manage version incompatibilities. Today, 57 packages in opam depend directly on cppo.

The project was recently handed off to ocaml-community in an effort to benefit from a wider community. I no longer wish to be in charge of supervising the project because it's too much for me, when combined with other projects. Ideally, we're looking for a person maintaining cppo with the support of their employer. The project is very stable. There are no new features in sight and very few bug reports. Maintenance would consist in roughly one day a month, generally spent answering incoming questions and working with users to solve their problems or review patches.

Let us know if you're interested!

Build-/Installation-Tools - not enogh of them?

Anil Madhavapeddy announced

Julia Lawall said:

Personally, I was in the end forced to install opam. But I don't want to force my users to install it (to figure out how to get it to work, I had to contact a member of Gallium - and my users don't have that opportunity). I would prefer that they can just use the ocaml that comes with their system package manager.

Since opam is the recommended method of installation, couldn't there at least be provided understandable instructions? That really seems like a tougher time than what is necessary.

Dear Julia, dear all,

Thanks for the helpful reminder that the opam and ocaml documentation needs a refresh. We’ve just gone through a fairly major release cycle with opam2 and dune1, and it’s to be expected that there are some inconsistencies.

To clear up one misconception on this thread — opam is not the only recommended mechanism to get the OCaml compiler. We go to some trouble to ensure that OCaml is packaged up natively on the OS package managers where possible, and there is a list on the ocaml.org installation page with popular distributions.

The problems begin when the user needs a particular version of OCaml, which is a common requirement for teaching. The system package managers typically lag in their versions (again, listed on the ocaml.org installation page). At this point, opam is indeed the simplest tool to bootstrap the precise version of the compiler and packages that you require, but at the expense of requiring an OCaml-specific tool.

Now, I recognise the need for improving the installation instructions, and particularly so with step-by-step guides to getting things installed. However, I would implore those with complaints to register the inaccuracies on the opam issue tracker [1], so that we can at least learn about them and fix them just as we do with any software bug.

[1] https://github.com/ocaml/opam/issues

It’s particularly frustrating to see these threads erupt with pent up (and valid) annoyances from users that we could have addressed if someone had reported it. Even better, please do send in PRs with some suggested text for the website, and take some load off our overworked maintainers :-)

For whoever said that my Ubuntu opam2 PPA is hard to Google, the process of creating it was roughly:

  • spend a few hours fighting with the Debian packaging scripts
  • get x86_64/i386/arm32/arm64/ppc64le all working in Launchpad
  • post on https://discuss.ocaml.org/t/opam-2-0-experimental-ppas/2446 to get no feedback whatsoever
  • eventually just push it live after one user gets back with feedback
  • wait for complaints to show up that its hard to find

Without feedback, it’s very difficult to know what to prioritise, and Ubuntu is just one of many, many Linux distributions that we have to worry about.

Japp Boender also wrote:

I'd like to chime in here - as the person responsible for doing most of the OCaml packaging for pkgsrc, I've noticed that I can hardly ever just update a package to a new version anymore - I'll generally also need to update the infrastructure as some sort of new build tool will have become involved, with all its attendant quirks, or someone will have decided to change the name of packages, or some other change will have occurred. Things seem to be in a constant state of flux, and thus packaging becomes quite time-consuming.

This is indeed the result of progress on the tooling side. There’s been a lot of work in the last few years on deprecating older tools (camlp4, ocamlbuild) and moving towards modern alternatives such as dune for build and opam2 for publishing.

Things are settling down now: dune is emerging as a well-maintained and portable alternative to the myriad of build systems that have been proposed in the past. The bootstrapping problems you mention should also have become a lot easier with opam2, since the solver is linked into the binary and no external aspcud is needed. If you do still run into any problems with fakeroots and such, please report them on the dune or opam issue tracker and we’ll take a look.

Behind the scenes, there are other benefits from a consistent build system: cross compilation and Windows support are greatly improved now.

Malcolm Matalka also wrote:

Is there a reasonable workflow for how to turn opam packages into packages for existing OS's though? Currently it does seem like users need to know about Ocaml to use things written in Ocaml, if only because most of the focus has been on opam. Other OSs tend to have old packages.

Our goal (with my dune and opam developer hat on) is to provide sufficient metadata within a dune description of a project to automatically generate opam and upstreamable OS packages (from templates). This will look something like “dune @package” and really reduce the burden on developers and OS maintainers. I’d like, for example, the native OpenBSD packages to be able to be replaced by a mechanically generated version.

See this issue for more: https://github.com/ocaml/dune/issues/1498

Julia Lawall also wrote:

The question is what to do after installing opam. Even if there is nothing to do after installing opam, it would be helpful to say that. If there is something to do after installing opam to get the ocaml compiler for example, as opposed to just the runtime system, it would be helpful to say that too. The documentation should be designed from the point of view of the person who has never used ocaml or opam, never wants to use them again, and has no contact with the ocaml community. Otherwise, people who want to try some ocaml software, but are not actually forced to use it, will just give up.

I completely agree with this sentiment. I’m personally motivated to make sure ocaml/opam meet these standards, as we are moving to teaching using OCaml in the Cambridge University Computer Lab from next year, so I’ll definitely be proposing some changes myself :-)

I’d love to hear from other teachers about the sort of environments you have to work in — we’ve been considering shifting to a combination of browser-based IDEs (for homogeneity across student laptops) and other systems.

Finally, this thread might seem like it is full of complaints, but the tooling is steadily improving and leaving behind silent and satisfied users. Dune in particular has made as big a difference to our community as opam did when it was first released five years ago. At ICFP this year, the OCaml Workshop was full of industrial users who were thankful for Dune improving their day-to-day development, as well as the ecosystem of new tools such as ocamlformat, merlin, odoc and so on, and the Coq project is moving to adopt it now as well! I’m not suggesting we rest on our laurels, but as the year end holiday approaches, I would be delighted if users to email their favourite tool developer a note of encouragement to continue to work on it. And then file an issue to get the docs fixed :-)

Ocaml Github Pull Requests

Gabriel Scherer and the editor compiled this list

Here is a sneak peek at some potential future features of the Ocaml compiler, discussed by their implementers in these Github Pull Requests.

Other OCaml News

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.