OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of March 31 to April 07, 2020.

Table of Contents

Making a music player in OCaml

Dracose asked

I'm interested in making my own music player in OCaml so I wanted to know whether there were any existing ones and/or examples of how to make one. Bear in mind, I am interested in the actual logic of how to read a music file (or a playlist) and listening to it, rather than the front-end part of a music player. (My knowledge of OCaml is intermediate)

Thomas Blanc suggested

Yotam Barnoy then said

Wow @PatJ I didn't know about liquidsoap. I added it to ocamlverse. This is what we have for the audio page now, in case it's helpful to the OP: https://ocamlverse.github.io/content/audio.html

gndl also replied

I experimented with several solutions in the playo project. One of the possible solutions is to use ocaml-gstreamer. If you find that the gstreamer framework is too annoying (which I can understand :-), you can use ocaml-ffmpeg. note however that, in the latest version of ocaml-ffmpeg, the audio device output no longer works. To overcome this drawback, you can use ocaml-portaudio.

The end of Camlp4

Continuing this old thread, Chet Murthy announced

Perhaps worth mentioning briefly that for anybody who -wants- to continue using camlp4, I'm (a) maintaining camlp5 and bringing it up-to-date with everything in ocaml 4.10.0 that I can think of, and (b) I'd be happy to help them port their dependency over to camlp5.

This is not to be construed as an argument for using camlp4/5.

OCamlformat 0.14.0

Etienne Millon announced

On behalf of the development team, I'd like to announce the release of ocamlformat version 0.14.0 :tada:.

Here are the main highlights of this release:

Support for OCaml 4.10

This means both that it compiles and runs using this version, but also that it can format 4.10-specific language features (module _ and multi-indices operators).

Preliminary support for invalid files

As OCamlformat operates on ASTs, it normally requires a valid input file. This release adds a --format-invalid-files option to detect invalid parts and print them verbatim. This feature is still experimental.

Preserving more concrete syntax

Starting with this release, OCamlformat is going to preserve more concrete syntax. For example, module M = functor (K : S) -> struct end and module M (K : S) = struct end are equivalent. In the past, both variants would be formatted as the latter. Now, the original syntax is preserved. In some cases, preserving was possible through the means of an option: for example, to choice between let%name x = e in body and [%name let x = e in body], was controlled by the extension-sugar option. This option is now deprecated and OCamlformat will now always preserve what was in the source file (this was the default behaviour).

Similarly, it was possible to control how special characters are escaped in string and character literals through the escape-strings and escape-chars options. They are being deprecated and the only possible behavior will be preserving the concrete syntax (as done by default).

The reason for this change is that we feel that ocamlformat should be just about formatting. The fact that this behavior was configurable is in part due to the fact that it operates on OCaml ASTs, but end users should not have to be surprised by their code being transformed on reformatting.

In the future, we plan to extend that to other similar constructs, such as using (~/)~ or begin~/~end, or spacing between module items.

Placement of doc comments

Placing doc comments (** ... *) is controlled by the doc-comments configuration option. It is always possible to put them before the item they refer to, and this is what the doc-comments=before option does. The alternative doc-comments=after will try to do its best to put them after, but in some cases it is not possible. For example, in a variant type declaration, a doc-comment put immediately after will be attached to the last constructor by documentation tools. Ocamlformat needs to preserve the meaning of programs, so in these cases, it will instead put the comment before. In the case of module declarations, putting the comment after might not be very useful if the corresponding module is very large.

This requires a complex rule to determine which comments will be put before and which comments will be put after. So in this version, we are deprecating this mechanism and replacing it with a simpler one controlled by doc-comments-val that applies only to val and external items. For these items, it is always possible to attach documents before or after them. For all other items, like type or module declarations, the doc comments will consistenly be put before.

Many bugs found by fuzzing

We hooked ocamlformat to AFL, looking for programs that parse correctly but trigger errors during formatting. This approach worked very well and more than 20 logical bugs were found with this technique.

Upgrading

To upgrade from ocamlformat 0.13.0, one needs to upgrade the ocamlformat binary and replace the version field in .ocamlformat files by 0.14.0 and then:

  • if you used doc-comments=after, you can replace it by doc-comments-val=after. This will move doc-comments on module items except val and external ones.
  • if you used doc-comments=before, you can remove it as it is now the default.
  • if you set escape-chars=preserve, escape-strings=preserve, or extension-sugar=preserve explicitly, you can

remove them safely (they were the default)

  • if you used another value for one of these options (such as escape-strings=hexadecimal), you will need to remove them as well. This will not trigger a diff, but ocamlformat will not enforce a particular concrete syntax for new code.

A note for new users

We encourage you to try ocamlformat, that can be installed from opam directly (opam install ocamlformat), but please remember that it is still beta software. We added a FAQ for new users that should help you decide if ocamlformat is the right choice for you.

Etienne Millon later added

This upgrade is likely to generate a huge diff on projects that use the default profile, so I would like to expand a bit on the reason.

According to the syntax rules used by the ocaml tools (the ocaml compilers, ocamldoc, odoc), it is always possible to put the doc-comment before an item.

Some teams prefer to put the documentation after. But that is not always possible. For example, type t = A | B (** doc *) will attach the doc-comment to B, not to t. The only way to attach the comment to t is by putting the comment before.

Enter ocamlformat: doc-comment placement is controlled by an option with two values, before or after. before will always place the comment before. after determines if it is possible to put the comment after, and if it is not, will put it before.

Some items cannot have comments after, like variant types (as described above). But there is another reason not to put comments after. In some cases, that can put the comment far from the thing it is documenting. Considering modules, the following is nice:

module M = L.M
(** doc *)

But this is not great is the structure is large:

module M = struct
  ...
  ...
end
(** doc *)

To summarize, when ocamlformat is configured to put comments after, it has to follow a complex heuristic to determine whether it has to fallback to before. In the case of a module, it depends on its shape, how many functor arguments are there, this kind of things (for various reasons, we don't know how large something is going to be in advance, so we have to look at its shape). The point is that it is complicated to understand and explain, and that fixing it always makes it more complex. Another aspect is that in the end, we want ocamlformat to be pretty stable when it reaches 1.0.0, and complex rules are at odds with this goal.

So, we have decided to simplify the rule: instead of looking deep in the AST, we just look at the kind of item this is. For val and external items, it is always possible to put the doc-comment after, so we follow exactly what the configuration option says.

As a user of the default profile, what this means for you: for items that are not val or external, and considered "simple" by the 0.13.0 heuristic, doc-comments are going to move from after to before.

Based on these reasons, you will understand that before is always simpler. You can opt into this by setting doc-comments-val=before. This will cause an even larger diff as all items are going to move before (that is: all items described just above, plus val and external items), but the rule gets extremely simple (everything is put before). It is possible that this option will become the default in the future, but we have not decided this yet (in this case, if you did not opt into it, you will see comments on val and external items move at that time).

ML Family Workshop 2020: Call for presentations

Leo White announced

We are happy to invite submissions to the ML Family Workshop 2020, to be held during the ICFP conference week on Thursday, August 27th.

The ML family workshop warmly welcomes submission touching on the programming languages traditionally seen as part of the "ML family" (Standard ML, OCaml, F#, CakeML, SML#, Manticore, MetaOCaml, etc.). The scope of the workshop includes all aspects of the design, semantics, theory, application, implementation, and teaching of the members of the ML family. We also encourage presentations from related languages (such as Haskell, Scala, Rust, Nemerle, Links, Koka, F*, Eff, ATS, etc), to exchange experience of further developing ML ideas.

Currently, the workshop is still scheduled to go ahead as planned in Jersey City, however it is likely that the ML workshop will end up being a virtual workshop this year. Either way provisions will be made to allow speakers to present their work remotely.

See our detailed CFP online on the ICFP website:

https://icfp20.sigplan.org/home/mlfamilyworkshop-2020

Important dates

  • Friday 15th May (any time zone): Abstract submission deadline
  • Friday 26th June: Author notification
  • Thursday 27th August: ML Family Workshop

Program committee

  • Youyou Cong (Tokyo Institute of Technology)
  • Gowtham Kaki (Purdue University)
  • Neel Krishnaswami (University of Cambridge)
  • Daan Leijen (Microsoft Research)
  • Koko Muroya (Kyoto University)
  • Atsushi Ohori (Tohoku University)
  • Jonathan Protzenko (Microsoft Research)
  • Gabriel Radanne (INRIA)
  • Claudio Russo (Dfinity)
  • Leo White (Jane Street) (Chair)
  • Jeremy Yallop (University of Cambridge)

Submission details

See the online CFP for the details on the expected submission format.

Submissions must be uploaded to the workshop submission website

https://ml2020.hotcrp.com/

before the submission deadline.

Announcing Sek, an efficient implementation of sequences

François Pottier announced

We are pleased to announce the first release of Sek, an OCaml library that offers an efficient implementation of sequences.

The library offers both ephemeral (mutable) sequences and persistent (immutable) sequences, and offers constant-time conversions between these flavors.

It supports all of the standard operations on stacks, queues, deques (e.g. push, pop at either end), catenable sequences (concat, split), and random access sequences (get, set).

Data is stored internally in chunks (fixed-capacity arrays), which is why this data structure is known as a chunK SEquence.

It is intended to achieve excellent time complexity and memory usage.

This is an initial release. The library has not been tested in production, but has received extensive unit testing, via afl-fuzz and ocaml+afl – which are remarkably effective tools, by the way!

This is work in progress; more features, such as iterators, will be added in the future.

To install Sek, just type

opam update && opam install sek

Documentation is online.

Feedback is welcome!

Arthur Charguéraud
François Pottier
with contributions by Émilie Guermeur

Yaron Minsky asked and Fabian replied

I’m particularly interested in how it compares to Base.Sequence and Seq in the OCaml distribution, but surely there are others as well.

This actually looks like an array/vector structure (supporting, among other things, fast access to the nth element), so a comparison with CCVector, CCFun_vec, BatVect, Clarity.Vector etc. would be more appropriate. The name is a bit unfortunate considering the naming used in the general ecosystem.

Some time ago, I added some crude benchmarks to containers' benchsuite. I'll see if I can add Sek when I find time.

gasche said

I think it really is a sequence library in the sense that in maintains an in-order sequence of items, and sequences can be joined/split efficiently. It also provides logarithmic random access, but this is probably not competitive with fixed-size arrays. It would be comparable to "persistent vector" libraries, ropes, finger trees, etc. The fact that the authors expose a Stack/Queue interface suggests that it has also been tuned to perform reasonably well in this case.

It does not provide any delayed computation of items, so in that regard it is not comparable to Sequence/Seq.

@charguer has designed similar datastructures in the past to represent the work-queues of concurrent workers (you want at least a fast "push" to add a new task and, when doing work-stealing, having a fast "split" is convenient). See Theory and Practice of Chunked Sequences, Umut Acar, Arthur Charguéraud, Mike Rainey, 2014, and A Work-Efficient Algorithm for Parallel Unordered Depth-First Search.

As far as I know, the OCaml implementation just released has not been tested/benchmarked for parallel algorithms. I would be curious to see an experiment of parallel graph traversal with this structure and Multicore-OCaml.

Announcing dune-deps: produces a project-centric dependency graph

Martin Jambon announced

I'm happy to announce the availability of dune-deps, a command-line tool that scans a dune project and gathers the dependencies into a graph. The output is in the dot format, supported by the dot command from graphviz.

It shows the dependencies between the following:

  • libraries defined by the project,
  • executables defined by the project,
  • direct dependencies on external libraries.

Dependencies are extracted by parsing dune files. As an example, here's what we obtain for the sources of opam, which has over 50K lines of code:

f6213fa7fda52521c6782988155ab23b997dafb8.png

The commands for this are:

# obtain the project's sources
$ git clone --depth=1 https://github.com/ocaml/opam.git

# extract dependencies and eliminate superfluous graph edges
$ dune-deps opam | tred > deps.dot

# render the graph
$ dot -Tpng deps.dot -o deps.png

A suggestion is to include such graph in your project's README.md.

OCaml Users and Developers Meeting 2020

Ivan Gotovchits announced

It is my pleasure to invite submissions to the OCaml Users and Developers Workshop 2020, which is again co-located with ICFP and will be held on Friday 28th August 2020 in Jersey City, NJ, USA.

The OCaml Users and Developers Workshop brings together the OCaml community, including users of OCaml in industry, academia, hobbyists and the free software community. Previous editions have been co-located with ICFP since 2012 in Copenhagen, Boston, Gothenburg, Nara, Oxford, St Louis and last year in Berlin, following OCaml Meetings in Paris in 2010 and 2011.

Important Dates

  • Talk proposal submission deadline: May 8th, 2020, AoE
  • Author Notification: June 26th, 2020
  • OCaml Workshop: August 28th, 2020

Scope

Presentations and discussions focus on the OCaml programming language and its community. We aim to solicit talks on all aspects related to improving the use or development of the language and its programming environment, including, for example (but not limited to):

  • compiler developments, new backends, runtime and architectures
  • practical type system improvements, such as GADTs, first-class modules, generic programming, or dependent types
  • new library or application releases, and their design rationales
  • tools and infrastructure services, and their enhancements
  • prominent industrial or experimental uses of OCaml, or deployments in unusual situations.

Presentations

The workshop is an informal meeting with no formal proceedings. The presentation material will be available online from the workshop homepage. The presentations may be recorded and made available at a later date.

The main presentation format is a workshop talk, traditionally around 20 minutes in length, plus question time, but we also have a poster session during the workshop – this allows to present more diverse work, and gives time for discussion. The program committee will decide which presentations should be delivered as posters or talks.

Submission

To submit a presentation, please register a description of the talk (about 2 pages long) at

https://ocaml2020.hotcrp.com/

providing a clear statement of what will be provided by the presentation: the problems that are addressed, the solutions or methods that are proposed.

LaTeX-produced PDFs are a common and welcome submission format. For accessibility purposes, we ask PDF submitters to also provide the sources of their submission in a textual format, such as .tex sources. Reviewers may read either the submitted PDF or the text version.

ML family workshop

The ML family workshop, held on the previous day, deals with general issues of the ML-style programming and type systems, focuses on more research-oriented work that is less specific to a language in particular. There is an overlap between the two workshops, and we have occasionally transferred presentations from one to the other in the past. Authors who feel their submission fits both workshops are encouraged to mention it at submission time and/or contact the Program Chairs.

Program Committee

  • Ivan Gotovchits, CMU, USA
  • Florian Angeletti, INRIA, France
  • Chris Casinghino, Draper Laboratory, USA
  • Catherine Gasnier, Facebook, USA
  • Rudi Grinberg, OCaml Labs, UK
  • Oleg Kiselyov, Tohoku University, Japan
  • Andreas Rossberg, Dfinity Stiftung, Germany
  • Marcello Seri, University of Groningen, Netherlands
  • Edwin Torok, Citrix, UK
  • Leo White, Jane Street, USA
  • Greta Yorsh, Jane Street, USA
  • Sarah Zennou, Airbus, France

COVID-19 Notice

While ICFP-20 is still scheduled to be held as planned, chances are high that it will be turned into a virtual conference. Which means a wider audience and reduced (hopefully) fees. We will keep you posted.

Questions and contact

Please send any questions to the chair: Ivan Gotovchits (ivg@ieee.org)

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.