OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of February 04 to 11, 2020.

Table of Contents

OCaml 4.10.0, first release candidate

octachron announced

The release of OCaml version 4.10.0 is imminent. We have created a release candidate that you can test.

The source code is available at these addresses:

The compiler can also be installed as an OPAM switch with one of the following commands.

opam switch create ocaml-variants.4.10.0+rc1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

or

opam switch create ocaml-variants.4.10.0+rc1+<VARIANT> --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

where you replace <VARIANT> with one of these:

  • afl
  • flambda
  • fp
  • fp+flambda

We want to know about any bugs. Please report them here: https://github.com/ocaml/ocaml/issues

If every goes well, the full release should follow in the next two or three weeks.

OCaml 4.10.0 changes

Language features

  • #7757, #1726: multi-indices for extended indexing operators: a.%{0;1;2} desugars to ( .%{ ;.. } ) a [|0;1;2|] (Florian Angeletti, review by Gabriel Radanne)
  • breaking change #1859, #9117: enforce safe (immutable) strings by removing the -unsafe-string option by default. This can be overridden by a configure-time option (available since 4.04 in 2016): –disable-force-safe-string since 4.08, -no-force-safe-since between 4.07 and 4.04. In the force-safe-string mode (now the default), the return type of the String_val macro in C stubs is const char* instead of char*. This change may break C FFI code. (Kate Deplaix)
  • #6662, #8908: allow writing "module _ = E" to ignore module expressions (Thomas Refis, review by Gabriel Radanne)

Runtime system

  • #8809: Add a best-fit allocator for the major heap; still experimental, it should be much better than current allocation policies (first-fit and next-fit) for programs with large heaps, reducing both GC cost and memory usage. This new best-fit is not (yet) the default; set it explicitly with OCAMLRUNPARAM="a=2" (or Gc.set from the program). You may also want to increase the space_overhead parameter of the GC (a percentage, 80 by default), for example OCAMLRUNPARAM="o=85", for optimal speed. (Damien Doligez, review by Stephen Dolan, Jacques-Henri Jourdan, Xavier Leroy, Leo White)
  • breaking change #8713, #8940, #9115, #9143, #9202, #9251: Introduce a state table in the runtime to contain the global variables. (The Multicore runtime will have one such state for each domain.)

    This changes the status of some internal variables of the OCaml runtime; in many cases the header file originally defining the internal variable provides a compatibility macro with the old name, but programs re-defining those variables by hand need to be fixed.

    (KC Sivaramakrishnan and Stephen Dolan, compatibility hacking by David Allsopp, Florian Angeletti, Kate Deplaix, Jacques Garrigue, Guillaume Munch-Maccagnoni and Nicolás Ojeda Bär, review by David Allsopp, Alain Frisch, Nicolas Ojeda Bar, Gabriel Scherer, Damien Doligez, and Guillaume Munch-Maccagnoni)

  • #8993: New C functions caml_process_pending_actions{,_exn} in caml/signals.h, intended for executing all pending actions inside long-running C functions (requested minor and major collections, signal handlers, finalisers, and memprof callbacks). The function caml_process_pending_actions_exn returns any exception arising during their execution, allowing resources to be cleaned-up before re-raising. (Guillaume Munch-Maccagnoni, review by Jacques-Henri Jourdan, Stephen Dolan, and Gabriel Scherer)
  • breaking change #8691, #8897, #9027: Allocation functions are now guaranteed not to trigger any OCaml callback when called from C. In long-running C functions, this can be replaced with calls to caml_process_pending_actions at safe points. Side effect of this change: in bytecode mode, polling for asynchronous callbacks is performed at every minor heap allocation, in addition to function calls and loops as in previous OCaml releases. (Jacques-Henri Jourdan, review by Stephen Dolan, Gabriel Scherer and Guillaume Munch-Maccagnoni)
  • breaking change #9037: caml_check_urgent_gc is now guaranteed not to trigger any finaliser. In long-running C functions, this can be replaced with calls to caml_process_pending_actions at safe points. (Guillaume Munch-Maccagnoni, review by Jacques-Henri Jourdan and Stephen Dolan)
  • #8619: Ensure Gc.minor_words remains accurate after a GC. (Stephen Dolan, Xavier Leroy and David Allsopp, review by Xavier Leroy and Gabriel Scherer)
  • #8667: Limit GC credit to 1.0 (Leo White, review by Damien Doligez)
  • #8670: Fix stack overflow detection with systhreads (Stephen Dolan, review by Xavier Leroy, Anil Madhavapeddy, Gabriel Scherer, Frédéric Bour and Guillaume Munch-Maccagnoni)
  • breaking change #8711: The major GC hooks are no longer allowed to interact with the OCaml heap. (Jacques-Henri Jourdan, review by Damien Doligez)
  • #8630: Use abort() instead of exit(2) in caml_fatal_error, and add the new hook caml_fatal_error_hook. (Jacques-Henri Jourdan, review by Xavier Leroy)
  • #8641: Better call stacks when a C call is involved in byte code mode (Jacques-Henri Jourdan, review by Xavier Leroy)
  • #8634, #8668, #8684, #9103 (originally #847): Statistical memory profiling. In OCaml 4.10, support for allocations in the minor heap in native mode is not available, and callbacks for promotions and deallocations are not available. Hence, there is not any public API for this feature yet. (Jacques-Henri Jourdan, review by Stephen Dolan, Gabriel Scherer and Damien Doligez)
  • #9268, #9271: Fix bytecode backtrace generation with large integers present. (Stephen Dolan and Mark Shinwell, review by Gabriel Scherer and Jacques-Henri Jourdan)

Standard library

  • #8760: List.concat_map : ('a -> 'b list) -> 'a list -> 'b list (Gabriel Scherer, review by Daniel Bünzli and Thomas Refis)
  • #8832: List.find_map : ('a -> 'b option) -> 'a list -> 'b option (Gabriel Scherer, review by Jeremy Yallop, Nicolás Ojeda Bär and Daniel Bünzli)
  • #7672, #1492: Add Filename.quote_command to produce properly-quoted commands for execution by Sys.command. (Xavier Leroy, review by David Allsopp and Damien Doligez)
  • #8971: Add Filename.null, the conventional name of the "null" device. (Nicolás Ojeda Bär, review by Xavier Leroy and Alain Frisch)
  • #8651: add '%#F' modifier in printf to output OCaml float constants in hexadecimal (Pierre Roux, review by Gabriel Scherer and Xavier Leroy)
  • #8657: Optimization in [Array.make] when initializing with unboxed or young values. (Jacques-Henri Jourdan, review by Gabriel Scherer and Stephen Dolan)
  • #8716: Optimize [Array.fill] and [Hashtbl.clear] with a new runtime primitive (Alain Frisch, review by David Allsopp, Stephen Dolan and Damien Doligez)
  • #8530: List.sort: avoid duplicate work by chop (Guillaume Munch-Maccagnoni, review by David Allsopp, Damien Doligez and Gabriel Scherer)

Other libraries

  • #1939, #2023: Implement Unix.truncate and Unix.ftruncate on Windows. (Florent Monnier and Nicolás Ojeda Bär, review by David Allsopp)

Code generation and optimizations

  • #8806: Add an [@@immediate64] attribute for types that are known to be immediate only on 64 bit platforms (Jérémie Dimino, review by Vladimir Keleshev)
  • #9028, #9032: Fix miscompilation by no longer assuming that untag_int (tag_int x) = x in Cmmgen; the compilation of (n lsl 1) + 1, for example, would be incorrect if evaluated with a large value for n. (Stephen Dolan, review by Vincent Laviron and Xavier Leroy)
  • #8672: Optimise Switch code generation on booleans. (Stephen Dolan, review by Pierre Chambart)
  • #8990: amd64: Emit 32bit registers for Iconst_int when we can (Xavier Clerc, Tom Kelly and Mark Shinwell, review by Xavier Leroy)
  • #2322: Add pseudo-instruction Ladjust_trap_depth to replace dummy Lpushtrap generated in linearize (Greta Yorsh and Vincent Laviron, review by Xavier Leroy)
  • #8707: Simplif: more regular treatment of Tupled and Curried functions (Gabriel Scherer, review by Leo White and Alain Frisch)
  • #8526: Add compile-time option -function-sections in ocamlopt to emit each function in a separate named text section on supported targets. (Greta Yorsh, review by Pierre Chambart)
  • #2321: Eliminate dead ICatch handlers (Greta Yorsh, review by Pierre Chambart and Vincent Laviron)
  • #8919: lift mutable lets along with immutable ones (Leo White, review by Pierre Chambart)
  • #8909: Graph coloring register allocator: the weights put on preference edges should not be divided by 2 in branches of conditional constructs, because it is not good for performance and because it leads to ignoring preference edges with 0 weight. (Eric Stavarache, review by Xavier Leroy)
  • #9006: int32 code generation improvements (Stephen Dolan, designed with Greta Yorsh, review by Xavier Clerc, Xavier Leroy and Alain Frisch)
  • #9041: amd64: Avoid stall in sqrtsd by clearing destination. (Stephen Dolan, with thanks to Andrew Hunter, Will Hasenplaugh, Spiros Eliopoulos and Brian Nigito. Review by Xavier Leroy)

Manual and documentation

  • #8718, #9089: syntactic highlighting for code examples in the manual (Florian Angeletti, report by Anton Kochkov, review by Gabriel Scherer)
  • #9101: add links to section anchor before the section title, make the name of those anchor explicits. (Florian Angeletti, review by Daniel Bünzli, Sébastien Hinderer, and Gabriel Scherer)
  • #9257, cautionary guidelines for using the internal runtime API without too much updating pain. (Florian Angeletti, review by Daniel Bünzli, Guillaume Munch-Maccagnoni and KC Sivaramakrishnan)
  • #8950: move local opens in pattern out of the extension chapter (Florian Angeletti, review and suggestion by Gabriel Scherer)
  • #9088, #9097: fix operator character classes (Florian Angeletti, review by Gabriel Scherer, report by Clément Busschaert)
  • #9169: better documentation for the best-fit allocation policy (Gabriel Scherer, review by Guillaume Munch-Maccagnoni and Florian Angeletti)

Compiler user-interface and warnings

  • #8833: Hint for (type) redefinitions in toplevel session (Florian Angeletti, review by Gabriel Scherer)
  • #2127, #9185: Refactor lookup functions. Included observable changes:

    • makes the location of usage warnings and alerts for constructors more precise
    • don't warn about a constructor never being used to build values when it has been defined as private

    (Leo White, Hugo Heuzard review by Thomas Refis, Florian Angeletti)

  • #8702, #8777: improved error messages for fixed row polymorphic variants (Florian Angeletti, report by Leo White, review by Thomas Refis)
  • #8844: Printing faulty constructors, inline records fields and their types during type mismatches. Also slightly changed other type mismatches error output. (Mekhrubon Turaev, review by Florian Angeletti, Leo White)
  • #8885: Warn about unused local modules (Thomas Refis, review by Alain Frisch)
  • #8872: Add ocamlc option "-output-complete-exe" to build a self-contained binary for bytecode programs, containing the runtime and C stubs. (Stéphane Glondu, Nicolás Ojeda Bär, review by Jérémie Dimino and Daniel Bünzli)
  • #8874: add tests for typechecking error messages and pack them into pretty-printing boxes. (Oxana Kostikova, review by Gabriel Scherer)
  • #8891: Warn about unused functor parameters (Thomas Refis, review by Gabriel Radanne)
  • #8903: Improve errors for first-class modules (Leo White, review by Jacques Garrigue)
  • #8914: clarify the warning on unboxable types used in external primitives (61) (Gabriel Scherer, review by Florian Angeletti, report on the Discourse forum)
  • #9046: disable warning 30 by default This outdated warning complained on label/constructor name conflicts within a mutually-recursive type declarations; there is now no need to complain thanks to type-based disambiguation. (Gabriel Scherer)

Tools

  • breaking change #6792, #8654 ocamldebug now supports programs using Dynlink. This changes ocamldebug messages, which may break compatibility with older emacs modes. (Whitequark and Jacques-Henri Jourdan, review by Gabriel Scherer and Xavier Clerc)
  • #8621: Make ocamlyacc a Windows Unicode application (David Allsopp, review by Nicolás Ojeda Bär)
  • breaking change #8834, ocaml: adhere to the XDG base directory specification to locate an .ocamlinit file. Reads an $XDG_CONFIG_HOME/ocaml/init.ml file before trying to lookup ~/.ocamlinit. On Windows the behaviour is unchanged. (Daniel C. Bünzli, review by David Allsopp, Armaël Guéneau and Nicolás Ojeda Bär)
  • #9113: ocamldoc: fix the rendering of multi-line code blocks in the 'man' backend. (Gabriel Scherer, review by Florian Angeletti)
  • #9127, #9130: ocamldoc: fix the formatting of closing brace in record types. (David Allsopp, report by San Vu Ngoc)
  • #9181: make objinfo work on Cygwin and look for the caml_plugin_header symbol in both the static and the dynamic symbol tables. (Sébastien Hinderer, review by Gabriel Scherer and David Allsopp)

Build system

  • #8840: use ocaml{c,opt}.opt when available to build internal tools On my machine this reduces parallel-build times from 3m30s to 2m50s. (Gabriel Scherer, review by Xavier Leroy and Sébastien Hinderer)
  • #8650: ensure that "make" variables are defined before use; revise generation of config/util.ml to better quote special characters (Xavier Leroy, review by David Allsopp)
  • #8690, #8696: avoid rebuilding the world when files containing primitives change. (Stephen Dolan, review by Gabriel Scherer, Sébastien Hinderer and Thomas Refis)
  • #8835: new configure option –disable-stdlib-manpages to disable building and installation of the library manpages. (David Allsopp, review by Florian Angeletti and Gabriel Scherer)
  • #8837: build manpages using ocamldoc.opt when available cuts the manpages build time from 14s to 4s (Gabriel Scherer, review by David Allsopp and Sébastien Hinderer, report by David Allsopp)
  • #8843, #8841: fix use of off_t on 32-bit systems. (Stephen Dolan, report by Richard Jones, review by Xavier Leroy)
  • #8947, #9134: fix/improve support for the BFD library (Sébastien Hinderer, review by Damien Doligez)
  • #8951: let make's default target build the compiler (Sébastien Hinderer, review by David Allsopp)
  • #8995: allow developers to specify frequently-used configure options in Git (ocaml.configure option) and a directory for host-specific, shareable config.cache files (ocaml.configure-cache option). See HACKING.adoc for further details. (David Allsopp, review by Gabriel Scherer)
  • #9136: Don't propagate Cygwin-style prefix from configure to Makefile.config on Windows ports. (David Allsopp, review by Sébastien Hinderer)

Internal/compiler-libs changes

  • #8828: Added abstractions for variants, records, constructors, fields and extension constructor types mismatch. (Mekhrubon Turaev, review by Florian Angeletti, Leo White and Gabriel Scherer)
  • #7927, #8527: Replace long tuples into records in typeclass.ml (Ulugbek Abdullaev, review by David Allsopp and Gabriel Scherer)
  • #1963: split cmmgen into generic Cmm helpers and clambda transformations (Vincent Laviron, review by Mark Shinwell)
  • #1901: Fix lexing of character literals in comments (Pieter Goetschalckx, review by Damien Doligez)
  • #1932: Allow octal escape sequences and identifiers containing apostrophes in ocamlyacc actions and comments. (Pieter Goetschalckx, review by Damien Doligez)
  • #2288: Move middle end code from [Asmgen] to [Clambda_middle_end] and [Flambda_middle_end]. Run [Un_anf] from the middle end, not [Cmmgen]. (Mark Shinwell, review by Pierre Chambart)
  • #8692: Remove Misc.may_map and similar (Leo White, review by Gabriel Scherer and Thomas Refis)
  • #8677: Use unsigned comparisons in amd64 and i386 emitter of Lcondbranch3. (Greta Yorsh, review by Xavier Leroy)
  • #8766: Parmatch: introduce a type for simplified pattern heads (Gabriel Scherer and Thomas Refis, review by Stephen Dolan and Florian Angeletti)
  • #8774: New implementation of Env.make_copy_of_types (Alain Frisch, review by Thomas Refis, Leo White and Jacques Garrigue)
  • #7924: Use a variant instead of an int in Bad_variance exception (Rian Douglas, review by Gabriel Scherer)
  • #8890: in -dtimings output, show time spent in C linker clearly (Valentin Gatien-Baron)
  • #8910, #8911: minor improvements to the printing of module types (Gabriel Scherer, review by Florian Angeletti)
  • #8913: ocamltest: improve 'promote' implementation to take skipped lines/bytes into account (Gabriel Scherer, review by Sébastien Hinderer)
  • #8908: Use an option instead of a string for module names ("_" becomes None), and a dedicated type for functor parameters: "()" maps to "Unit" (instead of "*"). (Thomas Refis, review by Gabriel Radanne)
  • #8928: Move contains_calls and num_stack_slots from Proc to Mach.fundecl (Greta Yorsh, review by Florian Angeletti and Vincent Laviron)
  • #8959, #8960, #8968, #9023: minor refactorings in the typing of patterns:

    • refactor the {let,pat}_bound_idents* functions
    • minor bugfix in type_pat
    • refactor the generic pattern-traversal functions in Typecore and Typedtree
    • restrict the use of Need_backtrack

    (Gabriel Scherer and Florian Angeletti, review by Thomas Refis and Gabriel Scherer)

  • #9030: clarify and document the parameter space of type_pat (Gabriel Scherer and Florian Angeletti and Jacques Garrigue, review by Florian Angeletti and Thomas Refis)
  • #8975: "ocamltests" files are no longer required or used by "ocamltest". Instead, any text file in the testsuite directory containing a valid "TEST" block will be automatically included in the testsuite. (Nicolás Ojeda Bär, review by Gabriel Scherer and Sébastien Hinderer)
  • #8992: share argument implementations between executables (Florian Angeletti, review by Gabriel Scherer)
  • #9015: fix fatal error in pprint_ast (#8789) (Damien Doligez, review by …)

Bug fixes

  • #5673, #7636: unused type variable causes generalization error (Jacques Garrigue and Leo White, review by Leo White, reports by Jean-Louis Giavitto and Christophe Raffalli)
  • #6922, #8955: Fix regression with -principal type inference for inherited methods, allowing to compile ocamldoc with -principal (Jacques Garrigue, review by Leo White)
  • #7925, #8611: fix error highlighting for exceptionally long toplevel phrases (Kyle Miller, reported by Armaël Guéneau, review by Armaël Guéneau and Nicolás Ojeda Bär)
  • #8622: Don't generate #! headers over 127 characters. (David Allsopp, review by Xavier Leroy and Stephen Dolan)
  • #8715: minor bugfixes in CamlinternalFormat; removes the unused and misleading function CamlinternalFormat.string_of_formatting_gen (Gabriel Scherer and Florian Angeletti, review by Florian Angeletti and Gabriel Radanne)
  • #8792, #9018: Possible (latent) bug in Ctype.normalize_type removed incrimined Btype.log_type, replaced by Btype.set_type (Jacques Garrigue, report by Alain Frisch, review by Thomas Refis)
  • #8856, #8860: avoid stackoverflow when printing cyclic type expressions in some error submessages. (Florian Angeletti, report by Mekhrubon Turaev, review by Leo White)
  • #8875: fix missing newlines in the output from MSVC invocation. (Nicolás Ojeda Bär, review by Gabriel Scherer)
  • #8921, #8924: Fix stack overflow with Flambda (Vincent Laviron, review by Pierre Chambart and Leo White, report by Aleksandr Kuzmenko)
  • #8892, #8895: fix the definition of Is_young when CAML_INTERNALS is not defined. (David Allsopp, review by Xavier Leroy)
  • #8896: deprecate addr typedef in misc.h (David Allsopp, suggestion by Xavier Leroy)
  • #8981: Fix check for incompatible -c and -o options. (Greta Yorsh, review by Damien Doligez)
  • #9019, #9154: Unsound exhaustivity of GADTs from incomplete unification Also fixes bug found by Thomas Refis in #9012 (Jacques Garrigue, report and review by Leo White, Thomas Refis)
  • #9031: Unregister Windows stack overflow handler while shutting the runtime down. (Dmitry Bely, review by David Allsopp)
  • #9051: fix unregistered local root in win32unix/select.c (could result in select returning file_descr-like values which weren't in the original sets) and correct initialisation of some blocks allocated with caml_alloc_small. (David Allsopp, review by Xavier Leroy)
  • #9073, #9120: fix incorrect GC ratio multiplier when allocating custom blocks with caml_alloc_custom_mem in runtime/custom.c (Markus Mottl, review by Gabriel Scherer and Damien Doligez)
  • #9209, #9212: fix a development-version regression caused by #2288 (Kate Deplaix and David Allsopp, review by Sébastien Hinderer and Gabriel Scherer )
  • #9218, #9269: avoid a rare wrong module name error with "-annot" and inline records. (Florian Angeletti, review by Gabriel Scherer, report by Kate Deplaix)
  • #9261: Fix a soundness bug in Rec_check, new in 4.10 (from #8908) (Vincent Laviron, review by Jeremy Yallop and Gabriel Scherer)

Luv — cross-platform concurrent I/O: a binding to libuv

Anton Bachin announced

I am pleased to announce the first release of Luv, a binding to libuv. libuv is the cross-platform C library inside Node.js that does I/O, wraps system calls, and drives Node's main loop.

let () =
  let timer = Luv.Timer.init () |> Stdlib.Result.get_ok in
  ignore (Luv.Timer.start timer 1000 (fun () ->
    print_endline "Hello, world!"));

  print_endline "Waiting...";
  ignore (Luv.Loop.run ())

https://github.com/aantron/luv

Luv exposes sockets, files, subprocesses, FS events, and many other OS APIs. Indeed, Luv is an independent alternative to the standard Unix, and does not depend on Unix. The easiest way to get an idea of all that is available is to quickly scroll through the list of modules in the API docs.

You are invited to glance through all of Luv's documentation:

libuv is focused primarily on event-driven programming, which makes it similar to Async or Lwt. However, it does not offer a promise data type — the API is written in terms of callbacks.

libuv also supports multithreading and multiprocessing. In particular, libuv allows running multiple event loops in multiple threads, which makes it a ready candidate for use with a multicore runtime.

A major advantage of libuv is that, because it is a core module of Node, it is well-maintained and supports many platforms. Luv inherits these properties.

Luv uses ctypes to greatly minimize the amount of C code in the repo. A thorough test suite checks return values, I/O effects, and interaction with the OCaml garbage collector and threading runtime. Luv vendors libuv to avoid version conflicts, and, when you install Luv, it automatically builds its internal libuv.

One of the main goals of Luv is to be easy to integrate into larger projects. For example, there may be an experimental version of Lwt based on it. To that end, Luv is as unopinionated as possible. It sticks closely to the libuv API, making no unnecessary design decisions, and deviating only where changes are necessary to integrate with OCaml, or where libuv has an awkward API due to the limitations of C.

And with all that… Happy systems programming :)

Musings on extended pattern-matching syntaxes

Continuing this old thread, Alexander Bashkirov announced

We reviewed this thread quite carefully along with features of other languages and summed up the results here.

We also started an active patterns RFC discussion.

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.