OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of June 13 to 20, 2023.

Table of Contents

Packaging an executable for homebrew install for non-OCaml users

James Geddes asked

Would anyone have an example I could look at of a homebrew “formula” for an OCaml program? I’m looking to distribute an executable to colleagues who are not OCaml users.

(There seem to be lots of moving parts and I’m not sure which one has stopped moving!).

Many thanks.

v0.16 release of Jane Street packages

Diana Kalinichenko announced

Dear OCaml developers,

We are pleased to announce the v0.16 release of Jane Street packages!

This release comes with 34 new packages and a plethora of new features, fixes, and enhancements.

The remainder of this post highlights the main changes since the v0.15 release.

Release-wide changes

  • Switch to OCaml 4.14

    We are switching Base and all our packages, except sexplib0, to OCaml 4.14 and above.

    Previously, Base supported the four latest OCaml versions. The prime reason for it was to ensure that a single ppxlib release would work across multiple compilers. However, ppxlib now depends solely on sexplib0, not Base.

    Meanwhile, we are making a considerable effort to integrate the latest OCaml features. We’ve already added features like the In_channel module from the 4.14 standard library to Base, and we’re working on switching to the 5.0 compiler internally.

    As a result, we decided to remove the strict requirement for Base’s backward compatibility. Instead, we hope to provide a more comprehensive Stdlib replacement and better testing for recent compiler versions. We plan to eventually add support for OCaml 5 primitives in Base, starting with releases v0.17 or v0.18.

  • 32-bit platform support

    As stated previously, we are no longer testing on 32-bit platforms. That said, we expect that our packages will mostly continue to function on 32-bit, and we strive to maintain compatibility. In particular, we will keep responding to bug reports and accepting 32-bit compatibility patches. We’re also hoping that Tarides’ work in this area will improve the testing of our packages across many platforms, including 32-bit bytecode.

  • Deprecation of Caml in favor of Stdlib

    We are deprecating our Caml alias for the OCaml standard library, in favor of using the default module, Stdlib. This allows writing code that references the standard library regardless of whether Base is opened or not.

  • Local allocations

    We’ve included annotations for local allocations in our public release code; however, these annotations are not recognized by the stock OCaml compiler. More information on local allocations can be found in our blog post at https://blog.janestreet.com/oxidizing-ocaml-locality/.

    For those interested in experimenting with local allocations, our OCaml fork with local allocation support can be found here. Please be aware that this is highly experimental and not guaranteed to function outside of Jane Street.

Future plans

  • Shadowing Stdlib

    We plan to stop shadowing Stdlib in Base. We also plan to add a flag to Dune to enable -nopervasives during compilation, which prevents Stdlib from being opened automatically. The previous practice of shadowing Stdlib in Base has caused issues, and we hope this new approach will prove more resilient.

  • OCaml extensions

    We intend to create an experimental Opam repository with our internal compiler for a more straightforward setup. This will allow users to experiment with our new language extensions before they are integrated into mainstream OCaml.

    At present, we convert our native syntax for OCaml extensions into annotations. However, once we transition to the experimental repo, we will release the unmodified code featuring the preferred local_ syntax.

Many, many other changes

Since our last release, we have focused on delivering improved changelogs for our users, and we plan to maintain these efforts moving forward. Most of our libraries now include changelog files that outline changes since the v0.15 release. Browse our GitHub repositories and access the respective CHANGES.md files for more information.

For examples, see changelogs for Base, Async_kernel, and VCaml.

New packages

base_trie – trie data structure library for Base.

bidirectional_map – a library for bidirectional maps and multimaps.

content_security_policy – a library for building content-security policies for the Web.

cpuid – a library for parsing CPU capabilities from the cpuid instruction.

csv – tools for working with CSVs on the command line

dedent – a library for improving the readability of multi-line string constants in code.

diffable – an interface for diffable data structures.

env_config – a helper library for retrieving configuration from an environment variable.

hardcaml_axi – Hardcaml AXI interface types.

hardcaml_c – Hardcaml C simulation backend.

hardcaml_handshake – Hardcaml handshake component.

indentation_buffer – a library for building strings with indentation.

jane_rope – string representation with cheap concatenation.

js_of_ocaml_patches – additions to js_of_ocaml’s standard library that are required by Jane Street libraries.

lru_cache – an LRU Cache implementation for Core.

man_in_the_middle_debugger – a man-in-the-middle debugging library for Async readers and writers.

n_ary – a library for N-ary datatypes and operations, generalizing enumerations, variants, tuples, and lists.

numeric_string – a comparison function that sorts numeric fragments of strings according to their numeric value.

of_json – a friendly applicative interface for Jsonaf.

ordinal_abbreviation – a minimal library for generating ordinal names of integers.

polling_state_rpc – an RPC which tracks the state on the client and server so it only needs to send diffs across the wire.

ppx_demo – a PPX that exposes the source code string of an expression/module structure.

ppx_derive_at_runtime – defining a new PPX deriver by naming a runtime module.

ppx_globalize – a PPX rewriter that generates functions to copy local values to the global heap.

ppx_sexp_conv – generation of S-expression conversion functions from type definitions.

ppx_stable_witness – a PPX extension for deriving a witness that a type is intended to be stable.

ppx_tydi – let expressions inferring pattern type from expression.

regex_parser_intf – interface shared by Re_parser and Re2.Parser

re_parser – typed parsing using regular expressions.

semantic_version – semantic versioning data structure.

stored_reversed – a library for representing a list temporarily stored in reverse order.

streamable – a collection of types suitable for incremental serialization.

tilde_f – provides a let-syntax for continuation-passing style.

torch – we are taking over the development of ocaml-torch from Laurent Mazare. Many thanks to him for his original work!

New release of Menhir (20230608)

François Pottier announced

Dear users of Menhir,

I am happy to announce a new release of Menhir, with several minor changes and bug fixes, listed below.

  opam update
  opam upgrade menhir

Happy parsing, François Pottier francois.pottier@inria.fr http://cambium.inria.fr/~fpottier/

2023/06/08

  • The new command line switch --dump-menhirLib <directory> causes the source files menhirLib.ml and menhirLib.mli to be created in the designated directory. This command can be useful to users with special needs who wish to use menhirLib but do not want to rely on it being installed somewhere in the file system. (Contributed by Nicolás Ojeda Bär.)
  • Changes in the public API of the library MenhirSdk. (Contributed by Frédéric Bour.)
    • The function Lr1.reductions, whose type is t -> (terminal * production list) list, is still present but is now deprecated. The function Lr1.get_reductions, whose type is t -> (terminal * production) list, should be used instead. The library offers a view of the automaton after conflict resolution, so, for each token, at most one production can be reduced.
    • The function Lr1.default_reduction : Lr1.t -> production option appears. This function indicates which states have a default reduction.
    • The function Cmly_read.read_channel appears.
    • The module Cmly_read.Lift appears.
  • The new experimental flag --specialize-token causes the code back-end to perform code specialization so as to guarantee that each token is examined exactly once (thus never more than once) by the parser. This causes an increase in code size and does not (yet) allow a performance increase. This feature is independent of the choice between -O 0, -O 1 and -O 2. It may disappear in the future.
  • Inconsistent type definitions used to be produced when --only-tokens and --inspection were passed on the command line. The type terminal was defined at the toplevel, whereas it should always be defined inside the submodule MenhirInterpreter. (Reported and fixed by Frédéric Bour.)
  • Malformed code and type definitions used to be produced when --external-tokens and --inspection were passed on the command line. Fixed. (Reported by Maxime Dénès.)
  • Enable OCaml warning 39 (unused rec flag) in the OCaml code produced by Menhir’s code back-end. Menhir now carefully emits let instead of let rec when a function is not recursive.

Dream-html - DSL to build HTML, integrated with Dream

Continuing this thread, Yawar Amin announced

release of minisat 0.5

Simon Cruanes announced

It didn’t take long, but here’s minisat 0.6 which actually switches to the C++ minisat from the C minisat that was used so far. That means new functions such as unsat_core are now available!

Which filters would you like to see on the OCaml blog at OCaml.org?

Zineb Jambin announced

Dear OCaml users,

I am currently working with the OCaml.org maintainers on improving the blog section of the OCaml.org website. Specifically, I am adding filters that enable you to quickly find relevant articles from the OCaml community.

To make this feature truly useful, it is essential that we add relevant and useful tags to the blog posts. I would like to hear your feedback on which specific tags you would like to see.

Your opinion is very important to us. Here are some ideas for possible tags/filters:

  • By authors
  • Industry vs Personal blog / kind of blog source
  • relating to libraries and packages

Feel free to add your own tag suggestions or comment on those that have already been proposed. Which filters are most important to you? Your feedback will help us improve the search experience and make ocaml.org more user-friendly for everyone.

Thank you in advance for your participation and contribution!

OCaml Platform Newsletter, May 2023

Thibaut Mattio announced

Welcome to the second issue of the OCaml Platform newsletter!

We’re excited to share the work we’ve done in May on improving OCaml developer experience with the OCaml Platform. Similar to the previous update, this issue is structured around the development workflow we’re currently exploring or improving.

The highlight of this month is the publication of the work-in-progress roadmap for the OCaml Platform. We published it to start gathering community feedback on the Design Principles and Persona of the Platform. The feedback will be used to establish our plans for the next three years. We’ve received tons of very insightful and constructive feedback already, and in the coming weeks and months, we’ll revise the roadmap based on that feedback. As a next step, we’ll share the first version of the proposed developer workflows.

Another important milestone this month is the release of Dune 3.8. The release comes with support for compiling OCaml projects to JavaScript using Melange, which has seen its first stable release this month! It also contains several important features and improvements that have been in the work for some time, like the new concurrent action and the composition of Coq rules.

As a last highlight, the first alpha of opam 2.2 is getting very close. There were some unexpected issues while preparing the release this month, but the opam team is still aiming for a release in June.

There’s a lot of other very exciting work to talk about, so let’s delve into it!

  • Building Packages
    • Dune Exploring Package Management in Dune
    • opam Native Support for Windows in opam 2.2
    • Dune Improving Dune’s Documentation
    • Dune Composing installed Coq theories
    • Dune Running Actions Concurrently
    • Dune Benchmarking Dune on Large Code Bases
  • Compiling to JavaScript
    • Dune Compile to JavaScript with Melange in Dune
  • Generating Documentation
    • Odoc Add Search Capabilities to odoc
    • Odoc Support for Tables in odoc
  • Editing and Refactoring Code
    • Merlin* Support for Project-Wide References in Merlin
    • Merlin* Improving Merlin’s Performance
    • OCaml LSP* Using Dune RPC on Windows
    • OCaml LSP* Upstreaming OCaml LSP’s Fork of Merlin
  • Formatting Code
    • OCamlFormat Closing the Gap Between OCamlFormat and ocp-indent

Releases

Here are the new versions of Platform tools we released in April. Have a look at the OCaml Changelog to read announcements and feature highlights!

Building Packages

  • Dune Exploring Package Management in Dune

    Contributors: @rgrinberg (Tarides), @Leonidas-from-XIV (Tarides), @gridbugs (Tarides), @kit-ty-kate (Tarides)

    Explorations continue on adding package management support to Dune. This month progress has been made on several fronts:

    • The work on the solver has been started, including vendoring the opam-0install solver for solving dependencies when generating Dune lockfiles. A work-in-progress implementation of lockfile generation is available on the main branch.
    • The source tree handling has undergone a refactor to eventually allow multiple context-specific lockfiles.
    • The source fetching implementation has seen improvements, including checksum handling and a better Fetch API. This results in a cleaner interface for building opam packages.
    • Work continues on prototyping the building of opam packages, which includes the addition of new Patch and Substitute actions. This has increased the subset of opam packages that can now be built.

    Activities:

  • opam Native Support for Windows in opam 2.2

    Contributors: @rjbou (OCamlPro), @kit-ty-kate (Tarides), @dra27 (Tarides), @emillon (Tarides), @Leonidas-from-XIV (Tarides)

    Throughout May, the opam team has focused on reviewing and fixing remaining PRs for the 2.2 alpha release. This effort debugged and resolved issues found during testing, and the team is now working through the handful PRs pending review.

    A new release, opam 2.1.5 point release, has also been rolled out, backporting several fixes from the upcoming 2.2 release, and an important security fix. You can read the announcement on the OCaml Changelog.

    Activities:

  • Dune Improving Dune’s Documentation

    Contributors: @emillon (Tarides)

    Two Dune libraries now have a documentation page on OCaml.org: both dune-build-info and dune-configurator now have their API documentation directly on their package page, and for Dune itself, a link to its official documentation has been included.

    A new action: directive has been added to the Dune Sphinx domain, allowing for improved cross-referencing. Finally, the old .org format Dune example docs have been converted to Markdown to standardize the documentation format further.

    The improvements to the documentation have been published as part of the Dune 3.8 release. The new structure and many improvements can be viewed on Dune documentation.

    Activities:

  • Dune Composing installed Coq theories

    Contributors: @Alizter and @ejgallego (IRIF)

    Last month, the PR that brings support for composing Coq theories with Dune was merged.

    This is now available in the release of Dune 3.8.0! From this point onwards, Coq users can utilize Dune to build Coq projects even if they depend on Coq projects that use other build systems!

  • Dune Running Actions Concurrently

    Contributors: @Alizter and @hhugo (Nomadic Labs)

    Last month, a couple of PRs were merged into Dune to add a new concurrent action and it was utilized to run inline tests concurrently.

    These patches are part of Dune 3.8.0 release, starting now, you can use the new concurrent action in your Dune rules:

    (rule
     (action
      (concurrent
       (run <prog> <args>)
       (run <prog> <args>))))
    
  • Dune Benchmarking Dune on Large Code Bases

    Contributors: @gridbugs (Tarides), @Leonidas-from-XIV (Tarides)

    The quality of the dune benchmark results has been improved, averaging out the variance seen in short tests by running them multiple times. This enhancement aims to reduce the effect of the background noise inherent in the environment.

    The dune benchmarks also exposed two broken packages: the hash of ppx_rapper (3.1.0) artefact has been updated as it had changed in place, and ocamlcodoc artefacts have been added to the opam-source-archives as the original URL is no longer reachable.

    Moreover, a stack overflow in dune-rpc-lwt exposed by the benchmarks has been fixed.

    Activities:

    • Ran short monorepo benchmarks multiple times – ocaml/dune#7798.
    • Fixed issues in monorepo benchmarks – ocaml/dune#7786.
    • Added ocamlcodoc to opam-source-archives and updated opam-repository.
    • Opened an issue about ppx_rapper.3.1.0 package changing in place.

Compiling to JavaScript

  • Dune Compile to JavaScript with Melange in Dune

    Contributors: @anmonteiro, @jchavarri (Ahrefs), @rgrinberg (Tarides)

    We’re thrilled to see the joint release of Dune 3.8.0 and Melange 1.0 this month!

    Melange is a compiler from OCaml to JavaScript with the vision of maintaining compatibility with OCaml and providing the best OCaml experience within the modern JavaScript ecosystem.

    Have a look at the Dune documentation and Melange documentation to learn how to get started using Melange to compile your OCaml projects to JavaScript.

    You can also refer to the template to get started.

Generating Documentation

  • Odoc Add Search Capabilities to odoc

    Contributors: @panglesd (Tarides), @EmileTrotignon (Tarides), @trefis (Tarides)

    The odoc team is making steady progress on adding a search bar to odoc’s generated documentation.

    In May, an interface for interaction between odoc and search engines was designed. Sherlodoc, which can now run in the browser, was updated to use the new interface exposed by odoc. Improvements were also made to sherlodoc itself to enable searching for constructors and record fields, as well as in docstrings. The larger database from all this extra indexing prompted work on profiling it and implementing optimizations.

    Anticipating the June update, the odoc PR that was open a few days ago can be checked out.

    In parallel, a working prototype of counting occurrences in odoc was developed. The aim is to add usage statistics in the generated index, so that search engines can use it to sort search results. It will also allow for a “jump-to-documentation” feature in the rendered source code that was merged in March.

  • Odoc Support for Tables in odoc

    Contributors: @gpetiot (Tarides), @panglesd (Tarides), @Julow (Tarides), @jonludlam (Tarides), @trefis (Tarides)

    The PRs adding support for a new syntax to create tables in Odoc have been merged!

    As a reminder, this new feature will enable the creation of tables using a syntax similar to Markdown:

    {t
      a  | b | c | d
      ---|:--|--:|:-:
      a  | b | c | d
    }
    

    Odoc will generate tables for different backends, including LaTex and HTML. This new syntax will be available in the upcoming release of Odoc 2.3.0. Stay tuned!

    Activities:

Editing and Refactoring Code

  • Merlin Support for Project-Wide References in Merlin

    Contributors: @voodoos (Tarides), @let-def (Tarides)

    Not much progress was made on the support for project-wide occurrences in Merlin this month due to the Merlin team’s focus on performance improvements, fixing user-reported bugs, compatibility with OCaml 5.1, and the release of Merlin 4.9

    The remaining issues, such as module aliases traversal and index filtering, were discussed, with the implementation of identified solutions set to commence soon.

  • Merlin Improving Merlin’s Performance

    Contributed by: @pitag-ha (Tarides), @3Rafal (Tarides), @voodoos (Tarides), @let-def (Tarides)

    The final stages of work on benchmarking Merlin are in progress, with the Merlin team focusing on integrating the developed benchmarking tooling into Merlin’s CI using current-bench. The PR on merl-an has been opened to add a new current-bench compatible backend and work has started on adding memory usage information to Merlin’s telemetry.

    On the performance optimisations front, the PPX phase cache in Merlin was completed and merged. Issues were opened upstream on how tools that use Merlin can benefit from the new PPX cache.

    Activities:

    • Implemented a sketch of current-bench backend in merl-anpitag-ha/merl-an#2.
    • Started work on adding memory usage information to Merlin’s telemetry.
    • Completed and merged the PPX phase cache PR – ocaml/merlin#1584. - Opened issues at dune and ocaml-lsp-server to keep record of the necessities to enable the PPX phase cache
    • Identified and fixed a memory “leak” related to the (F A).t syntax, causing uncontrolled memoization table growth – ocaml/merlin#1609.
  • OCaml LSP Using Dune RPC on Windows

    Contributors: @nojb (LexiFi)

    In May, a couple of patches that build on the support for watch mode on Windows introduced in Dune 3.7.0 were merged in Dune and OCaml LSP to allow OCaml LSP to use Dune RPC. The aim is to enable Windows users to leverage Dune RPC and receive build statuses and more exhaustive build errors in the editor when Dune is running in watch mode.

    Dune 3.8.0 was released with the above patches and a release of OCaml LSP will follow in the coming weeks.

  • OCaml LSP Upstreaming OCaml LSP’s Fork of Merlin

    Contributors: @voodoos (Tarides), @3Rafal (Tarides)

    The effort to upstream OCaml LSP’s fork of merlin continued. Necessary patches in Merlin have been reviewed and merged. Work also continued on the PR to use Merlin as a library in OCaml LSP.

    Activities:

Formatting Code

  • OCamlFormat Closing the Gap Between OCamlFormat and ocp-indent

    Contributors: @gpetiot (Tarides) and @EmileTrotignon (Tarides), @Julow (Tarides), @ceastlund (Jane Street)

    The effort continued to tune the janestreet profile so that it aligns better with the output of ocp-indent. Despite encountering difficulties with formatting “cinaps” comments, considerable progress was made in May and work will continue throughout June.

    Activities:

Congratulation to the OCaml team for the 2023 SIGPLAN programming languages software award! 🏆

Stephen De Gabrielle announced

Congratulation to the OCaml team for the 2023 SIGPLAN programming languages software award! 🏆​

https://www.sigplan.org/Awards/Software/

(Just announced at SIGPLAN: Awards Lunch at PLDI23)

Other OCaml News

From the ocaml.org blog

Here are links from many OCaml blogs aggregated at the ocaml.org blog.

Old CWN

If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.