OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of October 01 to 08, 2024.

Table of Contents

Releases of fpath-sexplib0, fpath-base, loc, file-rewriter, sexps-rewriter and provider

Mathieu Barbin announced

I wanted to announce the initial release of 6 utility packages to the opam-repository. They are dependencies to some other ongoing projects I have, perhaps some will find them useful.

These are very early days for this software. Please feel welcome to opening issues or discussions tickets if you are inclined.

Thank you @mseri , @avsm & @shonfeder for your help in making these libraries available!

Below you'll find short descriptions with links to the packages home pages. Thank you!

Fpath_sexplib0 only depends on fpath and sexplib0. It defines a single module, Fpath_sexplib0, which is designed to be opened to shadow the Fpath module to add small helpers and a sexp_of serializer to it. The package also introduces three new modules to the scope: Fpart, Absolute_path and Relative_path to increase type-safety when manipulating paths that are known to be relative or absolute.

Fpath_base further extends fpath-sexplib0 and adds a dependency on base. It is designed to be compatible with Base-style containers such as Map, Set, Hashtbl, Hash_set.

Loc is an OCaml library to manipulate code locations, which are ranges of lexing positions from a parsed file.

File_rewriter is an OCaml library for applying small rewrites to tweak or refactor your files. It provides a convenient interface to apply surgical textual substitutions on the fly, while navigating the contents of a file through an abstract representation containing code locations.

Sexps_rewriter is a specialized version of the file-rewriter library dedicated to rewriting sexp files, such as dune config files.

Provider is an OCaml library for creating Traits and Interfaces. It allows you to define the functionality needed by a library without committing to a specific implementation - in essence : dynamic dispatch. Provider is a pattern featured in the Eio project (Eio.Resource). I wanted to make it reusable in other projects - in particular I am currently using it as the parametrization story of vcs. This package had already been available for a little while already but was still unannounced.

Build a project without Stdlib

Mikhail announced

I decided to experiment with compiling a project without the standard library. Why? I don't know. But I could save ~50K. Just sharing my note about it.

You can find an example in my repository.

I found the -nostdlib and -nopervasives (undocumented) flags and after a lot of trying I was able to do what I wanted. It doesn't disable absolutely everything (lists and other types like option are available).

(flags
  :standard
  -nostdlib
  -nopervasives
  ; add runtime
  -cclib
  -lasmrun
  -ccopt
  "-L %{ocaml_where}"
  -ccopt
  "-lm -ldl")
(* stdlib.ml *)
external print_endline : string -> unit = "caml_print_endline" [@@noalloc]

(* main.ml *)
open Stdlib
let () = print_endline "hello from my stdlib"

Hello World program:

  with Stdlib without Stdlib
size 349K 302K

obatcher: Framework for building efficient concurrent services

Lee Koon Wen announced

Hot on the heels of the paper "Concurrent Data Structures Made Easy" appearing at OOPSLA 2024 on the 24th October, I'm pleased to announce release of obatcher - a picos compatible library for implementing efficient batched services in OCaml.

obatcher proposes a new way to approach the design and implementation of concurrent services. It's key benefits are:

  • Incremental optimization and parallelism of services
  • Easy to control and reason about concurrency
  • Retains atomic-style interface with your services while batching happens implicitly
  • Thread-safety for cheap!

Available on opam today, install with

opam install obatcher

For more details, check out the source and README on GitHub: obatcher.

Feedback, contributions, and discussions are welcome!

DBLP query program and library

Samuel Mimram announced

I am happy to announce the first realease of ocaml-dblp, which provides both a program and a library to query the DBLP bibliographic database. In practice, it is mostly useful for retrieving bibtex entries with commands such as

dblp bibtex girard locus solum

which will spit out

@article{DBLP:journals/mscs/Girard01,
  author       = {Jean{-}Yves Girard},
  title        = {Locus Solum: From the rules of logic to the logic of rules},
  journal      = {Math. Struct. Comput. Sci.},
  volume       = {11},
  number       = {3},
  pages        = {301--506},
  year         = {2001},
  url          = {https://doi.org/10.1017/S096012950100336X},
  doi          = {10.1017/S096012950100336X},
  timestamp    = {Wed, 01 Apr 2020 08:48:47 +0200},
  biburl       = {https://dblp.org/rec/journals/mscs/Girard01.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

(or, even better, use dblp bib to directly add this at the end of the .bib file in the current directory).

It might still need some polishing, feel free to reach out if you encounter some problems.

cudajit: Bindings to the cuda and nvrtc libraries

Lukasz Stafiniak announced

cudajit 0.5.0 is now available in the opam repository. It's organized into modules, and it adds support for CUDA events.

YOCaml, a framework for static site generator

Xavier Van de Woestyne announced

:wave: Hello everyone! We, the YOCaml development team, are very pleased to announce the release of version 2, freshly merged into opam-repository :champagne:!

YOCaml is a framework for describing static site generators (a very small applicative build-system whose API is tailor-made for creating web pages ) and its internal model is very similar to Hakyll (the 3th version), another Haskell framework. (a presentation was given to the OCaml user Group in Paris and you can find the video, in French, here).

Changes with 1.0.0

Historically, YOCaml was written very, very quickly to give examples of slightly exotic uses of the library Preface. Due to its experimental nature, the API was a bit laborious, but we did find some users! We took advantage of the redesign to stop relying on Preface (and yes, YOCaml was already more widely used than Preface), move to OCaml 5.x and take advantage of user-defined-effects and support dynamic dependencies. In other words, YOCaml 2.0.0 is not at all compatible with version 1…

Plugins and runtimes

The aim of YOCaml is to be very generic and to allow users to bring their own dependencies, but we've taken the opportunity to release it with several plugins and runtimes so that it can be used directly.

  • Runtimes

    A Runtime is an ‘execution context’ and generally exposes the primitive used to execute a YOCaml program. YOCaml 2 is bundled with 3 Runtimes:

    • Yocaml_unix: the default runtime, whose preview server is implemented on top of the brand new httpcats!
    • Yocaml_eio: a runtime iso to Unix but based on eio and whose preview server is described by cohttp_eio.
    • Yocaml_git: a parameterised runtime for generating a site directly in a git repository, which can be served, for example, by a Mirage (unipi), very well documented in this excellent article by @dinosaure!
  • Plugins
    • Yocaml_cmarkit provides a convenient API (via YOCaml) for converting Markdown files to HTML via the excellent cmarkit library.
    • Yocaml_omd provides a convenient API (via YOCaml) for converting Markdown files to HTML via the excellent OMD library (but we recommend yocaml_cmarkit).
    • yocaml_yaml provides a convenient API (via YOCaml) for reading Yaml via the excellent library ocaml-yaml
    • yocaml_otoml provides a convenient API (for YOCaml) for reading TOML via the excellent library Otoml
    • yocaml_mustache provides a convenient API (via YOCaml) for using Mustache as a template language via the excellent library ocaml-mustache
    • yocaml_jingoo provides a convenient API (via YOCaml) for using Jingoo as a template language via the excellent library jingoo
    • yocaml_syndication that gives tool to generate feeds ([Atom](https://en.wikipedia.org/wiki/Atom_(web_standard)), RSS 1 and 2 and OPML). The library is inspired by Syndic but does not depend directly on it.

A final word

YOCaml 2 was mainly written by xhtmlboi, helped by gr-im, mspwn and dinosaure with occasional support from maiste. It has already been used experimentally in a number of small projects:

You will also find extensively documented examples in the examples directory.

To conclude, we find (not very objectively) that YOCaml is a lot of fun to use, and it's very cool to make your site using as much OCaml as possible.

Happy Hacking!

oepub 0.1.0 : A library to parse epub files

EruEri announced

I humbly announce oepub a small library to parse epub files and to some extend create a list of chapters from the epub archive.

You can find the repository at Codeberg - Oepub

ppx_deriving_router — type safe routing for Dream and Melange

Andrey Popp announced

It's my pleasure to announce a new ppx for deriving Dream routers based on variant type declarations, ppx_deriving_router.

A small example. First we define routes (the signature showcases the generated code):

module Pages : sig
  ...

  val href : t -> string
  (** generate URL from the route *)

  val http_method : t -> [ `DELETE | `GET | `POST | `PUT ]
  (** HTTP method for the route *)

  val handle : (t -> Dream.handler) -> Dream.handler
  (** create a route handler *)
end = struct
  open Ppx_deriving_router_runtime.Primitives

  type t =
    | Home [@GET "/"]
    | About
    | Hello of { name : string; repeat : int option } [@GET "/hello/:name"]
    [@@deriving router]
end

Then we describe how we handle each route:

let handle =
  Pages.handle (fun route _req ->
      match route with
      | Home -> Dream.respond "Home page!"
      | About -> Dream.respond "About page!"
      | Hello { name; repeat } ->
          let name =
            match repeat with
            | Some repeat ->
                List.init repeat (fun _ -> name) |> String.concat ", "
            | None -> name
          in
          Dream.respond (Printf.sprintf "Hello, %s" name))

let () = Dream.run ~interface:"127.0.0.1" ~port:8080 handle

Using generated Pages.href function we can generate URLs for routes:

let () =
  assert (Pages.href Home = "/");
  assert (Pages.href About = "/about");
  assert (Pages.href (Hello { name = "world"; repeat = None }) = "/hello/world");
  assert (Pages.href (Hello { name = "world"; repeat = Some 3 }) = "/hello/world?repeat=3")

The URL matching is done by routes library.

There's also support for routes that track their response types and the ppx automatically derives JSON encoders and decoders for them (by using melange-json.ppx).

On top of that a separate ppx is provided for Melange which allows to construct type safe HTTP clients (route defintions are shared between server and client).

Happy hacking!

Mica, a PPX that automates differential testing for OCaml modules

Ernest Ng announced

I'm delighted to announce the initial release of Mica, a PPX deriver that automates differential testing for a pair of OCaml modules implementing the same signature. Users annotate module signatures with the directive [@@deriving mica], and at compile-time, Mica derives specialized property-based testing (PBT) code that checks if two modules implementing the signature are observationally equivalent. (Under the hood, Mica uses Jane Street's Core.Quickcheck PBT library.)

Mica was presented at the OCaml Workshop '24 (paper) and the ICFP '23 Student Research Competition (poster).

Note: Mica is currently a research tool and should not be used in production code, although contributions are very welcome!

Mica is available on Opam:

opam update 
opam install ppx_mica

(OCaml 5.1 or newer is required.)

Docs are available here, and a simple web app demo-ing Mica is available here.

Simplified Android cross-compiler with DkML

jbeckford announced

DkML has had a cross-compiler for years, but I have cleaned it up so that it is much easier to use for Android developers. It now works with a regular opam installation in a custom repository. Also included are patches to the OCaml compiler to work with Android NDK 21+ (currently Google is at NDK 27).

Try it out if you do Android development … just copy-and-paste the instructions below … but please read the notes and cautions below. And if you are still interested in Android development, tell me so I can decide if I'll merge the packages into the regular opam repository.

Trimmed slightly from the dkml-compiler Quick Start:

  • Docker container is used below for Windows and macOS users, and because it is easy to get the Android NDK from CircleCI.
  • Apple Silicon does not support 32-bit. The net effect is that Apple Silicon users cannot cross-compile android_arm32v7a.
$ docker run -it --rm cimg/android:2024.10.1-ndk

# Install opam if you don't have it
~/project$ sudo apt-get update && sudo apt-get install build-essential curl git patch rsync unzip -y
~/project$ echo /usr/local/bin | sudo bash -c "sh <(curl -fsSL https://opam.ocaml.org/install.sh) --version 2.2.1"

# Initialize opam if you haven't already. No sandboxing is needed in containers.
~/project$ opam init --cli=2.1 --no-setup --bare --disable-sandboxing

# Two Android options to set. ANDROID_PLATFORM is the minimum API level ("targetSdkVersion" in the Android manifest)
~/project$ opam var --cli=2.1 --global ANDROID_NDK=/home/circleci/android-sdk/ndk/27.1.12297006
~/project$ opam var --cli=2.1 --global ANDROID_PLATFORM=android-34

# PICK ONE: Android arm64-v8a switch
~/project$ opam switch create android34-ndk27-arm64-v8a --cli=2.1 \
  --packages dkml-base-compiler,dkml-host-abi-linux_x86_64,dkml-target-abi-android_arm64v8a,ocamlfind,conf-dkml-cross-toolchain \
  --repos default,diskuv-4d79e732=git+https://github.com/diskuv/diskuv-opam-repository.git#4d79e732

# PICK ONE: Android armeabi-v7a switch. You will need a 32-bit C/C++ compiler.
~/project$ sudo apt-get install gcc-multilib g++-multilib -y
~/project$ opam switch create android34-ndk27-armeabi-v7a --cli=2.1 \
  --packages dkml-base-compiler,dkml-host-abi-linux_x86,dkml-target-abi-android_arm32v7a,ocamlfind,conf-dkml-cross-toolchain \
  --repos default,diskuv-4d79e732=git+https://github.com/diskuv/diskuv-opam-repository.git#4d79e732

# PICK ONE: Android x86_64 switch
~/project$ opam switch create android34-ndk27-x86_64 --cli=2.1 \
  --packages dkml-base-compiler,dkml-host-abi-linux_x86_64,dkml-target-abi-android_x86_64,ocamlfind,conf-dkml-cross-toolchain \
  --repos default,diskuv-4d79e732=git+https://github.com/diskuv/diskuv-opam-repository.git#4d79e732

# THEN: Get and cross-compile your source code. Here we use Dune and assume 'android34-ndk27-arm64-v8a'
~/project$ opam install --cli=2.1 --switch android34-ndk27-arm64-v8a dune
~/project$ git clone https://github.com/avsm/hello-world-action-ocaml hello
~/project$ cd hello
~/project/hello$ opam exec --cli=2.1 --switch android34-ndk27-arm64-v8a -- \
  dune build -x android_arm64v8a world.exe

~/project/hello$ file _build/default*/world.exe
_build/default.android_arm64v8a/world.exe: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, with debug_info, not stripped
_build/default/world.exe:                  ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=1731ad9ce0fdeff69df28af0b1217e843eabe26e, for GNU/Linux 3.2.0, with debug_info, not stripped

# You can also directly use the ocamlfind -toolchain

~/project$ opam exec --cli=2.1 --switch android34-ndk27-arm64-v8a -- \
  ocamlfind ocamlc -config-var native_c_compiler
gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC  -D_FILE_OFFSET_BITS=64

~/project$ opam exec --cli=2.1 --switch android34-ndk27-arm64-v8a -- \
  ocamlfind -toolchain android_arm64v8a ocamlc -config-var native_c_compiler
/home/circleci/android-sdk/ndk/27.1.12297006/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android34-clang -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC  -D_FILE_OFFSET_BITS=64

DkML supports three out of the four supported Android ABIs. The three ABIs (all but x86) were chosen based on statistics for a large game on Aug 29, 2023:

Arch Percent
arm64-v8a 68.66
armeabi-v7a 30.38
x86_64 0.71
x86 0.26

and also Google's recommendation:

Note: While 64-bit-only devices will grow in popularity with phones joining Android Auto in this group, 32-bit-only devices will continue to be important for Android Go, Android TV, and Android Wear. Please continue supporting 32-bit ABIs; Google Play will continue serving 32-bit apps to 32-bit-only devices.

Finally, a word of CAUTION. The Android cross-compiler can never use OCaml 5+ because OCaml 5 will never bring back the 32-bit instruction set. That means if you don't want to drop a large percent of your users or drop new Android categories over the next five (?) years, you will have a critical dependency on DkML.

Other OCaml News

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe to the caml-list.