OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of September 03 to 10, 2024.

Table of Contents

Oxidizing OCaml — an update

Diana Kalinichenko announced

Hi everyone! Last year, we made a series of blogposts describing our plans to introduce Rust-like type system features to OCaml (see here). Now, we are sharing updates on everything we've done since last year for ICFP 2024. Please read our blogpost and check out our compiler extensions at our GitHub!

Toy Autograd Engine in OCaml with Apple Accelerate Backend

John Jewell announced

I have been venturing to learn a new language and I landed on OCaml after hearing a few interesting talks from Jane Street. I just made public a toy autograd engine in OCaml with an Apple Accelerate backend if anyone is interested:

https://github.com/jewelltaylor/camlgrad

I would really appreciate any feedback in terms of the OCaml code that I wrote so that I can improve. If anyone is willing to quickly take a look it would mean a lot :slight_smile:

New release of cppo, with multi-line macros and higher-order macros

François Pottier announced

It is my pleasure to announce a new release of cppo (v1.7.0) with two new features.

  • The new syntax #def ... #enddef allows a macro definition to span several lines, without backslashes. This syntax allows macro definitions to be nested.

    #def repeat_until(action,condition)
      action;
      while not (condition) do
        action
      done
    #enddef
    
  • A parameterized macro can take a parameterized macro as a parameter: this is a higher-order macro.

    #define TWICE(e)          (e + e)
    #define APPLY(F : [.], e) (let x = (e) in F(x))
    let forty_two =
      APPLY(TWICE,1+2+3+4+5+6)
    

For more details, please see the documentation.

OCamlPro's contributions to the 2024 ICFP in Milan

OCamlPro announced

Today, a quick head's up about our contributions to this year's International Conference on Functional Programming which is unraveling right now in Milan!

This year, our team presents two topics:

  • "Snapshottable Stores": A generic and efficient data structure for the implementation of backtracking algorithms, used particularly in automatic theorem provers and type checkers. This implementation in OCaml will soon be available on opam.
  • A presentation on opam, detailing the contents of the latest major release 2.2, which was released in July, as well as how the opam team operates.

Be sure to checkout the event, there are plenty of great presentations and video replays!

Until next time, which will be sooner than later with another one of our Flambda2 Snippets,

Kind regards, The OCamlPro Team

Flambda2 Ep. 3: Speculative Inlining, by OCamlPro

OCamlPro announced

As promised in our previous post about OCamlPro's contributions to this year's International Conference of Functional Programming, we back again with a new entry in our Flambda2 Snippet blog series!

Flambda2 Ep. 3: Speculative Inlining covers inlining in general as well as how our compiler handles it. We go in detail about how Speculative Inlining allows more significant optimisations to take place.

This blog entry is key for a smooth read of our next article which will cover Upwards and Downwards Traversals in Flambda2.

Happy to say that it's already quite far down the release pipeline!

Frustrating Interactions with the OCaml Ecosystem while developing a Synthesizer Library

Steve Sherratt announced

https://www.gridbugs.org/frustrating-interactions-with-the-ocaml-ecosystem-while-developing-a-synthesizer-library/

Last year I made a synthesizer library in OCaml and had some struggles using Dune and Opam, and also ran into several issues with libraries. I wrote a blog post about all the problems I encountered.

Cmdlang - Yet Another CLI Library (well, not really)

Mathieu Barbin announced

I hope you had a nice summer! Mine took an unexpected turn when, roughly at the same time, I wrote my first cmdliner subcommand and heard about climate for the first time. My experience with OCaml CLI so far had been centered around core.command.

When I read climate's terminology section and how it defines Terms, Arguments, and Parameters, something clicked. Seeing how climate's API managed to make positional and named arguments fit so nicely together, I thought: "Wow, for the first time, it seems I'll be able to write a CLI spec on a whiteboard without referring to some code I never seem to get right (I am looking at you, core.command's anonymous arguments)."

I got quite excited and thought: "Can I switch to climate today?" But reality checked: it's not on opam yet, still under construction, I'm not sure what the community will do, etc.

Implementing my own engine for an API resembling climate felt like a wasted effort, knowing about the work happening in climate. Still, having a 'a Param.t, 'a Arg.t, and 'a Command.t type that I would get to know and love felt too good to pass up.

I stared at the climate types for a while, and filled with happy thoughts about a bright CLI future, it occurred to me: can I use an API like climate but compile it down to existing libraries such as cmdliner or core.command? (and climate too!). I wrote down the following types:

Climate

'a Param.t     -> 'a Climate.Arg_parser.conv
'a Ast.Arg.t   -> 'a Climate.Arg_parser.t
'a Command.t   -> 'a Climate.Command.t

Cmdliner

'a Param.t     -> 'a Cmdliner.Arg.conv
'a Arg.t       -> 'a Cmdliner.Term.t
'a Command.t   -> 'a Cmdliner.Cmd.t

core.command

'a Param.t     -> 'a core.Command.Arg_type.t
'a Arg.t       -> 'a core.Command.Param.t
unit Command.t -> core.Command.t

… which I interpreted as stating the following theorem:

There exists an abstraction to encode OCaml CLIs that lives in the intersection of what's expressible in other well established libraries.

"One EDSL to command them all," so to speak. I couldn't resist the temptation to build actual terms for these types. That gave birth to cmdlang.

As a test, I switched one of my projects to cmdlang, with cmdliner as a backend. I liked the changes I made in the process. The 1-line bin/main.ml is now the only place that specifies which backend I want to use; the rest of the code is programmed solely against the cmdlang API. This means I'll be able to easily experiment with compiling down to climate in the future.

I am not against the multiplicity of solutions in general, but I tend to feel uneasy when incompatible libraries emerge, partitioning the ecosystem. As a community, we know too many examples of this. In this instance, I want to call the core.command vs cmdliner situation a … cli-vage.

I don't see my work on cmdlang as competing with these other libraries. Quite the contrary, it makes it easier for me to experiment with them without much changes while exploring the subject of CLI in general. Also, as a library author, if you wish to expose CLI helpers to your users, a library like cmdlang will give you a pleasant way to do so, as you can express your helpers with it, knowing your users will have the choice to translate them to the backend of their choice.

Before writing this post, I had a very pleasant chat with @gridbugs. I want to make it clear that I do not think cmdlang is competing with climate either. I think climate is a very promising library and I believe it will, in due time, deliver auto-completion to many - this has been a highly anticipated feature within the community. I wish to dedicate the initial work that I did on cmdlang to @gridbugs due to the impactful influence climate had on my work, and how it helped me improve my general understanding of declarative CLI libraries.

These are very early days for cmdlang. There are still areas I am fuzzy on, and I haven't really validated the whole design yet. I have put some thoughts in this Future Plans page. One thing that I did not initially include there would be to explore the feasibility of writing a mini-compiler for cmdlang targeting stdlib.arg as a runner. I am not sure how much you'd end up restricting cmdlang for this to work. I thought that'd be a fun project to tackle at a future point, as it would make a nice addition to the overall architecture of the project.

I'd welcome your input to help me shape the future of cmdlang if you have an interest in this project.

Thanks for reading!

zarr v0.1.0

zoj613 announced

Hi everyone, I'd like to announce the first release of zarr, an Ocaml implementation of the Zarr version 3 storage format specification for chunked & compressed multi-dimensional arrays, designed for use in parallel computing.

terminology-hierarchy.excalidraw.png

why?

The project was mainly inspired by the lack of functional programming language implementations of this specification as shown in this implementations table. Since I have been learning OCaml these past few months I figured I'd take on the challenge of producing the first functional programming implementation of Zarr, and it was a great learning experience!

Features

  • Supports creating n-dimensional Zarr arrays and chunking them along any dimension.
  • Compress array chunks using a variety of supported compression codecs.
  • Supports indexing operations to read/write views of a Zarr array.
  • Supports storing arrays in-memory or the local filesystem. It is also extensible, allowing users to easily create and use their own custom storage backends. See the example implementing a Zip file store for more details.
  • Supports both synchronous and concurrent I/O via Lwt and Eio.
  • Leverages the strong type system of Ocaml to create a type-safe API; making it impossible to create, read or write malformed arrays.
  • Supports organizing arrays into hierarchies using Groups.

Example

Below is a demo of the library's API for creating, reading and writing to a Zarr hierarchy.

open Zarr
open Zarr.Metadata
open Zarr.Node
open Zarr.Codecs
open Zarr.Indexing
open Zarr_sync.Storage
(* opens infix operators >>= and >>| for monadic bind & map *)
open FilesytemStore.Deferred.Infix

let store = FilesystemStore.create "testdata.zarr" in
let group_node = GroupNode.of_path "/some/group" in
FilesystemStore.create_group store group_node;
let array_node = ArrayNode.(group_node / "name");;
(* creates an array with char data type and fill value '?' *)
FilesystemStore.create_array
  ~codecs:[`Transpose [|2; 0; 1|]; `Bytes BE; `Gzip L2]
  ~shape:[|100; 100; 50|]
  ~chunks:[|10; 15; 20|]
  Ndarray.Char 
  '?'
  array_node
  store;
let slice = [|R [|0; 20|]; I 10; R [||]|] in
let x = FilesystemStore.read_array store array_node slice Ndarray.Char in
(* Do some computation on the array slice *)
let x' = Zarr.Ndarray.map (fun _ -> Random.int 256 |> Char.chr) x in
FilesystemStore.write_array store array_node slice x';
let y = FilesystemStore.read_array store array_node slice Ndarray.Char in
assert (Ndarray.equal x' y);

Installation

The library comes in several flavors depending on the synchronous / asynchronous backend of choice. To install the synchronous API, use

$ opam install zarr-sync

To install zarr with an asynchronous API powered by Lwt or Eio, use

$ opam install zarr-lwt
$ opam install zarr-eio

The documentation can be found here and the source code there

I'm happy to answer any questions regarding the library and more than welcome suggestions for improvements (especially performance!), issue reports as well as PR's.

Brr 0.0.7

Daniel Bünzli announced

There'a new release of Brr, an ISC licenced toolkit for programming browsers with the js_of_ocaml compiler.

This release has some changes to support work being done for wasm_of_ocaml; thanks to @vouillon for his patches. There are also other small fixes and additions, consult the release notes for the details and thanks to all the contributors.

A big thanks to my donators for supporting my work. I welcome the (not so[^1]) new donator Tarides.

Home page, Docs and manuals or odig doc brr

Install: opam install brr (PR)

Best,

Daniel

[^1]: Tarides has been generously donating for my work from the onset but used to do it via the Mirage organisation.

Ocsigen Server 6.0.0

Vincent Balat announced

We're delighted to announce a major new version of Ocsigen Server! This version 6.0.0 focuses on the use of Ocsigen Server as a library, without any configuration file, which is now much easier and brings the exact same features as the executable.

Example of use:

To add a Web server to your OCaml program, serving static files from directory `static`:

let _ = Ocsigen_server.start 
          [ Ocsigen_server.host [Staticmod.run ~dir:"static" ()]]

Install:

opam install ocsigenserver

Example of Dune file for this program:

(executable
 (public_name myproject)
 (name main)
 (libraries
  main
  ocsigenserver
  ocsigenserver.ext.staticmod))

Compile with:

dune build

Ocsigen Server can of course still be used as an executable taking its configuration from a file. This allows for non OCaml developers to use it and make their own configurations. It also makes it possible to distribute a binary version of your Web applications.

Ocsigen Server is build in modular and extensible way. The default opam packages comes with several extensions. In the example above, we are using Staticmod for serving static files. Other extensions makes it possible for example to configure redirections, to control the access to some sub-directory, to use a reverse proxy, to rewrite the request, compress the output etc.

The programming interface follows exactly the structure of the configuration file: instructions are tried in order until one generates a result, then some other instructions can be used to change the result (like compressing it or adding some headers).

Here is a more complex example of configuration:

let _ =
  Ocsigen_server.start
    ~ports:[`All, 8080]
    ~command_pipe:"local/var/run/mysite-cmd"
    ~logdir:"local/var/log/mysite"
    ~datadir:"local/var/data/mysite"
    ~default_charset:(Some "utf-8")
    [ Ocsigen_server.host
        ~regexp:"mydomain.com"
        [ Ocsigen_server.site ["subsite"]
            [ Accesscontrol.(
                if_
                  (and_
                     [ ip "122.122.122.122"
                     ; header ~name:"user-agent" ~regexp:".*FooBar.*"
                     ; method_ `POST ])
                  [forbidden] [])
            ; Authbasic.run ~realm:"myrealm"
                ~auth:(fun _u p -> Lwt.return (p = "toto"))
                ()
            ; Staticmod.run ~dir:"local/var/www/otherdir" () ]
        ; Ocsigen_server.site ["othersubsite"]
            [ Revproxy.run
                ~redirection:
                  (Revproxy.create_redirection ~full_url:false ~regexp:"(.*)"
                     ~keephost:true "http://localhost:8888/\\1")
                () ]
        ; Redirectmod.run
            ~redirection:
              (Redirectmod.create_redirection ~full_url:false ~regexp:"old(.*)"
                 "new\\1")
            ()
        ; Staticmod.run ~dir:"local/var/www/staticdir" ()
        ; Cors.run ~max_age:86400 ~credentials:true ~methods:[`POST; `GET; `HEAD]
            ~exposed_headers:
              [ "x-eliom-application"
              ; "x-eliom-location"
              ; "x-eliom-set-process-cookies"
              ; "x-eliom-set-cookie-substitutes" ]
            ()
        ; Deflatemod.run
            ~mode:
              (`Only
                [ `Type (Some "text", Some "html")
                ; `Type (Some "text", Some "javascript")
                ; `Type (Some "text", Some "css")
                ; `Type (Some "application", Some "javascript")
                ; `Type (Some "application", Some "x-javascript")
                ; `Type (Some "application", Some "xhtml+xml")
                ; `Type (Some "image", Some "svg+xml")
                ; `Type (Some "application", Some "x-eliom") ])
            () ] ]

In this example, the server defines one virtual host for domain mydomain.com. It will first check whether it is a request for directory subsite/, and if yes, will reject the request with 403 Forbidden if it is a POST request coming from user-agent FooBar at IP 122.122.122.122. If not, it will ask for a password before serving files from directory local/var/www/otherdir. Then we define another subsite othersubsite for which the requests will be transfered to another Web server running locally on port 8888, then rewrite the answer location header accordingly. Then, if the page is still not generated, the server will send a redirection if URLs starts with "old". Otherwise, it will try to serve files from directory local/var/www/staticdir. If the page has still not been found, a 404 Not found will be sent, otherwise, some CORS headers will be added, and the result will be compressed before being sent.

Compile this example with the following dune file:

(executable
 (public_name myserver)
 (name main)
 (libraries
  ocsigenserver
  ocsigenserver.ext.staticmod
  ocsigenserver.ext.authbasic
  ocsigenserver.ext.extendconfiguration
  ocsigenserver.ext.outputfilter
  ocsigenserver.ext.cors
  ocsigenserver.ext.accesscontrol
  ocsigenserver.ext.deflatemod
  ocsigenserver.ext.redirectmod
  ocsigenserver.ext.revproxy
 ))

Eliom and Ocsigen Start have also been updated for being used without configuration file and will be released very soon.

dream-html and pure-html

Yawar Amin announced

[ANN] dream-html & pure-html 3.6.1, 3.6.2

A double announcement:

3.6.1: when in XML rendering mode, correctly render empty-value attributes as having an empty string value. Thanks to @jonsterling !

3.6.2: automatically switch to XML rendering mode when rendering SVG and MathML tags inside HTML rendering mode.

Advanced Code Navigation coming to OCaml-LSP

PizieDust announced

Jump to Target

Currently, the standard LSP protocol only allows for generalized code navigation (`goto definition`, `goto declaration`, `goto implementation`, `goto type-definition`), which is not very useful when it comes to precise movements.

Coming to OCaml-lsp soon will be the ability to jump from one point in your code to another based on Merlin's Jump command.

Implementing this functionality took a bit of thinking as we wanted a solution that works for all supported editors (Vscode, Emacs and Vim) without any additional specific client implementations. We used a combination of call actions plus the LSP showDocumentRequest to move the cursor to the interesting position.

The call actions display are contextual and will display only if it's relevant to the code under the cursor.

Here is a demo in VSCode.

618bfd77db1a89256eeeaf69b6d3817dbfdd8dc0.gif

Other OCaml News

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe to the caml-list.