OCaml Weekly News

And to conclude the 3.9.x series:

The fix to sendfile in 3.9.2 was not quite enough so here is the last part of the fix. It brings compatibility with filesystems where sendfile is not available, in particular when ecryptfs is used.

We’re happy to announce the release of Dune 3.10.0. It comes with some internal fixes as well as some interesting features:

• some changes in dune describe that will allow a better implementation of opam-dune-lint to check the consistency between library and package dependencies
• more commands made available under dune show for a more consistent command line interface

Here is the full changelog:

Hi, I am pleased to announce that decimal 1.0.0 has been released on opam-repository: https://ocaml.org/p/decimal/1.0.0

(decimal is a somewhat-complete port to OCaml of the Python decimal module, contributions welcome to port more functionality!)

There is a breaking change in this release. We have removed the ?eng:bool parameter from the Decimal.to_string function, and replaced it with a more general-purpose ?format parameter: https://yawaramin.github.io/ocaml-decimal/decimal/Decimal/index.html#val-to_string

Now the fun part: @zbaylin has added a new ppx_decimal which enables a convenient literal syntax for decimals: e.g. 1.1m is automatically desugared to Decimal.of_string "1.1". Thanks Zach!

Also thanks @anmonteiro for showing me a dune trick to help everything build successfully in Opam CI.

Enjoy!

The lacking state of OCaml debuggers has been a rather hot topic recently, so I’m particularly excited to announce the release of earlybird 1.2.0.

Many of you might be familiar with OCaml earlybird: a debugger which supports Debug Adapter Protocol, allowing it to be used directly from VS Code’s builtin debugging GUI (among possibly other IDEs). This phenomenal tool was developed by @hackwaly.

Unfortunately, for a long time it wasn’t updated to support newer versions of OCaml and fell into despair. Since there’s still (or even moreso) need for OCaml debugging tools, it would be a real shame if earlybird was dead forever. Therefore, recently I volunteered to pick up its maintenance and get it up and running again.

I’m hereby announcing the release of earlybird 1.2.0 which supports OCaml 4.12, 4.13, 4.14 and 5.0. Moreover, I extended the VS Code OCaml Platform extension (since version 1.13) to directly be able to launch earlybird: https://github.com/ocamllabs/vscode-ocaml-platform/#debugging-ocaml-programs-experimental. This brings the handling of a DAP for OCaml into the same extension which handles the LSP for OCaml. Instructions are available at the above link!

Note that the OCaml Platform integration is still experimental. I invite everyone to try out the new release of earlybird via the new VS Code integration. The main purpose of this announcement is to get real-world testing for both sides and hopefully also get more people interested/involved.

Beware that this revival of earlybird comes purely out of my free time, so don’t expect any big developments now. So far I did what I could to get earlybird reasonably working again and I’ll do my best to keep it from degrading. I think it’s still valuable to have earlybird in the ecosystem until maybe something better comes along. However, if working on earlybird sparks anyone’s interest, I’ll gladly accept bug fixes and improvements, all towards a better OCaml ecosystem.

A final note about earlybird: it uses the (undocumented) protocol of ocamldebug to communicate with a bytecode executable. Thus, it comes with all the limitations that ocamldebug and its protocol have. Notably, don’t get too excited about OCaml 5.0 support: debugging is supported only until a domain is spawned (https://github.com/ocaml/ocaml/pull/11065#pullrequestreview-975383464).

The release of OCaml 5.1.0 is imminent. As a final step, we are publishing a release candidate to check that everything is in order before the release in the upcoming week(s).

If you find any bugs, please report them on OCaml’s issue tracker.

Compared to the beta release, this release contains one safe runtime fix, and two configuration tweaks.

The full change log for OCaml 5.1.0 is available on GitHub A short summary of the changes since the beta release is also available below.

With the help of many, but mostly @sabine, @professor.rose and @tmattio, I’ve attempted to revamp two of the “getting started” tutorials from ocaml.org

This PR contains the state of this work: https://github.com/ocaml/ocaml.org/pull/1431

In our opinion, the getting started tutorials should include three mandatory tutorials

1. Install
2. A Tour of OCaml
3. How to Write an OCaml Program

The fourth, dealing with installation options on Windows, being optional for apparent reasons.

This PR only covers parts 2 & 3 because they were mostly written from scratch. We’re also working on an update of the Install tutorial, but since it is not a rewrite, it felt more natural to put it in a separate PR. Although it is not a draft, I must say that this is an early version of the text. I hope you will understand.

Here is the main learning idea. That’s a breadth-first traversal of the rabbit holes.

A Tour of OCaml provides an overview of the basic language features. Here, the goal is not to address more topics but to reduce to the core of what can be learnt fast, only using utop, will always be useful later and helps building an overall view of OCaml.

How to Write an OCaml Program has the same approach, except instead of using utop, the reader should write files and command lines. It’s a compiler story, while the previous one was an interpreter story.

It is early August. In the northern hemisphere, we should enjoy the sea, the mountains, our friends or culture. I will be hiking soon. Reviewing tutorials is probably not the greatest summer read (Off topic: I’m in Martha Wells Murderbot Diaries and enjoying it). But if you’re in the mood for this, we’d love your feedback.

I am pleased to announce the first alpha release of Simple_httpd, available on github and opam. It is a library to produce web server and sites.

• Documentation: https://raffalli.eu/simple_httpd/simple_httpd
• Github : https://github.com/craff/simple_httpd

WARNING: currently we need the latest master of ocaml-ssl. It requires Linux and OCaml 5.0, if you have this, you can install with:

  opam pin add https://github.com/savonet/ocaml-ssl#master -k git
  opam pin add https://github.com/craff/simple_httpd -k git

And test the template site (very simple, as it is an empty shell to start from) with

  cd source_dir/template
  dune exec -- ./server.exe --log-folder ./log

The template is also documented at https://raffalli.eu/simple_httpd/simple_httpd/template.html

It aims at

• Being simple to use and rather complete (support ssl, chaml: an equivalent of php, but in OCaml and compiled, status and statistics, authentication, cookies, …).

• Being fast: our latencies and number of requests per seconds are very good, thanks to using linux epoll, eventfd, OCaml’s effects and domains, … The first page of the documentation shows some graphics, but here is a small comparison of latencies for a small 1kb file:

                   min        mean      50%       90%       95%      99%      max
    Simple_httpd  79.478µs 242.006µs 237.576µs 294.802µs 305.68µs 329.352µs  3.049ms
    Nginx        170.551µs 328.904µs 309.577µs 384.313µs 400.51µs 482.987µs 42.003ms
    Apaches      196.321µs 466.439µs 452.265µs 545.121µs 590.05µs 913.527µs  6.372ms
  

  And a small chaml (our equivalent of php) against php-fpm from apache and nginx:

    Simple_httpd 146.944µs 285.044µs 280.552µs 341.175µs 356.497µs 507.305µs  8.069ms
    Nginx        411.151µs 793.437µs 653.131µs 796.300µs 882.268µs     2.9ms 44.504ms
    Apache       688.765µs   2.342ms 950.647µs   1.201ms   1.321ms   5.844ms   1.171s
  

  These were obtained with vegeta at 1000 requests/s. Simple_httpd offers much more stable latencies under charge than nginx or apache.

  If you want your own measurments, you need to setup nginx/php on ports 7080 and 7443, an apache/php on port 80 and 443. Then, you can run [./bench.sh] from the [tests] folder of the source tree. I would be happy to have measurments for a big server with more than 20 cores.

• Currently only linux is supported.

Help, comments, bug reports, … would be greatly appreciated, as this is alpha release, it is time for you to propose change in the design of the library.

My website https://raffalli.eu and therefore simple_httpd documentation are powered by simple_httpd (do we name this bootstrap ;-) ?

It’s be interesting to compare simple_httpd with tiny_httpd + moonpool (see: https://github.com/c-cube/tiny-httpd-moonpool-bench/) :-). I wonder how much the two diverged, I wasn’t expecting simple_httpd to use epoll!

Very well indeed. Here is both latencies at 2500 req/s using vegeta:

• Tiny httpd+moonpool

    Latencies     [min, mean, 50, 90, 95, 99, max]  97.845µs, 197.1µs, 194.848µs, 251.333µs, 265.961µs, 470.347µs, 1.621ms
  

• Simple_httpd

    Latencies     [min, mean, 50, 90, 95, 99, max]  98.27µs, 176.478µs, 169.674µs, 229.443µs, 243.888µs, 279.937µs, 2.008ms
  

For the number of requests per seconds with wrk, we are at 139675 for tiny+moonpool and 167819 for simple_httpd.

Tiny is a bit better for the worst case. Simple_httpd is better in average and for all quantile. I think I am missing a preemptive scheduler that I could get if there where some way to perform an effect periodically in OCaml.

Moonpool is a very good idea actually to get a preemptive scheduler with domains !

I should also say that simple_httpd routing is now looking at the Host field, address and port, not only the path and method, It has logging by type and level (not just on/off) and maybe a few other extra feature that do not come to my mind now but have an impact for an hello request.

On the side of memory, VSZ=1032908Ko RSS=27492Ko for simple_https and VSZ=999288Ko RSS=401804Ko for tiny All this is very reasonnable but threads use much more resident memory.

I am happy to announce the release of a small tool I’ve implemented for explaining how a LR parser works for my students. The tool is called cairn, and lets you visualize the step by step building of the parsing tree of the input of a parser, and can be used with any grammar written with menhir (basically, you provide menhir generated modules and cmly file to a functor from cairn and you’re all set). It can either generate a text file or launch a small terminal user interface to explore the execution of the parser step by step. It is probably only useful on small inputs, and mainly for teaching purposes, but I thought it might be useful to share.

The homepage of the project is there, and it can be installed directly through opam with :

opam install cairn

A temporary list of accepted presentations is available:

https://icfp23.sigplan.org/home/ocaml-2023#event-overview

We are in discussion with the ML workshop to move some of these talks to the ML workshops, so the program may change slightly in the following weeks.

in November there’ll be the next retreat taking place in Marrakesh, Morocco. Happy to see old and new faces there, everyone is welcome (an interest in OCaml/MirageOS is nice to bring with you).

Further information at https://retreat.mirage.io

In case you have some questions, don’t hesitate to reach out to me.

And speaking of news, first part of blog post Kcas: Building a lock-free STM for OCaml is now online.