OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of October 13 to 20, 2020.

Table of Contents

Dialo is hiring frontend and backend OCaml developers (Remote)

Wojtek Czekalski announced

Dialo is an early stage company with an experienced founding team. Assembling a team that consists of the best and brightest is our top priority. In the immediate term we are building a visual programming language for conversational AI. Our long term vision is that personalized contact we are enabling will cause deeper relationships between users and businesses and turn all interactions into a unified long term customer journey.

The work is quite demanding when it comes to both ideation and implementation. We are aiming to provide a room for growth both technically and/or as a leader. For current open source maintainers we are willing to sponsor your work on OSS for 20% of time.

We use OCaml for frontend and backend (along with Python for machine learning, natural language processing). We are hiring people for different positions. Both people with extensive experience and newcomers are encouraged to apply. We try to find the sharpest people rather than checking boxes with particular skills.

The official job posting: https://dialo.recruitee.com/o/software-developer-ocamlreason We are also hiring for two other (related) positions:

Progress 0.1.0

Craig Ferguson announced

I'm pleased to announce the first release of Progress, now available on Opam.

Progress is a small library for quickly defining and using progress bars in OCaml programs. It aims to provide the following:

  • support for rendering multiple progress bars simultaneously;
  • responds dynamically to changes in terminal size;
  • allows user-defined progress bar layouts.

Defining your own progress bars

The example animation above uses a pre-provided progress bar layout that should meet many needs (Progress_unix.counter), but it's fairly easy to re-define it ourselves using the low-level Progress.Segment API:

let counter filename =
  let proportion i = Int64.to_float i /. 1_000_000. in
  let open Progress in
  Segment.(
    list
      [
        const filename;
        Units.bytes of_pp;
        Progress_unix.stopwatch ();
        bar ~mode:`ASCII proportion;
        using proportion (Units.percentage of_pp);
      ]
    |> box_winsize ~fallback:80  (* Dynamically scale to window size *)
    |> periodic 100              (* Re-render once every 100 updates *)
    |> accumulator Int64.add 0L  (* Accumulate progress updates *))
  |> make ~init:0L

The Segment combinators are similar to those of general-purpose pretty-printing libraries (e.g. pp and fmt), but are equipped with extra logic for "stateful" segments and segments that can have dynamic width. Together, these make for a convenient way to express common patterns when pretty-printing progress bars. For instance, the stateful segment periodic seen above can be used to ensure that very frequent updates from a hot-loop do not result in too much time spent re-rendering the output.

The library is not yet feature-complete, but should still be reasonably useful :slightly_smiling_face: Happy hacking!

Brr 0.0.1, a toolkit for programming browsers

Daniel Bünzli announced

I'd like to announce the first release of Brr.

The TL; DR is:

If you are looking for a productive way to program browsers with js_of_ocaml but without ppx and ghost OCaml objects, give Brr a try.

The details:

Brr is a toolkit for programming browsers in OCaml with the js_of_ocaml compiler. It provides:

  • Interfaces to a selection of browser APIs.
  • Note based reactive support (optional and experimental).
  • An OCaml console developer tool for live interaction with programs running in web pages.
  • A JavaScript FFI for idiomatic OCaml programming.

Brr is distributed under the ISC license. It depends on Note and on the js_of_ocaml compiler and runtime – but not on its libraries or syntax extension.

Brr is essentially what I need to be productive for browser programming with js_of_ocaml: an obvious FFI with JavaScript objects as abstract data types without OCaml object phantom types and binding documentation precisely linking into MDN.

The OCaml console is the hack on the cake. In the past I often found it frustrating to have OCaml programs running in my webpages and be greeted with a JavaScript prompt in the browser dev tools. Quite a bit of polishing could be done on that though. Some of which should likely directly be done upstream in the toplevel machinery (e.g. identifier completion, a better toploop API and support for easy pretty printer installation). It would also be nice if we could cut down on js_of_ocaml's toplevel compilation times ;–)

Parts of Brr have been seriously dogfooded in the past but that new incarnation is largely untested for now and certain APIs might need adjustements. Early adopters should study actual binding coverage, expect glitches and little breakages in the future.

The Note reactive functionality was also seriously used in the past but Note itself needs a new design round and I don't have the ressources to do it right now, expect breakage, don't pay too much attention to it for now.

My thanks to the js_of_ocaml developers for the nice ocaml to javascript compiler and a special shootout to Hugo Heuzard for not getting mad at me when pinging him directly for questions.

Happy browser compatibility bug hunting,

gasche asked

It's not really released, but I'm curious about Note now: this is a new FRP library from you, the author of React (the FRP library for OCaml, not the Javascript framework of the same name).

Would you say a few words on why you went for a different library? My guess would be that React depends on runtime mechanisms (weak pointers) that are not well-supported in Javascript-lang; but even if the guess is right, I'm not sure what would be the impact on the API or properties of the library.

Daniel Bünzli replied

Would you say a few words on why you went for a different library?

Note is the result from seeing people (and myself) struggling to use ~React~/FRP "correctly" over the years.

Some of this, I largely attribute to ergonomic problems with the API. It's my hope for Note to address most of these points (one thing that still needs to be done is replace fix points by a simple lazy infinitesimal delay combinator).

I don't think I could have made all these changes in React itself so I found it better to start a new library. Also I lost the trademark on the name :–)

Note also tries to provide a much simpler implementation. React's implementation was based on the FrTime Phd thesis. It's quite subtle and involved and, as you suggested, uses weak pointer. Note tries to avoid them since those are not available in the browser (but you have things like MutationObservers which I use as gc in Brr's Note-based reactive dom support).

However not using weak pointers has a semantic uncleanness cost whose impact I'm unsure yet – without discipline from the programmer it may lead to subtle and hard to track bugs when the reactive graph changes dynamically, which I'm a bit wary of.

When my brain dumped Note I wrote a few more technical points in the readme you can read them here.

New release of Conduit

Calascibetta Romain announced

Conduit 3.0.0

Hello everyone,

We're glad to announce the new release of conduit, a framework that allows to abstract over transfer protocols. One of its main advantages is allowing the implemententation of free-dependencies protocols.

Introduction

There are several ways to abstract over an implementation in OCaml. However, those solutions are often lost deep in the stack of protocols and allowing the user to choose the implementations of the sub-procotols implies growing complexity as we move up through the stack. (For example, allowing to abstract over the implementation of the TLS protocol from the implementation of the HTTP protocol)

One of those solutions, the functors, can rapidly become a hellish nightmare for the end-user. This is especially true in the case of MirageOS, which literally wants to abstract over everything!

This is why Conduit was implemented: it aims to provide to the user a cleaner abstraction mechanism which would allow the protocol developers to get rid of most of the responsibilities concerning the choice of sub-protocols (Like which TLS implementation use between OpenSSL or our great ocaml-tls library), while giving the end-users an easy way to compose the protocols of their choice and inject them in the stack via conduit.

Usage of Conduit

Such a framework allows us to separate the logic of a protocol from underlying implementation needed to communicate with a peer. The distribution of Conduit comes with a simple tutorial which explains step by step how to implement a ping-pong client & server and, most importantly, how to upgrade them with TLS.

With Conduit, we ensure the compatibility with MirageOS (and specially mirage-tcpip) while being useful for others. Of course, Conduit is not mandatory to ensure this compatibility, but it helps us for higher libraries such as ocaml-git/Irmin or Cohttp.

Specific improvements

  • Abstract and destruct it!

    The most requested feature on the new version of Conduit is the ability to destruct the [Conduit.flow][conduit-flow]. The ability to abstract the protocol comes with the abstract type Conduit.flow. The new version permits to destruct it to a well-known value (such as an UNIX socket):

    let handler flow = match flow with
      | Conduit_lwt.TCP.T (Value file_descr) ->
        let peer = Lwt_unix.getpeername file_descr in
        ...
      | flow -> ... (* other kind of protocol *)
    
    let run =
      Cohttp_lwt_unix.serve ~handler
        { sockaddr= Unix.inet_addr_loopback }
    
  • The dispatch of the protocol

    The second most interesting feature of Conduit is the full control over the dispatch between protocols by the end-user. From a concrete information such as an Uri.t, the end-user is able to describe how Conduit should choose the protocol (and with which value it should try to initiate the connection):

    let my_tls_config = Tls.Config.client ...
    
    let connect uri =
      let edn = Conduit.Endpoint.of_string
        (Uri.host_with_default ~default:"localhost" uri) in
      let resolvers = match Uri.scheme uri with
        | Some "https" ->
          let port = Option.value ~default:443 (Uri.port uri) in
          Conduit_lwt.add
            Conduit_lwt_tls.TCP.protocol
            (Conduit_lwt_tls.TCP.resolve ~port ~config:my_tls_config)
            Conduit.empty
        | Some "http" | None ->
          let port = Option.value ~default:80 (Uri.port uri) in
          Conduit_lwt.add
            Conduit_lwt.TCP.protocol
            (Conduit_lwt.TCP.resolve ~port)
            Conduit.empty in
      Conduit_lwt.resolve ~resolvers edn >>= fun flow ->
      ...
    
  • An explicit way to launch a server

    Conduit comes with a new API for the server-side, where everything becomes explicit: no dispatch, no hidden choice. It proposes now a simple function to start the usual server loop:

    let run handler =
      Conduit_lwt.serve ~handler
        Conduit_lwt.TCP.service
        { Conduit_lwt.TCP.sockaddr= Unix.(ADDR_INET (inet_addr_loopback, 8080)
        ; capacity= 40 }
    

Reverse-dependencies

Conduit is used by many libraries (~150 packages) and we spend 2 months to track this breaking-change. Currently, it's mostly about Cohttp and Irmin and both have a PR according the new version of Conduit. These packages will be released as soon as we can with the new version of Conduit.

Conclusion

Conduit is a piece required by many libraries but nobody really uses it. This new version wants to replace and redefine more concretely what Conduit is. The update is huge for us but small for people where we tried to keep the same global idea of the abstraction.

I would like to thank many people (MirageOS core team, Cohttp peoples, some not so famous guys of the Reason/OCaml eco-system) who followed us on this deep development (and tried and iterated on our version). It does not change too much our world, but it paves the way for a better MirageOS/OCaml eco-system.

As a french guy, I just would like to say: Conduit est mort, Vive Conduit!

Easy cross compilation using esy

EduardoRFS announced

I've been working on this for a couple of months now, and now it is ready for an initial announcement of my tools to cross compiling OCaml and ReasonML Native.

https://github.com/EduardoRFS/reason-mobile

What it can do

Out of box it can cross compile most dune and topkg, packages available on opam for a couple of platforms, there is also patches for popular packages.

You can also compile opam packages by making an wrapper, like

https://github.com/mirage/mirage-crypto/pull/84/files

Limitations

Your package should build with OCaml 4.10, and all the packages that are built for the host will also be build for the target, so sometimes you need to fix a package that you will not use directly.

Some packages you will need to pin to a dune-universe fork version

How to use it

## compile your project
esy

## generate the wrapper
esy add -D generate@EduardoRFS/reason-mobile:generate.json
esy generate android.arm64

## build for android.arm64
esy @android.arm64

Platforms

All of the following are tested from Linux and macOS, but I would suppose that FreeBSD should be also working as a build system.

Targets
android.arm64
android.x86_64
ios.arm64
ios.simulator.x86_64
linux.musl.arm64
linux.musl.x86_64

What I tested

In the past I was able to build Revery the UI framework for Android and iOS

But recently I did compile esy the package manager itself for all of the following platforms above from an Arch Linux x86_64 and macOS Catalina x86_64. Including iOS, with the right version of OCaml it will run inside of the new macOS ARM64 and inside of a jailbroken iPhone.

OCaml User Survey 2020

gasche announced

We are happy to announce the OCaml User Survey 2020. We are trying to get a better picture of the OCaml community and its needs. It would be very useful if you could fill the survey (10-15 minutes), and share it widely with other OCaml programmers!

The survey is run by the OCaml Software Foundation. Thanks in particular to our sponsors OCamlPro (@MuSSF) for preparing many of the questions, Jane Street (@Yaron_Minsky) for excellent feedback, and to Kim @K_N Nguyễn for his technical help.

This is our first year running the survey, we hope to continue in following years. There are many things to improve; please feel free to give us feedback! (There is a feedback question at the end of the survey, or you can post here, or send me a message/email.)

The survey was inspired by programming-language surveys ran by other communities. See for example past survey results for Go, Haskell, Rust, and Scala.

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.