OCaml Weekly News

For those that live in New York City and aren't aware, there's an NYC OCaml meetup: https://www.meetup.com/NYC-OCaml/

We had a fun informal gathering last week, there's a tech talk next week, and the announcements for Jane Street's tech talks also go (semi-indirectly) to the meetup.

Are there any mature structured-concurrency libraries for OCaml, similar to dill or trio?

If you want concurrency with OCaml, you'd better stick to monadic concurrency, aka lwt or async. Both are mature and battle-tested.

I'm aware of lwt and async. I mean something like this.

I'm very happy to announce a new major release of ocaml-git (2.0). This release is a 2-year effort to get a revamped streaming API offering a full control over memory allocation. This new version also adds production-ready implementations of the wire protocol: git push and git pull now work very reliably using the raw Git and smart HTTP protocol (SSH support will come soon). git gc is also implemented, and all of the basic bricks are now available to create Git servers. MirageOS support is available out-of-the-box.

Two years ago, we decided to rewrite ocaml-git and split it into standalone libraries. More details about these new libraries are also given below.

But first, let's focus on ocaml-git's new design. The primary goal was to fix memory consumption issues that our users noticed with the previous version, and to make git push work reliably. We also took care about not breaking the API too much, to ease the transition for current users.

> When is it usefult to use ocaml-git over standard git? Why is is it called a long lived git client?

The usual git is a set of command line tools for use by humans. ocaml-git is a set of libraries for building git repository manipulating and managing software in ocaml. If you're trying to write software in ocaml that manipulates a git repo at more than a trivial level or even has intimate knowledge of git internals, this is your library of choice.

> Thanks for the answer, do you also know of a practical use case of such a program?

ocaml-git was initialy developped for irmin. The idea behind it is to provide a way to have a persistant store for an unikernel/MirageOS. By this way, we need to apply on some assumptions:

• make a library to be able to link it (static link) with the rest of the OS
• use OCaml

ocaml-git provides some binaries (like ogit-write-tree, etc.) but it's only as little example of how to use this library. The goal is definitely not (at this stage) to provide a new CLI tool. Then, as a library, ocaml-git wants to be used in a server-context to be synchronized with differents endpoints (GitHub, MirageOS, local Git repository, etc.).

Finally, a practical example could be done with irmin when you want to have an access to the store of your unikernel without any access on it (SSH for example). You just need to push to a Git repository which is synchronized with your unikernel and then, your unikernel will load what you push (safely).

A good example is Canopy with is a static blog (unikernel). When you want to add a new unikernel, instead to remake your unikernel, you tell him to be synchronized with a GitHub repository which contains your articles - a real world example is the blog of @hannesm. Of course, we have others examples (like DNS server synchronized with a zonefile available on a Git repository).

> How complete is it? Can it determine the status of a working directory, which branch is checked out, how many files are added, deleted and unknown or is it more like an implementation of the data structures used within git?

> In particular I have this shell prompt script which is very slow and I would like to replace it without shelling out to a number of git commands while avoiding to also implement half of git itself.

We have full coverage for the on-disk and wire protocol, so reading the current branch is easy.

We also have support for working tree but this is not as well tested (and documented…) than the rest of the code, so you might hit performance and usability issues. We would be happy to support that better, so please feel free to report any issues!

I was trying to get started with practical OCaml via writing API clients and decided to experiment with https://deckofcardsapi.com/ as a starting point to discover the libraries.

However, the only pieces of documentation I've come across for these are the following

• https://discuss.ocaml.org/t/how-to-write-a-simple-web-client/1756
• The client_lwt.ml example in the cohttp github repo

But unfortunately, I haven't quite been able to make my way through these. So, I decided to ask for help from the OCaml community regarding the same.

Could you guide me a bit here ?

I hope the following will help, it's just a rough example. You can actually copy and paste it in utop. You will need cohttp-lwt-unix and yojson installed. For yojson refer to RWO, it is fine even if you don't use core.

#require "cohttp-lwt-unix";;
#require "yojson";;

(* A bit overkill to open everything *)
open Cohttp
open Cohttp_lwt_unix
open Lwt.Infix
open Yojson
;;

(* Make a get call and return the body parsed as json.
 * You could use atdgen to produce the right marshalling
 * functions into nice records, but for such small data
 * structures you might as well do it by hands.
 *
 * val json_body : string -> Basic.json Lwt.t
 *)
let json_body uri =
  Client.get (Uri.of_string uri)  >>= fun (_resp, body) ->
  (* here you could check the headers or the response status
   * and deal with errors, see the example on cohttp repo README *)
  body |> Cohttp_lwt.Body.to_string >|= Yojson.Basic.from_string
;;

(* In utop the promise is realised immediately, so if you run the
 * function above, you'll get the result immediately *)
json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1"
;;
(*Output:
- : Basic.json =
`Assoc
  [("shuffled", `Bool true); ("success", `Bool true); ("remaining", `Int 52);
   ("deck_id", `String "2svxenasd9qj")]
*)

(* In the code you will have more likely something like the following. *)
let shuffled_deck =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1"
;;
(* val shuffled_deck : Basic.json Lwt.t *)

(* To use it you just have to get use to the monadic bind, and once you
 * are ready, call `Lwt_main.run`. e.g. the kind of useless example below *)

(* val get_cards : Basic.json -> int -> Basic.json Lwt.t *)
let get_cards sdeck n =
  (* The use of Yojson is well explained in RWO *)
  let module YBU = Yojson.Basic.Util in
  let success = sdeck |> YBU.member "success" |> YBU.to_bool in
  if not success then Lwt.fail_with "Error while shuffling" else
    let deck_id = sdeck |> YBU.member "deck_id" |> YBU.to_string in
    Printf.sprintf "http://deckofcardsapi.com/api/deck/%s/draw/?count=%d" deck_id n
    |> json_body >>= fun cards ->
    Lwt.return cards

(* val do_something : unit -> string list Lwt.t *)
let do_something () =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1" >>= fun sdeck ->
  get_cards sdeck 3 >>= fun jcards ->
  let module YBU = Yojson.Basic.Util in
  let cards = jcards |> YBU.member "cards" |> YBU.to_list in
  List.map (fun c -> c |> YBU.member "code" |> YBU.to_string) cards
  |> Lwt.return

(* val do_something_else : unit -> unit Lwt.t *)
let do_something_else () =
  do_something () >>= fun cards ->
  List.iter (Printf.printf "%s\n") cards |> Lwt.return;;
;;

Lwt_main.run (do_something_else ()); print_endline "Done!"
;;

(* Be careful with the signatures, the example above can be re-run multiple
 * times, while the one below does nothing after the first run *)

(* val do_something_1 : string list Lwt.t *)
let do_something_1 =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1" >>= fun sdeck ->
  get_cards sdeck 3 >>= fun jcards ->
  let module YBU = Yojson.Basic.Util in
  let cards = jcards |> YBU.member "cards" |> YBU.to_list in
  List.map (fun c -> c |> YBU.member "code" |> YBU.to_string) cards
  |> Lwt.return

(* val do_something_else_1 : unit Lwt.t *)
let do_something_else_1 =
  do_something_1  >>= fun cards ->
  List.iter (Printf.printf "%s\n") cards |> Lwt.return;;
;;

Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run (do_something_else ()); print_endline "Done!"
;;

I forgot to mention that you can reduce the conversions boilerplate easily with ppx_deriving_yojson

There is also the example in Real world ocaml, althouh that uses cohttp-async. It should be relatively straightforward to translate.

Having a nice tutorial to add to the repo and the community documentation would be good. I am not a power user, but I can try to help. What you would loke to know?

Thanks @mseri for the link!

However, I'd really love to get started with lwt as I'm more inclined towards the mirageOS that relies on lwt .

For now, what I'd like to do is to just wrap the deckofcards API in a utop session as the starting point. If possible, could you please share the snippets that could help me accomplish these tasks ?

Here are the libraries I've identified so far

• yojson
• lwt
• cohttp-lwt

Are there any good resources to learn lwt apart from the main ocsigen docs ?

I recently published zeit which is client library for a JSON REST API using cohttp-lwt, yojson and ppx_deriving_yojson. You might be interested by the Client module.

Managing json APIs is what atdgen was designed for. You write your types and atdgen derives the OCaml code to convert your data from/to json safely. It also provides ways to deal with APIs that don't fit the OCaml type system perfectly (json adapters being the most flexible but last-resort method).

These are the type definitions that we used at my previous company for using the Google Calendar API: https://github.com/esperco/esper-gcal-api

Here is a sneak peek at some potential future features of the Ocaml compiler, discussed by their implementers in these Github Pull Requests.

• Format.pp_print_custom_break, a more general break hint
• A type for unification traces
• Full explanation for unsafe cycles in recursive modules
• Add a warning on type declarations "type t = ()"
• In error messages, print the source fragment responsible for the error
• Syntactic redirection of identifiers as a way to move libraries forward
• Disallow .~ in extended indexing operators
• Explicit stdlib design choices

Here are links from many OCaml blogs aggregated at OCaml Planet.

• ocaml-git 2.0
• OCamlPro’s TZScan grant proposal accepted by the Tezos Foundation – joint press release
• OCamlFormat 0.8
• OCaml Workshop 2018