OCaml Weekly News

On behalf of Nomadic Labs, I'm happy to announce the release of data-encoding version 0.5.

This new version brings several bug fixes, some increased test coverage, minor improvements in the API, and a major new feature:

Hi @raphael-proust! I have a question regarding the connection between data-encoding and json-data-encoding, also developed at Nomadic Labs. The latter seems tied to JSON, whereas the former is more flexible, supporting also binary encodings. However, since data-encoding also supports JSON, doesn't it subsume json-data-encoding completely?

The data-encoding library uses json-data-encoding for its JSON backend. It delegates conversion from OCaml values into and from JSON to the primitives provided in the interface of json-data-encoding.

In a way, yes, as an end-user you don't need to use json-data-encoding directly because you can use the Json module of data-encoding instead. There are three possible reasons why you might add json-data-encoding as a (non-transitive) dependency to your project and use it directly in your code:

• You want to keep the dependency set and the number of abstraction layers as small as possible. E.g., in order to reduce binary size.
• You want some static guarantees that some encodings are only every used for JSON. E.g., in your logging system.
• You need to define a JSON encoding which is rejected by data-encoding on grounds that it is invalid in binary. Note that
  • This is very specific to some combinators but basically some combinators will reject their inputs (raise Invalid_argument) because using the serialiser would lead to undecodable data. Most typically, this happens if you try to concatenate two fields of unknown length. Decoding the result becomes a guessing game as to were one field stops and where the next begins. These could easily be represented as an array in JSON which includes all the delimiters you need to decode it.
  • There are other workarounds (e.g., prefixing the fields with a length field), but going for the JSON encoding directly is a valid approach if you only need JSON.

Version 0.5.1 of the data-encoding has just been released.

This is a bugfix release making one of the library's internal checks more permissive. Without this fix (i.e., using version 0.5), some valid encodings are rejected (raising Invalid_argument) by the library.

You can update via opam: opam install data-encoding.0.5.1

The recent conversations about eio 0.1 and agnostic blocking have made me very curious about effect handlers. The multicore team has done an awesome job with their tutorials, examples and talks, but the laymen have been too quiet for such an exciting feature! Where are all the blog posts about how "you could have invented algebraic effects" and "one-shot continuations are like spaghetti"?

In any case, I'm hoping to tease some of you into trying them out with a simple tutorial about programming a roguelike with effect handlers :)

There's nothing new here besides the fun use-case! So if you already have an intuitive understanding of the syntax and motivations, you may be more interested by a deeper look at the scope of effect handlers – and a soft introduction to some less common features of the type system. (this link was previously posted deep into the eio thread)

I would be grateful if you spot any mistake! I'm also curious of other fun applications for effect handlers… and if you feel like sharing your own surprises and discoveries, I believe it could really help others learn faster :)

Great blog post! That seems like a very elegant implementation!

Funny you should make a rougelike :smiley: , I guess effect handlers + games might be popular for games, because I also had a blog post about effect handlers and their applications, in particular for games, although in my case it was for animations:

https://gopiandcode.uk/logs/log-bye-bye-monads-algebraic-effects.html

Note: the "upstream" status of effect handlers is a little uncertain/confusing right now. Your blog post (I didn't get a chance to read it yet, but it sounds very nice!) uses the experimental syntax of multicore-4.12+effects, but that syntax was intentionally not upstreamed, and it will not be part of OCaml 5.0.

I think there is a risk of confusion because the community is aware that Multicore OCaml has effect handlers, and also that Multicore OCaml has been merged upstream. So it can be tempting to believe that the upcoming OCaml release (or maybe one or two releases after that, we said the first Multicore release would be more like a preview) will support effect handlers as a language feature. It will not! Effects as a language feature were removed from Multicore OCaml before the upstream merge. And no one knows if/when they will be supported upstream.

So: I think that your blog posts on using effect handlers could have somewhere a short mention that the code is using an experimental extension of OCaml that is not supported by the upstream implementation.

The reasoning for this choice is that we want to give a chance to a type system for effect handlers, but that still need quite a bit more time than the Multicore runtime itself. We don't want to encourage the ecosystem to rely on untyped effects, if it means a lot of pain upgrading to typed effects later (or risk having to support both).

5.0 only contains basic support for effect handlers as a runtime primitive, but dos not support handlers as a language feature. I think they should be considered experimental: you can rely on them for their intended purpose of exposing a flexible interface for concurrent fibers, but uses beyond that may break in the future.

So, in a sense, we don't want people to use them. It's of course fine to use experimental features from experimental forks of the OCaml compiler (effect handlers, modular implicits or explicits, runtime type representations and what not), and the people working on these experimental features do benefit from other people trying them and giving them feedback. But we don't want people to depend on it in production, whatever that means. (For example, code using it is likely to get stuck on 4.12 forever and never see an upgrade to upcoming OCaml versions, although of course people could choose to port the experimental branch forward.)

Just in case anyone is actually interested in this: the project submission deadline has been extended from March 4th to March 23rd. So the updated timeline now looks as follows:

where 2. and 3. probably need to be done a bit in parallel.

Hi, some new developments. I have implemented a new Sdl_area widget where one can conveniently issue any SDL function (from the SDL Renderer API).

Here is (below) the new 'labelled graph' example. In this example I am using regular "label" widgets for creating the nodes, and I am using an Sdl_area for drawing the lines.

The nice things for labels to be regular widgets is that one can click on them. To demonstrate this, in this example they react to a click by jumping to another random location (with animation).

open Bogue
module W = Widget
module L = Layout

let n = 15 (* number of discs *)
let radius = 20
let width = 800
let height = 600

let c = Draw.find_color "#e5b92c"
let cb = Draw.find_color "#7b6b35"
let disc_style = Style.(
    create ~border:(
      mk_border ~radius (mk_line ~color:Draw.(opaque c) ~width:1 ~style:Solid ()))
      ~background:(color_bg Draw.(opaque cb)) ())

let background = L.style_bg Style.(
    of_bg (gradient ~angle:45. Draw.[opaque grey; opaque black]))

let fg = Draw.(opaque white)

let create_disc i (x,y) =
  let w = 2*radius + 1 in
  let bg = Box.create ~style:disc_style ~width:w ~height:w () in
  W.label ~fg (string_of_int i)
  |> L.resident ~background:(L.box_bg bg) ~x:(x-radius) ~y:(y-radius) ~w ~h:w

let move_disc (x,y) d =
  let (x0, y0) = L.xpos d, L.ypos d in
  L.animate_x d (Avar.fromto x0 x);
  L.animate_y d (Avar.fromto y0 y)

let random_center _ =
  radius + Random.int (width - 2*radius),
  radius + Random.int (height - 2*radius)

let area =
  let sdlw = W.sdl_area ~w:width ~h:height () in
  let sdla = W.get_sdl_area sdlw in
  let centers = Array.init n random_center in
  let color = Draw.(opaque grey) in
  let draw_lines renderer = let open Draw in
    for i = 0 to n - 2 do
      let x0, y0 = to_pixels centers.(i) in
      let x1, y1 = to_pixels centers.(i+1) in
      line renderer ~color ~thick:6 ~x0 ~y0 ~x1 ~y1
    done in
  Sdl_area.add sdla draw_lines;
  let discs = Array.mapi create_disc centers |> Array.to_list in
  (* move the disc when click on it *)
  List.iteri (fun i d ->
      W.on_click ~click:(fun _ ->
          centers.(i) <- random_center 0;
          Sdl_area.update sdla;
          let x,y = centers.(i) in
          move_disc (x - radius, y - radius) d) (L.widget d))
    discs;
  L.superpose ~w:width ~h:height ~background (L.resident sdlw :: discs)

let board = Bogue.make [] [area]

let () = Bogue.run board

This is Aya, one of the three Outreachy interns working on OCaml this winter :camel: After 3 very fast months, our internships are already coming to a close. We have had such a great time working on our projects and learning OCaml that we want to hold an event to mark the end of the internships, and we decided to open it up to the community :tada:

As you might have seen in the initial announcement, @pitag @shonfeder @gs0510 @tmattio and @pkel all volunteered to mentor us from December 2021 to now. Thank you all so so much for mentoring us and introducing us to OCaml :heart: :fire: It's been such an enjoyable experience!

We are inviting anyone who is interested to attend a virtual session of 3 short presentations on Friday, March 4th, 4-5pm CET (we will post the link to join on Thursday). There will be time for Q&A after each presentation, and the whole session will be recorded and posted online shortly after as well.

• @ayc9 will present on updating a standard PPX deriver (mentors: @pitag @shonfeder)
• @SaySayo will present on syntax highlighting and other updates to the vscode extension (mentors: @tmattio @gs0510)
• @JiaeK will present on building a basic monitoring dashboard for ocaml.org (mentors: @tmattio)

We hope you can make it!

-@ayc9 @SaySayo @JiaeK

I looked a bit into the kind of fiber abstraction and concurrency structure I would like to use with the new tools OCaml 5.0 is going to offer. You can find some results in affect's Fiber module.

This fiber abstraction supports terminating by returning values or abnormally (by aborting or via a spurious exception). Termination of a fiber is aligned on function scopes: all the fibers spawn by a fiber function have to terminate in order for it to terminate.

This means that if your fiber returns a value it waits for its spawns to terminate (in any way) before returning the value. And if your fiber returns abnormally (uncaught eception or explicit abort) it first aborts all its non-terminated spawns before returning abnormally – this provides affect's notion of cancellation.

Explicit fiber aborts raise the Abort exception in fibers. Combined with a disciplined use of Fun.protect and an optional finally handler specified at fiber spawn, this lets them release the ressources they may hold when it's time to say goodbye.

The module also provides a generic way of blocking and unblocking fibers that you can use to interface with your favourite event loop. It does so without requiring to fiddle with effects, you just need to make judicious use of Fiber.block and provide a suitable function to Fiber.run's built-in scheduler to let it know about fibers that can be unblocked.

A grab bag of comments:

1. The first goal of affect is to seek a concurrency and abort structure that are easy to understand, use and compose with event loops. Right now some efficiency and implementation aspects need to be improved. This will likely change the exposed set of primitive effects which doesn't feel exactly right yet (if you want to build your own scheduler).
2. I use abort rather than cancel terminology. From my non-native english speaker perspective, cancelling is more about not doing something that was planned but didn't happen yet. Aborting is more about stopping something that is going on. It also melds better with the uncaught exception case.
3. Say no to unit soups! Let fibers return values.
4. At that point I don't feel the need to add a promise/future abstraction to the toolbox. The whole point of direct style is to get rid of this async madness.
5. There's no synchronisation structure yet. Semaphores are always useful for throttling so I'll certainly add that at some point or a more fundamental primitive like an mvar.
6. The Funix module has a few fiber friendly Unix module functions for playing with timers and the network, see ping.ml for an example of use. In practice you want to be able to use something else than select(2) though. There are various ways one could go about this, see for example point 6. in these design notes.
7. The mouse.ml has a basic example on how to interface with the SDL event loop which provides another example on how one goes to interface Fiber with event loops.

I'm not fully convinced by everything yet. It will certainly need one or two more design rounds. If you try it, feel free to comment or make suggestions on the issue tracker.

Home page: https://erratique.ch/software/affect

API docs: https://erratique.ch/software/affect/doc/ (or odig doc affect)

Install:

opam switch create 5.0.0+trunk
opam pin add https://erratique.ch/repos/affect.git

@kayceesrk and I are delighted to announce that Segfault Systems, a spinout from IIT-Madras, is joining Tarides. Tarides has worked closely with Segfault Systems over the last couple of years, most notably on the award-winning Multicore OCaml project and the upstreaming plans for OCaml 5.0. This alliance furthers the goals of Tarides, bringing the compiler and benchmarking expertise of the Segfault team directly into the Tarides organisation, where it can be commercially funded and supported.

All of Segfault Systems’ existing responsibilities and open-source commitments will migrate over to Tarides, where work will continue towards the three main objectives in 2022:

• Releasing OCaml 5.0 with support for domains and effect handlers
• Supporting the ecosystem to migrate the OCaml community over to OCaml 5.0
• Improving developer productivity for OCaml 5.0 by releasing the best platform tools

This alliance will complement the commercial offerings of Tarides – already strengthened by the integration of OCaml Labs – and contribute to Tarides’ mission: empowering developers, communities, and organisations to adopt OCaml as their primary programming experience by providing training, expertise, and development services around the OCaml language.

Read the full announcement here, including details of our goals and the focus for 2022. This alliance brings the headcount of Tarides up to 60+ people, all working towards making OCaml the best language for any and every project. Join our team and reach out for commercial services at https://tarides.com/.

we are delighted to announce the OCaml User Survey 2022. With this survey, the OCSF is trying to get a better picture of the OCaml community and its needs. It would be very helpful if you could take a few minutes (10 to 15) to fill the survey and share it with other OCaml programmers.

https://forms.gle/oKy2Joz1cZhCPNtf6

The survey is run by the OCaml Software Foundation. It builds on the previous iteration issued in 2020. The results will be published here on discuss and on the website of the OCSF. We would like to particularly thank @cjr for his help as well as everyone who commented on the previous survey. We tried our best to take all remarks into account but surely missed something. Don't hesitate to give us your feedback (you can post here or send me a message/email).

The survey will remain opened until March 11th 2022 (AOE).