OCaml Weekly News

We are looking for talks about best-of-breed software technology, e.g.:

• functional programming
• persistent data structures and databases
• types
• formal methods for correctness and robustness
• abstractions for concurrency and parallelism
• metaprogramming
• probabilistic programming
• math and programming
• controlled side effects
• beyond REST and SOAP
• effective abstractions for data analytics
• … everything really that isn’t mainstream, but you think should be.

Presenters should provide the audience with information that is practically useful for software developers.

We're especially interested in experience reports. Other topics are also relevant, e.g.:

• demos and how-tos
• reports on problems that cutting-edge languages and tools should address but don't
• overviews of a given field

We accept proposals for presentations of 45 minutes (40 minutes talk + 5 minutes questions), as well as 90 minute tutorials for beginners. The language of presentation should be either English.

Your proposal should include (in your presentation language of choice):

• An abstract of max. 1500 characters.
• A short bio/cv
• Contact information (including at least email address)
• A list of 3-5 concrete ideas of how your work can be applied in a developer's daily life
• additional material (websites, blogs, slides, videos of past presentations, …)
• Don't be confused: The system calls a submission event.

https://bobcfc.active-group.de/bob2019-summer/cfp

BOB has Speaker Grants available to support speakers from groups under-represented in technology. We specifically seek women speakers and speakers who are not be able to attend the conference for financial reasons. Shepherding

The program committee offers shepherding to all speakers. Shepherding provides speakers assistance with preparing their sessions, as well as a review of the talk slides.

• Direct questions to contact at bobkonf dot de
• Proposal deadline: May 17, 2019
• Notification: May 31, 2019
• Program: June 14, 2019

• Matthias Fischmann, zerobuzz UG
• Matthias Neubauer, SICK AG
• Nicole Rauch, Softwareentwicklung und Entwicklungscoaching
• Michael Sperber, Active Group
• Stefan Wehr, factis research

Scientific Advisory Board

• Annette Bieniusa, TU Kaiserslautern
• Torsten Grust, Uni Tübingen
• Peter Thiemann, Uni Freiburg

More information here: http://bobkonf.de/2019-summer/programmkomitee.html

The current ppx world is composed of various components. I am quickly describing the ones I know in this section. The order doesn't correspond to the order in which these components where developed.

The main reason why ppx rewriters are so much pain is because the system is based on the compiler libraries. The compiler libraries are meant for experts and provide no stability guarantee. With such an unstable basis, it is no surprise that the whole system keeps breaking all the time.

ocaml-migrate-parsetree helped the situation by allowing to sandbox individual ppx rewriters into a protective layer. However, this sandboxing means that this method is not applicable when ppx rewriters need to inter-operate with each other in more sophisticated ways such as with ppx_deriving or ppxlib. Moreover, a user of ppx rewriters cannot use new language features until all the ppx rewriters it uses are based on the new version of the AST. Which means that all ppx rewriters still need to be updated and re-released after a new release of OCaml.

Finally, there are just too many projects doing the same thing which makes everything really confusing.

The plan is to provide a stable base for the whole system that doesn't break when a new compiler is released or require all ppx rewriters to be re-released. Because there are some complicated problems that cannot be solved without breaking API change, this new base will be released as new package that will be called simply "ppx". Although a large part of it will be imported from ppxlib.

We will ensure that one way or another, ppx rewriters based on ppx can be used in conjunction with ppx rewriters based on ppxlib, ocaml-migrate-parsetree, ppx_deriving, ppx_tools, … This will provide a smooth transition story from the old to the new world. I discussed with the authors of the various projects to make sure they are happy with the idea of this new project eventually replacing everything else, to make sure we are not creating "yet another standard". I also discussed with the other OCaml core developers to establish a strong link between "ppx" and the compier and make sure that the compiler will never break "ppx". In particular, it will become much easier to test the trunk of OCaml against all released opam packages.

I am hoping that the stability guarantee provided by this new base will be enough of an incentive for authors of ppx rewriters to switch to it. However, if you have any concern about this plan, please raise them here or to me privately as soon as possible.

The main difficulty of this project is to design a stable representation of the OCaml AST. What I mean by a stable AST is the following: a given file will always parse to exactly the same value no matter the version of the compiler. If this property is true, then one can have good confidence that an AST transformation written now will continue to be valid for a long time.

This is currently not true as the types used to represent the AST keep changing in breaking ways. For this reason, ppx rewriters will no longer see the AST used inside the compiler. Instead, they will work on a different AST that is more loose and allows to represent more than just the current language. In particular, this AST should be able to represent any future version of the language. However, the use of private types and construction functions will ensure that ppx rewriters can only construct valid AST fragments.

We are not far enough into this project to know the final representation of the AST. However, to illustrate the idea, here are two examples of what such an AST could look like (I omitted the locations to keep the examples simple):

(* Example 1: plain s-expressions *)

type t =
  | Atom of string
  | List of t list

type structure = private t
type expression = private t
...

(* Representation of [let x = 1 in x + 1]:

   {[
     List [Atom "let";
           List [Atom "binding";
                 List [Atom "ident"; Atom "x"];
                 List [Atom "int"; Atom "1"]];
           List [Atom "apply";
                 Atom "+";
                 List [List [Atom "ident"; Atom "x"];
                       List [Atom "int"; Atom "1"]]]]
   ]}
*)

(* Example 2: adding slightly more structure *)

type t =
  | Int of int
  | String of string
  | Ident of string
  | Let of t list * t
  | Binding of { lhs : t; rhs : t }
  | Apply of t * t list
  | ...

type structure = private t
type expression = private t
...

(* Representation of [let x = 1 in x + 1]:

   {[
     Let ([Binding { lhs = Ident "x"; rhs = Int "1" }],
           Apply (Ident "+", [Ident "x"; Int "1"]))
   ]}
*)

It is easy to see that such ASTs can easily be extended without breaking backward compatibility. Constructions functions would ensure that all values of type structure or expression that can be produced are valid AST fragments, i.e. ones that are part of the OCaml language.

For pattern matching, we will provide view patterns based on the ideas used in ppx_view so that programmers don't accidently write non-sensical patterns, i.e. patterns that can never match anything because they match on values that cannot be produced by the parser. Another way is by testing coverage: if one can reach 100% coverage then this is a proof that all the patterns are valid.

Using an AST that is not the one of the compiler might seem contrary to the philosophy of ppx. However, @alainfrisch mentioned to me that he did envision that ppx rewriters would use a different more stable AST when designing the original ppx feature. So I would say that we are making ppx what it was meant to be rather than diverging from it.

Ppx has a long and storied history leading to a complex stack that is difficult to maintain. We now want to clean it all and restart fresh with a strong base. While doing so, we are opening the discussion about this work early, and most importantly before the point of no return. So I definitely encourage anyone who is interested by all this or will be affected by these changes to chime in, ask for precisions, challenge the technical decisions and raise any concern so that together we can build a better, stronger and more unified ppx ecosystem!

Additionally, all this work will be done entirely in a pure open source fashion, which will make it easy for everyone to follow and/or contribute. In particular, help is most definitely welcome :) Ppx rewriters are used by a lot of people, so this work will benefit a large part of the OCaml and even Reason communities. So if you are new to OCaml and and motivated to make an impact, then this is definitely a project to consider.

When you write:

(executable
 (name prog)
 (libraries a b c)
 (variants x y z))

If a, b, c or one of their transitive dependency is a virtual library and no concrete implementation was found in the list of transitide dependencies, then dune will look at the variants attached to all installed libraries in order to automatically select a concrete implementation. As you might have guessed, the result depends on what is installed on the system. In particular, reinstalling a previously installed package could lead to a different result.

Anyone who is using the new library variants feature, i.e. any project with at least one of these fields in their dune files: variant, variants, default_implementation.

I grepped the dune-universe repository to see if this feature was already used by released packages. Fortunately, it doesn't seem to be the case.

For the record, dune-universe is a repository that embed a snapshot of the latest version of every package released in opam and using dune or jbuilder.

We decided to take the two following immediate actions:

1. we are narrowing the scope of library variants to something we are more confident can be safely supported
2. we are putting back the feature into development mode

We will soon release Dune 1.9.2 with these changes and mark Dune 1.9.0 and 1.9.1 as unvavaiable in the opam repository to prevent these versions from being used by newly released packages.

We are planning to brainstorm more about library variants to come up with a more robust design. Once we find a more satisfying one, we will implement it, test it and finally integrate it into the vanilla dune language. In the meantime, feedback on library variants is warmly welcome!