Hello
Here is the latest OCaml Weekly News, for the week of October 17 to 24, 2017.
Archive: https://discuss.ocaml.org/t/bob-tool-progress-or-repo/978/2
Deep in this thread, Anil Madhavapeddy announced:
Thanks for bringing this up; I've been meaning to write a followup post since a
busy ICFP week. As background, the [OCaml Platform 2017 slides are
online](https://speakerdeck.com/avsm/ocaml-platform-2017) for readers who
weren't at the conference (I haven't checked to see if the videos are online
yet).
We have two short-term priorities that we are working on for the remainder of
the year at OCaml Labs as relates to the Platform:
- *OPAM2:* We are now in the last stages of the opam2 release train, with just a
few things such as the repo migration tools, depext support and full Windows
support that are awaiting merge. There is also a rewrite of the CI system
ongoing that will scale out to much more testing and platform testing support
(including new architectures such as arm64 and operating systems such as Windows
and *BSD). This will take us through to November/December at least.
- *Docs/Build/Test:* The individual components such as docs (`odoc`), build
(`jbuilder`) and testing (`crowbar`) are being worked on for 1.0 releases. They
will all be released individually into opam as usual, and we will be posting on
this forum for feedback as each of them hits their respective milestones.
The above two chunks of work will come together to form Bob, which is comprised
of a new assemblage of the various libraries and compiler improvements that went
into them. This effort will mark the start of a fresh design cycle, involving
building prototypes with a new CLI and workflow. The intention here is to shed
the chains of `opam` CLI compatibility and embrace workflows that have emerged
from the community (both small-scale users and larger industrial users). We will
leave plenty of time for prototypes to be discussed and tested before committing
to a design.
As previous CLI tools have taught us, there is a 3-5 year lifetime (at least)
for infrastructure tools, and so we are keen to ensure that the opam 2.0 release
is a success (with a clear migration path from 1.0), and that Bob gives us a
solid foundation for the future with even more streamlined OCaml/Reason
workflows.
To answer your specific questions:
1) There is no official site to track Bob progress yet, but there will be after
we get some of the earlier priorities out of the way (opam 2.0 mainly)
2) See above :slight_smile:
3) No timescales yet beyond the design process starting later this year. As
always, all of the outputs from OCaml Labs will be open source and freely
available, and we very much welcome outside contributions.
Most importantly: do not let our roadmap block anyone else who wishes to
prototype a Bob variant. In the early days of opam we had plenty of similar
experiments, many of which eventually made their way into the mainline tool. I
hope for a similar diaspora of ideas from the Bob prototyping process!
Archive: https://discuss.ocaml.org/t/4-06-release-work-a-partial-progress-report/986/1
gasche announced:
As many of you know, the maintainers of the OCaml compiler distribution are hard
at work preparing the next release of the language, 4.06.0 -- the dream is to
have a release by the end of October. The release effort, however, is not
limited to the compiler distribution : there is a lot of work going behind the
scenes on updating the library ecosystem to changes in the new version. I'm
creating this topic to give a (partial) view of what's happening, so that people
have a better idea of what's the work involved -- and maybe they want to help!
(For a list of changes in 4.06, see the
[Changes](https://github.com/ocaml/ocaml/blob/4.06/Changes) file of the release
branch.)
## Compiler distribution
On the "public" side of the compiler distrubtion front, we have done two beta
releases of the distribution and @octachron released a beta version of the
manual:
- [beta1
announce](https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00006.html),
October 3rd (opam switch `4.06.0+beta1`)
- [beta2
announce](https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00019.html),
October 13th (opam switch `4.06.0+beta2`)
- [manual beta
release](https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00029.html),
October 16th; this beta manual is at
http://www.polychoron.fr/ocaml-beta-manual/4.06
On the development side (also public) of the compiler distribution, three
activities are happening thanks to maintainers and external contributors alike:
- We are busy reviewing and discussing the pull requests that we are considering
for inclusion in 4.06 (see the [4.06.0
milestone](https://github.com/ocaml/ocaml/pulls?q=is%3Aopen+is%3Apr+milestone%3A4.06.0)
on github); there are 10 PRs still open in this category. Note that being marked
for 4.06 does not mean "we know we want it in the release", but "we have to
decide what to do with it before the release". Since the feature freeze
mid-September, we are mostly working on bugfixes -- fixes for regressions in
4.06 or to features that are appearing in 4.06 are high-priority.
- We are also reviewing the bug reports on Mantis that are marked for
consideration before the 4.06 release (see the
[Roadmap](https://caml.inria.fr/mantis/roadmap_page.php)). There are currently 8
bugs open. Some of these bugs are reported as 4.06-regression (and thus high
priority), but many of those bugs have been around for longer. The current
mantis process is to encourage ourselves to review and re-triage, before each
release, as many of the open bugs as possible, to motivate
maintainers/contributors to fix those that seem easy (release work is also a
time where some people are more motivated to work on the OCaml codebase, so
fixing old bugs can be a useful way to channel this energy).
In theory, we could do a release as soon as these two categories have been
emptied (by merging/fixing or deciding to postpone), except for what comes next.
## The opam repository
There is a lot of work going on in the opam repository since the beta started,
aimed at making sure that the OCaml open-source ecosystem is as ready for 4.06
as possible. This works tends to be even more distributed than
compiler-distribution contributions (which are already fairly diverse), and I'm
personally very happy with the effort that many people have been pouring into
it.
A general goal is that when they switch to a new release, end-users that do
`opam install foo` should not see a build failure; the package should either
build correctly or be marked as non-installable. This means that we need to:
1. fix the packaging metadata of packages that fail to build to indicate that
they are incompatible with the release
2. get new version released (or sometimes patch an existing version) that are
compatible with the new OCaml release
Our goal for the OCaml language is to have strong backward-compatibility from
one language release to another, so in theory the work required to update
end-users code should be small. However:
- Some packages at the base of the ecosystem, in particular almost all of the
`ppx` universe, are tightly coupled to the compiler distribution, so they almost
always need an update (and them being broken prevents a *lot* of packages from
building correctly : they are deep down the dependency chain of almost
everyone). I think that `sexplib` is not yet 4.06.0-ready for example (or that
change in the last couple days), and this has a huge impact on the availability
of other packages.
- this release is a bit special is that the `-safe-string` option which was
introduced in 4.03 but optional so far is now the default. This means that
unless the build scripts are changed to explicitly use the `-unsafe-string`
option, all packages that mutate strings (rather than `bytes`) have to be fixed.
Thankfully, most of the time the fixes are fairly easy to make (use `Bytes`
instead of `String` for mutating operations, and `Bytes.{of,to}_string` to
convert in and out), so updating codebases is not a lot of work.
Some package authors are checking their own package and fixing it, and that is
very nice. Some contributors are more active than that: they actively help
updating the dependencies of their packages (or just packages they care about).
Sending an opam-repository PR to fix metadata, or sending an upstream patch to
fix a package are super-helpful, but a simpler thing that already helps is to
try to compile packages, and to report the breakage to maintainers of the
dependencies that are causing the build failure. (Maintainers can work in a
demand-driven way, they fix stuff when people complain.)
For example, I have personally interacted with @jpdeplaix who pinged a couple
packages I follow, and fixed a bunch of opam metadata; his work will have a very
nice impact on the quality of the final release. Many other people are
contributing (and it's hard to have a global view of the work involved without
subscribing to the development repository of all OCaml packages), see for
example the [opam-repository contributors this
month](https://github.com/ocaml/opam-repository/graphs/contributors?from=2017-10-03&to=2017-10-30&type=c).
I have been monitoring the buildability of the whole opam-repository ecosystem
using a personal instance of @lefessan's
[opam-builder](https://github.com/OCamlPro/opam-builder) tool (the temporary
output is
[there](http://gallium.inria.fr/~scherer/tmp/opam-builder/opam-builder.html); I
dislike the new interface but I don't have the time to improve it right now),
and Damien Doligez uses his own
[opamcheck](https://github.com/janestreet/opamcheck) tool to systematically
build opam packages in a different way (we talked, and we may try to join forces
after the release). I hoped to obtain a good/clear enough output that I could
encourage people to look at it to find packages to fix, but I haven't had the
time for this release. In any case I can tell you that the global health of the
4.06 packages has been improving sharply this month, thanks to a couple
regression fixes on the compiler-distribution side and a lot of work to
fix/update the packages from everyone.
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00045.html
Malcolm Matalka asked:
I have a question in two parts:
1. Would this be a good idea? Why? (I'll describe why I think it is)
2. If it were a good idea, is it feasible to do?
Full question:
Despite exceptions being a part of the language, there is a trend in
many libraries to try to avoid using them. While I cannot find it, I
recall someone (Daniel maybe?) saying that the standard API advice is
that exceptions should not cross API boundaries.
The short reason for why people seem to want to avoid exceptions (which
I agree with) is that they side step the type system for helping you
understand if your code is correct and handles all situations the code
might experience. Since the exn type is open, it means that one can add
any exception they want so it's not even known what exceptions you might
get ahead of time.
Another aspect of exceptions, which might be more of my personal
experience, is that exceptions tend to be pretty useless after the
fact. For example, forgetting to handle a Not_found exception is an
exercise in pain. Maybe I'm just bad at this, but many exceptions just
aren't that useful. End_of_file is another one that, IMO, makes the
program flow pretty awkward and if you have multiple files you're
reading from at the same time quite ugly. I tend to use wrappers that
give me an option based API. Maybe I just bad at solving these problems
though and I'm the problem.
The consequence of this is that even though I put a lot of effort in my
code trying to avoid exceptions, I can never actually know that I have
succeeded unless I'm very defensive and wrap all foreign calls in some
exception handling code. There are APIs for this, but if I mess up then
I'm in a bad spot.
My proposal is that exceptions becomes a closed type and they reflect
what Java calls "errors", which are things your program logic should
generally not handle but can choose to if it wants to (I think we call
these failures in Ocaml). The two specific exceptions I can think if
that should exist are: Assertion_failure and Out of Memory. Another one
that I think might be nice but is open for debate is a
Not_implemented_failure, I use something like this often while building
a system. I'm sure there are a few more that people can think of are
meaningful, but the point is these represent pretty bad situations that
the program logic shouldn't handle except in special situations.
David Allsopp replied:
Without wishing to open old debating wounds too much, the argument of exceptions
as errors tends to come down as to whether the thing signalled by an exception
is truly exceptional. Not_found, for example, in some scenarios is as unexpected
or impossible as Invalid_argument. Historically, they're (ab)used for
performance reasons, but some of the overhead of that is being addressed in
flambda. Note that for some arguable design mistakes - e.g. End_of_file, you can
use exception matching to get around this, e.g.
match input_line ch with
| data -> ...
| exception End_of_file -> ...
which means that the old pattern
let data = try Some (input_line ch) with End_of_file -> None
is only needed if you need to compile with OCaml < 4.02
If you haven't come across it,
https://caml.inria.fr/pub/old_caml_site/ocamlexc/ocamlexc.htm is an interesting
piece of older research around dealing with handling exceptions.
What your proposal does overlook slightly is the use of exceptions for actual
flow control. See for example, an oldish post of Alain Frisch's at
https://www.lexifi.com/blog/static-exceptions. However, uses of exceptions like
this may at some point be subsumed by Algebraic Effects which are being worked
on by various people, mostly with multicore OCaml in mind. There's lots of links
to that in https://github.com/ocamllabs/ocaml-multicore/wiki as well as other
literature elsewhere online.
Ivan Gotovchits then added:
It's also a question of efficiency, signaling an absence of data with an
exception is usually more efficient, than signaling the presence of data by
wrapping it in some data constructor, as the latter needs an allocation.
Thus a function that raised an exception is more basic, than a function
that returns an optional value, as the former can be translated into the
latter, but not vice versa if you take an allocation into account.
Simon Cruanes also added:
Exceptions are also very useful as control structures sometimes. For
example:
- exiting from a traversal (graph, tree, `Foo.iter` functions) early
- being able to return a result from any point in a very large algorithm
(e.g. returning "sat" or "unsat" in a SAT solver, where the main
algorithm can span several hundreds or thousands of lines).
Gabriel Scherer finally said:
Participants to this discussion may be interested in the article "Catch me
if you can", by David Teller, Arnaud Spiwack and Till Varoquaux, 2008:
https://hal.inria.fr/inria-00432575
> This is the year 2008 and ML-style exceptions are everywhere. Most modern
> languages, whether academic or industrial, feature some variant of this
> mechanism. Languages such as Java even feature static coverage-checking for
> such exceptions, something not available for ML languages, at least not
> without resorting to external tools. In this document, we demonstrate a
> design principle and a tiny library for managing errors in a functional
> manner, with static coverage-checking, automatically-inferred, structurally
> typed and hierarchical exceptional cases, with a reasonable run-time
> penalty. Our work is based on OCaml and features monads, polymorphic
> variants, compile-time code rewriting and trace elements of black magic.
Archive: https://discuss.ocaml.org/t/ann-horned-worm-0-3-1-simple-use-web-app-server/987/1
Koga Kazuo announced:
I am pleased to announce horned_worm is available on OPAM!
This package is greatly inspired by [Suave.IO](https://suave.io) and
[GIRAFFE](https://github.com/dustinmoris/Giraffe) of F#.
But I want to use OCaml, so I wrote this!
Let's go >=> OCaml!
repo: https://github.com/kkazuo/horned_worm
Archive: https://discuss.ocaml.org/t/ann-ocaml-mariadb-0-8-2/989/1
Andre Nathan announced:
I've released a new version of OCaml-MariaDB. This version fixes a bug in the
return value of NULL fields, reported by Donovan Mueller.
Archive: https://discuss.ocaml.org/t/ann-ocamlformat-open-source-released/990/1
Josh Berdine announced:
OCamlFormat open-source released
I'm pleased to
[announce](http://fbinfer.com/blog/2017/10/20/ocamlformat-released.html) the
first public release of OCamlFormat.
OCamlFormat is a tool to automatically format [OCaml](https://ocaml.org/) code.
It follows the same basic design as refmt for
[Reason](https://reasonml.github.io/) code, but for OCaml. In particular, it
works by parsing source code using the OCaml compiler's standard parser,
deciding where to place comments in the parsetree, and printing the parsetree
and comments in a uniform style.
At Facebook, we currently use this for the OCaml code of
[Infer](https://github.com/facebook/infer) to enable developers to stop thinking
about line breaking, indentation, parenthesization, etc., to minimize stylistic
nit-picking during code review, and to make it as visually obvious as possible
when the parser's interpretation of code does not match the programmer's. We use
this both with integration with editors as well as a pre-commit hook.
Development is taking place on
[github](http://github.com/ocaml-ppx/ocamlformat). License is MIT.
See the [github page](http://github.com/ocaml-ppx/ocamlformat) for more info on
installation, documentation, contributing, etc.
Archive: https://discuss.ocaml.org/t/ocaml-rocks-and-few-announcements/992/1
Gerson Moraes announced:
# OCaml rocks! and few announcements
I am finally ready to announce the creation of [OCaml rocks!](https://ocaml.rocks)
It is a new blog for sharing coding experience, new ideas on automation and to
make introductions on interesting projects - *personal or from anywhere in the
community*. And while on the subject, there you will also find information about
to projects that recently landed on opam:
## Sugar
[Sugar](https://github.com/gersonmoraes/ocaml-sugar) is a monadic library to
help you build an error handling layer. It unifies many uses for error aware
expressions and integrates well with theading libraries. A proper introduction
on the library can be found
[here](https://ocaml.rocks/post/2017-10-error-handling-with-sugar-part1/). Its
documentation in [online](https://gersonmoraes.github.io/ocaml-sugar/), but you
can also read a beginner friendly intro on the
[subject](https://ocaml.rocks/post/2017-10-error-handling-intro/).
## Dryunit
[Dryunit](https://github.com/gersonmoraes/dryunit) is a detection tool for
traditional testing. It started as a PPX, but after a while the project was
simplified to be just a command line. Writing the extension was an interesting
learning process though.
Tests are detected with the helper of OCaml parser and preprocessed test suites
are cached to avoid performance penalties. Calling an extra shell for each test
file might seem unnecessary, but that leaves you free to implement tests using
any feature in OCaml, writing tests anyway you want. An introduction can be
found [here](https://ocaml.rocks/post/2017-10-dryunit-and-ocaml-testing/).
*Feedback and suggestions for future posts are welcome!*
Archive: https://discuss.ocaml.org/t/ann-decompress-0-7/995/1
Calascibetta Romain announced:
I'm happy to announce a new release of
[`decompress.0.7`](https://github.com/mirage/decompress/releases/tag/v0.7)
available for installation via OPAM.
Decompress is a library which provide an implementation of
[zlib](https://zlib.net/) in pure OCaml. This library provides the _same_
functions as [`camlzip`](https://github.com/xavierleroy/camlzip) (which is a
binding with the official implementation of `zlib`) and a more [low-level
API](https://mirage.github.io/decompress/api.docdir/Decompress.html) to keep the
control about the memory allocation.
A little example with an explanation is provided
[here](https://github.com/mirage/decompress/blob/master/bin/easy.ml) to
understand how to use `decompress`.
The goal of `decompress` is to provide a good `zlib` implementation which can be
compiled (by the ocaml compiler way) to some exotic back-end and keep the same
behaviour. In this way, `decompress` is one key on the Mirage OS project.
Obviously, `decompress` can be compiled by `js_of_ocaml` (tested on the last
ICFP to simulate a Git repository in your web browser).
This release is much more about fixed bug and a good example of [how to use
AFL](https://github.com/mirage/decompress/blob/master/fuzz/fuzz.ml). We fixed 2
bugs:
*
[eaeb1de](https://github.com/mirage/decompress/commit/eaeb1ded4ecf61cca07ccb2196a2d3de2d1f178d)
the first deflate algorithm (which uses the CPS style but appears only when the
input is less than 12 bytes)
*
[a6f6b2b](https://github.com/mirage/decompress/commit/a6f6b2ba0a3ba6dadf2ea13ef9a85aa71b106aca)
a variable name overlap in the dictionary inflate algorithm
About performance, we already did some works with @yallop and @samoht
(_defuntorization_, immutable state, etc.) in the
[0.5](https://github.com/mirage/decompress/releases/tag/0.5) release. And, with
[`landmarks`](https://github.com/LexiFi/landmarks), the ADLER-32 checksum seems
to be the biggest bottle-neck.
Finally, `decompress` is used by some projects like
[datakit](https://github.com/moby/datakit) or
[ocaml-git](https://github.com/mirage/ocaml-git) and we can consider (from the
[0.4](https://github.com/mirage/decompress/releases/tag/0.4) release) a stable
API - this new version exports the ADLER-32 implementation only. A package for
[npm](https://www.npmjs.com/) is in the pipe with this current version.
The next plan now is to focus `decompress` on the performance. Then, a plan to
implement `gzip` could be interesting.
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00060.html
Bruno Bernardo announced:
The next OUPS meetup will take place on Wednesday, Nov 8th, 7pm.
This meetup is hosted and sponsored by Facebook. It will take
place in its Parisian premises at 6 rue Ménars, Paris 2e.
https://her.is/2gCvNcT
The meetup event page is:
https://www.meetup.com/ocaml-paris/events/244437286/
Please register before the *1st of November*. We need your real name
(not a pseudonym) *and* an e-mail address. (Send me your details by
e-mail or via a direct message on meetup.com.)
Please note that registration is *really* mandatory: you will not be
able to enter the building if you are not in the list of
participants. Furthermore, you will be asked to show an *ID*.
Regarding the program, there will be four talks, followed by drinks
and food:
- Hugo Venturini: OCaml @ Facebook: overview
- Louis Gesbert: OPAM 2
- Frédéric Bour: a Qt binding for OCaml
- Jacques-Henri Jourdan: a statistical memory profiler.
The meetup organizers.
PS: Registration deadline is Nov. 1st, real name + e-mail, bring an ID!
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2017-10/msg00065.html
François Berenger asked and Milo Davis replied:
> Just to be sure, it seems there are no OCaml bindings for this
> library:
>
> https://www.csie.ntu.edu.tw/~cjlin/liblinear/
>
> Am I right or has anyone something under wraps that
> he would be willing to share with the OCaml community?
>
> My googling for such bindings was unsuccessful.
There are bindings for libsvm available here
https://bitbucket.org/ogu/libsvm-ocaml/. It's not lib linear, but it
should have the functionality you're looking for.
Here are links from many OCaml blogs aggregated at OCaml Planet,
http://ocaml.org/community/planet/.
Pearl No.4 - Kth Smallest in the Union of 2 Sorted Collections
http://typeocaml.com/2017/10/19/pearl-no-4-double-binary-search/
Coq 8.7.0 is out
https://coq.inria.fr/news/139.html
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.