Hello

Here is the latest OCaml Weekly News, for the week of September 22 to 29, 2015.

1. New release of Menhir (20150921)
2. Allocation profiler for bytecode interpreter
3. Core Suite 113.00.00
4. ppx_deriving 3.0
5. Jane Street is hiring interns!
6. ocaml-4.02.3 for iOS
7. ocaml-webmachine
8. ocaml-d3
9. polymorphic sets?
10. Stb_truetype, Stb_image, Stb_image_write 0.1
11. Assimp for OCaml 0.1

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00146.html

A new release of Menhir (20150921) is available via opam (just run "opam
install menhir") and from http://gallium.inria.fr/~fpottier/menhir/ .

The relevant chunk of the CHANGES file is appended at the end of this message.
Here is a summary of the most important changes:

- The Coq back-end (designed and implemented by Jacques-Henri Jourdan) has
  been documented. Yes, Menhir can produce a provably correct parser, written
  in Coq's programming language.

- The incremental API has been changed (in an incompatible way), and a new
  inspection API has appeared. Expect more incompatible changes in the future,
  as these APIs are not yet mature.

- Added support for anonymous rules. This allows writing, e.g.,
    list(e = expression SEMI { e })
  whereas previously one should have written
    list(terminated(e, SEMI)).

Best regards,

--
François Pottier
Francois.Pottier@inria.fr
http://gallium.inria.fr/~fpottier/

2015/09/21:
Re-established some error messages concerning the mis-use of $i which
had disappeared on 2015/06/29.

2015/09/11:
Fixed the mysterious message that would appear when a nonterminal symbol
begins with an uppercase letter and --infer is turned on. Clarified the
documentation to indicate that a (non-start) nonterminal symbol can begin
with an uppercase letter, but this is not recommended.

2015/08/27:
New option --inspection (added last January, documented only now). This
generates an inspection API which allows inspecting the automaton's stack,
among other things. This API can in principle be used to write custom code
for error reporting, error recovery, etc. It is not yet mature and may
change in the future.

2015/07/20:
Added the command line options --unused-token <symbol> and --unused-tokens.

2015/06/29:
Changed the treatment of the positional keywords $i. They are now
rewritten into variables of the form '_i' where 'i' is an integer.
Users are advised not to use variables of this form inside semantic
actions.

2015/02/11:
Added support for anonymous rules. This allows writing, e.g.,
list(e = expression SEMI { e })
whereas previously one should have written
list(terminated(e, SEMI)).

2015/02/09:
Moved all of the demos to ocamlbuild (instead of make).

2015/01/18:
Incompatible change of the incremental API.
The incremental API now exposes shift events too.

2015/01/16:
Fixed a couple bugs in Makefile and src/Makefile which would cause
compilation and installation to fail with "TARGET=byte". (Reported
by Jérémie Courrèges-Anglas and Daniel Dickman.)

2015/01/01:
Incompatible change of the incremental API.
The entry point main_incremental is now named Incremental.main.

2014/12/29:
Incompatible change of the incremental API.
The API now exposes reduction events.
The type 'a result is now private.
The type env is no longer parameterized.
[handle] is renamed to [resume].
[offer] and [resume] now expect a result, not an environment.

2014/12/22:
Documented the Coq back-end (designed and implemented by Jacques-Henri Jourdan).
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00149.html

As someone who does not yet have the expertise to clearly see the cost of
abstractions by looking at the source code, the lack of a freely available and
easily usable allocation profiler is often a hindrance for writing
high-performance OCaml code. I have implemented a simple allocation profiler
for the bytecode interpreter [1]. Part of the motivation to write one for
bytecode instead of native is to profile the multicore OCaml compiler, which
doesn’t support native compilation yet. I find it to be quite useful in
practice to get an overview of allocation bottlenecks before applying targeted
optimisations. 

The profiler is quite naive at this point. The tooling support is
non-existent; one has to manually search through the relevant text files to
figure out the source of allocations. I am interested in understanding how to
make this better. As a first step, I would like to do something similar to
`ocamlprof`. I plan to keep improving the allocation profiler for vanilla
OCaml as the multicore OCaml development continues.

Kind Regards,
KC

[1] http://kcsrk.info/ocaml/profiling/2015/09/23/bytecode-allocation-profiler/
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00167.html

I am pleased to announce the release of Core 113.00.00!

The following packages were upgraded:

- async
- async_extended
- async_extra
- async_inotify
- async_kernel
- async_parallel
- async_rpc_kernel
- async_smtp
- async_ssl
- async_unix
- bignum
- bin_prot
- comparelib
- core
- core_extended
- core_kernel
- core_profiler
- custom_printf
- email_message
- fieldslib
- incremental
- jenga
- ocaml_plugin
- pa_bench
- pa_ounit
- pa_structural_sexp
- patdiff
- pipebang
- re2
- rpc_parallel
- sexplib
- type_conv
- typerep
- typerep_extended

Files for this release are available on our website and all packages
are in opam:

  https://ocaml.janestreet.com/ocaml-core/113.00/files/

The changelog is large enough that I won't reproduce it here, see:

  https://ocaml.janestreet.com/ocaml-core/CHANGES.md
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00179.html

I am glad to announce ppx_deriving 3.0. It brings two
major changes: hygiene and ppx_type_conv compatibility.
It is installable via opam[1].

First, hygiene. Earlier versions of ppx_deriving:
  1) assumed that all the modules that the deriver
     requires are not shadowed with a different signature;
  2) included user-specified code such as from [@default]
     attributes in a way that exposed deriver-internal
     identifiers to such code.

Now, ppx_deriving includes a mechanism for hygienic
unquoting[2], as well as a module[3] containing references
to most standard library modules under their proper names.
This allows it to coexist with e.g. `open Core.Std`.

Second, ppx_type_conv compatibility. You probably don't
know it yet, but Jane Street has converted all their syntax
extensions to ppx, so now there's ppx_type_conv, ppx_sexp_conv,
etc. I've coordinated the development of ppx_deriving with
Jane Street, so now ppx_deriving and ppx_type_conv use
the same syntactic conventions. In fact, they share
the [@@deriving] syntax without any effort from the user!

Furthermore, I have been looking at the sources of Jane Street's
ppx_core, ppx_driver, ppx_type_conv, etc and I find these
projects exceptionally well designed; further, they tackle some
important problems that ppx_deriving has explicitly left out of
scope, such as attribute hygiene.

While these libraries have a somewhat larger API, I think anyone
writing a new deriver should consider using ppx_type_conv.
I cannot outright recommend that since I have not actually
tried it (yet?), but it sounds worthwhile.

[1]: http://opam.ocaml.org/packages/ppx_deriving/ppx_deriving.3.0/
[2]: https://whitequark.github.io/ppx_deriving/Ppx_deriving.html#2_Hygiene
[3]: https://whitequark.github.io/ppx_deriving/Ppx_deriving_runtime.html
      

Jeremie just air-dropped about 25k lines of PPX code on github and opam, so
OCaml's PPX tools are finally out there. You can find them here:

https://github.com/janestreet
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00182.html

Jane Street is actively hiring summer interns for our offices in New
York, London and Hong Kong.

Interning at Jane Street is a great learning experience.  If you're
interested in seeing how functional programming is applied to real
world problems at big scale, there's no better place.

Look here

    https://blogs.janestreet.com/?p=962

to get a sense of the kinds of projects summer interns do.  (I'm
preparing an update for this summer, but haven't gotten it done quite
yet.)  As is reflected there, many of our intern projects make their
way out as open-source projects.  Interns also learn about Jane
Street's trading business through lectures and interactive training
sessions.  Plus, there are a lot of fun social activities throughout
the summer, and interns get to visit one of the other two offices for
a week.

Feel free to redistribute this to any students you think might be
interested.  And if you're interested directly, you can apply here:

    http://janestreet.com/apply

And as usual, we're also hiring developers for fulltime positions in
NYC, Hong Kong and London.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00186.html

I rebased our development branch to 4.02.3, and did few quick tests, and
everything seems to work. At this point I'd like to invite all iOS
developers to check out our work, and help us finding the last bugs:

https://github.com/gerdstolpmann/ocaml/tree/gs-4.02.3+ios

Problems should be discussed here:

http://caml.inria.fr/mantis/view.php?id=6986

What you get is a cross compiler for either armv7, armv7s, or arm64. See
the instructions below how to install.

This branch is based on Jeffrey Scofield's work to port ocaml to (32
bit) iOS. We (Gerd, Jeffrey and Mark) added support for armv7s and
arm64, and tried to better adapt to the coding standards of the ocaml
sources. There is an effort to merge this branch into mainline OCaml.

So how to install:

 - You need a host ocaml compiler for Mac, and this compiler must also
   be version 4.02.3, and the word size must match the word size of the
   target (i.e. 64 bit for arm64, and 32 bit for armv7/armv7s). Please
   take these requirements seriously - they are checked. The
   host compiler must be in your PATH. (NB. For getting a 32 bit
   host compiler please take http://caml.inria.fr/mantis/view.php?id=6900
   into account.)
 - Clone:
   git clone https://github.com/gerdstolpmann/ocaml/tree/gs-4.02.3+ios
 - Look at the checked-out build.sh script. Modify the variables for
   platform, target, and SDK version.
 - Configure and build: ./build.sh -prefix /where/you/want/it
 - Install: make install

What you get:

 - The installed compilers remain dependent on the host installation
   (so don't delete it after building). In particular, the "ocamlrun"
   executable of the host is still used.
 - The installed runtime is for the target, in particular all the
   libraries (such as libocamlrun.a, and libasmrun.a)
 - The native-code compiler ocamlopt generates executables, but you
   can also emit object files (-output-obj) that can be linked with
   any existing app. You'll also need libasmrun.a in this case.
 - For completeness, you can also use ocamlc -custom to generate
   stand-alone bytecode executables. I guess this is less interesting,
   and I'm not sure whether Apple would allow this format in the app
   store.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00212.html

I'm happy to announce the initial release of webmachine:

https://github.com/inhabitedtype/ocaml-webmachine

Webmachine provides a layer on top of cohttp that implements a
state-machine-based HTTP request processor. It's particularly well-suited for
writing RESTful APIs. As the name suggests, this is an OCaml port of the
webmachine[1] project.

Here are well-commented async- and lwt-based examples to help you get started.

https://github.com/inhabitedtype/ocaml-webmachine/blob/master/examples/hello_async.ml
https://github.com/inhabitedtype/ocaml-webmachine/blob/master/examples/hello_lwt.ml

Issues and pull requests welcomed.

-Spiros E.

[1]: https://github.com/webmachine/webmachine
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00213.html

I'm happy to announce the initial release of ocaml-d3:

https://github.com/seliopou/ocaml-d3

ocaml-d3 provides OCaml bindings to D3.js[1], via js_of_ocaml. Using this
library will enable you to create type-safe, composable widgets using HTML,
SVG, and CSS. The binds at the moment only cover the Core Selection API. For
visualization uses cases, the additional library code will have to be
manipulated directly with the js_of_ocaml FFI.

These are the second languages bindings I've written for D3.js. The first were
for the Elm programming language[3]. Both are quite similar to each other in
terms of their treatment of operations and types. However, due to the
computational model of Elm, the bindings for that language do not allow you to
directly get a handle on DOM nodes. The OCaml version lifts this restriction.
But, I would suggest using that feature sparingly.

Here's a TodoMVC example to help you get started:

https://github.com/seliopou/ocaml-d3/blob/master/examples/todo.ml

Issues and pull requests welcomed.

-Spiros E.

[1]: http://d3js.org/
[3]: https://github.com/seliopou/elm-d3
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00215.html

Is it possible to implement a polymorphic sets module over the Set
module provided in OCaml's standard library?

By polymorphic set, I mean a set whose elements could be of type 'a
(like for lists) and the equality funciton would be the one provided by
OCaml.

So one would have for instance

val add : 'a -> 'a t -> 'a t

etc.

Is that possible somehow?
      

I don't think this is possible.

In Batteries, we ported (duplicated) the existing implementation of OCaml's
Set and Map modules to give them a "concrete" definition, with no abstraction
and with each function parametrized over a comparison operation, on top of
which both a functorized and a polymorphic interface are built. You may be
interested in the code:

https://github.com/ocaml-batteries-team/batteries-included/blob/master/src/batSet.ml
https://github.com/ocaml-batteries-team/batteries-included/blob/master/src/batMap.ml

Note that the polymorphic sets (or maps) are less statically-safe than their
functorized counterpart, because they are parametrized over their comparison
function at creation time (a better choice, I think, that enforcing the use
polymorphic comparison, think of records with a function parameter for
example), but then you can mix two sets with same carrier type but different
ordering, and the ordering of the result may not be what you expect. To
counter this, the documentation of the polymorphic interface is careful to
precisely define, for functions taking two set arguments, which of the
comparison functions is used in the result set; so the semantics is
maybe-surprising but well-defined. 

On a related note: I believe that a good base library should provide two
separate kinds of modules: concrete modules implementing a particular data
structure (AVL trees, red-black trees, HAMT), and abstract modules build on
top of it that use whatever implementation is best today to implement useful
interfaces (set, bag, associative map...).
The abstract modules can only be extended with definitions given in terms of
the abstract interface, but concrete modules allow them to easily define
abstract modules extended with functions relying on the underlying concrete
data-structure-manipulation primitives.

Currently with the standard library you only have abstract module, so you have
to re-implement them on your own or break the abstraction boundary in unsafe
ways.
      

> The only point in time I've had troubles is when you have a map where its
> values and keys range over a type that can recursively contain that map of
> the same type. This required a polymorphic compare (from Core) to pull off.

Core_kernel's maps and sets solve this problem, and address Gabriel's
concern as well, by providing type witnesses that distinguish maps
built with different comparators.  There's an introduction to the use
of these modules here:

https://realworldocaml.org/v1/en/html/maps-and-hash-tables.html

and you can see the current APIs here:

https://github.com/janestreet/core_kernel/blob/master/src/core_map.mli

Sets are handled similarly.For sure, better inlining (as is coming in
Flambda!) can make this approach better.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00220.html

I'm happy to announce the release of bindings to three _stb_ libraries for
OCaml.

Stb libraries are made by Sean Barrett, http://nothings.org/ .
They are small, self-contained and highly portable: no dependencies, one C
file.
They have a well-defined scope and are released into the public domain.

The OCaml bindings are licensed under CC-0.

Stb_truetype is a truetype (*.ttf) font rasterizer.
Stb_image is a JPEG, HDR, PNG, BMP, TGA, PSD, GIF, PIC, PNM image loader.
Stb_image_write is a PNG, BMP, TGA, HDR image writer.

These libraries are originally targeted at game engines.
They offer decent performances while not being optimized to the extreme. In
practice, I observed loading times slightly slower than libjpeg-turbo and
faster than libjpeg.
The writer doesn't try to compress optimally: you will be able to easily
produce image files, but not the smallest possible.

Being self contained, they are easy to deploy and make almost no assumptions
on the host system (compared to e.g. CamlImages).
I have successfully used them in conjunction with Tgls,
https://github.com/dbuenzli/tgls , but they can be used in any situation where
bigarrays and native C code are allowed.

However, be aware that they don't try to be robust against file corruption
(loading might fail) and attacks / forging (libraries are obviously less
audited than the common libpng, libjpeg, ...).

They are all available as Opam packages:

opam install stb_image stb_image_write stb_truetype

Webpages:

https://github.com/def-lkb/stb_image
https://github.com/def-lkb/stb_image_write
https://github.com/def-lkb/stb_truetype

Issues and pull requests welcomed, although this is a hobby project: I make no
guarantee on my ability to respond. 
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-09/msg00222.html

I'm happy to announce the release of bindings to Assimp for OCaml.

Assimp is the Open Asset Import Library, an open-source libraries for
importing 3d scenes in various popular formats. It is released under
BSD-license.

The OCaml bindings are licensed under CC-0.

Assimp version 3 or greater is required, it is available in recent versions of
Debian (libassimp-dev), Ubuntu, Archlinux...

Basic scene data and post-processing functions are exposed. The bindings are
not complete but fully usable.

Install through opam:

opam install assimp

Webpage:

https://github.com/def-lkb/assimp

Issues and pull requests welcomed, although this is a hobby project: I make no
guarantee on my ability to respond. 
      

Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt