Hello

Here is the latest OCaml Weekly News, for the week of July 07 to 14, 2015.

1. stog 0.15.0 and friends
2. Native compiler for oCaml on System Z
3. mqueue 0.5.3
4. Scanning objects outside the OCaml heap
5. Other OCaml News

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-07/msg00023.html

I'm glad to announce the release 0.15.0 of Stog, a static website
compiler: https://zoggy.github.io/stog/index.html .

Changes are available from the release page:
  https://zoggy.github.io/stog/posts/release-0.15.0.html

One of the news is the introduction of the stog-tmpl tool, to generate
website skeletons. It was used for example to create website of
Ocf [http://zoggy.github.io/ocf/].

The stog package should be available soon in opam.

Two plugins are also released and numbered 0.15.0:
- stog-rdf [https://zoggy.github.io/stog/plugins/rdf.html]
    to specify RDF triples within documents, to produce a RDF graph for
    the whole generated site. It also permits executing Sparql queries
    to include data in the generated pages.

- stog-writing [https://zoggy.github.io/stog/plugins/writing.html]
    add new rewrite rules to use footnotes and bibliographies in
    your documents.

They will also be available in opam.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-07/msg00030.html

> Hello. I'm with the IBM Linux on Z Open Source Ecosystem Center of
> Competency in Toronto. I would like to provide (and maintain) a port of the
> OCaml native compiler for Linux on System Z (a.k.a IBM 390 or s390x). Can
> someone advise me on whom to contact to begin discussions on how to make the
> port available to the wider OCaml community? Ideally it would be great if it
> could ultimately become part of the standard OCaml distribution. Thanks for
> any advice you can give.

A possible way might be to not directly target SystemZ but to leverage on some
of the efforts which
either have translated the Ocaml bytecode to C, or to consider writing some
JIT backend for Ocaml;
I'm thinking of patching the Ocaml native compiler to use LLVM or more
probably the very new
GCCJIT https://gcc.gnu.org/onlinedocs/jit to leverage on the optimization
abilities of GCC middle-end & back-end etc.

There are indeed some issues, in particular compatibility between the garbage
collector and what such JIT libraries are able to generate (tail recursion is
less of an issue at least in GCCJIT).
      

Targeting LLVM or GCCJIT is an interesting but separate project.
At least one attempt were made in the past to target LLVM, running into
problems with the (exact) GC interface of OCaml.  Maybe LLVM improved
recently in this department, and maybe GCCJIT can handle it too.  If
not, I think it's a dead end.
      

> For posterity, and for the sake of curiosity, might someone in the
> know succinctly explain what the (exact) GC interface road blocks
> were? Thank you!

There were in fact at least two serious attempts at a LLVM port:
- one by Colin Benner,
https://github.com/colinbenner/ocamlllvm
http://oud.ocaml.org/2012/abstracts/oud2012-paper7.pdf
(the paper above briefly discusses GC issues)
- one by Raphaël Amiard,
https://github.com/raph-amiard/CamllVM
https://www.irill.org/blog/camllvm-a-llvm-based-runtime-for-ocaml
(in the spirit of University Pierre et Marie Curie, this compiler goes from
OCaml bytecode to LLVM programs, instead of being a harder-to-deploy
native-code compiler backend)

Both cited the GC interface and the slowness of exceptions (relative to the
exceptionnally fast OCaml runtime exception handling on which many programs
rely). These difficulties had been correctly foreplanned by Xavier Leroy in
his 2009 caml-list message
http://caml.inria.fr/pub/ml-archives/caml-list/2009/03/3a77bfcca0f90b763d127d1581d6a2f1.en.html

There has been recent developments on the GC support front in the LLVM
community, notably the work of Philip Reames on "statepoints" as an
alternative approach to GC support (that should support eg. passing roots in
registers):
http://www.philipreames.com/Blog/2014/10/21/statepoints-vs-gcroot-for-representing-call-safepoints/

Regarding exceptions, I would guess a necessary evil would be to do more
static analyses inside the OCaml backend to detect when exception usage can be
refined to less generic constructs (eg. the experiments of Alain Frisch on
"static raise": https://www.lexifi.com/blog/static-exceptions ). Degraded
exceptions are also a source of performance issues on other backends such as
js_of_ocaml.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-07/msg00037.html

I’m please to announce the first release of “mqueue", OCaml bindings to POSIX
message queues.

The project page is on the forge: http://mqueue.forge.ocamlcore.org/ 
You can also find the API there:
http://mqueue.forge.ocamlcore.org/doc/Mqueue.html
The library is available via opam.
      

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2015-07/msg00020.html

> I'm interested in implementing a custom memory management strategy
> that I can prove is safe in a language I will compile to OCaml. I'm
> thinking I'd like to allocate freeable memory outside the OCaml heap,
> but as OCaml values. Since these values can point to OCaml
> heap-resident data, they need to be scanned. But I don't want to have
> to explicitly register/deregister them as roots to the GC, or track
> all mutations to the values, if I can help it. Rather, having a single
> "persistent root" to my memory area, for purposes of scanning, would
> be ideal.
>
> Thanks in advance for any help,

There is an undocumented and unexported way to do something like that:
the caml_scan_roots_hook.
You can register a callback in this global variable that takes a
scanning action (another callback) to call
on every of your roots.

See https://github.com/ocaml/ocaml/blob/trunk/byterun/caml/roots.h#L34
for the definition
and https://github.com/ocaml/ocaml/blob/trunk/asmrun/roots.c#L270 as an
example of how the scanning
action should be called.

Since those roots are scanned only at the end of a gc major cycle, you
must handle mutation specificaly (like
mutations in the major heap). You will have to track them as you must
not have unnoticed pointers from outside
of the minor heap to the minor heap. You will have to addapt caml_modify
as it suppose that values not in the minor
are in the major heap. Since it is declared using a weak symbol, this
can be redefined without changing the
runtime. See http://caml.inria.fr/mantis/view.php?id=6084 .

Take care, as this is not exported in the official interface, this could
break at any version change of the runtime.
Have fun with your potential hard to debug segfaults ;)
      

> Does anyone know: Is there any way to make outside memory scannable by the
> GC?

The library of delimited control delimcc

        http://okmij.org/ftp/continuations/caml-shift.tar.gz

in native mode has to perform a custom scan of pieces of memory stored
outside the OCaml heap. Captured delimited continuation is essentially
a portion of stack. Native OCaml uses stack for boxed and unboxed
values. Thus the captured continuation is a curious data structure
that contains both boxed (OCaml) and unboxed (machine) values. Delimcc
stores captured continuation outside the OCaml heap and has to scan
them at GC time.

The file stacks-native.c in caml-shift.tar.gz shows how the
interaction with GC is done. BTW, when installing a new GC call-back,
don't forget the previously installed one.
      

Thanks to Alp Mestan, we now include in the OCaml Weekly News the links to the
recent posts from the ocamlcore planet blog at http://planet.ocaml.org/.

ocaml-fileutils 0.5.0 released:
  https://forge.ocamlcore.org/forum/forum.php?forum_id=925

Converting a code base from camlp4 to ppx:
  https://blogs.janestreet.com/converting-a-code-base-from-camlp4-to-ppx/
      

Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt