Hello
Here is the latest Caml Weekly News, for the week of November 16 to 23, 2010.
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/8652e1563fcfdc01#
In this very long thread, Wolfgang Draxinger asked and Sylvain Le Gall suggested:
> So, what're the developments regarding SMP multithreading OCaml?
There are various development regarding this subject (most recent
first):
- Plasma (MapReduce in OCaml)
http://plasma.camlcity.org/plasma/index.html
- OC4MC (OCaml for MultiCore)
http://www.algo-prog.info/ocmc/web/
- ocamlp3l
http://camlp3l.inria.fr/eng.htm
- jocaml
http://jocaml.inria.fr/
- ocamlmpi
http://forge.ocamlcore.org/projects/ocamlmpi/
All these projects try to tackle the challenge of SMP from different
point of view. Maybe you'll find what your answer in one of them.
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/a3f3d9ee6deb7344#
Daniel de Rauglaudre announced:
New release of "ledit" : 2.02.
Ledit is a line editor to be used before an interactive command,
allowing you to edit input lines (cut, paste, move cursor forward
and backward and in history, balance parentheses and so on).
Examples:
$ ledit ocaml
$ ledit ocamldebug
$ ledit ed
and so on...
New feature : TAB for 'complete-file-name'
When you type 'tab', ledit tries to complete what is before the cursor
considered as a path, and displays the possible files if there are
several of them, just like common shells do.
Useful when you have forgotten the file name you want to specify.
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/437242c094e335c3#
Goswin von Brederlow asked and Jérémie Dimino replied:
> I'm looking for stubs for
>
> ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
> ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>
> Specifically I need those to send (among normal messages) an
> Unix.file_descr over a Unix Domain Socket.
>
> Does anyone know of a module that has them?
The Lwt_unix module has them [1].
Cheers,
Jérémie
[1] http://ocsigen.org/lwt/doc/api/Lwt_unix.html#VALrecv_msg
Dave Scott also replied:
We have some bindings for those:
https://github.com/xen-org/xen-api-libs/blob/master/stdext/unixext.mli
external send_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag
list -> Unix.file_descr -> int = "stub_unix_send_fd_bytecode"
"stub_unix_send_fd"
external recv_fd : Unix.file_descr -> string -> int -> int -> Unix.msg_flag
list -> int * Unix.sockaddr * Unix.file_descr = "stub_unix_recv_fd"
You might prefer to extract the relevant functions from the code -- there's a
lot of other misc stuff in that repo which you're probably not interested in.
We use those functions quite a lot so hopefully they'll work for you.
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/a8b3d15a3cba60f0#
Benedikt Meurer announced:
OCamlJit 2.0 is a new Just-In-Time engine for Objective Caml 3.12.0 on desktop
processors (x86/x86-64). It translates the OCaml byte-code used by the
interpreter (ocamlrun and ocaml) to x86/x86-64 native code on-demand and runs
the generated native code instead of interpreting the byte-code. It is
designed to run with minimal compilation overhead (translating only what is
being executed, avoiding costly code generation and optimization techniques),
while being 100% compatible with the byte-code runtime (including
serialization and hashing of closures, etc.).
OCamlJit 2.0 was specifically designed for desktop processors and is not
really portable to anything else in its current shape, because the target
audience are people using the interactive top-level and the byte-code
interpreter for rapid prototyping/development (which is unlikely to happen on
anything else but x86/x86-64). The implementation currently requires a system
that adheres to the SysV ABI, which includes Linux, BSD, OS X, but excludes
Win32/Win64 (patches/ideas are welcome). It was tested on Linux/x86 (Debian),
Linux/amd64 (CentOS) and Mac OS X 10.6 (64bit). The x86 implementation
requires SSE2 capable processors (otherwise it falls back to the byte-code
interpreter), so it won't speedup your OCaml programs running on 486 CPUs. :-)
OCamlJit 2.0 runs most benchmarks at 2-6 times faster than the byte-code
interpreter. The interactive top-level benefits twice when used with the JIT
engine: (a) the compiler stages are JIT compiled and (b) the generated
byte-code is JIT compiled. A tech report describing a slightly earlier
prototype and including performance measures of OCamlJit 2.0 on Mac OS X
(64bit) is available at:
http://arxiv.org/abs/1011.1783
The source code is available from the Git repository (master branch) at:
http://gitorious.org/ocamljit2/ocamljit2
Installation is similar to installation of Objective Caml, just run
./configure -prefix /path/to/ocamljit2 [options]
followed by
make world opt
make install
This will install a fully working Objective Caml 3.12.0 to /path/to/ocamlji2,
where /path/to/ocamljit2/bin/ocamlrun and
/path/to/ocamljit2/lib/libcamlrun_shared.so include the JIT engine in addition
to the byte-code interpreter (fallback to the byte-code interpreter is
necessary for debugging with ocamldebug). The configure script prints a line
indicating whether the JIT engine is enabled or not (if not, it'll be just a
regular OCaml 3.12 installation).
Comments are welcome.
bluestorm asked and Benedikt Meurer replied:
> To those of you who are lazy but still curious, I just read the report, and
> here are the answers to the question I had:
Thanks for posting these points, should have done this in my original post...
> 1. Is that project related to Basile Starynkevitch's venerable OCamlJIT ?
>
> Yes, OcamlJIT was apparently a major inspiration for this work. The overall
> design is similar, and in particular the objective of absolute compatibility
> with the existing bytecode (even when it may hurts performances) was retained.
> Unfortunately, due to bitrotting and different architectures (OcamlJIT x86 vs.
> OcamlJIT2 x86_64), there is no direct performance comparison, though the
> reports seems to indicate similar to better performances. Several points were
> also improved (better (but still suboptimal) register usage, clever hacks to
> speed up mapping between bytecode and native code adresses...).
With the x86 port being close to complete, direct comparison with OcamlJIT
will be possible (dunno if OcamlJIT will work with 3.12 tho). I'll do
appropriate comparisons once everything is in place.
> 2. Does it use LLVM ?
>
> No, it doesn't, but it's justified. Apparently, an early prototype showed than
> LLVM compilation/generation overhead was too high. A (debatable) design goal
> of OcamlJIT2.0 is to be absolutely faster than ocamlrun, even on *very*
> short-running programs (`ocamlc -help`).
This is indeed debatable, atleast for "ocamlc -help". But this was not the
main concern with LLVM. LLVM overhead is acceptable for long running
computations, but everything else is slowed down noticably. It should also be
noted that my LLVM prototype was rather quick&dirty, so it may indeed be
possible to get a LLVM based JIT which is on par with the byte-code
interpreter for common applications. But why would one want to do this? Long
running computations can be speed up very well using ocamlopt (no need to
perform them using the interactive top-level).
> OCamlJIT2.0 uses its own macro to generate x86_64 assembly directly
> (apparently using a dedicated library wasn't worth it). A drawback of this
> strategy is its inherent non-portability.
It used AsmJit before, which is also limited to x86/x86-64; the
non-portability is actually a design decision (as explained in my original
post, portability to non-desktop machines was not a goal for the current
implementation).
Alain Frisch suggested:
This looks like a very interesting project!
Does performance really matter that much for rapid prototyping/development? I
can imagine other uses of the toplevel where performance matters more, like
theorem provers embedded in the OCaml toplevel.
Anyway, another option to get a more efficient interactive top-level is to
turn to native code.
There is actually already a native top-level in the distribution, even though
it is undocumented and unmaintained. You can build it with the "make ocamlnat"
target. The implementation is based on the same approach as native dynlink.
The toplevel embeds the native compiler; for each phrase, the toplevel
produces assembly code, calls the assembler and the linker to produce a
dynamic/shared library, and then dynamically load and execute the resulting
code. This gives some latency, but it's not so bad in practice, and you get
the full speed of native code.
A further step to improve this native toplevel is to avoid the call to the
external assembler and linker. To do that, one basically needs to replace the
assembly code emitters (emit.mlp/emit_nt.mlp) with native code emitters and do
the relocation/dynamic loading by hand, which is quite easy.
As it turns out, LexiFi uses on a daily basis such direct binary code emitters
for x86/amd64 on Windows, and that would be easy to port to other x86/amd64
platforms. The x86 version was developed internally, and the amd64 version was
done by Fabrice Le Fessant. There is also some code to wrap the binary code
into COFF objects (and flexdll has a stand-alone mode to produce .cmxs files
without an external linker), but that would be useless for a native toplevel.
The goal was to have a compiler which can be more easily embedded in our
applications and deployed to our customers, without depending on any external
tool.
If you Benedikt, or someone else, is willing to work on a native top-level
without the need to call an external assembler/linker, we are ready to extract
the binary code emitters from our code base and share them with the community
(with an open-source license to be determined). This requires some packaging
work on our side, so we're going to do it only if there is interest in the
project.
(Another nice side project, for having a good native toplevel, would be to
extend ocamlopt with some control over the trade-off between speed of code
generation, and the performance of the generated code. One way to do it is to
force the interference graph to remain not too big, by spilling early, in
order to avoid quadratic behavior in the register allocator.)
Thanks to Alp Mestan, we now include in the Caml Weekly News the links to the
recent posts from the ocamlcore planet blog at http://planet.ocamlcore.org/.
RegSTAB 1.4.6 released:
http://forge.ocamlcore.org/forum/forum.php?forum_id=730
ledit 2.02:
http://caml.inria.fr/cgi-bin/hump.cgi?contrib=301
OCamlODE 0.6:
http://caml.inria.fr/cgi-bin/hump.cgi?contrib=454
Idea for OCaml browser extension:
http://www.elehack.net/michael/blog/2010/11/ocaml-extension
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.