Previous week Up Next week


Here is the latest Caml Weekly News, for the week of September 09 to 16, 2008.

  1. New release of Menhir
  2. ocamlnet and kernel poll

New release of Menhir


François Pottier announced:
It is my pleasure to announce a new release of Menhir, with the following
main improvements over previous versions:

 --table       Menhir now supports producing table-based LR automata, in
               the tradition of yacc, bison, and ocamlyacc. This makes
		the generated parsers up to 5x smaller, and somewhat slower.

 --interpret   Menhir can now be used not just as a compiler, but also as
               an interpreter. It will read sentences off the standard
		input channel, parse them as per your grammar, and report
		an outcome. This should help debug grammars.

These new features were implemented by Guillaume Bau, Raja Boujbel, and
François Pottier. We would like to gratefully acknowledge the generous support
of Jane Street Capital LLC, who funded this endeavor through an ocaml summer

As usual, the new release is available either through GODI, or as source code
at the following URL.

Enjoy! Comments and bug reports are welcome.
Julia Lawall asked and François Pottier replied:
> I was very excited about this option, because it drops the size of the 
> generated code for my parser from 61K LOC to 28K LOC.  But unfortunately 
> it gives me:
> Unbound module MenhirLib.TableInterpreter.Make
> when I try to compile.

Yup. I didn't write the details in the announcement; maybe I should have.

When a parser is produced using --table, it is not quite stand-alone: it
must be linked with a new library, called MenhirLib. (This is analogous
to ocaml's Parsing module, which is part of the standard library.)

If you are using ocamlfind, this is quite easy: just add "-package menhirLib"
to your ocamlc/ocamlopt flags (for compiling and for linking), and add
"-linkpkg" to your ocamlc/ocamlopt flags (for linking).

If you do not wish to rely on ocamlfind, then things become slightly more
complicated, since you must tell ocamlc/ocamlopt where MenhirLib is installed.
Fortunately, Menhir itself can help you: it has three new command-line
switches, of the form --suggest-*, which cause it to print suggested flags.
The details are in Menhir's reference manual. The sample Makefile
(demos/Makefile.shared) offers an illustration.

ocamlnet and kernel poll


Joel Reymont asked and Gerd Stolpmann replied:
> Suppose I want to build a server that runs on top of ocamlnet and  
> handles 10k+ connections.
> ocamlnet seems to use select exclusively.

There's also netplex in ocamlnet, which is a kind of fork framework. I
don't know which protocol you want to use. For instance, you can arrange
that netplex starts 100 processes and every process handles 100

> Any suggestions on how to add kernel poll? Is this possible even?

I'm in the middle of changing ocamlnet here, so we can have more
flexibility for choosing the kind of event polling. Look at the svn

There's a lot of new stuff in the src/netsys directory. In the future,
all polling will be based on the pollset class type
(src/netsys/netsys_pollset.mli). There is currently an implementation
that uses the poll() syscall (src/netsys/netsys_pollset_posix.mli) and
another one for Win32 (src/netsys/netsys_pollset_win32.mli - limited to
sockets and named pipes for now).

As this is a class type you can also go ahead, and implement it for
every kernel mechanism you like, and just you your class instead of the
classes provided by ocamlnet. I'll add some more mechanisms later (any
help is welcome).

You can turn these pollsets into event_system by using
src/equeue/unixqueue2.mli. This is not very much tested, however, and
there is no support for multi-threading yet in this module. Many
ocamlnet modules allow to inject whatever event_system you like to have
(should be all modules in the future).

All this is experimental for now. I have tested it only with small
programs, not with large production systems. But maybe it's an option
for you to use the svn version.
Markus Mottl also replied:
The Core-library that we developed at Jane Street, and which is also
in Godi, contains a module "Linux_ext", which has a fully-featured
interface to epoll (besides lots of other Linux-specific goodies).

There you just call "Epoll.create" to get a file descriptor on which
to listen for I/O-events.  Then you only need to add other file
descriptors you want to monitor with "Epoll.add", specifying flags for
the kind of events you want to wait for.  "Epoll.modify" and
"Epoll.del" modify event flags of and remove monitored descriptors
respectively.  "Epoll.wait" allows you to wait for received events
(similar to "select").

Note, however, that "select" is usually more efficient for small (=
tens) numbers of descriptors!
Mattias Engdegård also replied:
There is an ocaml wrapper for libevent:

With that many connections, you may actually benefit from multiple
processors, which in the OCaml world usually means multiple processes.

There's nothing wrong with serving myriads of connections from a
single process, especially when using a strongly typed language, but
parallelism can be useful. On the other hand, one process per
connection may be inefficient as well - a hybrid N:M solution is
probably best.

Using folding to read the cwn in vim 6+

Here is a quick trick to help you read this CWN if you are viewing it using vim (version 6 or greater).

:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'<1':1

If you know of a better way, please let me know.

Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt