Hello
Here is the latest Caml Weekly News, for the week of December 12 to 19, 2006.
We are proud to announce the latest release of the OMake Build System -
OMake version 0.9.8.
OMake is a build system designed for scalability and portability. It
uses a syntax similar to make utilities you may have used, but it
features many additional enhancements, including the following.
- Support for projects spanning several directories or directory
hierarchies.
- Fast, reliable, automated, scriptable dependency analysis using MD5
digests, with full support for incremental builds.
- Fully scriptable, includes a library that providing support for
standard tasks in C, C++, OCaml, and LaTeX projects, or a mixture
thereof.
Often, a configuration file is as simple as a single line
.DEFAULT: $(OCamlProgram prog, foo bar baz)
which states that the program "prog" is built from the files foo.ml,
bar.ml, and baz.ml. This one line will also invoke the default
standard library scripts for discovering implicit dependencies in
OCaml files.
- Full native support for rules that build several files at once.
- Portability: omake provides a uniform interface on Linux/Unix
(including 64-bit architectures), Win32, Cygwin, Mac OS X, and other
platforms that are supported by OCaml.
- Built-in functions that provide the most common features of programs
like grep, sed, find, and awk. These are especially useful on Win32.
- Active filesystem monitoring, where the build automatically restarts
whenever you modify a source file. This can be very useful during
the edit/compile cycle.
- A built-in command-interpreter osh that can be used interactively.
OMake preserves the style of syntax and rule definitions used in
Makefiles, making it easy to port your project to OMake. There is no
need to code in Perl (cons), or Python (scons). However, there are a few
things to keep in mind:
1. Indentation is significant, but tabs are not required.
2. The OMake language is functional: functions are first-class and
there are no side-effects apart from I/O.
3. Scoping is dynamic.
OMake is licensed under a mixture of the GNU GPL license (OMake engine
itself) and the MIT-like license (default configuration files).
OMake version 0.9.8 in major feature enhancements and bugfixes release.
The changes in this version include:
- The conversion from OMake expressions to shell command lines went
through a major redesign to make it more consistent and less
ad-hoc.
- OMake documentation was significantly reorganized.
- OCaml build rules: Added experimental improved dependency scanner
using the upcoming "ocamldep -modules" feature; added support for
the Menhir parser-generator.
- Added a number of new options to control OMake's output and
verbosity; OMake is now more silent by default.
- Many more improvements and bug fixes.
For a more verbose change log, please visit
http://omake.metaprl.org/changelog.html#0.9.8 .
Source and binary packages of OMake 0.9.8 may be downloaded from
http://omake.metaprl.org/download.html . In addition, OMake
may be obtained via the GODI packaging system.
To try it out, run the command "omake --install" in a project directory,
and modify the generated OMakefile.
OMake 0.9.8 is still an alpha release. While we have made an effort
to ensure that it is bug-free, it is possible some functions may not
behave as you would expect. Please report any comments and/or bugs to
the mailing list omake@metaprl.org and/or at http://bugzilla.metaprl.org/
As part of my master's thesis I've written a few patches to the Ocaml
3.09.2 compiler.
You can find them under: http://rainbow.mimuw.edu.pl/~pw189451/ocaml/
Patch:
export.patch - lets you specify signatures of submodules of packed
module to be included in signature of the packed module.
It adds command line argument -export "<module_name1> <module_name2>..."
extsig.patch - makes compiler to search for signature of the given name
in .cmi files.
You can write: "import Sig" in mli file and it makes compiler to
look for sig.cmi file and load signature from the file.
sigops.patch - lets you create new signature based on source signature
by specifying which fields of the signature should/should not be exported.
You can write "include Sig without (val foo and type bar)" or
"include Sig using (val rhu and type barb)"
retyping.patch - lets you retype bytecode module to use new signature
being subtype of the signature of the module
It adds command line argument "ocamlc Mod.cmo -retype -o
new/Mod.cmo" - if there is Mod.cmi in "new" directory it will try to
make it signature of the new/Mod.cmo module.
whole.patch - patch being sum of all the above mentioned patches.
could somebody tell me how to compile ocaml sources into a shared
object that can be loaded with dlopen? The following message
seems to tell me that this is possible:
http://caml.inria.fr/pub/ml-archives/caml-list/2001/12/e2092b52999522eaa0bbc099b07cf4e1.en.html
I tried
ocamlopt -output-obj -o ocaml-dlex.so dlex.ml
but on
dlopen ("/home/tews/src/ocaml/ocaml-dlex.so", RTLD_LAZY);
I get
dlopen error: /home/tews/src/ocaml/ocaml-dlex.so: only ET_DYN and ET_EXEC can be loaded
Passing -ccopt -shared to ocamlopt changed nothing. In fact it
seems that -output-obj discards all -ccopt options!
Richard Jones answered:It's tricky, but I managed to compile the bytecode interpreter and bytecode into a shared object. See the source for mod_caml.so: http://download.savannah.nongnu.org/releases/modcaml/ (actually there's a much newer version in CVS, but I don't think the linking issue changed at all). For AMD64, see also: http://merjis.com/developers/mod_caml/amd64Oleg also answered:
It is possible: MetaOCaml in the native mode does exactly that. To be
more precise, evaluating `.! .<code here>.', that is, `running' the
code expression, is equivalent to invoking ocamlopt (as a library) to
compile `code here', making a shared object, and loading it into the
running program.
For more details (the code, a few tests and the Makefile), you might
want to look into the directory `natdyn' of the MetaOCaml
distribution.
Yet it is awfully tricky -- perhaps trickier than one may realize. The
problems may occur long after everything is deemed working. For one
thing, Ocaml native runtime makes an assumption that the data segment
of a program is contiguous. With the dynamically linked code this
assumption is no longer true, and subtle bugs emerge (see below).
After you have loaded a shared object, chances are you'd like to
invoke an OCaml function from there. That is also tricky: the stack
frame has to be set up appropriately. If not -- everything works,
until the major garbage collection. At which point, you get the
segmentation fault.
Here's an example of a subtle bug (which also may be present in the
`Ancient' extension announced on this list some time ago).
let v = Array.create 256 iv in
Array.length v
where iv is a _float_ datum that is a literal in a loaded
shared object, or had been evicted into the Ancient heap. Now, what is
the reported length of the allocated float array of 256 elements? 128.
Exactly half of what is allocated. Weird, eh?
This problem has actually happened, please see the MetaOCaml list for
discussion. It took a couple of days just to understand the problem.
The natdyn part of MetaOCaml does work, I believe correctly. However,
the implementation takes advantage of the fact that some code just
cannot be part of MetaOCaml quotations. In particular, MetaOcaml
quotations cannot contain any declarations (of data types, of
exceptions) and any module expressions (in particular, modules
requiring initialization). These limitations of natdyn can be remedied
(OTH, since natdyn was needed quite urgently, one may be justified in
taking advantage of the domain knowledge to simplify the problem).
there is now a release candidate of the upcoming ocamlnet-2.2 library "The Best of 2006": http://www.ocaml-programming.de/packages/ocamlnet-2.2rc1.tar.gz Developers interested in the upcoming 2.2 version can have look at it, and experienced developers are invited to test it, and to help finding the remaining bugs and problems. This version will be the final release if no serious bugs are found. The release will happen before December 31. In the rest of this (long) email, I'll explain certain aspects of this version: 1. What's new in ocamlnet-2.2 2. Release notes 3. How you can help testing 4. Resources 5. Credits Gerd ------------------------------------------------------------ 1. What's new in ocamlnet-2.2 ------------------------------------------------------------ Ocamlnet now includes equeue, netclient, and rpc These libraries were previously distributed as separate software packages. All four libraries form now together the new ocamlnet-2.2. This allows much deeper integration of their functionality. Building servers with Netplex The framework Netplex simplifies the development of server applications that require the parallel execution of requests. It focuses on multi-processing servers but also allows multi-threading servers. Netplex manages the start and stop of processes/threads, and dynamically adapts itself to the current workload. Netplex allows it to integrate several network protocols into one application, and as it also supports SunRPC as protocol, one can consider it even as component architecture. Furthermore, it has infrastructure to read configuration files and to log messages. Ocamlnet includes add-ons for Netplex to build SunRPC servers, web servers, and web application servers (the latter over the protocols AJP, FastCGI, or SCGI). The revised API for web applications The library netcgi2 is a revised version of the old cgi API (now also called netcgi1). The changes focus on restructuring the library in order to improve its modularity. It is hoped that beginners find more quickly the relevant functions and methods. The API is essentially the same, but the support for cookies has been enhanced. The connectors allowing a web server to talk with the application have been completely redeveloped - all four connectors (CGI, AJP, FastCGI, SCGI) support the same features. The connector for SCGI is new. The connector for AJP has been upgraded to protocol version 1.3. There are Netplex add-ons for the connectors. The old API is still available, but its features are frozen. It is recommended to port applications to netcgi2. Improvements for SunRPC applications It is now possible to use the SunRPC over SSL tunnels. All features are available, including asynchronous messages. As a side effect of this, the SunRPC implementation is now transport-independent, i.e. it is sufficient to implement a few class types to run RPC over any kind of transport. Furthermore, a few details have been improved. SunRPC servers can now implement several RPC programs or program versions at the same time. SunRPC clients can now connect to their servers in the background. A few bugs have been fixed, too. Shared memory As multi-processing has become quite important due to Netplex, Ocamlnet supports now the inter-process communication over shared memory. The implementation is brand-new and probably not very fast, but shared memory makes sometimes things a lot easier for multi-processing applications. Old things remain good Of course, this version of Ocamlnet supports the long list of features it inherited from its predecessors. This includes an enhanced HTTP client, a Telnet client, a (still incomplete) FTP client, a POP client, and an SMTP client. The shell library is an enhanced version of Sys.command. The netstring library is a large collection of string routines useful in the Internet context (supports URLs, HTML, mail messages, date strings, character set conversions, Base 64, and a few other things). ------------------------------------------------------------ 2. Release notes ------------------------------------------------------------ Stability In general, the stability of this version is excellent. About 90 % of the code has been taken over from previous versions of ocamlnet, equeue, netclient, and rpc, and this means that this code is already mature. About 10 % of the code has been newly developed: - netcgi2 is a revised version of the cgi library. Large parts are completely new. - netplex is the new server framework. Fortunately, it could already be used in a production environment, and it has proven excellent stability there. - netcgi2-plex combines netcgi2 and netplex. - nethttpd has now the option to use netcgi2 as cgi provider (configure option -prefer-netcgi2). - netshm adds shared memory support. - equeue-ssl and rpc-ssl add SSL support to the RPC libraries. Known Problems There are known problems in this release which won't be solved until the release of 2.2 but in a later version: - There is no good concept to manage signals. This is currently done ad-hoc. For now, this does not make any problems, or better, there is always the workaround that the user sets the signal handlers manually if any problems occur. - The new cookie implementation of netcgi2 should replace the old one in netstring. Users should be prepared that Netcgi.Cookie will eventually become Nethttp.Cookie in one of the next releases. - In netcgi2-plex, the "mount_dir" and "mount_at" options are not yet implemented. - In netclient, aggressive caching of HTTP connections is still buggy. Do not use this option (by default, it is not enabled). - The FTP client is still incomplete. ------------------------------------------------------------ 3. How you can help testing ------------------------------------------------------------ It would be great if experienced developers tested the libraries, especially the new and revised ones. Discussions should take place in the Ocamlnet mailing list (see resources section below). It is important to know that this version of Ocamlnet also includes the libraries formerly distributed as equeue, netclient, and rpc. If you upgrade an O'Caml installation, you _must_ remove old versions of these libraries prio to the installation of the new Ocamlnet. For GODI users, there is a convenient way of installing ocamlnet2. First install GODI as usual (either for O'Caml 3.08 or 3.09). Then, change godi.conf, and add the line GODI_BUILD_SITES += http://www.ocaml-programming.de/godi-build/branch-ocamlnet2/ update packages and rebuild. You can install godi-ocamlnet godi-ocamlnet-gtk1 godi-ocamlnet-gtk2 godi-ocamlnet-tcl godi-ocamlnet-ssl where the latter four packages contain add-ons that need further libraries to be installed. The packages godi-equeue*, godi-rpc, godi-netclient are only fake packages that include godi-ocamlnet as predecessors. ------------------------------------------------------------ 4. Resources ------------------------------------------------------------ On online version of the reference manual can be found here: http://ocamlnet.sourceforge.net/manual-2.2/ The current development version is available in Subversion: https://gps.dynxs.de/svn/lib-ocamlnet Note that the ocamlnet file tree in Sourceforge refers to ocamlnet-1 only. There is a mailing list for Ocamlnet development: http://sourceforge.net/mail/?group_id=19774 In case of problems, you can also contact me directly: Gerd Stolpmann <gerd@gerd-stolpmann.de> ------------------------------------------------------------ 5. Credits ------------------------------------------------------------ A number of people and institutions helped creating this new version: - Christophe Troestler wrote the revised CGI library - The California State University sponsored the development of Netplex and the SSL support for SunRPC. Special thanks to Eric Stokes who convinced the University, and David Aragon for his business support. - All companies who hired me this year and made it possible that I can make a living from O'Caml development. Without that it would have been impossible to put so much energy into this. Special thanks go to Yaron Minsky and Mika Illouz.
For information, an OCaml program, demexp, won the 2nd prize of latest International Free Software Contest (Trophées du Libres) in Soissons, France, latest November (category Public Services, /Application pour structures publiques et collectivités/). http://www.tropheesdulibre.org/ Apparently, the jury was kind enough to accept the esoteric language we used. :-)
Here is a quick trick to help you read this CWN if you are viewing it using vim (version 6 or greater).
:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'<1':1
zM
If you know of a better way, please let me know.
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.