Hello

Here is the latest Caml Weekly News, for the week of June 10 to 17, 2008.

1. Uuidm 0.9.0
2. Fortress talk in New York City
3. Bitmatch 1.9 released
4. GODI News: RocketBoost Beta
5. ocamlopt generates binaries with executable stacks

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/3f4414dd02a9435f#

I'd like to announce the existence of the following snippet of code.

Uuidm is an OCaml module implementing 128 bits universally unique identifiers
version 3, 5 (name based with MD5, SHA-1 hashing) and 4 (random based)
according to RFC 4122.

Uuidm is made of a single, independent, module and distributed under the new
BSD license.

Module home page : http://erratique.ch/software/uuidm
			

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/8a6f259fdfbe1809#

There will be a talk on Fortress ( a new OO/Functional language from Sun) on
Wednesday June 25 at 6:30pm in Manhattan.

Abstract:

The Java Programming Language revolutionized programming with two simple 
concepts:   "Write once run anywhere", and Garbage Collection.  This led 
to a big step up in programmer productivity.  Project Fortress does it 
again.    This time we give you multiprocessor performance without 
having to code threads, locks, or load balancing.  Can you say "Implicit 
Parallelism" and "Transactional Memory"?  We also give you a growable 
language (small fixed core), strong static typing (more errors caught at 
compile time), and mathematical notation.   This talk will focus on the 
strengths of Fortress as a programming language, as well as a deep dive 
into implementation issues.

More information is available at http://lisp.meetup.com/59
			

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/1a588466ab306b81#

This is a preview/testing release:

 http://code.google.com/p/bitmatch/

The major change is the introduction of persistent patterns.  This
allows you to write importer tools, so you can import bitstring
descriptions from your own languages or other sources.

 http://et.redhat.com/~rjones/bitmatch/html/Bitmatch_persistent.html

I'm working on a C language importer, which will be able to import C
structures, unions, enums and constants (eg. out of header files).
This will be based on bitmatch & CIL, and there's an early prototype
included in the 1.9 release.

The syntax for this is simple and lightweight.  For example, to
implement the equivalent of the 'df' command (showing free blocks on a
filesystem) you could do something like this:

 (* bitmatch-import-c ext3.c > ext3.bmpp *)
 open bitmatch "ext3.bmpp"

 let () =
   let fd = Unix.openfile "/dev/sda1" in
   let bits = Bitmatch.bitstring_of_file_descr_max fd 4096 in

   bitmatch bits with
   | { :ext3_super_block } ->
       printf "free blocks = %ld\n" s_free_blocks_count
   | { _ } ->
       printf "/dev/sda1 is not an ext3 filesystem\n"
			

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/8beef489399229cc#

In the past weeks I've worked hard to finish the next GODI release,
focusing on portability. A beta version of the new bootstrap and
godi_console, called RocketBoost, is now available, and it would be
great if it were tested at large.

There are not many new features in this release, so there is no reason
to switch to it if you already have GODI.

The big news is that GODI now supports the MinGW port of OCaml for
Windows (besides the Cygwin port). This means that it is now possible to
create native Windows applications with GODI. Note, however, that the
porting effort is still in its beginning - GODI itself works, but most
packages aren't ported yet, and won't work.

The MinGW support has become possible because large parts of the GODI
core have been rewritten. In particular, the C programs accompanying
godi_console are now coded in O'Caml (godi_make, godi_add, etc.), so
porting was possible by enhancing a few O'Caml libraries. This
refactoring of GODI also increases the portability in the Unix world -
effectively it should now run on all platforms where OCaml runs, where
gcc is available, and where POSIX-compliant shell utilities are
available.

If you would like to test it, follow these instructions. For all ports,
you need the bootstrap tarball from:

http://download.camlcity.org/download/godi-rocketboost-20080615.tar.gz



--- Installation for Unix ---

For Unix platforms (including MacOSX), just download
godi-rocketboost-20080615.tar.gz, unpack it, run ./bootstrap, and follow
the instructions.



--- Installation for Windows ---

For Windows you always need Cygwin, even for the MinGW port (it is
needed as scripting environment, please don't question that). Get it
from cygwin.com. Install everything that is needed (binutils, bzip2,
diffutils, file, findutils, gawk, gcc-core, gcc-mingw-core, grep, groff,
gzip, m4, make, man, mingw-runtime, patch, rxvt, sed, tar, w32api, wget
- hope this list is complete).

IMPORTANT: When you install software you need for GODI, choose paths
that do not contain space characters. This is incompatible with the
shell scripts. So don't install into "My Files".

Download godi-rocketboost-20080615.tar.gz, and unpack it:

$ tar xzf godi-rocketboost-20080615.tar.gz
$ cd godi-rocketboost-20080615

Now invoke bootstrap. You probably want to set the path where it is
going to be installed with -prefix. Furthermore, you can now select
whether you want the Cygwin or the MinGW port. For the latter, pass
"-w32port mingw" as extra argument.

IMPORTANT: Remember that GODI itself relies on Cygwin scripting. If you
pass paths to GODI (including godi_console, godi_add, etc.) these are
Cygwin paths, even if you build the MinGW port.

$ ./bootstrap -prefix /home/gerd/godi -w32port mingw

This will build boot_console. Then you need an Internet connection, and
do:

$ export PATH=/home/gerd/godi/bin:/home/gerd/godi/sbin:$PATH
$ ./bootstrap_stage2

This will install the minimal GODI core.

IMPORTANT: godi_console isn't ported to the Windows console window, and
for now only supports terminal windows that can deal with ANSI control
codes. Use Cygwin's rxvt as terminal (or any other capable terminal like
xterm). Furthermore, there is an issue with the MinGW port setting some
terminal features. As workaround, you have to set the TTY environment
variable to the Cygwin tty device, i.e. TTY=`tty`.

Note that the MinGW port is only available for ocaml-3.10. Passing
"-section 3.09" to bootstrap won't work.


--- List of problems ---

The final part of this message lists some problems that popped up during
the MinGW porting effort. I have often found workarounds, but maybe the
reader knows better solutions.

* Cygwin interoperability

A lot of the porting effort was about Cygwin interoperability.
godi_console (which is a native Win32 binary) can translate Cygwin paths
to native Windows paths (by reading the Cygwin mount table in the
registry). However, some problems turned out to be hard:

Starting a Cygwin binary with CreateProcess from godi_console turned out
to be unreliable when godi_console was itself called from a Cygwin
program. So the calling chain Cygwin_pgm -> Win32_pgm -> Cygwin_pgm
did not always work (symptom: the called program immediately exits). A
workaround was to use cmd.exe as intermediate instance:
Cygwin_pgm -> Win32_pgm -> cmd.exe -> Cygwin_pgm

Maybe related to that, it wasn't possible to run stty without explicit
device argument. (godi_console needs to call it to talk to the Cygwin
tty driver.) stty (and tty, too) always complained that stdin wasn't a
tty. The workaround is to ask the user to set the environment variable
TTY to the tty device.

* Cygwin vs. Windows file names

The approach is that GODI itself is seen as a Cygwin program - although
godi_console is a native program. As already mentioned, godi_console is
capable of translating Cygwin paths without using Cygwin.

At some time, it is required to translate paths to the Windows style,
because programs like ocaml, ocamlfind, etc. are Windows executables
without Cygwin path translation capability. In GODI Makefiles, there is
now always the variable $(LOCALBASE_NATIVE), set to the Windows path
(using forward slashes). The porting effort was effectively quite low -
in a few places I had to use ${LOCALBASE_NATIVE} instead of
${LOCALBASE}.

* CR/LF

In godi_console, it was required to look carefully over the code, and to
switch between ASCII and binary channel modes where needed. As native
Windows executable, godi_console uses CR/LF as EOL characters.

It turned out that Cygwin's bash has sometimes problems with CR/LF.
Especially, something like

stdlib=`ocamlc -where`

does not work - the CR character remains as part of the result value.
The workaround was to use Cygwin's alternate shell, ash, in these cases,
which handles that better.

* cygpcre DLL, and DLL lookup by PATH

Cygwin includes a PCRE DLL. Of course, this DLL cannot be used in
non-Cygwin programs, so GODI always installs its own PCRE DLL.

It turns out that the MinGW install of PCRE produces a DLL with the same
name as the Cygwin version (cygpcre-0.dll, as far as I remember). As
DLL's are looked up by PATH, and we _need_ the Cygwin directories in
PATH, the problem arises how to direct MinGW executables to use the
right DLL.

For now the workaround is that the MinGW version has a different version
number (cygpcre-7.dll). It would be nice, however, to get rid of the
"cyg" prefix for a clearer separation of DLL spaces.

* Ocaml does not install ocamldep.opt.exe

in the MinGW port. Maybe an oversight?

* No support for ocamlmklib

A lot of work has been put into working around the missing ocamlmklib.
The difficulty is that stub libraries need to be compiled with different
flags when a DLL is to be produced, i.e. the C compiler is invoked
differently in this case. This is incompatible with many build Makefiles
that can only produce one version of .o files from a .c file. Because of
this difficulty, ocamlmklib isn't available.

For ocamlnet, I worked around by using two scripts, stubcc, and
mkstublib, available from

https://godirepo.camlcity.org/svn/godi-build/trunk/godi/godi-ocamlnet/files/

Effectively, stubcc calls the C compiler twice, and produces a .d.o
intended for DLL inclusion, and a normal .o for the static library. I'm
wondering whether such a tool could be included in the ocaml
distribution, or whether ocamlc could have a special -for-stublib option
so that it invokes gcc twice for .c files.

* Windows "execv" is broken with respect to "wait"

When a windows process calls execv to invoke another binary, it is
signaled to the parent process that the program is finished. (The
correct POSIX behavior is to wait until the second binary is finished.)
In many places, it was easy to work around the problem by using the
spawn calls. At one place, this is not possible, and godi_console has to
use "execv" to avoid deadlocks. As consequence, it may happen that the
caller of godi_console thinks that the program is done although it is
still running.

No workaround yet.

* No real argv support in Windows

There are no clear rules how to quote arguments in Windows (cmd.exe).
(Actually, there is no list of arguments (argv) in Windows, but only a
command line string.) In GODI, it sometimes happens that quoted
arguments need to be quoted (and more complicated), and this did not
work in Windows. The workaround is to write complex commands to
temporary files, and to run these by /bin/sh.

* No signals in Windows

There is no clear way how to signal a process to terminate itself. As a
consequence, aborting builds is broken in godi_console.

* argv.(0) is broken

I don't know who is to blame for this, either Windows, or Cygwin. At
least it wasn't possible to pass a name as argv.(0) that differed from
the executable name.

The workaround was to use wrapper programs.



Anyway, I hope you like the new port. And remember: be patient. Windows
is very slow when used for shell-style scripting.
			

Archive: http://groups.google.com/group/fa.caml/browse_frm/thread/761a37673bd9ebcc#

> I posted a patch which should fix the issue that ocamlopt generates
> binaries with executable stacks:
> 
>  http://caml.inria.fr/mantis/view.php?id=4564
> 
> However this patch affects every assembly target, far more than I
> could possibly test.  Could people using OCaml on non-Linux platforms
> have a look at the patch, or even test it for me?

I'm pretty sure this patch is Linux-specific.  My fear is that it
might be specific to particular versions of binutils and/or particular
Linux distributions...  I smell a portability nightmare!

Note that in 3.11, the "configure" script will have options to specify
how to call the assembler (for ocamlopt-generated assembly code and
for the hand-written asm files in the runtime system).  So it might be
sufficient to configure the Gentoo packages with e.g.

configure -as "as --noexecstack" -aspp "gcc -c -Wa,--noexecstack"

This could be one of the rare cases where addressing the issue at the
level of the packages is safer than by changing the source distribution.
			

Using folding to read the cwn in vim 6+

Here is a quick trick to help you read this CWN if you are viewing it using vim (version 6 or greater).

:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'<1':1
zM

If you know of a better way, please let me know.

Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt