Hello
Here is the latest Caml Weekly News, for the week of June 22 to 29, 2010.
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/348dabaeb60a6709#
Gerd Stolpmann announced:
I'm very proud to announce the public availability of Plasma MapReduce,
a map/reduce compute framework, and PlasmaFS, the underlying distributed
filesystem.
Plasma MapReduce is a distributed implementation of the map/reduce
algorithm scheme. In a sentence, map/reduce performs a parallel List.map
on an input file, sorts and splits the output by some criterion into
partitions, and runs a List.fold_left on each partition. Only that it
does not do that sequentially, but in a distributed way, and chunk by
chunk. Because of this Plasma MapReduce can process very large files,
and if run on enough computers, this also will work in reasonable time.
Of course, map and reduce are Ocaml functions here.
This all works on top of a distributed filesystem, PlasmaFS. This is a
user-space filesystem that is primarily accessed over RPC (but it is
also mountable as NFS volume). Actually, most of the effort went here.
PlasmaFS focuses on reliability and speed for big blocksizes. To get
this, it implements ACID transactions, replicates data and metadata with
two-phase commit, uses a shared memory data channel if possible, and
monitors itself. Unlike other filesystems for map/reduce, PlasmaFS
implements the complete set of usual file operations, including random
reads and writes. It can also be used as unspecialized global
filesystem.
Both pieces of software are bundled together in one download. The
project page is
http://projects.camlcity.org/projects/plasma.html
This is an early alpha release (0.1). A lot of things work already, and
you can already run map/reduce jobs. However, it is in no way complete.
For discussions on specifics of Plasma there is a separate mailing list:
https://godirepo.camlcity.org/mailman/listinfo/plasma-list
Archive: http://groups.google.com/group/fa.caml/browse_thread/thread/475408faa9427aaa#
José Romildo Malaquias asked, Michael Ekstrand said, and Nicholas Kidd suggested:
> > Is there a symbol type in OCaml, with a constant time comparison
> > function? Something like symbols from Scheme and LISP or atoms from
> > Prolog. Useful in compiler construction.
>
> Not directly. As I see it, you have two decent options:
>
> - Use/write a symbol table which "interns" symbols to integers. The
> resulting integers can be compared.
> - Use/write a symbol table which interns symbols to unique string
> instances, so SymTbl.intern "foo" returns the existing string object if
> one already exists, and the string object passed in if it's never been
> seen before. The resulting strings can be compared with == rather than
> = in constant time.
>
> Either of these options would be fairly similar to how symbols work
> under the hood in a Lisp implementation, I believe.
Use the Ocaml hash-consing library.
http://gallium.inria.fr/ml2006/accepted/5.html
http://www.lri.fr/~filliatr/ftp/publis/hash-consing2.pdf
bluestorm also suggested:
If your set of symbol is closed, you can use a variant type (sum type).
type symbols =
| A
| B
If you really need open symbols, you can use [polymorphic variants].
Let tag_a foo = (`A, foo)
[polymorphic variants]
http://caml.inria.fr/pub/docs/manual-ocaml/manual006.html#toc36
However, you won't have convenience functions such as string_of_symbol; you
would have to define them yourself.
Thanks to Alp Mestan, we now include in the Caml Weekly News the links to the
recent posts from the ocamlcore planet blog at http://planet.ocamlcore.org/.
LLVM Backend for DDC.:
http://www.mega-nerd.com/erikd/Blog/CodeHacking/DDC/llvm_backend.html
Post-OWASP AppSec Research:
http://dutherenverseauborddelatable.wordpress.com/2010/06/28/post-owasp-appsec-research/
FP-Syd #25.:
http://www.mega-nerd.com/erikd/Blog/FP-Syd/fp-syd-25.html
Plasma: Map/Reduce for Ocaml:
http://blog.camlcity.org/blog/plasma1.html
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.