Hello
Here is the latest Caml Weekly News, for the week of January 30 to February 13, 2007.
The LSD mount utilities are written in OCaml, so I hope the
announcement is not OT here.
Don't hesitate either to flame me or ask questions if I'm OT or the
utilities don't work as advertised.
Since those need to be SUID programs help in reviewing them or
discussion of security aspects would be appreciated.
Regards, Markus
_oOo_
About
-----
The LSD mount utilities allow to automate mounting of encrypted
filesystems in linux with cryptsetup and loopback devices and enable
users (not only root) to mount encrypted filesystems when appropriate
entries to /etc/fstab have been added by root.
The LSD mount utilities are licensed/distributed under the the terms
of the GPL Version 2 (no later version). We'll usually be happy though
to relicense under other OS licenses, but you have to ask and get the
licensing change from us in writing.
The LSD mount utilities can be downloaded from
http://software.m-e-leypold.de/lsd-mount-utilities.
What does it do? How does it work?
----------------------------------
In Linux /sbin/mount calls /sbin/mount.$FSTYPE and /sbin/umount calls
/sbin/umount.$FSTYPE if those programs exist. This leads to the
observation, that the filesystem types either in /etc/fstab or passed
to mount with -t rather more characterize a mounting mechanism than a
file system type.
With the LSD mount utilities this mechanism is used to delegate
mounting of encrypted loopback devices to mount.lcrypt which automates
the steps necessary to set up those devices (like: modprobe, losetup,
cryptsetup).
mount.lcrypt on the other side should know the filesystem of the
decrypted device from a mount option. (This does not work yet,
presently all lcrypt devices have the decrypted filesystem type ext2).
Mounting directly
-----------------
Pass '-t lcrypt' to mount:
mount -t lcrypt /data/encrypted-image /mnt
You'll be asked for the passphrase.
With fstab / allow user mounting
---------------------------------
Use lcrypt as filesystem type. Use the option 'user', if non
privileged users should be able to mount the device:
/data/encrypted-image /secret-mnt lcrypt rw,noauto,user,exec 0 0
Any user can now use
mount /data/encrypted-image
and is then asked for the passphrase.
> Can you (or anyone familiar with both) summarize how this binding > compares to Markus Mottl's pcre-ocaml? Thanks. As Markus says, it does implement the POSIX API for regular expression matching and extraction of sub-patterns... but note that that just [here] controls the "level" of the interface: there is a regcomp function for compiling the patterns and a regexec call for matching a compiled pattern against a specific string. The full "language" of PCRE for matching and pattern specification is accepted. Further, LablPCRE adds an even lighter-weight regmatch function when all that is desired is a boolean answer on a match, as well as a set of substring extraction and [error] status info accessor functions. LablPCRE makes an effort to be a good GC-citizen by not holding references to anything it doesn't need to. :) Finally, LablPCRE supports PCRE 7.0 (or any of the releases back to 6.1).
The "0.9pre1" release of the LablScintilla OCaml binding for Scintilla is now available, fully supporting Linux and Windows builds and Scintilla versions 1.70 - 1.72 (current). LablScintilla provides a complete binding for the Scintilla editing component, a widely used multi-platform open source package. While billed as a "source code editing component", Scintilla is much more: providing control of complex layouts with multiple font faces and backgrounds, plus supplying numerous fine-grained events covering input and text modification tracking, it can easily provide the foundation for general-purpose text editors or programming IDEs. The majority of this binding has been in use internally for well over a year, but as it has had little opportunity for external feedback, a little exposure seemed prudent prior to going for a "1.0" version. This release has been built and tested using OCaml 3.09.3 on Fedora Core 6 and Windows XP, supports findlib and "hands-off" building and installing (no "configure" script or manual file editing required), and has pre-built binaries for [native] Windows XP. The full package is licensed under the "new" BSD license, and may be downloaded here: http://www.rftp.com/Downloads.shtml
> I found some strange difference between the native and bytecode
> compilers, when Marshaling functional values:
> [damien@mostha]$ cat lift.ml
> let r = ref 0
> let f =
> fun () -> incr r; print_int !r; print_newline()
> let () = match Sys.argv.(1) with
> | "w" -> Marshal.to_channel stdout f [Marshal.Closures]
> | "r" ->
> let g = (Marshal.from_channel stdin: unit -> unit) in
> g (); f ()
> | _ -> assert false
> [damien@mostha]$ ocamlc lift.ml; ( ./a.out w | ./a.out r )
> 1
> 1
> [damien@mostha]$ ocamlopt lift.ml; ( ./a.out w | ./a.out r )
> 1
> 2
> [damien@mostha]$ ocamlc -version
> 3.09.2
> In the bytecode version, the reference [r] gets marshaled along with
> [f] so that the calls [f()] and [g()] respectively affect the initial
> reference of the reader, and the (fresh) marshaled reference.
> On the contrary in the native version, it seems that [f] is not
> `closed': its code address is directly sent, and the call [g()]
> affects the initial reference of the reader.
Interesting phenomenon. According to the usual definition of closure,
the correct solution is probably the bytecode one. But this definition
seems hardly applicable in practice, since it would also mean bringing
all dependencies with you. This is not the case even with the bytecode
version. For instance if you move "let r = ref 0" to r.ml, and replace
the first line of your program by "open R", you get the same behaviour
as for native code.
So as a first approximation, the real specification is: local
variables are transmitted with the closure, but global ones are not.
The trouble being that the definition of global is different for
bytecode and native code. With bytecode, definitions from the same
module are local, while they are global for native code.
Moreover, I believe that, through optimizations, variables that look
local may turn up to be global.
I'm not sure what would be the right fix.
A more complete specification would be a good idea.
A flag to disable optimizations would be rather costly.
For now, a rule of the thumb would be:
* if you want your variable to be handled as global, even in bytecode,
either receive it as parameter (after marshalling) or put it in
another compilation unit.
* if you want your variable to be handle as local, even in native
code, then define or redefine it locally inside your function.
let r = ref 0
let f =
let r = r in
fun () -> incr !r; print_int !r; print_newline()
For the time being this seems to work.
Maybe it is better just to assume that you should not mix closure
marshalling with mutable variables. In either case, the semantics
seems fishy. It seems more reasonable to make such functions receive
their mutable state explicitly, and choose either to send it
(obtaining a "fork" behaviour) or not.
Almost one year after my last message on this list where I asked for help compiling ocamlgsl with MingW, I succeeded. I want to thank Olivier Andrieu and Ulrich Steinmetz for their support and to provide the necessary steps here (since some of them are not obvious at all): First of all, it seems not to be possible to link ocamlgsl dynamically. So when compiling it, remove the line DYNAMIC_LINKING=... from the Makefile. After this, I was able to compile the library and could even generate native binaries of my programs which used it. However, whenever functions from Gsl_rnd where called, I received lots of compile warnings, and in many cases the compiled binaries were not working (crashing or doing nothing). The solution was to link the gsl library _statically_ to my binaries. This could be done by adding the following arguments to ocamlopt: -ccopt -static. However, this links _every_ C library you use to your binary, and you may end up in lots of "undefined reference" errors, since all libraries that are required by your C libraries have to be linked as well, and you may have to spend hours or days with resolving dependencies and receive a giant executable. To prevent this, I suggest not to use this global "static" flag but to link _only_ the gsl library statically and all other libraries dynamically. For this, one has to link gsl at the right position in the ocamlopt arguments. Here is a way that is working: ocamlopt (...other options...) -cclib -Wl,-lmlgsl,-dn,-lgsl,-dy,-lgslcblas (...) Finally, one has to link the pthread library by -cclib -lpthread With these tricks I succeeded, and even Gsl_rng is working.
Intro
=====
Inspired by Simon Peyton Jones' proposal of simple views for Haskell
[1,2], I incorporated something very close to his proposal into Micmatch
[3]. It's an experimental feature of Micmatch 0.697 [4,5] which is
available in GODI.
What are views?
===============
Views allow pattern matching on data that don't have a type that allows
pattern matching. Useful uses of views in OCaml include the following:
* pattern matching over object fields (methods w/o arguments)
* pattern matching over lazy values, such as lazy lists
* filtering sets of elements (e.g. Even/Odd, Positive, Short_list, ...)
* generally: matching abstract values when you only have functions to
obtain information about their structure.
* parametrization of pattern matching (patterns with gaps filled at runtime)
In general the use of views avoids breaking match-with blocks into several
pieces when the patterns are complex.
There are several kinds of views, each with their own advantages and
difficulties. The main disadvantage of the solution that we chose here is
that it prevents the detection of redundant or missing cases whenever a
view pattern is used. The advantages are the flexibility and simplicity of
use: several overlapping views can be used in the same match-with
block (e.g. match x with (%Even | %Positive) -> ... | ... -> ...).
Example
=======
(* Define a view function view_XY *)
let view XY = fun obj -> try Some (obj#x, obj#y) with _ -> None
(* Test if a list of objects starts with coordinates x=0 and y=0 *)
let starts_from_origin = function
%XY (0, 0) :: _ -> true
| _ -> false
You can see that a view is simply defined as a function that takes the
subject value and returns an option. This is the case of views with
an argument. Views without arguments must be defined as predicates (return
a bool).
Efficiency
==========
Don't use simple views such as %Even or %Positive if you are concerned
about speed at this particular point in your program.
References
==========
[1] SPJ's email: http://www.haskell.org/pipermail/haskell/2007-January/019014.html
[2] Proposal for Haskell: http://hackage.haskell.org/trac/ghc/wiki/ViewPatterns
[3] Micmatch home: http://martin.jambon.free.fr/micmatch.html
[4] Views in Micmatch: http://martin.jambon.free.fr/micmatch-manual.html#htoc10
[5] Wiki for your comments: http://ocaml.pbwiki.com/Micmatch
Here is a quick trick to help you read this CWN if you are viewing it using vim (version 6 or greater).
:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'<1':1
zM
If you know of a better way, please let me know.
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.