Hello
Here is the latest Caml Weekly News, for the week of December 06 to 13, 2011.
As there has been a huge spike of traffic on the Caml mailing list this week, I have had to select just a few messages to reproduce here to keep this summary short. If you want to read everything that has been said, here are a some links to the threads. I apologize in advance if I missed a message that should have been included. The initial threads, from which I'll quote a few messages below, were about forking OCaml: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00018.html https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00106.html They spawned another thread about the community in general: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00046.html Another thread was about whether or not using C as a backend: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00141.html Finally, Gabriel Scherer wrote a long email about Camlp4 and Camlp5, which spawned some discussion about their maintenance status and possible alternatives: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00214.html
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00018.html
Benedikt Meurer said:During the last year or two it seems that time and interest in OCaml maintenance from the official OCaml development team is diminishing. It takes several months to get a patch reviewed (if at all), which is quite frustrating for OCaml contributors and even worse for OCaml users. I suspect that this is one of the top reasons why there are only a few active contributors to OCaml (and the number of active users, at least on the mailing list, is declining). I understand that INRIA does not necessarily pay people for full time maintenance jobs on OCaml (and Coq), and the official dev team is probably already doing as much as possible to maintain OCaml. Given that OCaml is such a nice language with a lot of useful frameworks available, it is too sad to see it loosing ground just because of it's closed development process and lack of time of the official team. I'd therefore propose to open up OCaml development to a wider range of developers / contributors, to ensure that OCaml will be ready for the (functional programming) future. There are already various "OCaml forks" in the wild, with different goals and patch sets, so simply starting another fork would be rather useless. Instead I'd suggest to bundle efforts in a new "OCaml community fork", which is always based on the most recent upstream OCaml release (starting point would be 3.12.1 for now), and takes care to review and integrate pending patches as well as developing and testing new features. Let's say we'd name the fork "OCaml-ng", then we'd try to release a new patch set every month or two, based on the official OCaml release, i.e. "ocaml-3.12.1+ng201112" and so on, to get early testing and feedback (should work together closely with the Debian/Ubuntu/etc. OCaml maintainers). With this process, OCaml upstream could merge (tested) patches from OCaml-ng once they proved working in the wild, and thereby 1. maintenance overhead for INRIA people is reduced, 2. maintenance status of OCaml would be way better, 3. there would be a lot less frustration for possible contributors, and 4. users benefit from a better and more up to date OCaml. Now that does of course raise a few questions: 1. What is the opinion of the official development team / INRIA on this? 2. Who would help with the community fork? 3. What about infrastructure?Xavier Leroy replied:
> Maybe we can get back to my original proposal and restart on the > right foot this time? All right. I have a number of concerns about your proposal, most of which have already been mentioned by others. Whether you call it a fork or not, the mere fact of having two separate code bases for exactly the same software components raises more issues than it solves: - It complicates the lives of OCaml users, packagers, and 3rd-party library developers: what version should they use? what will be the basis for the packagers's distribution-specific patches? what happens if a library is compatible with one version but not the other? what if the community effort runs out of steam in a few years? - It means additional efforts with some duplication. The synchronization between the "community" and the "reference" code bases will take work. From the core team's standpoint, that's about as much work as dealing with individual suggestions and contributions. At the same time, the community team will spend non-negligible time organizing and administering itself, at least initially. - It would be a PR disaster. I struggled (and still have to struggle) with my INRIA management to get a little bit of support for OCaml's development, e.g. Xavier Clerc's part-time assignment to work on OCaml. This support is fragile and can easily disappear if there's a perception that "the community" will take care of that anyway. Likewise, I don't quite see how to explain the situation to the members of the Caml consortium. For these reasons, I feel that the cure is worse than the illness. I am, however, much more sympathetic to your other proposal, namely: > Why not accept a model similar to i.e. the NetBSD project, with a > lot of committers (experts in their own areas) and 2-3 people to > keep an eye on the overall direction? Except for the "lot of committers" part, this is pretty much how the core Caml team has been working, and there is definitely room for more contributors. As I mentioned earlier, there are a number of areas where we're lacking manpower, e.g. Windows-related stuff and tools such as the debugger or Camlp4, but many other areas of the system would benefit from more contributors too. Smaller contributions are most welcome as well, such as commenting on the PRs, testing changes, new features and release candidates, PR triaging, helping to identify the top little things to be resolved by the next release, etc. That's a much more low-key approach, but one that is more likely to succeed. Volunteers are welcome to contact us as caml AT inria.fr.Kakadu asked and Fabrice Le Fessant replied:
> Does anybody have news about OCamlPro? Yes, OCamlPro is working on different projects to improve OCaml (namespaces, better inlining, debugging, multicore gc, etc.), but the main focus is currently on improving development tools (edition, refactoring, navigation, documentation, etc.), while minimizing the modifications on the compiler itself. The plan is to release a first version by the beginning of 2012, but it will be complementary to OCaml, not a replacement. For the OCaml distribution itself, OCamlPro will have a different release cycle for its own version, targetting industrial users, but the high quality testing process required to do that is still under construction.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00071.html
Maxence Guesdon announced:I just started bindings for the Redland's librdf library[1]. The project is here: http://ocaml-rdf.forge.ocamlcore.org/ It's far from being complete but a lot of functions are already bound. Hopefully, this will help OCaml to become "the programming tool of choice for discriminating semantic web hackers." :-) Cheers, Maxence [1] http://librdf.org
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00092.html
William Le Ferrand asked and Stéphane Glondu suggested:> We are building a cache in ocaml and we're wondering if it would make > sense to store ocaml values outside the reach of the gc. (gc on a 20GB > cache hangs the process for a second or so). Have you heard of the ancient [1] library? [1] http://git.annexia.org/?p=ocaml-ancient.git;a=summaryGaius Hammond also suggested:
Alternatively (depending on the datatypes), try in in-memory[1] SQLite[2] database: [1] http://www.sqlite.org/inmemorydb.html [2] http://www.ocaml.info/home/ocaml_sources.htmlGerd Stolpmann also suggested:
this can be made working, and there is already a lot of support in Ocamlnet (module Netsys_mem, espeically init_value). I'm now using this all the time for populating shared memory blocks (which need to be outside the ocaml heap) with normal ocaml values. In Ocamlnet I'm using simply a bigarray for keeping the reference to the off-heap memory area (remember, bigarrays are just pointers to C arrays). This allows it to program the whole algorithm accessing such values in Ocaml. A warning though: Experience shows also that this type of programming is very error-prone, as you always have to take the memory representation of ocaml values into account.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00082.html
Jérémie Dimino announced:I have made a patch for ocaml 3.12.1 which allow to have a generic printing function. The patch is available here: http://www.dimino.org/ocaml-3.12.1-generic-print.patch Here is how to use it: external show : 'a -> string = "%show" let () = print_endline (show ([1; 2; 3], Some "foo")) this will print: ([1; 2; 3], Some "foo") It is also possible to define custom printers, for example: module StringMap : sig include Map.S with type key = string val string_of_t : ('a -> string) -> 'a t -> string end = struct include Map.Make(String) let string_of_t (type a) string_of_a (m : a t) = show (bindings m) end let () = print_endline (show (StringMap.singleton "x" 1)) will print: [("x", 1)] Of course it is limited to what the compiler knows, for example the following function will always returns "[<poly>; <poly>]": let f x = show [x; x] But i think it is already very useful for debugging. The git repo is here: http://www.dimino.org/gitweb/?p=ocaml-3.12.1-print.git;a=summaryAlex Rubinsteyn asked and Jérémie Dimino replied:
> Cool! Thanks for writing this. Can you explain how defining a custom > printer works? The new language features still bewilder me. If you want to define a custom printer for a type named foo, you have to define a function named string_of_foo. It must be in the same module as foo. It takes as arguments printers for its type variables, the value to print and returns a string. For example, if your type is: type ('a, 'b) foo a custom printer for foo will have the signature: val string_of_foo : ('a -> string) -> ('b -> string) -> ('a, 'b) foo -> string For example: module Id : sig type t val string_of_t : t -> string val create : unit -> t end = struct type t = int let string_of_t x = "<id=" ^ string_of_int x ^ ">" let next = ref 0 let create () = incr next; !next end You may also want to use "show" in your custom printer. In order to tell show to use the printers given as arguments, you must use type parameters like that: let string_of_foo (type a) (type b) string_of_a string_of_b (x : (a, b) foo) = ... For example to export a generated printer for an abstract type: module M : sig type ('a, 'b) t val string_of_t : ('a -> string) -> ('b -> string) -> ('a, 'b) t -> string end = struct type ('a, 'b) t = A of 'a | B of 'b let string_of_t (type a) (type b) string_of_a string_of_b (x : (a, b) t) = show x endFrançois Bobot asked and Jérémie Dimino replied:
> 1) If I understand well the "%show" command is expanded into the ocaml > code (lambda-code) which can print the type. So, for instance, in the > case of a list, your code generates a recursive function "aux" and it > applies this new "aux" function to the argument of "%show". Do you plan > to add in the module List a function : > > val string_of_list : ('a -> string) -> 'a list -> string > > and to generate only the call to this function instead of generating the > "aux" function every time? Or do you do it purposely? It is not in the lambda-code: i get the typed tree, generate for every occurrence of %show a parsetree using types and type this parsetree. The result is inserted in place of %show. For the rest, yes, it is how it works. Of course it is better if the printer is defined once and for all in its module. I did not modify the standard library because i did not wanted to rebuild all my libraries. > 2) Could you imagine to generalize it to Format.formatter or to > out_channel (without creating a string and concatenating)? Romain Bardou > add in the mantis tracker (I can't give you the bugtracking number since > mantis "is currently offline for maintenance") a feature wish for a new > conversion specification that can print anything. Do you think you can > fulfill is dream? For using formatter, yes, we can do that. This patch is more a proof of concept. If we want to really use it we can do something more generic than just generating printers. For example we could generate type representation values from which we would generate printers, marshallers, ... For the new conversion specification i don't think it is possible with the way the stdlib implements printf. The best you can do is something like: printf "%a" show value However it is possible with batteries' formatted printing (Print.printf p"%s" ...). > 3) Is it impossible to reuse/merge with the code of the top level that > print ocaml values? No, the toplevel use outcome trees (defined in Outcometree). > Thank you for providing this code. Funny the way you find the user > defined function. Be careful to not implement type classes in Ocaml ;) :-)Martin Jambon asked and Jérémie Dimino replied:
> I understand that this feature could be replaced in the future by a more > complete solution, but we would be happy if it were provided as an > "experimental extension" of OCaml. If you want to use it i can maintain it (and make it a bit better) until a real solution is added to OCaml. I am already using it myself for debugging. I uploaded it here: https://github.com/diml/ocaml-3.12.1-printGabriel Scherer suggested an alternative approach:
Martin, in the meantime, you can use Extlib's (Std.dump : 'a -> string) function, which is also integrated into Batteries. `dump` does not require any modification to the compiler or toolchain. For those that are not familiar with it, 'dump' uses the low-level representation of OCaml values to print a sensible string representing a value. As it only uses information that is available at runtime, it is much less precise that Jeremie's printer, as for example None, false, [] and 0 will all be printed as 0 (they have the same runtime representation). Of course, the "right" way to print/marshal data without changing the language is to build a printer for your type from printer combinators. Such combinators are available for example: - in Jane Street Core library ( a Sexpable interface in each datatype module ) - in Xavier Clerc's Kaputt testing framework : http://kaputt.x9c.fr/distrib/api/Utils.html You can also use metaprogramming (.. camlp4) to generate printer functions from datatypes description automatically, using such combinators. See: - Markus Mottl's 'type-conv': http://hg.ocaml.info/release/type-conv - Jeremy Yallop's 'deriving': http://code.google.com/p/deriving/ Of course, printing values magically is still easier: you don't have to build the printer yourself, passing subtype printers when necessary, etc. I think Std.dump is reasonable for quick hacks or debugging usage.Edgar Friendly asked and Alain Frisch replied:
> I'm interested in having compile-time reflection like this, but maybe > with a bit more generality. You might be interested in my proposal to extend OCaml with a notion of runtime representation of types: http://www.lexifi.com/blog/runtime-types A generic value pretty-printer can be written in "user-land" using this extension (in a type-safe way).Gerd Stolpmann then said and Alain Frisch replied:
> Want it! Want it! Want it! > > Any plans for including this into the official compiler? Since the reception was not bad when I presented it to the Caml Consortium meeting, yes, I'm proposing this as an extension to the official compiler. The plan is to create a branch in the OCaml SVN (I don't know when I'll be able to do it, hopefully before end of January) and follow the same approach as for GADTs (i.e. ask the community for some feedback, and discuss the proposal amongst the core team). I cannot commit on the final outcome, of course.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00156.html
Deep in this thread, Benedikt Meurer announced:You can follow the progress here: https://github.com/colinbenner/ocamlllvm It does work for some simple examples already, but it's still very early prototype quality and requires a patched LLVM. LLVM as such is not a bad idea for the compiler backend, but getting things to work with stuff compiled by the regular OCaml backends is the difficult part. We'll see how that turns out.Gabriel Scherer asked and Benedikt Meurer replied:
> - there are two different repos, ocamlllvm and ocaml-llvm (which has a > commit history that make it looks like it is where the real > development happen); which one should one follow? A wild guess after > only a quick look is that the ocaml-llvm repo did not build upon your > ocamlnat changes, and ocamllvm is about merging the changes on top of > it; but I really have no idea. ocaml-llvm was the initial attempt, based on the upstream OCaml sources. But we decided to ditch the surrounding OCaml stuff, esp. the broken make-based build system in favor of something simple, based on oasis and ocamlbuild (just like I did with ocamlnat). So the one to follow is ocamlllvm. > - you mention a "patched" LLVM; where can the patches be fetched? Just drop Colin a mail and ask him for the current patch (should be for 2.7 or 2.8, IIRC). > Do > you plan to present changes in such a way that it can be submitted > upstream? Yes, the long time goal (maybe impossible to reach with just one bachelor thesis) is to have a simple set of patches for LLVM (mostly calling conventions, maybe some additional intrinsics for the weird OCaml exception model), which can be merged upstream. > I think it is natural that you have to make changes to LLVM, > the GHC people (which now have an experimental LLVM backend) also did, > and I was under the impression that the LLVM people where quite > welcoming of their changes, they are glad to see LLVM being used in a > non-Clang-centric project. I think your patches could bring value to > LLVM, independently of the success of the ambitious ocaml backend > attempt. Hm, I'm not sure. It's really easy to generate LLVM code for OCaml in general, the problem is getting things to interact with legacy OCaml code, with exception handling being one of the most important issue. The required stuff will be very platform specific and very specific to OCaml, and we don't even know if it's going to work.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00163.html
Damien Doligez announced (please find the attached file at the archive link):I have implemented emacs-lisp code that correctly recognizes all comments and strings in OCaml code, and interfaced it with the fontification features of Emacs. The code is committed in the SVN repository, but if you want to try it before the next release, I'm including it in attachment. If you want to know why this is not a trivial problem, consider the following OCaml program. ======================================================= let l' _ = ();; let _' _ = ();; let l' = ();; let b2_' = ();; let a'a' = ();; let f2 _ _ = ();; let f3 _ _ _ = ();; (* ==== easy stuff ==== *) (* a comment *) (* "a string" in a comment *) (* "another string *)" in a comment *) (* not a string '"' in a comment *) "a string";; '"';; (* not a string *) (* ==== hard stuff ==== *) l'"' not not a string ";; _'"' also not not a string";; f2 0l'"';; (* not not not a string *) f2 0_'"';; (* also not not not a string *) f3 0.0l'"' not not not not a string ";; f3 0.0_'"';; (* not not not not not a string *) f2 0b1_'"';; (* not not not a string *) f3 0b2_'"' not not not not a string ";; f3 0b02_'"';; (* not not not not not a string *) '\'';; (* a char *) ' ';; (* a char *) '^M ';; (* also a char [replace ^M with one CR character] *) a'a';; (* not a char *) type ' a' t = X;; (* also not a char *) ======================================================= Obligatory troll: try to do the same in vim!Yoann Padioleau then said:
There is perfect syntax coloring in Codemap for OCaml :) https://github.com/facebook/pfff/wiki/CodeMap It even colorizes differently identifiers for functions, variables, types, etc.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00204.html
Deep in some thread, Edgar Friendly said:Sylvain has worked with me to enable auto-installation of oasis-db packages via odb[2]. There's not a large repo of packages[1], but most of it is auto-installable (run odb to get a list), as long as you have ocaml and findlib to start from. The dependencies are automatically handled and installed from source. I even have confirmation that it works under windows, although it definitely needs cygwin there. The repo could use some love with people uploading[3] new packages to it. Just provide a tarball and optionally a link to the tarball on some other website. Oasis dependencies are detected automatically, packages w/o an _oasis file need to have deps specified manually. Odb is able to install findlib-enabled projects that use ([./configure &&] make && make install), (omake && omake install) and oasis' setup.ml for building. This last option is able to execute whatever sequence of configure, make and install commands are needed for your project, without replacing the current infrastructure. Files uploaded to the repo are available with odb --unstable, they have to be manually approved to become part of the default --testing. I hope that in time, the --stable option can become default. Odb also installs by default to ~/.odb/lib. --sudo tells it to use sudo for global install, --have-perms tells it to install to the global repo without sudo. If anyone wants to auto-detect permissions on the ocamlfind global, we can get rid of --have-perms. I'm on #ocaml most of the time, and will get back to you if I'm away. Tell me how things work or don't. E. [1] http://oasis.ocamlcore.org/dev/browse [2] http://github.com/thelema/odb [3] http://oasis.ocamlcore.org/dev/uploadrixed asked and Edgar Friendly replied:
> What I'd really like is a way to mix any version I want of the packages I > install, especially experimental versions for the packages I want to test or > contribute to. > I stopped using GODI some time ago because I wanted master of ocaml and > batteries but stable versions of everything else. So I ended up rolling my > own makefile-based installation/upgrade tool which is both annoying and > archaic. > > Is this in the planned feature list? This is possible currently, by using the --stable, --testing and --unstable flags when installing different packages. Of course, the downside of this is that there's no guarantee or test of compatibility between packages and different versions of OCaml (and possibly each other). Oasis packages can have versioned dependencies, which helps, but the only real guarantee I think that odb can make is that the stable repo all works with each other. Maybe the testing repo too, sometimes, but definitely not the unstable repo. The above all assumes that the versions you want are packaged and in oasis-db. Auto installation from repositories (or local directories) has been considered, but there's some more code to add, as currently the oasis-db server parses _oasis files for deps, and odb is designed to be extremely lightweight. On that note, one more thing about odb - it has no configuration or database of what packages are installed. It doesn't even have the ability to remove a package (not that this couldn't be hacked up in not too much time for library packages). If you ask it to install a package, it checks for deps by asking findlib what's installed. It doesn't try to control things as much as GODI seems to. It just does the minimum necessary to get the job done. This means it plays as nicely as possible with packages installed by whatever other means you wish, even GODI.Gabriel Scherer asked and Edgar Friendly replied:
> Could you (or Sylvain) make a more precise picture of how exactly the > community could help in the Oasis-DB effort? My opinion is that oasis-db+odb is good enough for wider use. I don't know what plans Sylvain has for the oasis-db server side, but what's currently provided is good enough to make many people's lives easier. > Is the priority to upload package (then maybe the warning on the > webpage advising not to do it seriously should be changed), or are > there other things we could help with, for example development > aspects? Who/where should we ask for advice/help when we have issues? Any current work in packaging things for oasis-db will almost certainly be not lost by future updates. Worst case, if the database side of the server gets re-designed, all it has to do is re-import the tarballs and everything will be fine. AFAIK, only Sylvain can change the warning. As well, he's the one that knows the server side of things. I have experience dealing with some of its ... subtleties, but no ability to fix. For the client end (downloading, installation) - that's entirely me, and I'm happy to work with people on IRC (which I find myself doing often with beginners) or over email. > A few months ago we tried to organize an "OCaml packaging sprint", in > which in particular Hezekiah M. Carty was of great help. Do you think > we could restart a similar effort in the short future? Probably more useful would be for people to try packaging the libraries that they use for themselves. I've not made grand announcements so that its users can grow slowly, without huge expectations. > It would really help, I think, if: > - there was a list somewhere of things other people can contribute I've just started this wiki page: https://github.com/thelema/odb/wiki/SoftwareToPackage listing the status of many pieces of software under oasis-db. Extend.Andrei Formiga asked and Gerd Stolpmann replied:
> I think a good package system (with associated repository) and better > documentation are the two biggest things that can help OCaml's > adoption. It's true that there are languages that have become > successful without a package management system, but it has become > increasingly expected that languages have one. OCaml does not have > marketing or hype, so it has to win over new users by not creating > barriers to adoption. Plus it's much easier to work on a daily basis, > even for veterans. This is already true with GODI, which saves me a > lot of time when the library I need to install is available in its > repo. > > The question is: what should be done? What must be done to enable > OASIS-DB? Or should everyone adopt GODI? What's the situation between > these two systems? Maybe having some kind of rough roadmap would help > there. The plan is that OASIS-DB exports its packages in a format that is understood by GODI. OASIS-DB would then appear as a second source for packages. For users there would be practically no difference - godi_console just fetches packages from a second site, too. For package developers this will mean that there is a choice. More complicated packages will probably remain native GODI ones (because of the unlimited scripting) whereas the average library will be well expressable in OASIS-DB.
Archive: https://sympa-roc.inria.fr/wws/arc/caml-list/2011-12/msg00241.html
Alain Frisch said:The mingw port of OCaml was not in a good shape, because of changes in Cygwin: - We used to rely on the normal Cygwin gcc compiler, using the -mno-cygwin flag. This is no longer available for recent versions of gcc shipped in Cygwin. There is still a gcc-3.exe, but it is not clear whether it will be supported in the future. - There are now two modern versions of gcc, available in cygwin, which supports compiling in "mingw" mode (32-bit mode): * A packaged version of the compiler from the MinGW.org project: i686-pc-mingw32-gcc.exe * A packaged version of the compiler from the mingw-w64 project: i686-w64-mingw32-gcc Future versions of the OCaml mingw port should be based on one of these two versions. I'd be interested to hear if anyone in the community has any advice on which one to choose. Feel free to comment on this list, or on the bug tracker: http://caml.inria.fr/mantis/view.php?id=5179 Currently, flexdll (version 0.27) and OCaml (SVN trunk version) have been adapted to work with the mingw-w64 version (32-bit only, for now), but if there are arguments in favor of the other one, it should not be difficult to switch (supporting both is not technically difficult, but it might create useless confusion). It would also be very useful to get some community feedback about the current version (OCaml trunk + flexdll version 0.27).
Thanks to Alp Mestan, we now include in the Caml Weekly News the links to the recent posts from the ocamlcore planet blog at http://planet.ocamlcore.org/. PDF Explained, now available from O'Reilly: http://coherentpdf.com/blog/?p=54 Coq's sumor Type: http://seb.mondet.org/blog/post/coqtests-03-sumors.html Batteries moving to 2.0: https://forge.ocamlcore.org/forum/forum.php?forum_id=818 1D root finding: https://forge.ocamlcore.org/projects/root1d/ Runtime Types: http://www.lexifi.com/blog/runtime-types HoTT Equivalences: http://math.andrej.com/2011/12/07/hott-equivalences/ 1D optimization: https://forge.ocamlcore.org/projects/optimization1d/ How to make the "impossible" functionals run even faster: http://math.andrej.com/2011/12/06/how-to-make-the-impossible-functionals-run-even-faster/
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.