Hello

Here is the latest OCaml Weekly News, for the week of December 02 to 09, 2014.

1. What are people using when writing distributed applications in OCaml?
2. Other OCaml News

Archive: https://sympa.inria.fr/sympa/arc/caml-list/2014-12/msg00021.html

If you know about some distributed application that went into production
and that can happily withstand the fire of production and real world
loads, I would be happy to here about which library the system was
using.

I have heard about MPI and zmq for example, which both have
ocaml bindings. However, I also know that there are several RPC
implementations out there (one in ocamlnet, one in mirage, one in
core), but I am not sure about which one is the most mature and
production-tested.
      

I personally use ocaml-zmq bindings, and I am working with the piqi
author (and my work) to try and release my ocaml RPC implementation.
      

At Issuu, we started with an ZMQ / Piqi based messaging (RPC and
broadcast). However we are leaning more and more to using AMQP for
messaging and RPC where throughput and latency is not as critical
(we still handle 50.000 messages/sec through AMQP though).

Amqp handles RPC style messages nicely, and handles client disconnects
and retransmissions in a useful way.

Having a central message broker also decouples the services even more
as the micro-services only need to agree on which Amqp to use and does
not need to know the name of each other.
      

The one in OCamlnet was extensively tested at Mylife.com (around 2009),
where we built up a 24/7 distributed service over several hundred
machines. Part of that were also some HA services with failover
possibility, which is worth to mention because it means that the error
reporting and timeout handling was working.

At present, I'm not aware of a mass deployment, though.
      

It depends on what level you're talking.  My distributed systems apps
tend to be functorized over their transport, so I can switch out what
works and what doesn't pretty easily.

I tend to avoid RPC frameworks, I find they focus more on the
computation than on the data, but depending on what you're doing they
could be appropriate.

Unless you're under heavy load, simple HTTP calls with some JSON is
probably sufficient.
      

I guess Francois is asking because for some reason HTTP is
inappropriate. I can understand that because it is a really bloated
protocol, and you usually only use it when you need to (e.g. you have to
call it from Javascript or the like, or have to go through firewalls).
Also, there are several aspects you cannot control well, e.g. when TCP
connections are created and kept alive (or not). Multicast messaging is
completely impossible.

I recommend to not to see RPC systems under their framework aspect, but
simply as transport libraries. In particular, if you don't like the
serialization scheme, there is usually the alternative that you take
your own serializer and just send the data as (binary) strings on the
RPC level (which is always possible and simple to do). From my
experience it is essential that you have good control over the low-level
things (e.g. connection control, timeouts, retrials, error handling
etc), because this is what the distributed system keeps running. Sure it
is good when an RPC library implements some automatisms (e.g. for error
handling), but it is even better when there is also a manual mode, as
distributed systems sometimes require unusual ways of dealing with that
stuff.
      

Thanks to Alp Mestan, we now include in the OCaml Weekly News the links to the
recent posts from the ocamlcore planet blog at http://planet.ocaml.org/.

Full Time: Software Developer (Functional Programming) at Jane Street in New York, NY; London, UK; Hong Kong:
  http://jobs.github.com/positions/0a9333c4-71da-11e0-9ac7-692793c00b45

Parsing regular expressions:
  http://shayne-fletcher.blogspot.com/2014/12/parsing-regular-expressions.html

Senior Software Engineer at McGraw-Hill Education (Full-time):
  http://functionaljobs.com/jobs/8771-senior-software-engineer-at-mcgraw-hill-education

OCaml, Objective C, Rule 4:
  http://psellos.com/2014/12/2014.12.objc-rule-four.html

Recursion Reloaded:
  http://typeocaml.com/2014/12/04/recursion-reloaded/
      

Old cwn

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt