Hello
Here is the latest OCaml Weekly News, for the week of January 30 to February 06, 2018.
Archive: https://discuss.ocaml.org/t/modules-moved-out-of-async-extra/1506/1
Stephen Weeks announced:
At Jane Street, we recently removed a number of modules from the
`Async_extra` library:
- `Cpu_usage`
- `Dynamic_port_writer`
- `File_tail`
- `Persistent_singleton`
- `Sequencer_table`
- `Tcp_file`
- `Unpack_sequence`
- `Versioned_typed_tcp`
These modules are now available as separate standalone libraries that
depend on `Async`. Please let us know if you would like us to release
any of them.
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2018-01/msg00079.html
Evgeny Roubinchtein asked and David Allsopp replied:
> Dear OCaml users and developers,
> My current employer ships an executable whose source code is written in OCaml,
> and is trying to understand the security implications and mitigating actions
> (if any) of CVE-2017-9779. For the purposes of this discussion, only the
> native code compiler (ocamlopt) is relevant.
>
> The questions I would hope to get answers to are:
>
> 1. Which versions of the OCaml compiler produce executables which are affected
> by the vulnerability/ies described in CVE-2017-9779?
All versions prior to 4.04.2 (that's "All" as in "since 1.00", not just "4.04.0
and 4.04.1").
> 2. What mitigation/s (if any) are suggested?
The issue only affects bytecode executables linked in custom runtime mode where
there is a limited attack possible on CAML_DEBUG_SOCKET. Since you're using
ocamlopt, you're fine.
Note that we updated the manual for 4.05.0 in GPR#1213
(https://github.com/ocaml/ocaml/pull/1213/files#diff-d79da36dce83d24f7fd0cadefe8c97a1R319)
to note that custom runtime bytecode executables should never be setuid or
setgid.
Sorry for the delay with the reply - it turns out several of us developers
involved in the discussion had taken out non-disclosure agreements with our own
memories. The CVE was reserved when it was thought that attack allowed arbitrary
privilege escalation, but when we determined it wasn't that serious, we released
with just CVE-2017-9772 (which affects ocamlc and ocamlopt but in 4.04.0 and
4.04.1 *only*). We've forgotten to update the CVE text itself, which we're now
in the process of dealing with.
All best,
David (on behalf of the OCaml devs)
Archive: https://discuss.ocaml.org/t/web-scraping-in-ocaml/1511/1
Perry E. Metzger:
What are the preferred tools for doing web scraping in OCaml? I'm interested
both in what's available for automating interaction (like the Mechanize packages
for Ruby and Python) as well as the HTML parsing side.
Gabriel Radanne replied:
For the first part, there is [mechaml](https://github.com/yannham/mechaml) (by
@yannham) and for the second part, you can use
[lambdasoup](https://github.com/yannham/mechaml) (by @antron)!
I'm sure both authors would be very happy to have feedback. ;)
Simon Cruanes then added:
The url for lambdasoup is https://github.com/aantron/lambda-soup :wink:
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2018-01/msg00080.html
Nicolas Cannasse announced:
Haxe Foundation is recruiting a compiler developer!
Haxe compiler is a cross platform compiler allowing to target different
platforms using a single high level mixed OO/Functional programming
language. The Haxe compiler is entirely written in OCaml and outputs code
for JavaScript, C++, HashLink, Python, C#, Java, PHP, Lua, Neko and Flash.
More details about the position are available on our blog post here:
https://haxe.org/blog/hf-is-recruiting/
You can contact me directly for any question.
Archive: https://discuss.ocaml.org/t/caqti-0-9-0-release-compatibility-layer-for-database-clients/1522/1
Petter A. Urkedal announced:
A am please to finally announce the first release of
[Caqti](https://github.com/paurkedal/ocaml-caqti) to our main opam repository.
Caqti is an RDBMS client library for OCaml, currently shipped with drivers for
MariaDB, PostgreSQL, and Sqlite3. In additional to serving as a compatibility
layer, it provides monadic concurrency with support for Lwt and Async.
TL;DR (yes it is; hopefully some of you will read anyway): There is [an
example](https://github.com/paurkedal/ocaml-caqti/blob/master/tests/bikereg.ml)
linked from the GitHub page.
## Design
Caqti accepts a URI which provides the location, authentication, and
configuration for a particular database, and returns a pool of first-class
modules implementing a common signature used to execute queries. The pooled
modules includes the connection object, so they are meant to be used only in
local contexts. Automatic loading of drivers is provided by linking against
`caqti-dynload`, otherwise link against the needed drivers.
The client can define fixed requests (queries) at the module level. When
executed, Caqti prepares and caches the prepared statements within the module
corresponding to the connection. Caching of prepared statements can be disabled
in case the query is generated on the fly.
Caqti handles encoding and decoding of results based on types specified along
with the queries. Most common datatypes are supported except for the time-of-day
(`time`) type. The elementary types are represented as an open variant, which
can be extended by providing conversion functions between the types implemented
by drivers. On top of the elementary types, on can form tuples and custom types
converted to and from tuples, though these custom types should be considered
experimental for now.
On the other hand, Caqti does not provide a DSL for defining queries. Queries a
basically strings, though in the form of templates in order to ease generation
of queries strings and to support both `?`-style and `$n`-style parameters
independent of which database driver is used. The primitive request constructor
actually takes a function which receives information about the driver and
returns a template, in order to allow the client code to dispatch between
different RDBMS where needed.
In other words, the focus of Caqti is to provide the most basic compatibility
layer to allow writing code which works across RDBMS. It is okay to use it
directly, but there is also room for higher level interfaces like code
generators or DSLs implemented in plain OCaml or with the help of PPXes. I have
seen several good approaches, but I'll leave that for the discussion and future
work.
## Status
The API underwent some redesign this fall, but is largely based on code which I
have used myself for a few years. Lwt and PostgreSQL is the best tested
components, since I've used them in all my production code, though with help
from @andrenth, I believe the MariaDB component is fairly solid too now. Both
PostgreSQL and MariaDB drivers are based on asynchronous calls provided by the
respective C bindings. The Sqlite3 driver is a fairly straight-forward
preemptive wrapper around the already stable C bindings.
There may still be issues with conversion of certain values depending on
databases, especially when the type cannot be inferred. Handling conversion as
best as possible is one of the goals of Caqti, so please report it in the issue
tracker or submit a PR with a test case.
This being the first properly announced release, I think it is also a good time
for you to report back if we need adjustments to the API. Consider that also a
warning, though I hope we can avoid substatial redesign.
The API documentation is not on-line at the moment, but can be generated with
`odig odoc`.
Please ignore the `Caqti1_*` modules and `*.v1` findlib libraries, they will be
deprecated (as soon as I rewrite the epiSQL code generator, which is not
released in opam-repository yet).
## Thanks
Thanks to Markus Mottl, Alain Frisch, Christian Szegedy, and Andre Nathan for a
great job on providing bindings to the C libraries, which requires special
expertise in memory management, debugging, and packaging.
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2018-02/msg00002.html
David MENTRÉ announced:
My company, Mitsubishi Electric R&D Centre Europe, is developing research
prototypes in OCaml. For one of those prototypes, we would like to make a
graphical interface using web technologies, typically using Ocsigen framework.
We could do it by ourself but in order to speed up our developments we would
like to recruit somebody on a short term contract.
Our strong budget and time constraint is that the work should be finished by end
of this March. Therefore we are looking for an available OCaml developer that
could start working right now.
If you are interested, please contact me, preferable at email:
d.mentre@fr.merce.mee.com.
Archive: https://discuss.ocaml.org/t/ann-json-of-jsonm-1-0-0-released/1526/1
Stephen Bleazard announced:
json_of_jsonm 1.0.0 has been released. This provides a layer on top of jsonm to
encode and decode text to a `json` type. Key features include:
* The `json` type is compatible with and a subset of yojson's `json` type
* Both string and channel interfaces are provided
* The API supports Pervasives.result and exceptions to handle errors
* A functor is provided to implement additional I/O mechanisms. The README
includes an Async example.
* The string module defines a standard `t`, `of_string` and `to_string` interface
jpdeplaix then said:
Nice !
Just in case anybody else wanted to know more as well, here is the repository:
https://github.com/stevebleazard/ocaml-json-of-jsonm/
Jesse Haber-Kucharsky asked and Stephen Bleazard replied:
> Would you please describe how this library differs from
> [ezjsonm](https://github.com/mirage/ezjsonm)?
json_of_jsonm is designed provide a thin layer on top of jsonm that allows
conversion to and from a reasonably standard `json` type. In addition, where
appropriate, the API attempts to follow standard conventions (the string module
defines a type `t` and `{of,to}_string` functions for example) and provide
exception and `result` based eror reporting
Finally, additional I/O mechanisms can easily be defined with an example Async
module shown in the examples section of the README.md
ezjsom in comparison provides a number of useful functions that extend the jsonm
library as weill as adding Lwt support. However, it follows jsonm typing and
does not support I/O methods other than those already provided by the library
Which you use will depend on your needs, for me json_of_jsonm is used to handle
thinks like RESTful APIs using Async libraries but others may find ezsjonm's
access functions more useful.
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2018-02/msg00003.html
Louis Gesbert announced:
We are pleased to announce the first Release Candidate for opam 2.0.0.
I'd like to thanks all the beta testers, and welcome all interested to take this
last chance to report any issues with the new version.
See the details at https://opam.ocaml.org/blog/opam-2-0-0-rc/
Archive: https://discuss.ocaml.org/t/ann-easy-xlsx/1529/1
Brendan Long announced:
I just finished cleaning up a library I've been working on for reading Excel
documents (XLSX / SpreadsheetML):
https://github.com/brendanlong/ocaml-ooxml
I've tried to separate out the various bits, so you if you want to read
Microsoft's ["Open Packaging Conventions"
files](https://en.wikipedia.org/wiki/Open_Packaging_Conventions) (the generic
packaging for all Microsoft Office docs), there's `open_packaging`. If you want
to read SpreadsheetML files (i.e. Excel files) there's `spreadsheetml`, and if
you just want to open an Excel file and get the data out, there's `easy_xlsx`,
which exposes an XLSX file as a list of sheets (sheet name, rows) where each row
is a list of values (date, datetime, number, string, time of day).
The interface isn't exactly ideal, since CamlZip can only open files, and date
and time formats aren't built into the standard library, so this requires
`core_kernel`. It's also not particularly efficient right now, but pull requests
are welcome!
If you have files where `easy_xlsx` doesn't do the right thing, please open an
issue and attach the file.
I would upload it to opam but I `opam prepare` is giving me empty folders, and
I've run out of time to spend on it this weekend.
Archive: https://discuss.ocaml.org/t/im-preparing-a-talk-about-ocaml-and-web-programming/1532/3
In this thread, Yaron Minsky said:
I don't know that I'd recommend it to the masses, but Incr_dom is at least an
interesting approach. We use it for building a variety of production UIs at Jane
Street, and people seem pretty happy with it thus far.
It's most interesting if you're interested in building performant, incrementally
updating UIs that may use lots of data.
https://github.com/janestreet/incr_dom
Marek Kubica then said:
I particularly enjoyed
[webmachine](https://github.com/inhabitedtype/ocaml-webmachine) to write a
REST-service. The state tree it uses (inspired by Erlang's Webmachine and
Clojure's Liberator) makes it easier to make sure to return the proper codes and
responses while the type system helps you to write your code correctly in the
first place.
Edit: [Here's how such a service might look
like](https://github.com/Leonidas-from-XIV/reklama).
Archive: https://sympa.inria.fr/sympa/arc/caml-list/2018-02/msg00011.html
Gabriel Scherer announced:
Dear OCaml users,
The release of OCaml version 4.06.1 is still imminent. We have
created a second release candidate for your testing pleasure. Please
try it and let me know whether it works for you. It is available as
opam switches 4.06.1.+rc2 (or +rc2+flambda, etc.)
opam switch 4.06.0+rc2
or as source archives at
http://caml.inria.fr/pub/distrib/ocaml-4.06/
The list of changes since 4.06.0 is included at the end of this
email.
The only change compared to 4.06.1+rc1 is that we backported some
pattern-matching changes (GPR#1550) already present in the development
version (trunk, 4.07.0+dev), which are necessary to prevent some
compile-time errors when matching on extensible datatypes or GADTs.
Happy hacking,
-- Gabriel Scherer for the OCaml team.
Changes:
- MPR#7661, GPR#1459: fix faulty compilation of patterns
using extensible variants constructors
(Luc Maranget, review by Thomas Refis and Gabriel Scherer, report
by Abdelraouf Ouadjaout and Thibault Suzanne)
- MPR#7702, GPR#1553: refresh raise counts when inlining a function
(Vincent Laviron, Xavier Clerc, report by Cheng Sun)
- MPR#7704, GPR#1559: Soundness issue with private rows and pattern-matching
(Jacques Garrigue, report by Jeremy Yallop, review by Thomas Refis)
- MPR#7705, GPR#1558: add missing bounds check in Bigarray.Genarray.nth_dim.
(Nicolás Ojeda Bär, report by Jeremy Yallop, review by Gabriel Scherer)
- MPR#7713, GPR#1587: Make pattern matching warnings more robust
to ill-typed columns; this is a backport of GPR#1550 from 4.07+dev
(Thomas Refis, review by Gabriel Scherer, report by Andreas Hauptmann)
- GPR#1470: Don't commute negation with float comparison
(Leo White, review by Xavier Leroy)
- GPR#1538: Make pattern matching compilation more robust to ill-typed columns
(Gabriel Scherer and Thomas Refis, review by Luc Maranget)
Archive: https://discuss.ocaml.org/t/which-xml-libraries-do-you-recommend/1535/1
Yoriyuki Yamagata asked:
I want to parse ucdxml (the XML version of Unicode Character Database). I find
three XML parsing libraries
- pxp
- xml-light
- xmlm
in opam. Which libraries do you recommend? I need only parsing it to some useful
OCaml data structure, no need to search, validate etc. One problem, however, is
that I do not know the specification of this XML file so I want to experiment
the format of XML while developing a parser. For example, I want to print the
value of the output of the parser. The doc is voluminous and difficult to
understand, so I do not want to understand it completely before starting
development.
Daniel Bünzli replied:
The only one I would not recommend is `xml-light` unless you control the format
you are parsing yourself (unless it has been updated it's not standard compliant
and will choke on valid xml). Note however that a parser for the ucdxml already
exists [here](http://erratique.ch/software/uucd).
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.