# OCaml Weekly News

Hello

Here is the latest OCaml Weekly News, for the week of March 26 to April 02, 2019.

## Cstruct.4.0.0: sexplib goes optional

A headsup about a forthcoming backwards incompatible change to the cstruct 4.0.0 release. We've been reducing the required dependencies of the core Mirage libraries to make the 'hello world' unikernels as small as possible. As part of that, cstruct (along with its good friends ipaddr and uri) have moved sexplib into a separate optional module.

From the changes file:

• Sexplib is now an optional library for the base Cstruct module. A new Cstruct_sexp module has been introduced with the serialiser functions, contained within the cstruct-sexp opam package.

To convert old code, simply use Cstruct_sexp.t instead of Cstruct.t in a record type for which you are using [@@deriving sexp]. This is a type alias to Cstruct.t but also has the right sexp-conversion functions in scope. There is an example of this in the ppx_test/with-sexp directory in the source repo.

When you have converted and released your library, add an opam constraint of cstruct {>="4.0.0"} to your own opam packages to ensure that they pick up this version of the library.

The opam repository PR with the full changeset is at : https://github.com/ocaml/opam-repository/pull/13748

## New library - uritemplate 0.1.0

### Corin Chaplin announced

I am happy to announce my first library published to opam: uritemplate! The library is a OCaml implementation of URI templates (RFC6570).

Currently it only provides basic functionality but is compliant to level 4 of the specification. I am looking at adding more functionality, such as a Template.t type, so templates can be parsed and then templated multiple times.

The repository is here, the README and documentation is a bit lacking at the moment, but any feedback would be very much appreciated :slight_smile:

## Check opam's health for the upcoming OCaml release (4.08)

### Kate announced

Like last month, I'm here to announce new features and news from check.ocamllabs.io.

To recall, this website is an interface to find out which opam packages are broken with a given set of OCaml compilers. This month's features are especially useful to look for missing or broken packages for the upcoming OCaml 4.08.

What's new:

• A new type of failures has appeared: internal failures (in white). Those were previously mistakenly categorized as a normal build failure but are usually far from it. It includes: solver failures, unavailable external dependencies, temporary misc server failures, …
• Not available packages are now not skipped but have now logs attached to them. It makes the total check run longer but is extremely useful to understand why does packages are not available (e.g. which dependencies are blocking, …)

Minor features:

• I have adopted a new name for the underlying tool: opam-health-check
• Under advice from @kkazuo, @grayswandyr and @Ronan in the previous post, I have adopted a new colour palette less ambiguous to both colorblinds and non-colorblinds (at least I hope)
• Slack integration (see more below)

What's next:

• Due to the ever growing number of opam packages and solver shenanigans, some solver failures can appear, especially on a server with a lot of packages building in parallel. For now opam-health-check uses the SYMPHONY solver instead of the builtin solver to mitigate that. However some failures still appear and I'm looking to use a custom branch of opam that uses Z3 instead.
• I'm still meaning to add a revdeps counter soon as it is especially useful for new compiler releases (suggested by @bluddy a while ago, see previous post)
• Discuss and email integration, to send a message every time a run has finished (every 3-4 days). For now only Slack has just been integrated.

As OCaml 4.08 will be somewhat soon released, I'm inviting everyone (especially maintainers), with some spare time and will power, to have a look at the main page with those filters:

For maintainers you can filter yourself and get only the packages you are maintaining, using the form for this purpose.

Happy hunt.

Side note: for now the diff page is going to show garbage data due to a server failure during the previous run

Is there a portable way in ocaml to turn echoing off on standard input from the terminal to read e.g. passwords? By portable I mean that it works for Windows, Unix and Mac.

### Matthew Ryan replied

The usual way to do this (for any language) is using ANSI escape sequences. Code 8 sets the terminal to conceal characters and code 0 resets the attributes, making them visible again.

For example, in a unix shell you can test this with echo and read:

echo -e '\x1b[8m'; read varname; echo -e '\x1b[0m'

To do the same from OCaml, you can output "\x1b[8m", read the password, and then output "\x1b[0m" afterwards to switch printing back on.

I believe that this will work on Windows 10, but earlier versions may not have the necessary ANSI support.

### Daniel Bünzli also replied

Just for refence another way (that will in no way work on Windows) is to temporarily tweak the terminal attributes. The ocamlunix book shows how to do this [here][1].

In contrast to ANSI escapes which I believe is just a rendering trick that snippet will not allow to cut and paste the input password to recover it, whether that's a property you care about or not is up to your use case.

Best,

Daniel

### Chet Murthy said

A little Googling turns up that the author of the Unix library (Xavier, IIRC) provided support for termios(3). So you can already do what you want in Ocaml with no extra C ugly bits.

Here's a little ocaml program to demonstrate, and after it, some strace output showing the way it calls ioctl(2) to manipulate the line discipline (relevant bits bolded in HTML format mail).

let main () =
let open Unix in
let tios = tcgetattr stdin in
Printf.printf "c_echo: %b\n" tios.c_echo ;
Printf.printf "c_echoe: %b\n" tios.c_echoe ;
Printf.printf "c_echok: %b\n" tios.c_echok ;
Printf.printf "c_echonl: %b\n" tios.c_echonl ;
flush Pervasives.stdout ;
tios.c_echo <- false ;
tcsetattr stdin TCSANOW tios ;
let tios = tcgetattr stdin in
Printf.printf "AFTER c_echo: %b\nSleeping 10 sec ....\n" tios.c_echo ;
flush Pervasives.stdout ;
Unix.sleep 10;
tios.c_echo <- true ;
tcsetattr stdin TCSANOW tios ;
()
;;

main() ;;

$strace -eioctl ./noecho ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0 c_echo: true c_echoe: true c_echok: true c_echonl: false ioctl(0, TCGETS, {B38400 opost isig icanon *echo* ...}) = 0 ioctl(0, TCGETS, {B38400 opost isig icanon *echo* ...}) = 0 ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig icanon *-echo *...}) = 0 ioctl(0, TCGETS, {B38400 opost isig icanon *-echo *...}) = 0 ioctl(0, TCGETS, {B38400 opost isig icanon *-echo* ...}) = 0 AFTER c_echo: false Sleeping 10 sec .... ioctl(0, TCGETS, {B38400 opost isig icanon *-echo* ...}) = 0 ioctl(0, TCGETS, {B38400 opost isig icanon *-echo* ...}) = 0 ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig icanon *echo* ...}) = 0 ioctl(0, TCGETS, {B38400 opost isig icanon *echo* ...}) = 0 +++ exited with 0 +++  ### Jeremie Dimino also replied If you don't mind the extra dependency, there is a complete example in the lambda-term library: The main differences with the other solutions mentioned in this thread are that this version works on Windows, supports line edition and supports customisation such as displaying nothing or displaying stars. ## http2/af: An HTTP/2 implementation for OCaml ### Antonio Nuno Monteiro announced I'm pleased to announce the product of a few months of work: an HTTP/2 implementation written entirely in OCaml: http2/af. http2/af is based on the concepts in http/af, using Angstrom and Faraday for the parsing and serialization of the HTTP/2 framing layer, respectively. It also preserves the same API as http/af wherever possible, for familiarity reasons. The notable exception is the addition of a Reqd.push function that implements the HTTP/2 server push functionality. http2/af currently provides a server implementation, as well as runtime implementations for Lwt (UNIX) and Mirage (which I implemented at the last Mirage Hack Retreat in Marrakech). The repo for http2/af is here. The packages are not yet released to OPAM, a task I'm planning to complete in the next few days. Happy to hear any feedback / answer questions here or in Github issues (https://github.com/anmonteiro/http2af/issues). ### Antonio Nuno Monteiro later added There were some licensing issues in this work related to oversights and ignorance on my part. I’m copying a text from a Reddit post of mine that tries to clarify the situation below. Thanks everyone who brought all the licensing issues to my attention. To clarify: I never intended to “steal” any code, I.e. passing work as mine without proper attribution to the original authors. The truth is that, while I’ve been doing open source for a little while, this is the first time that I’ve done any derivative work, and not being a lawyer, I really had no idea what exactly I needed to do (my thought process being that if my code didn’t bear any resemblance at all to the original code then I wouldn’t need to preserve the copyright headers). It has been brought to my attention that I was wrong and I have therefore rectified the situation. To clarify further what h2 is in comparison to http/af: h2 is an implementation of HTTP/2 (a totally different protocol) that tries to share the http/af types and API wherever possible. I understand now that this is derivative work because it shares the same underlying architecture, but I didn’t know that coming into it at first. To all the affected parties: my apologies, I hope you can forgive me. ## Release of OCamlFormat 0.9 ### Guillaume Petiot announced Release of OCamlFormat 0.9 We are pleased to announce the release of OCamlFormat (available on opam). There have been numerous changes since the last release, so here is a comprehensive list of the new features and breaking changes to help the transition from OCamlFormat 0.8. #### Dependencies OCamlFormat now requires: • ocaml >= 4.05 (up from 4.04.1) • dune >= 1.1.1 • octavius >= 1.2.0 • uutf OCamlFormat_Reason now requires: • ocaml >= 4.05 • dune >= 1.1.1 • ocaml-migrate-parsetree >= 1.0.10 (up from 1.0.6) • octavius >= 1.2.0 • uutf • reason >= 3.2.0 (up from 1.13.4) #### New preset profiles The ocamlformat profile aims to take advantage of the strengths of a parsetree-based auto-formatter, and to limit the consequences of the weaknesses imposed by the current implementation. This is a style which optimizes for what the formatter can do best, rather than to match the style of any existing code. General guidelines that have directed the design include: • Legibility, in the sense of making it as hard as possible for quick visual parsing to give the wrong interpretation, is of highest priority; • Whenever possible the high-level structure of the code should be obvious by looking only at the left margin, in particular, it should not be necessary to visually jump from left to right hunting for critical keywords, tokens, etc; • All else equal compact code is preferred as reading without scrolling is easier, so indentation or white space is avoided unless it helps legibility; • Attention has been given to making some syntactic gotchas visually obvious. ocamlformat is the new default profile. The conventional profile aims to be as familiar and "conventional" appearing as the available options allow. The default profile is ocamlformat with break-cases=fit. default is deprecated and will be removed in version 0.10. #### OCamlFormat diff tool ocamlformat-diff is a tool that uses OCamlFormat to apply the same formatting to compared OCaml files, so that the formatting differences between the two files are not displayed. Note that ocamlformat-diff comes in a separate opam package and is not included in the ocamlformat package. The file comparison is then performed by any diff backend. The options' documentation is available through ocamlformat-diff --help. The option --diff allows you to configure the diff command that is used to compare the formatted files. The default value is the vanilla diff, but you can also use patdiff or any other similar comparison tool. ocamlformat-diff can be integrated with git diff, as explained in the online documentation. #### Formatting docstrings Previously, the docstrings (** This is a docstring *) could only be formatted like regular comments, a new option --parse-docstrings has been added so that docstrings can be nicely formatted. Here is a small example: (** {1 Printers and escapes used by Cmdliner module} *) val subst_vars : subst:(string -> string option) -> Buffer.t -> string -> string (** [subst b ~subst s], using [b], substitutes in [s] variables of the form "$(doc)" by their [subst] definition. This leaves escapes and markup
directives \$(markup,...) intact.
@raise Invalid_argument in case of illegal syntax. *)


Note that this option is disabled by default and you have to set it manually by adding --parse-docstrings to your command line or parse-docstrings=true to your .ocamlformat file. If you get the following error message:

Error: Formatting of (** … *) is unstable (e.g. parses as a list or not depending on the margin), please tighten up this comment in the source or disable the formatting using the option –no-parse-docstrings.

It means the original docstring cannot be formatted (e.g. because it does not comply with the odoc syntax) and you have to edit it or disable the formatting of docstrings.

Of course if you think your docstring complies with the odoc syntax and there might be a bug in OCamlFormat, feel free to file an issue on github.

#### Print the configuration

The new --print-config flag prints the configuration determined by the environment variable, the configuration files, preset profiles and command line. Attributes are not considered.

It provides the full list of options with the values they are set to, and the source of this value. For example ocamlformat --print-config prints:

profile=ocamlformat (file .ocamlformat:1)
quiet=false (profile ocamlformat (file .ocamlformat:1))
max-iters=10 (profile ocamlformat (file .ocamlformat:1))
comment-check=true (profile ocamlformat (file .ocamlformat:1))
wrap-fun-args=true (profile ocamlformat (file .ocamlformat:1))
type-decl=compact (profile ocamlformat (file .ocamlformat:1))
space-around-collection-expressions=false (profile ocamlformat (file .ocamlformat:1))
single-case=compact (profile ocamlformat (file .ocamlformat:1))
sequence-style=separator (profile ocamlformat (file .ocamlformat:1))
parse-docstrings=true (file .ocamlformat:4)
parens-tuple-patterns=multi-line-only (profile ocamlformat (file .ocamlformat:1))
parens-tuple=always (profile ocamlformat (file .ocamlformat:1))
parens-ite=false (profile ocamlformat (file .ocamlformat:1))
ocp-indent-compat=false (profile ocamlformat (file .ocamlformat:1))
module-item-spacing=sparse (profile ocamlformat (file .ocamlformat:1))
margin=77 (file .ocamlformat:3)
let-open=preserve (profile ocamlformat (file .ocamlformat:1))
let-binding-spacing=compact (profile ocamlformat (file .ocamlformat:1))
let-and=compact (profile ocamlformat (file .ocamlformat:1))
infix-precedence=indent (profile ocamlformat (file .ocamlformat:1))
indicate-nested-or-patterns=space (profile ocamlformat (file .ocamlformat:1))
indicate-multiline-delimiters=true (profile ocamlformat (file .ocamlformat:1))
if-then-else=compact (profile ocamlformat (file .ocamlformat:1))
field-space=tight (profile ocamlformat (file .ocamlformat:1))
extension-sugar=preserve (profile ocamlformat (file .ocamlformat:1))
escape-strings=preserve (profile ocamlformat (file .ocamlformat:1))
escape-chars=preserve (profile ocamlformat (file .ocamlformat:1))
disable=false (profile ocamlformat (file .ocamlformat:1))
cases-exp-indent=4 (profile ocamlformat (file .ocamlformat:1))
break-struct=force (profile ocamlformat (file .ocamlformat:1))
break-string-literals=wrap (profile ocamlformat (file .ocamlformat:1))
break-sequences=false (profile ocamlformat (file .ocamlformat:1))
break-separators=before (profile ocamlformat (file .ocamlformat:1))
break-infix-before-func=true (profile ocamlformat (file .ocamlformat:1))
break-infix=wrap (profile ocamlformat (file .ocamlformat:1))
break-fun-decl=wrap (profile ocamlformat (file .ocamlformat:1))
break-collection-expressions=fit-or-vertical (profile ocamlformat (file .ocamlformat:1))
break-cases=fit (file .ocamlformat:2)


If many input files are specified, only print the configuration for the first file. If no input file is specified, print the configuration for the root directory if specified, or for the current working directory otherwise.

#### Parentheses around if-then-else branches

A new option parens-ite has been added to decide whether to use parentheses around if-then-else branches that spread across multiple lines.

If this option is set, the following function:

let rec loop count a =
if count >= self#len
then a
else
let a' = f cur#get count a in
cur#incr ();
loop (count + 1) a'


will be formatted as:

let rec loop count a =
if count >= self#len
then a
else (
let a' = f cur#get count a in
cur#incr ();
loop (count + 1) a' )


#### Parentheses around tuple patterns

A new option parens-tuple-patterns has been added, that mimics parens-tuple but only applies to patterns, whereas parens-tuples only applies to expressions. parens-tuple-patterns=multi-line-only mode will try to skip parentheses for single-line tuple patterns, this is the default value. parens-tuple-patterns=always always uses parentheses around tuples patterns.

For example:

(* with parens-tuple-patterns=always *)
let (a, b) = (1, 2)

(* with parens-tuple-patterns=multi-line-only *)
let a, b = (1, 2)


#### Single-case pattern-matching expressions

The new option single-case defines the style of pattern-matching expressions with only a single case. single-case=compact will try to format a single case on a single line, this is the default value. single-case=sparse will always break the line before a single case.

For example:

(* with single-case=compact *)
try some_irrelevant_expression
with Undefined_recursive_module _ -> true

(* with single-case=sparse *)
try some_irrelevant_expression
with
| Undefined_recursive_module _ -> true


#### Space around collection expressions

The new option space-around-collection-expressions decides whether to add a space inside the delimiters of collection expressions (lists, arrays, records).

For example:

(* by default *)
type wkind = {f : 'a. 'a tag -> 'a kind}
let l = ["Nil", TCnoarg Thd; "Cons", TCarg (Ttl Thd, tcons)]

(* with space-around-collection-expressions *)
type wkind = { f : 'a. 'a tag -> 'a kind }
let l = [ "Nil", TCnoarg Thd; "Cons", TCarg (Ttl Thd, tcons) ]


#### Break separators

The new option break-separators decides whether to break before or after separators such as ; in list or record expressions, * in tuples or -> in arrow types. break-separators=before breaks the expressions before the separator, this is the default value. break-separators=after breaks the expressions after the separator. break-separators=after-and-docked breaks the expressions after the separator and docks the brackets for records.

For example:

(* with break-separators=before *)
type t =
{ foooooooooooooooooooooooo: foooooooooooooooooooooooooooooooooooooooo
; fooooooooooooooooooooooooooooo: fooooooooooooooooooooooooooo }

(* with break-separators=after *)
type t =
{ foooooooooooooooooooooooo: foooooooooooooooooooooooooooooooooooooooo;
fooooooooooooooooooooooooooooo: fooooooooooooooooooooooooooo }

(* with break-separators=after-and-docked *)
type t = {
foooooooooooooooooooooooo: foooooooooooooooooooooooooooooooooooooooo;
fooooooooooooooooooooooooooooo: fooooooooooooooooooooooooooo
}


#### Not breaking before bind/map operators

The new option break-infix-before-func decides whether to break infix operators whose right arguments are anonymous functions specially. This option is set by default, if you disable it with --no-break-infix-before-func, it will not break before the operator so that the first line of the function appears docked at the end of line after the operator.

For example:

(* by default *)
f x
>>= fun y ->
g y
>>= fun () ->
f x >>= fun y -> g y >>= fun () -> f x >>= fun y -> g y >>= fun () -> y ()

(* with break-infix-before-func = false *)
f x >>= fun y ->
g y >>= fun () ->
f x >>= fun y -> g y >>= fun () -> f x >>= fun y -> g y >>= fun () -> y ()


#### Break toplevel cases

There is a new value for the break-cases option: toplevel, that forces top-level cases (i.e. not nested or-patterns) to break across lines, otherwise breaks naturally at the margin.

For example:

let f =
let g = function
| H when x y <> k -> 2
| T | P | U -> 3
in
fun x g t h y u ->
match x with
| E -> 4
| Z | P | M -> (
match y with
| O -> 5
| P when h x -> (
function
| A -> 6 ) )


#### Number of spaces before docstrings

The new option doc-comments-padding controls how many spaces are printed before doc comments in type declarations. The default value is 2.

For example:

(* with doc-comments-padding = 2 *)
type t = {a: int  (** a *); b: int  (** b *)}

type t = {a: int (** a *); b: int (** b *)}


#### Ignore files

An .ocamlformat-ignore file specifies files that OCamlFormat should ignore. Each line in an .ocamlformat-ignore file specifies a filename relative to the directory containing the .ocamlformat-ignore file. Lines starting with # are ignored and can be used as comments.

Here is an example of such .ocamlformat-ignore file:

#This is a comment
dir2/ignore_1.ml


#### Tag-only docstrings

The new option doc-comments-tag-only controls the position of doc comments only containing tags. doc-comments-tag-only=default means no special treatment is done, this is the default value. doc-comments-tag-only=fit puts doc comments on the same line if it fits.

For example:

(* with doc-comments-tag-only = default *)

(** @deprecated  *)
open Module

(* with doc-comments-tag-only = fit *)

open Module (** @deprecated  *)


#### Fit or vertical mode for if-then-else

There is a new value for the option if-then-else: fit-or-vertical. fit-or-vertical vertically breaks all branches if they do not fit on a single line. Compared to the compact (default) value, it breaks all branches if at least one of them does not fit on a single line.

For example:

(* with if-then-else = compact *)
let _ =
if foo then
let a = 1 in
let b = 2 in
a + b
else if foo then 12
else 0

(* with if-then-else = fit-or-vertical *)
let _ =
if foo then
let a = 1 in
let b = 2 in
a + b
else if foo then
12
else
0


#### Check mode

A new --check flag has been added. It checks whether the input files already are formatted. This flag is mutually exclusive with --inplace and --output. It returns 0 if the input files are indeed already formatted, or 1 otherwise.

#### Break function declarations

The new option break-fun-decl controls the style for function declarations and types. break-fun-decl=wrap breaks only if necessary, this is the default value. break-fun-decl=fit-or-vertical vertically breaks arguments if they do not fit on a single line. break-fun-decl=smart is like fit-or-vertical but try to fit arguments on their line if they fit. The wrap-fun-args option now only controls the style for function calls, and no more for function declarations.

For example:

(* with break-fun-decl = wrap *)
let ffffffffffffffffffff aaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb
cccccccccccccccccccccc =
g

(* with break-fun-decl = fit-or-vertical *)
let ffffffffffffffffffff
aaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbb
cccccccccccccccccccccc =
g

(* with break-fun-decl = smart *)
let ffffffffffffffffffff
aaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb cccccccccccccccccccccc =
g


#### Disable configuration in files and attributes

Two new options have been added so that .ocamlformat configuration files and attributes in OCaml files do not change the configuration. These options can be useful if you use some preset profile and you do not want attributes and .ocamlformat files to interfere with your preset configuration. --disable-conf-attrs disables the configuration in attributes, and --disable-conf-files disables .ocamlformat configuration files.

#### Preserve module items spacing

There is a new value for the option module-item-spacing: preserve, that will not leave open lines between one-liners of similar sorts unless there is an open line in the input.

For example the line breaks are preserved in the following code:

let cmos_rtc_seconds = 0x00
let cmos_rtc_seconds_alarm = 0x01
let cmos_rtc_minutes = 0x02

let x = o

let log_other = 0x000001
let log_cpu = 0x000002
let log_fpu = 0x000004


#### Breaking changes

• When --disable-outside-detected-project is set, disable ocamlformat when no .ocamlformat file is found.
• Files are not parsed when ocamlformat is disabled.
• Disallow - with other input files.
• The wrap-fun-args option now only controls the style for function calls, and no more for function declarations.
• The default profile is now named ocamlformat.
• The deprecated syntax for .ocamlformat files: option value is no more supported anymore and you should use the option = value syntax instead.

#### Miscellaneous bugfixes

• Preserve shebang (e.g. #!/usr/bin/env ocaml) at the beginning of a file.
• Improve the formatting when ocp-indent-compat is set.
• UTF8 characters are now correctly printed in comments.
• Add parentheses around a constrained any-pattern (e.g. let (_ : int) = x1).
• Emacs: the temporary buffer is now killed.
• Lots of improvements on the comments, docstrings, attributes formatting.
• Lots of improvements on the formatting of modules.
• Lots of improvements in the Reason support.
• Do not rely on the file-system to format sources.
• The --debug mode is more user-friendly.

#### Credits

This release also contains many other changes and bug fixes that we cannot detail here.

Special thanks to our maintainers and contributors for this release: Jules Aguillon, Mathieu Barbin, Josh Berdine, Jérémie Dimino, Hugo Heuzard, Ludwig Pacifici, Guillaume Petiot, Nathan Rebours and Louis Roché.

If you wish to get involved with OCamlFormat development or file an issue, please read the contributing guide, any contribution is welcomed.

## Other OCaml News

### From the ocamlcore planet blog

Here are links from many OCaml blogs aggregated at OCaml Planet.

## Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.