OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of August 24 to 31, 2021.

Table of Contents

First beta release of OCaml 4.13.0

octachron announced

The release of OCaml 4.13.0 is getting closer.

The set of new features has been stabilized, and core opam packages already work with this release (see http://check.ocamllabs.io). After two alpha releases, we have created a first beta version to help you update your softwares and libraries ahead of the release.

If you find any bugs, please report them here:

https://github.com/ocaml/ocaml/issues

The full release of OCaml 4.13.0 is currently planned for September.

Happy hacking, Florian Angeletti for the OCaml team.

Installation instructions

The base compiler can be installed as an opam switch with the following commands

opam update
opam switch create 4.13.0~beta1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

If you want to tweak the configuration of the compiler, you can switch to the option variant with:

opam update
opam switch create <switch_name> --packages=ocaml-variants.4.13.0~beta1+options,<option_list>
--repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

where <option_list> is a comma separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch:

opam switch create 4.13.0~beta1+flambda+nffa
--packages=ocaml-variants.4.13.0~beta1+options,ocaml-option-flambda,ocaml-option-no-flat-float-array
--repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git

All available options can be listed with opam search ocaml-option.

The source code for the beta release is also available at these addresses:

Changes since the last alpha release

  • 10549: Stack overflow detection and naked pointers checking for ARM64 (Xavier Leroy, review by Stephen Dolan)
  • 10442, 10446: Fix regression in the toplevel to #directory caused by corrections and improvements to the Load_path in 9611. #directory now adds the path to the start of the load path again (so files in the newly added directory take priority). (David Allsopp, report by Vasile Rotaru, review by Florian Angeletti and Nicolás Ojeda Bär)
  • 10450, 10558: keep %apply and %revapply primitives working with abstract types. This breach of backward compatibility was only present in the alpha releases of OCaml 4.13.0 . (Florian Angeletti, review by Thomas Refis and Leo White)
  • 10380: Correct handling of UTF-8 paths in configure on Windows (David Allsopp, review by Sébastien Hinderer)
  • 10543: Fix Ast_mapper to apply the mapping function to the constants in "interval" patterns c1..c2. (Guillaume Petiot, review by Gabriel Scherer and Nicolás Ojeda Bär)
  • 10550, 10551: fix pretty-print of gadt-pattern-with-type-vars (Chet Murthy, review by Gabriel Scherer)
  • 10497: Styling changes in the post-processed HTML manual (webman) (Wiktor Kuchta, review by Florian Angeletti)

Windows-capable Diskuv OCaml 0.1.0

jbeckford announced

I am pleased to announce the first preview release of the Diskuv OCaml ("DKML") distribution. Parts of the diskuv / diskuv-ocaml README.md are reproduced below:

Diskuv OCaml is an OCaml distribution focused on a) secure, cross-platform software development and b) ease of use for language learners and professional developers.

Documentation is available at https://diskuv.gitlab.io/diskuv-ocaml/

The preview versions 0.1.x run on 64-bit Windows and:

  1. Includes an installer for the initial multi-hour Windows compilation process, including the installation of Git and Visual Studio Build Tools if needed: Intro-install-world.png
  2. Includes a UNIX-compatible runtime environment for building OCaml applications with common tools like `make`, `opam` and `dune`:

    [PS Z:\source\diskuv-ocaml-starter] cd ~/DiskuvOCamlProjects/diskuv-ocaml-starter
    [PS Z:\source\diskuv-ocaml-starter] ./make build-dev
    [PS Z:\source\diskuv-ocaml-starter] _build/default/bin/main.exe
    > 1
    > 2
    > 3
    > 94.5
    > Total: 100.5
    [PS Z:\source\diskuv-ocaml-starter] ./make shell-dev
    
    [diskuv-ocaml-starter]$ echo You are now running a UNIX shell.
    > You are now running a UNIX shell.
    [diskuv-ocaml-starter]$ opam switch --short
    > C:\Users\you\AppData\Local\Programs\DiskuvOCaml\1\system
    > Z:\source\diskuv-ocaml-starter\build\dev\Debug
    > Z:\source\diskuv-ocaml-starter\build\dev\Release
    > diskuv-boot-DO-NOT-DELETE
    [diskuv-ocaml-starter]$ dune utop
    > ──────────┬─────────────────────────────────────────────────────────────┬──────────
    >           │ Welcome to utop version 2.8.0 (using OCaml version 4.12.0)! │
    >           └─────────────────────────────────────────────────────────────┘
    >
    > Type #utop_help for help about using utop.
    >
    > ─( 06:26:11 )─< command 0 >─────────────────────────────────────────{ counter: 0 }─
    
    utop #> let square x = x * x ;;
    > val square : int -> int = <fun>
    utop #> square 2 ;;
    > - : int = 4
    utop #> square (square 2) ;;
    > - : int = 16
    utop #> #quit ;;
    
  3. Works with the OCaml recommended Visual Studio Code plugin: diskuv-ocaml-starter.vscode-screenshot.png

Please visit our documentation at https://diskuv.gitlab.io/diskuv-ocaml/

https://img.shields.io/twitter/follow/diskuv?style=social

Acknowledgements

The Diskuv OCaml distribution would not be possible without many people's efforts!

In alphabetical order some of the critical pieces were provided by:

  • Andreas Hauptmann (fdopen@) - Maintained the defacto Windows ports of OCaml for who knows how long
  • INRIA for creating and maintaining OCaml
  • Jane Street and the contributors to dune
  • OCaml Labs and the contributors for the Visual Studio Code extension for OCaml
  • OCamlPro, Jane Street and the contributors to opam
  • Yaron Minsky, Anil Madhavapeddy and Jason Hickey for the book "Real World OCaml"

For OCaml package maintainers two things in particular may be of interest:

  • Instructions for VirtualBox for how to run Windows virtual machines (legally and free!) on Linux and macOS: Windows 10 on macOS/Linux with VirtualBox
  • Using core_kernel.v0.14.2 as a prototypical example, there is a detailed walkthrough and resolution for the common problems you may experience getting your package to build on a native Windows toolchain: MSVC + MSYS2 toolchain

Feedback is appreciated! I'll post critical bug fixes and important documentation updates in the Windows-capable Diskuv OCaml 0.1.0 topic thread until the next release (0.1.1 or 0.2.0).

v3.OCaml.org: A roadmap for OCaml's online presence

Anil Madhavapeddy announced

The current ("v2") ocaml.org website was written a decade ago (presentation) and – while it has served us well – its age is showing as our community has grown. To put things in context, the opam package manager didn't actually exist when we built the current iteration of the website.

TL;DR: preview the new package search at https://v3.ocaml.org/packages

In November 2020, the current maintainers put our heads together to rebuild the website to modern standards ("v3"). When it came to determining the goals of the website refresh, we were driven by the need to integrate the technological developments of the last decade – for example, the adoption of opam as the package management system (evidenced by close to 20,000 packages in the opam repository), and the emergence of common tools and workflows that benefit both individual developers as well as larger-scale industrial codebases.

In October 2020, the core OCaml team also issued a user survey to solicit information from our user community about priorities for the OCaml ecosystem. Here are the original results, original discuss thread, concrete suggestions and some analysis. Where appropriate, we reference the user survey results in the information below to try to ensure our decisions are as inclusive as possible.

v3.ocaml.org features

The v3 version of the website is a rewrite from scratch. This is mostly because the web technologies used in the 2011 version are outdated, and it is difficult to forward port them to a standard that meets modern accessibility requirements. Our own OCaml tooling ecosystem has also moved on tremendously in the last decade, so it was time for a fresh start. The new site features:

Integrated documentation and package management

When v2 was developed, the opam package manager didn't exist yet and so we ended up with a fragmented ecosystem (opam.ocaml.org is separate and distinct from ocaml.org, and other package managers like GODI also had separate websites). Since then, opam metadata has become the standard in our ecosystem with over 20000 packages published.

The v3 site combines the existing package management subdomain with a new central documentation source (formerly codenamed docs.ocaml.org) for all versions of all opam packages. This content is held directly within the new ocaml.org site and not a set of subdomains, reflecting a clear community direction. Our goal is for the live website to contain published package API docs within 30 minutes of the package being merged.

Separation of data editing from HTML/CSS generation

The v2 site combined (e.g.) fetching external newsfeeds with the HTML generation, which made it expensive to scale with new content as every build would take longer and longer.

The v3 site keeps a structured store of data as OCaml values, which is then used to generate the frontend website. We use ReScript/OCaml to generate the site content, and Ocurrent to automate the data pipelines. The data source repository is known as ood (OCaml.Org Data). Finally, the static HTML is combined with the dynamic API documentation generation and served via a Dream-based webserver.

For users wishing to submit content fixes to the website, they need to do nothing more than clone ocaml/ood and submit the fix there, without a full website build.

Publishing original content

The v2 site syndicated from external sources, which worked well when the web (circa 2011) was centred around RSS/Atom feeds and blogging engines. The modern web (circa 2021) has moved towards social networking chatter, which greatly reduces the value of syndicating content. We also now have multiple discussion forums such as discuss.ocaml.org where original content is frequently posted.

There will there be original content on the v3 site, so that we can publish editorial content from around the community directly on ocaml.org. Original content can include interviews, featured news stories, informative forum posts, online talks and other media from past conferences, along with many other types of content.

To facilitate this without tying ourselves to a third-party website, we have deployed an instance of the open-source Peertube video hosting application, over at watch.ocaml.org. This provides an open API (free of adverts and third-party cookies) to view video material about OCaml through its rich history. The OCaml Workshop 2021 videos are being published on watch.ocaml.org as their primary source: https://watch.ocaml.org/video-channels/ocaml2021/videos

Accessible, sustainable and respectful of privacy

The v2 site grew organically over a decade, and so a multiplicity of ways to navigate it emerged. The v3 site design takes into account modern web design principles, restructuring the old content in accordance with methods that present it more compellingly. It is a total redesign that modernises the look and feel of the site to work well on mobile and other responsive targets, as well as being easier to navigate and more accessible.

In terms of technology, the frontend switches to Tailwind CSS and uses React/Rescript/NextJS to generate most of the site HTML. The backend data sources all remain in pure OCaml code, meaning that we retain the ability to migrate to new frontend technologies without a full rewrite in the future.

The v3 website has a simple privacy policy: there are no third-party cookies or tracking systems in place beyond server-side log analysis. This allows us to be compliant with GDPR rules, but also be accessible to the widest global audience.

In reflection of our strong committment to sustainability, we are also working on a "carbon policy" that will track and publish our energy usage across the full ocaml.org infrastructure (including the build cluster). This adds up to around ~1000 CPU cores of different architectures hosted around the world in various data centres, and we are aiming to be carbon neutral via the use of renewable energy and similar initiatives.

A quick tour of some nice bits of v3.ocaml.org

(This is accurate as of 4th Aug 2021)

  • The Industrial Users page is driven from the ood data here.
  • The Academic Excellence page has a world map of users as well and is driven from ood data here.
  • The opam package search for 'vg' show a docs build (there are some broken images in the docs, which is a known issue). Try clicking around the different versions of the package as well, or searching for your own!
  • The v3 tutorials are all checked with mdx to make sure that they are consistent with the latest version of OCaml.
  • The events page shows past OCaml Workshops. Clicking on the 2020 one shows videos embedded from watch.ocaml.org and served directly from our new site instead of a third-party provider.

There are still lots of unfinished pages (most notably the front page is totally placeholder until we finish more of the leaf pages), but this should hopefully give you a good view on where the site is going!

2020 Survey Actions

The v3 website was partly driven by the feedback from our community as well as the core team's discretion. We will now go through the feedback items.

Individual quotes from different answers
Pain point when Learning the OCaml Language
  • "Lack of search engine friendly content. Other mainstream languages have plenty of short tutorials available on the web": The few OCaml does have are not discoverable easily": On the new site we're making the core tutorials easier to find. Community members are also working on new tutorial systems for beginners, and the website can link to these as they are published.
  • "Up to date docs and learning materials and books": The Language page will help users find this information in the future.
  • "Poor documentation" and "Most libraries have very sparse documentation": There are many, many comments like this one, and while we cannot solve this directly, having integrated and up-to-date API docs directly in OCaml.org should incentivise people to do a better job of adding ocamldoc comments to their code.
I am satisfied with OCaml's package repositories, what would you change
  • "Create a opam search repo". This now exists, via v3.ocaml.org/packages and the search bar.
From the discuss thread
  • Yawaramin wrote: "Or Outreachy internships or other funded dev work at less than senior level? There are libraries out there in other languages that would be tremendously beneficial to port over to OCaml…". Action: the OCaml Software Foundation and OCaml Labs have now run multiple Outreachy programs, with the most recent working on v3.ocaml.org itself. We need community help to add more mentors to scale this program – please volunteer!
  • Anil's comment touches on all aspects (ocaml.org, mdx (the tutorials now use), 'Our new CI will also feature macOS and Windows' … getting there). Action: the v3 infrastructure uses all the best-of-breed tools the community has been developing in recent years. Tutorials are checked in CI via mdx, for example, so that they won't bitrot with a new release of OCaml that changes outputs.
Items nearly achieved
  • 42 people (6.2%) when answering "If one piece of the ecosystem could magically be made state-of-the-art, I would ask for" answered with a language website, 121 (18%) answered documentation for user libraries and 73 (10.8%) communication to outsiders. - In question no 20, the preferred interaction methods differed for each group (beginner, intermediate, advanced, expert). In order to address this, we have integrated links to all the main interaction methods on "Around The Web". "Around the Web" also serves as a common interaction method for all groups.
Works in progress
  • When asked "What do you think are the main pain points that prevent OCaml adoption for new projects?" 310 (50.4%) said Too hard to hire and find developers and 181 (29.4%) said Too hard to learn – v3 incorporates an opportunities section and sets up a much more comprehensive environment for writing and expanding tutorials and documentation.
  • When asked if OCaml best practises are understood roughly 40% said they were neutral, disagreed or strongly disagreed! Action: There's been a phenomenal amount of work here including @cdaringe's discuss post and @tmattio's WIP workflows PR to ood.
  • When asking whether or not people could easily find OCaml jobs, just shy of 50% responded with Neutral, Disagree, Strongly Disagree and I don't know. Action: We have a jobs category on the discussion forum, which will be syndicated to the Opportunities page on the new website.
Long-term goals
  • OFronds + OCaml Playground/Per package toplevel – making it easier and easier for beginners to get started in OCaml. We have a proof-of-concept of a toplevel built using JSOO, which we are aiming to integrate for every package that supports such a build.

Next Steps in 2021

Fall 2021

The v3 website is going into "public alpha" in August. This means that several pages aren't yet complete at all, integration is continuing, and the design will be refined over the coming months. The overall structure and sitemap is ready, and the technology choices have all been ratified and deployed.

The best way to give feedback would be to start a thread on discuss.ocaml.org. Bear in mind that the site is in an early stage, so be constructive and kind in your criticism (but also do not hold back!). If you feel that private feedback is more appropriate, then feel free to contact me directly on anil@recoil.org and I will forward your feedback to the team.

Winter 2021

As community feedback settles and the features are all implemented, the site will move into public beta. At this point, we will be working on refining the tail of bugs and preparing to swap websites.

Going live

When going live, the switch will happen as follows:

  • the existing ocaml.org site will be snapshotted as static HTML and deployed on v2.ocaml.org with a header marking the site as "deprecated"
  • we generate a sitemap of static HTML on v2.ocaml.org and, where the link doesn't resolve on v3.ocaml.org, create a redirect to v2.ocaml.org. This will prevent dead links from existing content.
  • v3.ocaml.org is aliased to ocaml.org via a DNS swap.
  • opam.ocaml.org will continue to be operated indefinitely, since existing clients use that as the package mirror. The web content will gradually migrate to ocaml.org links in the course of 2022.
How can you contribute to the v3 effort?

There is an awful lot left to be done to get this site live, as always!

  • Frontend developer: The ReScript/NextJS-based frontend issue tracker has a maintained set of issues about what pages are remaining. There is also always general polish to be done as many of the pages are in their first design iteration. See the repository CONTRIBUTING.md file for more information on how to build the site.
  • OCaml developer: The ocaml/ood repository is where all the data that drives the site is stored. There are schemas there for videos, talks, events, workshops, papers, releases and so on. There's a lot of data from the past 25 years to catch up on there, so please do feel free to create an issue on the tracker that you are working on something and start adding it. The OCaml proficiency level required here is fairly simple.
  • OCaml tooling hacker: If you are familiar with the OCaml Platform tools like opam and dune, then you may want to look at the dynamic v3.ocaml.org-server that drives the site and does the documentation rendering. The backend infrastructure is fairly complex, but all of the built artefacts are available over HTTP and the server retrieves them on startup, so you should be able to develop from the comfort of your own laptop.
  • Language translator: We have got the infrastructure in place to handle translated content. If you would be willing to help maintain a translation to a different language, we'd love to hear from you as the site content settles.

Team

The immediate team (as of Aug 2021) working on the v3 site consists of:

  • Ashish Agarwal (Solvuu)
  • Kanishka Azimi (Solvuu)
  • Richard Davison (Solvuu)
  • Patrick Ferris (OCaml Labs)
  • Gemma Gordon (OCaml Labs)
  • Isabella Leandersson (OCaml Labs)
  • Anil Madhavapeddy (University of Cambridge)
  • Thibault Mattio (Tarides)
  • Tim McGilchrist (Tarides)

The documentation generation is courtesy of the odoc development team, with the site generation done by:

  • Jon Ludlam (OCaml Labs)
  • Jules Aguillon (Tarides)
  • Lucas Pluvinage (Tarides)
  • Thibault Mattio (Tarides)

We had three wonderful Outreachy interns help us this year:

  • Diksha Gupta is working on automating and expanding v3.ocaml.org's new and experimental peertube instance for hosting OCaml-related video content. This required adding functionality into ood to track the videos that have been published.
  • Odinaka Joy is prototyping and building an online package service accessible through a graphQL endpoint in v3.ocaml.org-server. During the internship Joy has also prototyped js_of_ocaml web applications for consuming the data from the endpoint.
  • Shreya Kumari Gupta has been a driving force in porting more content, pages and designs into v3 including adding all of the academic institutions that use OCaml into ood.

The site of course depends on a very large number of community published libraries in opam, as well as the Rescript compiler. There's a lot of work left to do to improve the site and we warmly welcome more volunteers to assist us. If you'd like to join the ocaml.org website team, get in touch with Ashish Agarwal or Anil Madhavapeddy and we'd be pleased to help you onboard. You can also look at the various issue trackers for labelled issues on tasks that need to be completed.

Thank you to Xavier Leroy, Gabriel Scherer, David Allsopp, Thomas Gazagnaire, Frederic Bour, Florian Angeletti, Ashish Agarwal and many others for feedback on this post, and for Patrick Ferris and Bella Leandersson for helping to write it.

Patrick Ferris said

As a part of the team that started thinking about v3.ocaml.org back in Autumn 2020, it is incredibly rewarding and encouraging to see such a positive response and constructive feedback to the new, work-in-progress version of the site. A lot of deliberation and effort has gone into what we currently have and now that is publicly available, hopefully will gain community support and momentum to improve it further. I thought I'd answer some of the questions people have here.

Also, there is a bit of an overlap in content with https://ocamlverse.github.io/ for some things (eg best practices, community) but the (to me) most valuable feature is missing: The ecosystems overview, where I can find packages sorted thematically. Could such a section also have a place in the packages subpage somewhere? Alternatively, maybe opam can allow to “tag” packages in the future so one could see all packages for graphics, databases etc.

Indeed. Much like the OCaml compiler itself, the "main" ocaml.org website has a commitment to backwards compatibility as well as new content and features. What I mean by this is we are trying to incorporate everything in https://github.com/ocaml/ocaml.org as well as creating new content/features. This inevitably makes things a little slower, but best practices are in the works.

Afaik opam already has tags they are just not used that widely or in a standardised way for indicating things like graphics, databases etc. (in fact the server reads them). If some consensus was reached on this it should be a fairly straightforward and useful feature to add. In the meantime we could certainly add curated lists of packages to ood :)).

as someone who hasn’t used, but has been considering jsoo vs rescript for the FE of a project, I’m curious why ReScript was chosen over JSOO for the website FE?

When making the technology choices, the main reasoning was "use the right tool for the job". In my opinion (as an avid jsoo user) for what we wanted jsoo makes little sense. The Rescript frontend is essentially bindings to Nextjs (plus any other libraries from the JS universe that are needed). Rescript has good JSX support, integrates well with the npm world, has (arguably…) better FFI support with JS etc. When we need jsoo (e.g. the experimental plan to have integrated toplevels for packages) then we'll reach for it.

That's not to say you can't use jsoo for front-end work, for example there are bindings to reactjs and FRP libraries like brr + note. Another very flexible way for building static sites with a sprinkling of JS when needed is using OCaml for HTML generation and alpine.js for bits of javascript (e.g. the preview app in ood).

Importantly, now that the data lives very much separately we're also not tied to any particular front-end. In fact this allows us to share all of the data in ocaml.org (I think there's more than people realise, that was certainly the case for me!) for example I wrote an in-complete "terminal rendering" of the data. Ood could equally be reused perhaps directly in vscode!

Software engineer at Wolfram MathCore

Leonardo Laguna Ruiz announced

there is a position at the company that I work, Wolfram MathCore. You can find the full description here: https://www.wolframmathcore.com/2021/08/23/wolfram-mathcore-job-posting-senior-software-engineer/

For this position we use OCaml as the main programming language.

If you have any questions you can contact Jan Brugård at janb@wolfram.com or send me a message here in discuss.

memprof-limits (first official release): Memory limits, allocation limits, and thread cancellation, with interrupt-safe resources

Guillaume Munch-Maccagnoni announced

My talk at the OCaml workshop about memprof-limits (and reasoning about exceptions in general) is now available online:

https://watch.ocaml.org/videos/watch/bc297e85-82dd-4baf-8556-4a3a934978f9

OCaml Workshop 2021: Live Stream

Frédéric Bour announced

Goodbye Core_kernel

Jérémie Dimino announced

Hi, just a heads up that the Core_kernel library from Jane Street will eventually disappear, leaving place to only Base and Core. More details in this blog post on our website:

https://blog.janestreet.com/goodbye-Core_kernel/

This change is good but is not backward compatible. We have a few ideas to make the transition smoother in opam, but we haven't settled on a concrete plan yet.

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.