Concurrent System Programming with Effect
Handlers
Stephen Dolan
1
, Spiros Eliopoulos
3
, Daniel Hillerström
2
, Anil Madhavapeddy
1
,
KC Sivaramakrishnan
1
(
), and Leo White
3
1
University of Cambridge
2
The University of Edinburgh
3
Jane Street Group
Abstract. Algebraic effects and their handlers have been steadily gain-
ing attention as a programming language feature for composably express-
ing user-defined computational effects. While several prototype imple-
mentations of languages incorporating algebraic effects exist, Multicore
OCaml incorporates effect handlers as the primary means of expressing
concurrency in the language. In this paper, we make the observation that
effect handlers can elegantly express particularly difficult programs that
combine system programming and concurrency without compromising
performance. Our experimental results on a highly concurrent and scal-
able web server demonstrate that effect handlers perform on par with
highly optimised monadic concurrency libraries, while retaining the sim-
plicity of direct-style code.
1 Introduction
Algebraic effect handlers are a modular foundation for effectful program-
ming, which separate the operations available to effectful programs from their
concrete implementations as handlers. Effect handlers provide a modular alter-
native to monads [25, 35] for structuring effectful computations. They achieve the
separation between operations and their handlers through the use of delimited
continuations, allowing them to pause, resume and switch between different com-
putations. They provide a structured interface for programming with delimited
continuations [10], and can implement common abstractions such as state, gen-
erators, async/await, promises, non-determinism, exception handlers and back-
tracking search. Though originally studied in a theoretical setting [27, 28], ef-
fect handlers have gained practical interest with several prototype implemen-
tations in the form of libraries, interpreters, compilers and runtime representa-
tions [4, 5, 9, 12, 15, 16, 20, 21].
However, the application space of effect handlers remains largely unexplored.
In this paper we explore the applicability of effect handlers to concurrent system
sk826@cl.cam.ac.uk
programming in Multicore OCaml. While Multicore OCaml supports shared-
memory parallel programming, this paper restricts its focus to concurrency i.e.
overlapped execution of tasks, leaving parallelism outside our scope.
2 Motivation
Multicore OCaml [8] incorporates effect handlers as the primary means of ex-
pressing concurrency in the language. The modular nature of effect handlers al-
lows the concurrent program to abstract over different scheduling strategies [9].
Moreover, effect handlers allow concurrent programs to be written in direct-
style retaining the simplicity of sequential code as opposed to callback-oriented
style (as used by e.g. Lwt [34] and Async [24]). In addition to being more read-
able, direct-style code tends to be easier to debug; unlike callback-oriented code,
direct-style code uses the stack for function calls, and hence, backtraces can be
obtained for debugging. Indeed, experience from Google suggests that as well as
making the code more compact and easier to understand (particularly important
when thousands of developers touch the code), direct-style code can perform as
well or better than callback-oriented code [3].
Some of the benefits of direct-style code can be achieved by rewriting direct-
style functions into callbacks, using syntactic sugar such as Haskell’s do-notation
for monads or F#’s async/await [32]. However, this separates functions which use
such rewriting from those that do not, leading to awkward mismatches and code
duplication: for instance, Haskell provides mapM, filterM and foldM because the
ordinary map, filter and foldl functions do not work with monadic arguments.
By contrast, effect handlers do not introduce an incompatible type of function.
In Multicore OCaml, the user-level thread schedulers themselves are ex-
pressed as OCaml libraries, thus minimising the secret sauce that gets baked
into high-performance multicore runtime systems [31]. This modular design al-
lows the scheduling policy to be changed by swapping out the scheduler library
for a different one with the same interface. As the scheduler is a library, it can live
outside the compiler distribution and be tailored to application requirements.
However, the interaction between user-level threading systems and the oper-
ating system services is difficult. For example, the Unix write() system call may
block if the underlying buffer is full. This would be fine in a sequential program
or a program with each user-level thread mapped to a unique OS thread, but
with many user-level threads multiplexed over a single OS thread, a blocking
system call blocks the entire program. How then can we safely allow interaction
between user-level threads and system services?
Concurrent Haskell [23], which also has user-level threads, solves the prob-
lem with the help of specialised runtime system features such as safe FFI calls
and bound threads. However, implementing these features in the runtime system
warrants that the scheduler itself be part of the runtime system, which is incom-
patible with our goal of writing thread schedulers in OCaml. Attempts to lift the
scheduler from the runtime system to a library in the high-level language while
retaining other features in the runtime system lead to further complications [31].
Our goals then are:
– Retain the simplicity of direct-style code for concurrent OCaml programs.
– Allow user-level thread schedulers to be written in OCaml as libraries.
– Allow safe interaction between user-level threads and the operating system.
– Perform as well as or better than existing solutions.
We observe that algebraic effects and their handlers can meet all of these
goals. In particular, we introduce asynchronous effects and their handlers, and
show how they elegantly solve the interaction between user-level threads and
operating system services. This paper makes the following contributions:
– We introduce effect handlers for Multicore OCaml and illustrate their utility
by constructing a high-performance asynchronous I/O library that exposes
a direct style API (Section 3).
– We show how asynchronous effects provide a clean interface to difficult-to-
use operating system services, such as signal handling and asynchronous
notification of I/O completion, and demonstrate how effect handlers enable
scoped interrupt handling (Section 4).
– We evaluate the performance of effect handlers in OCaml by implementing
a highly scalable web server and show that Multicore OCaml effect handlers
are efficient (Section 5).
After the technical content of the paper in Sections 3, 4, and 5, we discuss
related work in Section 6 and our conclusions in Section 7.
3 Algebraic effects and their handlers
Since the primary motivation for adding effect handlers in Multicore OCaml is
concurrency, we introduce effect handlers in constructing an asynchronous I/O
library which retains the simplicity of direct-style programming
4
.
3.1 Concurrency
We start with an abstraction for creating asynchronous tasks and waiting on
their results. We use the term fiber to indicate a lightweight user-level thread to
distinguish it from kernel threads.
val async : (α → β) → α → β promise
(* [ async f v] spawns a fiber to run [f v] asyn c h ronou s l y . *)
val await : α promise → α
(* Block until the result of a promise is available . Raise s
exception [e ] if the promise raises [ e ]. *)
val yield : unit → unit
(* Yield control to other fibers . *)
4
A comprehensive list of example programs written using effect handlers in Multicore
OCaml is available at https://github.com/kayceesrk/effects-examples
Multicore OCaml extends OCaml with the ability to declare user-defined
effects with the help of the effect keyword. Since async, await and yield are
effectful operations, we declare them as follows:
ef fect Async : (α → β) * α → β p r omise
ef fect Await : α promise → α
ef fect Yield : unit
The first declaration says that Async is an effect which is parameterised by a
pair of a thunk and a value, and returns a promise as a result when performed.
Await is parameterised by a promise and returns the result. Yield is a nullary
effect that returns a unit value. To be precise, these declarations are operations of
a single built-in effect type α eff in Multicore OCaml. Indeed, these declarations
are syntactic sugar for extending the built-in extensible variant type α eff:
type _ eff +=
| Async : (α → β ) * α → β pro m ise eff
| Await : α promise → α eff
| Yield : unit eff
Effects are performed with the perform : α eff → α primitive, which per-
forms the effect and returns the result. We can now define the functions async,
await and yield as:
let async f v = p erform ( Async (f ,v ))
let await p = perform ( Await p)
let yield () = perform Yield
These effects are interpreted by an effect handler, as shown in Figure 1.
A promise (lines 1–6) is either completed successfully Done v, failed with an
exception Error e or still pending Waiting l, with a list of fibers waiting on it
for completion. The function run (line 8) is the top-level function that runs the
main concurrent program. run_q is the queue of concurrent fibers ready to run.
The effect handler itself is defined in the lines 17–38. An effect handler comprises
of five clauses – a value clause, an exception clause, and three clauses that handle
the effects Async, Await and Yield.
Effect clauses are of the form effect e k where e is the effect and k is the
continuation of the corresponding perform delimited by this handler. k is of type
(α , β) continuation, representing a continuation waiting for a value of type
α and returning a value of type β when resumed. There are two primitives
operating on continuations: continue k x resumes the continuation k where it
left off, returning the value x from perform, while discontinue k exn resumes the
continuation k by raising the exception exn from perform.
In the case of an Async (f,v) effect (lines 28–31), we create a new promise
value p which is initially waiting to be completed. We set up the original fibers,
represented by continuation k, to resume with the promise using the continue
primitive. Finally, we recursively call fork to run the new fiber f v. Since Mul-
ticore OCaml uses so-called deep handlers, the continuation k references its sur-
rounding handler, and so we need not write another match expression when
continue-ing k (See Kammar et al. [15] for more on deep vs. shallow handlers).
1 type α _promise = Done of α | Error of exn
2 | Waiti n g of (α, unit ) cont i n uation list
3
4 type α prom i se = α _p r o m i s e ref
5
6 let run main v =
7 let run_q = Queue . create () in
8 let enque u e f = Queue . push f run_q in
9 let run_next () =
10 if Queue . is_empty run_q then ()
11 else Queue . pop run_q ()
12 in
13 let rec fork : α β. α promise → (β → α) → β → unit =
14 fun p f v →
15 match f v with
16 | v →
17 let Waiti n g l = ! p in
18 List . iter ( fun k →
19 enq u eue ( fun () → continue k v) ) l ;
20 p := Done v;
21 run_next ()
22 | exception e →
23 let Waiti n g l = ! p in
24 List . iter ( fun k →
25 enq u eue ( fun () → dis c o n tinue k e ) ) l ;
26 p := Error e;
27 run_next ()
28 | eff ect ( Async (f ,v )) k →
29 let p = ref ( Waitin g []) in
30 enqueue (fun () → continue k p ) ;
31 fork p f v
32 | eff ect ( Await p) k →
33 match !p with
34 | Done v → continue k v
35 | Error e → d iscontinue k e
36 | Waiti n g l → p := Waiting ( k :: l) ; run_next ()
37 | eff ect Yield k →
38 enqueue (fun () → continue k () );
39 run_next ()
40 in
41 fork ( ref ( Waiting []) ) main v
Fig. 1: A simple scheduler, implemented with effects
