With Haskell now more popular than ever, a great deal of programmers deal with lazy evaluation in their daily lives. They’re aware of the pitfalls of lazy I/O, know not to use `foldl`

, and are masters at introducing bang patterns in the right place. But very few programmers know the magic behind lazy evaluation—graph reduction.

This post is an abridged adaptation of Simon Peyton Jones’ and David R. Lester’s book, *“Implementing Functional Languages: a tutorial.”*, itself a refinement of SPJ’s previous work, 1987’s *“The Implementation of Functional Programming Languages”*. The newer book doesn’t cover as much material as the previous: it focuses mostly on the evaluation of functional programs, and indeed that is our focus today as well. For this, it details three abstract machines: The G-machine, the Three Instruction Machine (affectionately called Tim), and a parallel G-machine.

In this post we’ll take a look first at a stack-based machine for reducing arithmetic expressions. Armed with the knowledge of how typical stack machines work, we’ll take a look at the G-machine, and how graph reduction works (and where the name comes from in the first place!)

This post is written as a Literate Haskell source file, with Cpp conditionals to enable/disable each section. To compile a specific section, use GHC like this:

Stack machines are the base for all of the computation models we’re going to explore today. To get a better feel of how they work, the first model of computation we’re going to describe is stack-based arithmetic, better known as reverse polish notation. This machine also forms the basis of the programming language FORTH. First, let us define a data type for arithmetic expressions, including the four basic operators (addition, multiplication, subtraction and division.)

```
data AExpr
= Lit Int
| Add AExpr AExpr
| Sub AExpr AExpr
| Mul AExpr AExpr
| Div AExpr AExpr
deriving (Eq, Show, Ord)
```

This language has an ‘obvious’ denotation, which can be realised using an interpreter function, such as `aInterpret`

below.

```
aInterpret :: AExpr -> Int
aInterpret (Lit n) = n
aInterpret (Add e1 e2) = aInterpret e1 + aInterpret e2
aInterpret (Sub e1 e2) = aInterpret e1 - aInterpret e2
aInterpret (Mul e1 e2) = aInterpret e1 * aInterpret e2
aInterpret (Div e1 e2) = aInterpret e1 `div` aInterpret e2
```

Alternatively, we can implement the language through its *operational* behaviour, by compiling it to a series of instructions that, when executed in an appropriate machine, leave it in a *final state* from which we can extract the expression’s result.

Our abstract machine for aritmethic will be a *stack* based machine with only a handful of instructions. The type of instructions is `AInstr`

.

The state of the machine is simply a pair, containing an instruction stream and a stack of values. By our compilation scheme, the machine is never in a state where more values are required on the stack than there are values present; This would not be the case if we let programmers directly write instruction streams.

We can compile a program into a sequence of instructions recursively.

```
aCompile :: AExpr -> [AInstr]
aCompile (Lit i) = [Push i]
aCompile (Add e1 e2) = aCompile e1 ++ aCompile e2 ++ [IAdd]
aCompile (Mul e1 e2) = aCompile e1 ++ aCompile e2 ++ [IMul]
aCompile (Sub e1 e2) = aCompile e1 ++ aCompile e2 ++ [ISub]
aCompile (Div e1 e2) = aCompile e1 ++ aCompile e2 ++ [IDiv]
```

And we can write a function to represent the state transition rules of the machine.

```
aEval :: ([AInstr], [Int]) -> ([AInstr], [Int])
aEval (Push i:xs, st) = (xs, i:st)
aEval (IAdd:xs, x:y:st) = (xs, (x + y):st)
aEval (IMul:xs, x:y:st) = (xs, (x * y):st)
aEval (ISub:xs, x:y:st) = (xs, (x - y):st)
aEval (IDiv:xs, x:y:st) = (xs, (x `div` y):st)
```

A state is said to be *final* when it has an empty instruction stream and a single result on the stack. To run a program, we simply repeat `aEval`

until a final state is reached.

```
aRun :: [AInstr] -> Int
aRun is = go (is, []) where
go st | Just i <- final st = i
go st = go (aEval st)
final ([], [n]) = Just n
final _ = Nothing
```

A very important property linking our compiler, abstract machine and interpreter together is that of *compiler correctness*. That is:

As an example, the arithmetic expression $2 + 3 \times 4$ produces the following code sequence:

You can interactively follow the execution of this program with the tool below. Pressing the Step button is equivalent to `aEval`

. The stack is drawn in boxes to the left, and the instruction sequence is presented on the right, where the `>`

marks the currently executing instruction (the “program counter”, if you will).

`amc-prove`

is a smallish tool to automatically prove (some) sentences of constructive quantifier-free^{1} first-order logic using the Amulet compiler’s capability to suggest replacements for typed holes.

In addition to printing whether or not it could determine the truthiness of the sentence, `amc-prove`

will also report the smallest proof term it could compute of that type.

- Function types
`P -> Q`

, corresponding to $P \to Q$ in the logic. - Product types
`P * Q`

, corresponding to $P \land Q$ in the logic. - Sum types
`P + Q`

, corresponding to $P \lor Q$ in the logic `tt`

and`ff`

correspond to $\top$ and $\bot$ respectively- The propositional bi-implication type
`P <-> Q`

stands for $P \iff Q$ and is interpreted as $P \to Q \land Q \to P$

Amulet will not attempt to pattern match on a sum type nested inside a product type. Concretely, this means having to replace $(P \lor Q) \land R \to S$ by $(P \lor Q) \to R \to S$ (currying).

`amc-prove`

’s support for negation and quantifiers is incredibly fiddly. There is a canonical empty type, `ff`

, but the negation connective `not P`

expands to `P -> forall 'a. 'a`

, since empty types aren’t properly supported. As a concrete example, take the double-negation of the law of excluded middle $\neg\neg(P \lor \neg{}P)$, which holds constructively.

If you enter the direct translation of that sentence as a type, `amc-prove`

will report that it couldn’t find a solution. However, by using `P -> ff`

instead of `not P`

, a solution is found.

```
? not (not (P + not P))
probably not.
? ((P + (P -> forall 'a. 'a)) -> forall 'a. 'a) -> forall 'a. 'a
probably not.
? ((P + (P -> ff)) -> ff) -> ff
yes.
fun f -> f (R (fun b -> f (L b)))
```

`amc-prove`

is bundled with the rest of the Amulet compiler on Github. You’ll need Stack to build. I recommend building with `stack build --fast`

since the compiler is rather large and `amc-prove`

does not benefit much from GHC’s optimisations.

```
% git clone https://github.com/tmpim/amc-prove.git
% cd amc-prove
% stack build --fast
% stack run amc-prove
Welcome to amc-prove.
?
```

Here’s a small demonstration of everything that works.

```
? P -> P
yes.
fun b -> b
? P -> Q -> P
yes.
fun a b -> a
? Q -> P -> P
yes.
fun a b -> b
? (P -> Q) * P -> Q
yes.
fun (h, x) -> h x
? P * Q -> P
yes.
fun (z, a) -> z
? P * Q -> Q
yes.
fun (z, a) -> a
? P -> Q -> P * Q
yes.
fun b c -> (b, c)
? P -> P + Q
yes.
fun y -> L y
? Q -> P + Q
yes.
fun y -> R y
? (P -> R) -> (Q -> R) -> P + Q -> R
yes.
fun g f -> function
| (L y) -> g y
| (R c) -> f c
? not (P * not P)
yes.
Not (fun (a, (Not h)) -> h a)
(* Note: Only one implication of DeMorgan's second law holds
constructively *)
? not (P + Q) <-> (not P) * (not Q)
yes.
(* Note: I have a marvellous term to prove this proposition,
but unfortunately it is too large to fit in this margin. *)
? (not P) + (not Q) -> not (P * Q)
yes.
function
| (L (Not f)) ->
Not (fun (a, b) -> f a)
| (R (Not g)) ->
Not (fun (y, z) -> g z)
```

You can find the proof term I redacted from DeMorgan’s first law here.

Technically, amc-prove “supports” the entire Amulet type system, which includes things like type-classes and rank-N types (it’s equal in expressive power to System F). However, the hole-filling logic is meant to aid the programmer while she codes, not exhaustively search for a solution, so it was written to fail early and fail fast instead of spending unbounded time searching for a solution that might not be there.↩︎

Amulet, unlike some other languages, has records figured out. Much like in ML (and PureScript), they are their own, first-class entities in the language as opposed to being syntax sugar for defining a product constructor and projection functions.

Being entities in the language, it’s logical to characterize them by their introduction and elimination judgements^{1}.

Records are introduced with record literals:

$\frac{\mathrm{\Gamma}\u22a2\stackrel{\u203e}{e\downarrow \tau}}{\mathrm{\Gamma}\u22a2\{\stackrel{\u203e}{\mathtt{x}=e}\}\downarrow \{\stackrel{\u203e}{\mathtt{x}:\tau}\}}$

And eliminated by projecting a single field:

$\frac{\mathrm{\Gamma}\u22a2r\downarrow \{\alpha \mathrm{\mid}\mathtt{x}:\tau \}}{\mathrm{\Gamma}\u22a2r\mathrm{.}\mathtt{x}\uparrow \tau}$

Records also support monomorphic update:

$\frac{\mathrm{\Gamma}\u22a2r\downarrow \{\alpha \mathrm{\mid}\mathtt{x}:\tau \}\phantom{\rule{1em}{0ex}}\mathrm{\Gamma}\u22a2e\downarrow \tau}{\mathrm{\Gamma}\u22a2\{r\text{}\mathtt{w}\mathtt{i}\mathtt{t}\mathtt{h}\text{}\mathtt{x}=e\}\downarrow \{\alpha \mathrm{\mid}\mathtt{x}:\tau \}}$

Unfortunately, the rather minimalistic vocabulary for talking about records makes them slightly worthless. There’s no way to extend a record, or to remove a key; Changing the type of a key is also forbidden, with the only workaround being enumerating all of the keys you *don’t* want to change.

And, rather amusingly, given the trash-talking I pulled in the first paragraph, updating nested records is still a nightmare.

```
> let my_record = { x = 1, y = { z = 3 } }
my_record : { x : int, y : { z : int } }
> { my_record with y = { my_record.y with z = 4 } }
_ = { x = 1, y = { z = 4 } }
```

Yikes. Can we do better?

Amulet recently learned how to cope with functional dependencies. Functional dependencies extend multi-param type classes by allowing the programmer to restrict the relationships between parameters. To summarize it rather terribly:

```
(* an arbitrary relationship between types *)
class r 'a 'b
(* a function between types *)
class f 'a 'b | 'a -> 'b
(* a one-to-one mapping *)
class o 'a 'b | 'a -> 'b, 'b -> 'a
```

As of today, Amulet knows the magic `row_cons`

type class, inspired by PureScript’s class of the same name.

```
class
row_cons 'record ('key : string) 'type 'new
| 'record 'key 'type -> 'new (* 1 *)
, 'new 'key -> 'record 'type (* 2 *)
begin
val extend_row : forall 'key -> 'type -> 'record -> 'new
val restrict_row : forall 'key -> 'new -> 'type * 'record
end
```

This class has built-in solving rules corresponding to the two functional dependencies:

- If the original
`record`

, the`key`

to be inserted, and its`type`

are all known, then the`new`

record can be solved for; - If both the
`key`

that was inserted, and the`new`

record, it is possible to solve for the old`record`

and the`type`

of the`key`

.

Note that rule 2 almost lets `row_cons`

be solved for in reverse. Indeed, this is expressed by the type of `restrict_row`

, which discovers both the `type`

and the original `record`

.

Using the `row_cons`

class and its magical methods…

- Records can be extended:

- Records can be restricted:

And, given a suitable framework of optics, records can be updated nicely:

It’s worth pointing out that making an optic that works for all fields, parametrised by a type-level string, is not easy or pretty, but it is work that only needs to be done once.

```
type optic 'p 'a 's <- 'p 'a 'a -> 'p 's 's
class
Amc.row_cons 'r 'k 't 'n
=> has_lens 'r 'k 't 'n
| 'k 'n -> 'r 't
begin
val rlens : strong 'p => proxy 'k -> optic 'p 't 'n
end
instance
Amc.known_string 'key
* Amc.row_cons 'record 'key 'type 'new
=> has_lens 'record 'key 'type 'new
begin
let rlens _ =
let view r =
let (x, _) = Amc.restrict_row @'key r
x
let set x r =
let (_, r') = Amc.restrict_row @'key r
Amc.extend_row @'key x r'
lens view set
end
let r
: forall 'key -> forall 'record 'type 'new 'p.
Amc.known_string 'key
* has_lens 'record 'key 'type 'new
* strong 'p
=> optic 'p 'type 'new =
fun x -> rlens @'record (Proxy : proxy 'key) x
```

Sorry for the short post, but that’s it for today.

Record fields $\mathtt{x}$ are typeset in monospaced font to make it apparent that they are unfortunately not first-class in the language, but rather part of the syntax. Since Amulet’s type system is inherently bidirectional, the judgement $\Gamma \vdash e \uparrow \tau$ represents type inference while $\Gamma \vdash e \downarrow \tau$ stands for type checking.↩︎

Compositional type-checking is a neat technique that I first saw in a paper by Olaf Chitil^{1}. He introduces a system of principal *typings*, as opposed to a system of principal *types*, as a way to address the bad type errors that many functional programming languages with type systems based on Hindley-Milner suffer from.

Today I want to present a small type checker for a core ML (with, notably, no data types or modules) based roughly on the ideas from that paper. This post is *almost* literate Haskell, but it’s not a complete program: it only implements the type checker. If you actually want to play with the language, grab the unabridged code here.

```
module Typings where
import qualified Data.Map.Merge.Strict as Map
import qualified Data.Map.Strict as Map
import qualified Data.Set as Set
import Data.Foldable
import Data.List
import Data.Char
import Control.Monad.Except
```

We’ll begin, like always, by defining data structures for the language. Now, this is a bit against my style, but this system (which I shall call ML_{$\Delta$} - but only because it sounds cool) is not presented as a pure type system - there are separate grammars for terms and types. Assume that `Var`

is a suitable member of all the appropriate type classes.

```
data Exp
= Lam Var Exp
| App Exp Exp
| Use Var
| Let (Var, Exp) Exp
| Num Integer
deriving (Eq, Show, Ord)
data Type
= TyVar Var
| TyFun Type Type
| TyCon Var
deriving (Eq, Show, Ord)
```

ML_{$\Delta$} is *painfully* simple: It’s a lambda calculus extended with `Let`

since there needs to be a demonstration of recursion and polymorphism, and numbers so there can be a base type. It has no unusual features - in fact, it doesn’t have many features at all: no rank-N types, GADTs, type classes, row-polymorphic records, tuples or even algebraic data types.

I believe that a fully-featured programming language along the lines of Haskell could be shaped out of a type system like this, however I am not smart enough and could not find any prior literature on the topic. Sadly, it seems that compositional typings aren’t a very active area of research at all.

The novelty starts to show up when we define data to represent the different kinds of scopes that crop up. There are monomorphic $\Delta$-contexts, which assign *types* to names, and also polymorphic $\Gamma$-contexts, that assign *typings* to names instead. While we’re defining `newtype`

s over `Map`

s, let’s also get substitutions out of the way.

```
newtype Delta = Delta (Map.Map Var Type)
deriving (Eq, Ord, Semigroup, Monoid)
newtype Subst = Subst (Map.Map Var Type)
deriving (Eq, Show, Ord, Monoid)
newtype Gamma = Gamma (Map.Map Var Typing)
deriving (Eq, Show, Ord, Semigroup, Monoid)
```

The star of the show, of course, are the typings themselves. A typing is a pair of a (monomorphic) type $\tau$ and a $\Delta$-context, and in a way it packages both the type of an expression and the variables it’ll use from the scope.

With this, we’re ready to look at how inference proceeds for ML_{$\Delta$}. I make no effort at relating the rules implemented in code to anything except a vague idea of the rules in the paper: Those are complicated, especially since they deal with a language much more complicated than this humble calculus. In an effort not to embarrass myself, I’ll also not present anything “formal”.

```
infer :: Exp -- The expression we're computing a typing for
-> Gamma -- The Γ context
-> [Var] -- A supply of fresh variables
-> Subst -- The ambient substitution
-> Either TypeError ( Typing -- The typing
, [Var] -- New variables
, Subst -- New substitution
)
```

There are two cases when dealing with variables. Either a typing is present in the environment $\Gamma$, in which case we just use that with some retouching to make sure type variables aren’t repeated - this takes the place of instantiating type schemes in Hindley-Milner. However, a variable can also *not* be in the environment $\Gamma$, in which case we invent a fresh type variable $\alpha$^{2} for it and insist on the monomorphic typing $\{ v :: \alpha \} \vdash \alpha$.

```
infer (Use v) (Gamma env) (new:xs) sub =
case Map.lookup v env of
Just ty -> -- Use the typing that was looked up
pure ((\(a, b) -> (a, b, sub)) (refresh ty xs))
Nothing -> -- Make a new one!
let new_delta = Delta (Map.singleton v new_ty)
new_ty = TyVar new
in pure (Typing new_delta new_ty, xs, sub)
```

Interestingly, this allows for (principal!) typings to be given even to code containing free variables. The typing for the expression `x`

, for instance, is reported to be $\{ x :: \alpha \} \vdash \alpha$. Since this isn’t meant to be a compiler, there’s no handling for variables being out of scope, so the full inferred typings are printed on the REPL- err, RETL? A read-eval-type-loop!

```
> x
{ x :: a } ⊢ a
```

Moreover, this system does not have type schemes: Typings subsume those as well. Typings explicitly carry information regarding which type variables are polymorphic and which are constrained by something in the environment, avoiding a HM-like generalisation step.

```
where
refresh :: Typing -> [Var] -> (Typing, [Var])
refresh (Typing (Delta delta) tau) xs =
let tau_fv = Set.toList (ftv tau `Set.difference` foldMap ftv delta)
(used, xs') = splitAt (length tau_fv) xs
sub = Subst (Map.fromList (zip tau_fv (map TyVar used)))
in (Typing (applyDelta sub delta) (apply sub tau), xs')
```

`refresh`

is responsible for ML_{$\Delta$}’s analogue of instantiation: New, fresh type variables are invented for each type variable free in the type $\tau$ that is not also free in the context $\Delta$. Whether or not this is better than $\forall$ quantifiers is up for debate, but it is jolly neat.

The case for application might be the most interesting. We infer two typings $\Delta \vdash \tau$ and $\Delta' \vdash \sigma$ for the function and the argument respectively, then unify $\tau$ with $\sigma \to \alpha$ with $\alpha$ fresh.

```
infer (App f a) env (alpha:xs) sub = do
(Typing delta_f type_f, xs, sub) <- infer f env xs sub
(Typing delta_a type_a, xs, sub) <- infer a env xs sub
mgu <- unify (TyFun type_a (TyVar alpha)) type_f
```

This is enough to make sure that the expressions involved are compatible, but it does not ensure that the *contexts* attached are also compatible. So, the substitution is applied to both contexts and they are merged - variables present in one but not in the other are kept, and variables present in both have their types unified.

```
let delta_f' = applyDelta mgu delta_f
delta_a' = applyDelta mgu delta_a
delta_fa <- mergeDelta delta_f' delta_a'
pure (Typing delta_fa (apply mgu (TyVar alpha)), xs, sub <> mgu)
```

If a variable `x`

has, say, type `Bool`

in the function’s context but `Int`

in the argument’s context - that’s a type error, one which that can be very precisely reported as an inconsistency in the types `x`

is used at when trying to type some function application. This is *much* better than the HM approach, which would just claim the latter usage is wrong. There are three spans of interest, not one.

Inference for $\lambda$ abstractions is simple: We invent a fresh monomorphic typing for the bound variable, add it to the context when inferring a type for the body, then remove that one specifically from the typing of the body when creating one for the overall abstraction.

```
infer (Lam v b) (Gamma env) (alpha:xs) sub = do
let ty = TyVar alpha
mono_typing = Typing (Delta (Map.singleton v ty)) ty
new_env = Gamma (Map.insert v mono_typing env)
(Typing (Delta body_delta) body_ty, xs, sub) <- infer b new_env xs sub
let delta' = Delta (Map.delete v body_delta)
pure (Typing delta' (apply sub (TyFun ty body_ty)), xs, sub)
```

Care is taken to apply the ambient substitution to the type of the abstraction so that details learned about the bound variable inside the body will be reflected in the type. This could also be extracted from the typing of the body, I suppose, but *eh*.

`let`

s are very easy, especially since generalisation is implicit in the structure of typings. We simply compute a typing from the body, *reduce* it with respect to the let-bound variable, add it to the environment and infer a typing for the body.

```
infer (Let (var, exp) body) gamma@(Gamma env) xs sub = do
(exp_t, xs, sub) <- infer exp gamma xs sub
let exp_s = reduceTyping var exp_t
gamma' = Gamma (Map.insert var exp_s env)
infer body gamma' xs sub
```

Reduction w.r.t. a variable `x`

is a very simple operation that makes typings as polymorphic as possible, by deleting entries whose free type variables are disjoint with the overall type along with the entry for `x`

.

```
reduceTyping :: Var -> Typing -> Typing
reduceTyping x (Typing (Delta delta) tau) =
let tau_fv = ftv tau
delta' = Map.filter keep (Map.delete x delta)
keep sigma = not $ Set.null (ftv sigma `Set.intersection` tau_fv)
in Typing (Delta delta') tau
```

Parsing, error reporting and user interaction do not have interesting implementations, so I have chosen not to include them here.

Compositional typing is a very promising approach for languages with simple polymorphic type systems, in my opinion, because it presents a very cheap way of providing very accurate error messages much better than those of Haskell, OCaml and even Elm, a language for which good error messages are an explicit goal.

As an example of this, consider the expression `fun x -> if x (add x 0) 1`

(or, in Haskell, `\x -> if x then (x + (0 :: Int)) else (1 :: Int)`

- the type annotations are to emulate ML_{$\Delta$}’s insistence on monomorphic numbers).

```
Types Bool and Int aren't compatible
When checking that all uses of 'x' agree
When that checking 'if x' (of type e -> e -> e)
can be applied to 'add x 0' (of type Int)
Typing conflicts:
· x : Bool vs. Int
```

The error message generated here is much better than the one GHC reports, if you ask me. It points out not that x has some “actual” type distinct from its “expected” type, as HM would conclude from its left-to-right bias, but rather that two uses of `x`

aren’t compatible.

```
<interactive>:4:18: error:
• Couldn't match expected type ‘Int’ with actual type ‘Bool’
• In the expression: (x + 0 :: Int)
In the expression: if x then (x + 0 :: Int) else 0
In the expression: \ x -> if x then (x + 0 :: Int) else 0
```

Of course, the prototype doesn’t care for positions, so the error message is still not as good as it could be.

Perhaps it should be further investigated whether this approach scales to at least type classes (since a form of ad-hoc polymorphism is absolutely needed) and polymorphic records, so that it can be used in a real language. I have my doubts as to if a system like this could reasonably be extended to support rank-N types, since it does not have $\forall$ quantifiers.

**UPDATE**: I found out that extending a compositional typing system to support type classes is not only possible, it was also Gergő Érdi’s MSc. thesis!

**UPDATE**: Again! This is new. Anyway, I’ve cleaned up the code and thrown it up on GitHub.

Again, a full program implementing ML_{$\Delta$} is available here. Thank you for reading!

Olaf Chitil. 2001. Compositional explanation of types and algorithmic debugging of type errors. In Proceedings of the sixth ACM SIGPLAN international conference on Functional programming (ICFP ’01). ACM, New York, NY, USA, 193-204. DOI.↩︎

Since I couldn’t be arsed to set up monad transformers and all, we’re doing this the lazy way (ba dum tss): an infinite list of variables, and hand-rolled reader/state monads.↩︎

Jesus, it’s been a while. Though my last post was almost 6 months ago (give or take a few), I’ve been busy working on Amulet, which continues to grow, almost an eldritch abomination you try desperately, but fail, to kill.

Since my last post, Amulet has changed a ton, in noticeable and not-so-noticeable ways. Here are the major changes to the compiler since then.

No language is good to use if it’s inconvenient. So, in an effort to make writing code more convenient, we’ve removed the need for `;;`

after top-level declarations, and added a *bunch* of indentation sensitivity, thus making several keywords optional: `begin`

and `end`

are implicit in the body of a `fun`

, `match`

, or `let`

, which has made those keywords almost entirely obsolete. The body of a `let`

also need not be preceded by `in`

if meaning is clear from indentation.

To demonstrate, where you would have

One can now write

Moreover, we’ve added shorthand syntax for building and destructuring records: `{ x, y, z }`

is equivalent to `{ x = x, y = y, z = z }`

in both pattern and expression position.

Whereas `{ x with a = b }`

would extend the record `x`

to contain a new field `a`

(with value `b`

), it’s now *monomorphic update* of the record `x`

. That is: `x`

must *already* contain a field called `a`

, with the same type as `b`

.

This lets you write a function for updating a field in a record, such as the one below, which would previously be impossible. Supporting polymorphic update is not a priority, but it’d be nice to have. The way PureScript, another language with row-polymorphic records, implements polymorphic update does not fit in with our constraint based type system. A new type of constraint would have to be introduced specifically for this, which while not impossible, is certainly annoying.

The impossibility of supporting polymorphic update with regular subsumption constraints $a \le b$ stems from the fact that, when faced with such a constraint, the compiler must produce a coercion function that turns *any* $a$ into a $b$ *given the types alone*. This is possible for, say, field narrowing—just pick out the fields you want out of a bigger record—but not for update, since the compiler has no way of turning, for instance, an `int`

into a `string`

.

Changes to how we handle subsumption have made it possible to store polymorphic values in not only tuples, but also general records. For instance:

```
let foo = {
apply : forall 'a 'b. ('a -> 'b) -> 'a -> 'b =
fun x -> x
} (* : { apply : forall 'a 'b. ('a -> 'b) -> 'a -> 'b } *)
```

`foo`

is a record containing a single polymorphic application function. It can be used like so:

A feature I’ve desired for a while now, `let`

expressions (and declarations!) can have a pattern as their left-hand sides, as demonstrated above. These can be used for any ol’ type, including for cases where pattern matching would be refutable. I haven’t gotten around to actually implementing this yet, but in the future, pattern matching in `let`

s will be restricted to (arbitrary) product types only.

```
type option 'a = Some of 'a | None
type foo = Foo of { x : int }
let _ =
let Some x = ... (* forbidden *)
let Foo { x } = ... (* allowed *)
```

Even more “in-the-future”, if we ever get around to adding attributes like OCaml’s, the check for this could be bypassed by annotating the declaration with (say) a `[@partial]`

attribute.

Unfortunately, since Amulet *is* a strict language, these are a bit limited: They can not be recursive in *any* way, neither directly nor indirectly.

A verification pass is run over the AST if type-checking succeeds, to forbid illegal uses of recursion (strict language) and, as an additional convenience, warn when local variables go unused.

For instance, this is forbidden:

And this gives a warning:

Plans for this include termination (and/or productivity) (as a warning) and exhaustiveness checks (as an error).

`main`

Since pattern-matching `let`

s are allowed at top-level, there’s no more need for `main`

. Instead of

Just match on `()`

at top-level:

This gets rid of the (not so) subtle unsoundness introduced by the code generator having to figure out how to invoke `main`

, and the type checker not checking that `main`

has type `unit -> 'a`

, and also allows us to remove much of the silly special-casing around variables called `main`

.

A bit like Scala’s, these allow marking a function’s parameter as implicit and having the type checker find out what argument you meant automatically. Their design is based on a bit of reading other compiler code, and also the paper on modular implicits for OCaml. However, we do not have a ML-style module system at all (much to my dismay, but it’s being worked on), much less first class modules.

Implicit parameters allow ad-hoc overloading based on dictionary passing (like type classes, but with less inference).

```
type show 'a = Show of 'a -> string
let show ?(Show f) = f
let implicit show_string =
Show (fun x -> x)
let "foo" = show "foo"
```

Here, unification makes it known that `show`

is looking for an implicit argument of type `show string`

, and the only possibility is `show_string`

, which is what gets used.

There is a built-in type `lazy : type -> type`

, a function `force : forall 'a. lazy 'a -> 'a`

for turning a thunk back into a value, and a keyword `lazy`

that makes a thunk out of any expression. `lazy 'a`

and `'a`

are mutual subtypes of eachother, and the compiler inserts thunks/`force`

s where appropriate to make code type-check.

```
let x && y = if x then force y else false
let () =
false && launch_the_missiles ()
(* no missiles were launched in the execution of this program *)
```

Literal patterns are allowed for all types, and they’re tested of using

`(==)`

.Amulet only has one type constructor in its AST for all its kinds of functions:

`forall 'a. 'a -> int`

,`int -> string`

and`show string => unit`

are all represented the same internally and disambiguated by dependency/visibility flags.Polymorphic recursion is checked using “outline types”, computed before fully-fledged inference kicks in based solely on the shape of values. This lets us handle the function below without an annotation on its return type by computing that

`{ count = 1 }`

*must*have type`{ count : int }`

beforehand.Combined with the annotation on

`x`

, this gives us a “full” type signature, which lets us use checking for`size`

, allowing polymorphic recursion to happen.

```
type nested 'a = Nested of nested ('a * 'a) * nested ('a * 'a) | One of 'a
let size (x : nested 'a) =
match x with
| One _ -> { count = 1 }
| Nested (a, _) -> { count = 2 * (size a).count }
```

The newtype elimination pass was rewritten once and, unfortunately, disabled, since it was broken with some touchy code.

Operator declarations like

`let 2 + 2 = 5 in 2 + 2`

are admissible.Sanity of optimisations is checked at runtime by running a type checker over the intermediate language programs after all optimisations

Local opens are allowed, with two syntaxes: Either

`M.( x + y)`

(or`M.{ a, b }`

) or`let open M in x + y`

.Amulet is inconsistent in some more ways, such as

`type : type`

holding.There are no more kinds.

This post was a bit short, and also a bit hurried. Some of the features here deserve better explanations, but I felt like giving them an *outline* (haha) would be better than leaving the blag to rot (yet again).

Watch out for a blog post regarding (at *least*) implicit parameters, which will definitely involve the changes to subtyping involving records/tuples.

Dependent types are a very useful feature - the gold standard of enforcing invariants at compile time. However, they are still very much not practical, especially considering inference for unrestricted dependent types is equivalent to higher-order unification, which was proven to be undecidable.

Fortunately, many of the benefits that dependent types bring aren’t because of dependent products themselves, but instead because of associated features commonly present in those programming languages. One of these, which also happens to be especially easy to mimic, are *inductive families*, a generalisation of inductive data types: instead of defining a single type inductively, one defines an entire *family* of related types.

Many use cases for inductive families are actually instances of a rather less general concept, that of generalised algebraic data types, or GADTs: Contrary to the indexed data types of full dependently typed languages, these can and are implemented in several languages with extensive inference, such as Haskell, OCaml and, now, Amulet.

Before I can talk about their implementation, I am legally obligated to present the example of *length indexed vectors*, linked structures whose size is known at compile time—instead of carrying around an integer representing the number of elements, it is represented in the type-level by a Peano^{1} natural number, as an *index* to the vector type. By universally quantifying over the index, we can guarantee by parametricity^{2} that functions operating on these don’t do inappropriate things to the sizes of vectors.

```
type z ;;
type s 'k ;;
type vect 'n 'a =
| Nil : vect z 'a
| Cons : 'a * vect 'k 'a -> vect (s 'k) 'a
```

Since the argument `'n`

to `vect`

(its length) varies with the constructor one chooses, we call it an *index*; On the other hand, `'a`

, being uniform over all constructors, is called a *parameter* (because the type is *parametric* over the choice of `'a`

). These definitions bake the measure of length into the type of vectors: an empty vector has length 0, and adding an element to the front of some other vector increases the length by 1.

Matching on a vector reveals its index: in the `Nil`

case, it’s possible to (locally) conclude that it had length `z`

. Meanwhile, the `Cons`

case lets us infer that the length was the successor of some other natural number, `s 'k`

, and that the tail itself has length `'k`

.

If one were to write a function to `map`

a function over a `vect`

or, they would be bound by the type system to write a correct implementation - well, either that or going out of their way to make a bogus one. It would be possible to enforce total correctness of a function such as this one, by adding linear types and making the vector parameter linear.

```
let map (f : 'a -> 'b) (xs : vect 'n 'a) : vect 'n 'b =
match xs with
| Nil -> Nil
| Cons (x, xs) -> Cons (f x, map f xs) ;;
```

If we were to, say, duplicate every element in the list, an error would be reported. Unlike some others, this one is not very clear, and it definitely could be improved.

```
Occurs check: The type variable jx
occurs in the type s 'jx
· Arising from use of the expression
Cons (f x, Cons (f x, map f xs))
│
33 │ | Cons (x, xs) -> Cons (f x, Cons (f x, map f xs)) ;;
│ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

This highlights the essence of GADTs: pattern matching on them reveals equalities about types that the solver can later exploit. This is what allows the programmer to write functions that vary their return types based on their inputs - a very limited form of type-term dependency, which brings us ever closer to the Calculus of Constructions corner of Barendregt’s lambda cube^{3}.

The addition of generalised algebraic data types has been in planning for over two years—it was in the original design document. In a mission that not even my collaborator noticed, all of the recently-added type system and IR features were directed towards enabling the GADT work: bidirectional type checking, rank-N polymorphism and coercions.

All of these features had cover stories: higher-ranked polymorphism was motivated by monadic regions; bidirectional type checking was motivated by the aforementioned polymorphism; and coercions were motivated by newtype optimisation. But, in reality, it was a conspiracy to make GADTs possible: having support for these features simplified implementing our most recent form of fancy types, and while adding all of these in one go would be possible, doing it incrementally was a lot saner.

While neither higher-ranked types nor GADTs technically demand a bidirectional type system, implementing them with such a specification is considerably easier, removing the need for workarounds such as boxy types and a distinction between rigid/wobbly type variables. Our algorithm for GADT inference rather resembles Richard Eisenberg’s Bake^{4}, in that it only uses local equalities in *checking* mode.

Adding GADTs also lead directly to a rewrite of the solver, which now has to work with *implication constraints*, of the form `(Q₁, ..., Qₙ) => Q`

, which should be read as “Assuming `Q₁`

through `Qₙ`

, conclude `Q`

.” Pattern matching on generalised constructors, in checking mode, captures every constraint generated by checking the right-hand side of a clause and captures that as an implication constraint, with all the constructor-bound equalities as assumptions. As an example, this lets us write a type-safe cast function:

```
type eq 'a 'b = Refl : eq 'a 'a
(* an inhabitant of eq 'a 'b is a proof that 'a and 'b are equal *)
let subst (Refl : eq 'a 'b) (x : 'a) : 'b = x ;;
```

Unfortunately, to keep inference decidable, many functions that depend on generalised pattern matching need explicit type annotations, to guide the type checker.

When *checking* the body of the function, namely the variable reference `x`

, the solver is working under an assumption `'a ~ 'b`

(i.e., `'a`

and `'b`

stand for the same type), which lets us unify the stated type of `x`

, namely `'a`

, with the return type of the function, `'b`

.

If we remove the local assumption, say, by not matching on `Refl`

, the solver will not be allowed to unify the two type variables `'a`

and `'b`

, and an error message will be reported^{5}:

```
examples/gadt/equality.ml[11:43 ..11:43]: error
Can not unify rigid type variable b with the rigid type variable a
· Note: the variable b was rigidified because of a type ascription
against the type forall 'a 'b. t 'a 'b -> 'a -> 'b
and is represented by the constant bq
· Note: the rigid type variable a, in turn,
was rigidified because of a type ascription
against the type forall 'a 'b. t 'a 'b -> 'a -> 'b
· Arising from use of the expression
x
│
11 │ let subst (_ : t 'a 'b) (x : 'a) : 'b = x ;;
│ ~
```

Our intermediate language was also extended, from a straightforward System F-like lambda calculus with type abstractions and applications, to a System F_{C}-like system with *coercions*, *casts*, and *coercion abstraction*. Coercions are the evidence, produced by the solver, that an expression is usable as a given type—GADT patterns bind coercions like these, which are the “reification” of an implication constraint. This lets us make type-checking on the intermediate language fast and decidable^{6}, as a useful sanity check.

The two new judgements for GADT inference correspond directly to new cases in the `infer`

and `check`

functions, the latter of which I present here for completeness. The simplicity of this change serves as concrete evidence of the claim that bidirectional systems extend readily to new, complex features, producing maintainable and readable code.

```
check (Match t ps a) ty = do
(t, tt) <- infer t
ps <- for ps $ \(p, e) -> do
(p', ms, cs) <- checkPattern p tt
let tvs = Set.map unTvName (boundTvs p' ms)
(p',) <$> implies (Arm p e) tt cs
(local (typeVars %~ Set.union tvs)
(extendMany ms (check e ty)))
pure (Match t ps (a, ty))
```

This corresponds to the checking judgement for matches, presented below. Note that in my (rather informal) theoretical presentation of Amulet typing judgements, we present implication constraints as a lexical scope of equalities conjoined with the scope of variables; Inference judgements (with double right arrows, $\Rightarrow$) correspond to uses of `infer`

, pattern checking judgements ($\Leftarrow_\text{pat}$) correspond to `checkPattern`

, which also doubles as $\mathtt{binds}$ and $\mathtt{cons}$, and the main checking judgement $\Leftarrow$ is the function `check`

.

$\frac{\mathrm{\Gamma};\mathcal{Q}\u22a2e\Rightarrow \tau \phantom{\rule{1em}{0ex}}\mathrm{\Gamma}\u22a2{p}_{i}{\Leftarrow}_{\text{pat}}\tau \phantom{\rule{1em}{0ex}}\mathrm{\Gamma},\mathtt{b}\mathtt{i}\mathtt{n}\mathtt{d}\mathtt{s}({p}_{i});\mathcal{Q},\mathtt{c}\mathtt{o}\mathtt{n}\mathtt{s}({p}_{i})\u22a2{e}_{i}\Leftarrow \sigma}{\mathrm{\Gamma};\mathcal{Q}\u22a2\mathtt{m}\mathtt{a}\mathtt{t}\mathtt{c}\mathtt{h}\text{}e\text{}\mathtt{w}\mathtt{i}\mathtt{t}\mathtt{h}\text{}\{{p}_{i}\to {e}_{i}\}\Leftarrow \sigma}$

Our implementation of the type checker is a bit more complex, because it also does (some) elaboration and bookkeeping: tagging terms with types, blaming type errors correctly, etc.

This new, complicated feature was a lot harder to implement than originally expected, but in the end it worked out. GADTs let us make the type system *stronger*, while maintaining the decidable inference that the non-fancy subset of the language enjoys.

The example presented here was the most boring one possible, mostly because two weeks ago I wrote about their impact on the language’s ability to make things safer.

Peano naturals are one particular formulation of the natural numbers, which postulates that zero (denoted

`z`

above) is a natural number, and any natural number’s successor (denoted`s 'k`

above) is itself natural.↩︎This is one application of Philip Wadler’s Theorems for Free technique: given a (polymorphic) type of some function, we can derive much of its behaviour.↩︎

Amulet is currently somewhere on the edge between λ2 - the second order lambda calculus, System F, and λP2, a system that allows quantification over types and terms using the dependent product form, which subsumes both the ∀ binder and the → arrow. Our lack of type functions currently leaves us very far from the CoC.↩︎

See his thesis. Our algorithm, of course, has the huge simplification of not having to deal with full dependent types.↩︎

And quite a good one, if I do say so! The compiler syntax highlights and pretty-prints both terms and types relevant to the error, as you can see here.↩︎

Even if we don’t do it yet—work is still ongoing to make the type checker and solver sane.↩︎

Ever since its inception, Amulet has strived to be a language that *guarantees* safety, to some extent, with its strong, static, inferred type system. Through polymorphism we gain the concept of *parametricity*, as explained in Philip Wadler’s Theorems for Free: a function’s behaviour does not depend on the instantiations you perform.

However, the power-to-weight ratio of these features quickly plummets, as every complicated type system extension makes inference rather undecidable, which in turn mandates more and more type annotations. Of the complex extensions I have read about, three struck me as particularly elegant, and I have chosen to implement them all in Amulet:

- Generalised Algebraic Data Types, which this post is about;
- Row Polymorphism, which allows being precise about which structure fields a function uses; and
- Rank-N types, which enables the implementation of many concepts including monadic regions.

Both GADTs and rank-N types are in the “high weight” category: inference in the presence of both is undecidable. Adding support for the latter (which laid the foundations for the former) is what drove me to re-write the type checker, a crusade detailed in my last post.

Of course, in the grand scheme of things, some languages provide way more guarantees than Amulet: For instance, Rust, with its lifetime system, can prove that code is memory-safe at compile time; Dependently-typed languages such as Agda and Idris can express a lot of invariants in their type system, but inference is completely destroyed. Picking which features you’d like to support is a game of tradeoffs—all of them have benefits, but some have exceedingly high costs.

Amulet was originally based on a very traditional, HM-like type system with support for row polymorphism. The addition of rank-N polymorphism and GADTs instigated the move to a bidirectional system, which in turn provided us with the ability to experiment with a lot more type system extensions (for instance, linear types)—in pursuit of more guarantees like parametricity.

In a sense, generalised ADTs are a “miniature” version of the inductive families one would find in dependently-typed programming (and, indeed, Amulet can type-check *some* uses of length-indexed vectors, although the lack of type-level computation is a showstopper). They allow non-uniformity in the return types of constructors, by packaging “coercions” along with the values; pattern matching, then, allows these coercions to influence the solving of particular branches.

Since this is an introduction to indexed types, I am legally obligated to present the following three examples: the type of equality witnesses between two other types; higher-order abstract syntax, the type of well-formed terms in some language; and *vectors*, the type of linked lists with statically-known lengths.

As is tradition in intuitionistic type theory, we define equality by postulating (that is, introducing a *constructor* witnessing) reflexivity: anything is equal to itself. Symmetry and transitivity can be defined as ordinary pattern-matching functions. However, this demonstrates the first (and main) shortcoming of our implementation: Functions which perform pattern matching on generalised constructors *must* have explicitly stated types.^{1}

```
type eq 'a 'b =
| Refl : eq 'a 'a ;;
let sym (Refl : eq 'a 'b) : eq 'b 'a = Refl ;;
let trans (Refl : eq 'a 'b) (Refl : eq 'b 'c) : eq 'a 'c = Refl ;;
```

Equality, when implemented like this, is conventionally used to implement substitution: If there exists a proof that `a`

and `b`

are equal, any `a`

may be treated as a `b`

.

Despite `a`

and `b`

being distinct, *rigid* type variables, matching on `Refl`

allows the constraint solver to treat them as equal.

```
type z ;; (* the natural zero *)
type s 'k ;; (* the successor of a number *)
type vect 'n 'a = (* vectors of length n *)
| Nil : vect z 'a
| Cons : 'a * vect 'k 'a -> vect (s 'k) 'a
```

Parametricity can tell us many useful things about functions. For instance, all closed, non-looping inhabitants of the type `forall 'a. 'a -> 'a`

are operationally the identity function. However, expanding the type grammar tends to *weaken* parametricity before making it stronger. Consider the type `forall 'a. list 'a -> list 'a`

—it has several possible implementations: One could return the list unchanged, return the empty list, duplicate every element in the list, drop some elements around the middle, among *many* other possible behaviours.

Indexed types are beyond the point of weakening parametricity, and start to make it strong again. Consider a function of type `forall 'a 'n. ('a -> 'a -> ordering) -> vect 'n 'a -> vect 'n 'a`

—by making the length of the vector explicit in the type, and requiring it to be kept the same, we have ruled out any implementations that drop or duplicate elements. A win, for sure, but at what cost? An implementation of insertion sort for traditional lists looks like this, when implemented in Amulet:

```
let insert_sort cmp l =
let insert e tl =
match tl with
| Nil -> Cons (e, Nil)
| Cons (h, t) -> match cmp e h with
| Lt -> Cons (e, Cons (h, t))
| Gt -> Cons (h, insert e t)
| Eq -> Cons (e, Cons (h, t))
and go l = match l with
| Nil -> Nil
| Cons (x, xs) -> insert x (go xs)
in go l ;;
```

The implementation for vectors, on the other hand, is full of *noise*: type signatures which we would rather not write, but are forced to by the nature of type systems.

```
let insert_sort (cmp : 'a -> 'a -> ordering) (v : vect 'n 'a) : vect 'n 'a =
let insert (e : 'a) (tl : vect 'k 'a) : vect (s 'k) 'a =
match tl with
| Nil -> Cons (e, Nil)
| Cons (h, t) -> match cmp e h with
| Lt -> Cons (e, Cons (h, t))
| Gt -> Cons (h, insert e t)
| Eq -> Cons (e, Cons (h, t))
and go (v : vect 'k 'a) : vect 'k 'a = match v with
| Nil -> Nil
| Cons (x, xs) -> insert x (go xs)
in go v ;;
```

These are not quite theorems for free, but they are theorems for quite cheap.

```
type term 'a =
| Lit : int -> term int
| Fun : ('a -> 'b) -> term ('a -> 'b)
| App : term ('a -> 'b) * term 'a -> term 'b
```

In much the same way as the vector example, which forced us to be correct with our functions, GADTs can also be applied in making us be correct with our *data*. The type `term 'a`

represents well typed terms: the interpretation of such a value need not be concerned with runtime errors at all, by leveraging the Amulet type system to make sure its inputs are correct.

```
let eval (x : term 'a) : 'a =
match x with
| Lit l -> l
| Fun f -> f
| App (f, x) -> (eval f) (eval x)
```

While equalities let us bend the type system to our will, vectors and terms let *the type system* help us, in making incorrect implementations compile errors.

Rank-N types are quite useful, I’m sure. To be quite honest, they were mostly implemented in preparation for GADTs, as the features have some overlap.

A use case one might imagine if Amulet had notation for monads would be an implementation of the ST monad^{2}, which prevents mutable state from escaping by use of rank-N types. `St.run action`

is a well-typed program, since `action`

has type `forall 's. st 's int`

, but `St.run action'`

is not, since that has type `forall 's. st 's (ref 's int)`

.

```
let action =
St.bind (alloc_ref 123) (fun var ->
St.bind (update_ref var (fun x -> x * 2)) (fun () ->
read_ref var))
and action' =
St.bind (alloc_ref 123) (fun var ->
St.bind (update_ref var (fun x -> x * 2)) (fun () ->
St.pure var))
```

Types are very powerful things. A powerful type system helps guide the programmer by allowing the compiler to infer more and more of the *program*—type class dictionaries in Haskell, and as a more extreme example, proof search in Agda and Idris.

However, since the industry has long been dominated by painfully first-order, very verbose type systems like those of Java and C#, it’s no surprise that many programmers have fled to dynamically typed languages like ~~Go~~ Python—a type system needs to be fairly complex before it gets to being expressive, and it needs to be *very* complex to get to the point of being useful.

Complexity and difficulty, while often present together, are not nescessarily interdependent: Take, for instance, Standard ML. The first-order parametric types might seem restrictive when used to a system with like Haskell’s (or, to some extent, Amulet’s^{3}), but they actually allow a lot of flexibility, and do not need many annotations at all! They are a sweet spot in the design space.

If I knew more about statistics, I’d have some charts here correlating programmer effort with implementor effort, and also programmer effort with the extent of properties one can state as types. Of course, these are all fuzzy metrics, and no amount of statistics would make those charts accurate, so have my feelings in prose instead:

Implementing a dynamic type system is

*literally*no effort. No effort needs to be spent writing an inference engine, or a constraint solver, or a renamer, or any other of the very complex moving parts of a type checker.However, the freedom they allow the implementor they take away from the programmer, by forcing them to keep track of the types of everything mentally. Even those that swear by dynamic types can not refute the claim that data has shape, and having a compiler that can make sure your shapes line up so you can focus on programming is a definite advantage.

On the opposite end of the spectrum, implementing a dependent type system is a

*lot*of effort. Things quickly diverge into undecidability before you even get to writing a solver—and higher order unification, which has a tendency to pop up, is undecidable too.While the implementor is subject to an endless stream of suffering, the programmer is in some ways free and some ways constrained. They can now express lots of invariants in the type system, from correctness of

`sort`

to correctness of an entire compiler or an operating system kernel, but they must also state very precise types for everything.In the middle lies a land of convenient programming without an endlessly suffering compiler author, a land first explored by the ML family with its polymorphic, inferred type system.

This is clearly the sweet spot. Amulet leans slightly to the dependently type end of the spectrum, but can still infer the types for many simple and complex programs without any annotations-the programs that do not use generalised algebraic data types or rank-N polymorphism.

In reality, the details are fuzzier. To be precise, pattern matching on GADTs only introduces an implication constraint when the type checker is applying a checking judgement. In practice, this means that at least the return type must be explicitly annotated.↩︎

Be warned that the example does not compile unless you remove the modules, since our renamer is currently a bit daft.↩︎

This is

*my*blog, and I’m allowed to brag about my projects, damn it.↩︎