GitHub

purplesyringa

Impressions of Teal

About two months ago, I found the Teal programming language, which describes itself as a statically-typed dialect of Lua. It transpiles to Lua and runs typecheck on the developer’s machine, so it fills roughly the same niche as TypeScript. It’s used by quite a few projects, including LuaRocks and inspect.

For better or worse, I am trying to write Lua without feeling like it’s year 2010, and Teal promised to be suitable for this purpose.

This post is a retrospective on my experience: what went right and what, in my opinion, could be improved. Mostly the latter.

Disclaimer: while I stumbled upon multiple issues, I deeply respect everyone who works on this project, and I consider it a valuable tool. No offence intended.

Annotations

I think the main selling point of Teal is just how simple its annotations are. Look at the example on the website:

local function add(a: number, b: number): number
    return a + b
end

local s: number = add(1, 2)
print(s)

It’s concise and non-intrusive, like TypeScript. Compare it to LuaLS annotations, which offers a competitive Lua typing project:

---@param a number
---@param b number
---@return number
local function add(a, b)
    return a + b
end

---@type number
local s = add(1, 2)
print(s)

Nasty!

Soundness

This is where its obvious advantages end, though. As we’ll find out in a bit, the simplicity is quite fragile.

I’m a Rust-pilled developer, so I expect some degree of soundness from type systems. “Soundness” means that if the type checker says the program is fine, you shouldn’t encounter type errors in runtime. Sounds easy?

But soundness is fundamentally difficult to achieve in dynamic languages due to variance. The issue arises when mutability and subtypes collide. Here’s an example:

The only real way to solve this is with readonly annotations, like in TypeScript. Teal doesn’t offer anything of similar functionality. It also doesn’t have nil safety, not even opt-in, so e.g. all table accesses return T, not T | nil.

I won’t hold this specific issue against Teal, as even TypeScript supports bivariance by default and doesn’t have nil safety for arrays, but it does hint that Teal considers soundness – which I’d call the main point of typing – an afterthought.

Unions

Teal has very rudimentary support for discriminated unions (or sum types, or enums with fields). For comparison, here’s what that looks like in Rust:

enum Enum {
    Variant1 {
        a: String,
    },
    Variant2 {
        a: i32.
        b: i32,
    },
}

…and in TypeScript, which simulates them with union types:

interface Variant1 {
    type: "variant1"
    a: string
}

interface Variant2 {
    type: "variant2"
    a: number
    b: number
}

type Enum = Variant1 | Variant2

Note that we have to explicitly add a field of a literal type to disambiguate types in runtime. Teal supports neither first-class sum types, nor literal types, so you have to tell it how to disambiguate variants explicitly – otherwise you can’t make a type union:

local record Variant1
where self.type == "variant1"
    type: string
    a: string
end

local record Variant2
where self.type == "variant2"
    type: string
    a: number
    b: number
end

local type Enum = Variant1 | Variant2

where denotes a type predicate. It can contain arbitrary code, and the compiler doesn’t validate that the definition actually makes sense. So you have to write the equivalent of unsafe code every time you want to define a discriminated union!

Furthermore, Teal can’t validate that newly created instances of Variant* have the right types, because type is just a string. To enforce that, you have to use single-variant enums to simulate some aspects of literal types:

local enum Variant1Type "variant1" end
local record Variant1
where self.type == "variant1" -- still required, but at least type-checked
    type: Variant1Type
    a: string
end

...

-- Error: string "variatn1" is not a variant of Variant1Type
local x: Variant1 = {
    type = "variatn1", -- hypothetical misspelling
    a = "b",
}

Suffice to say, this is messier than it should be. But all the underlying mechanisms are already there, so hopefully Teal supports sum types as a first-class citizen one day.

is

Teal adds two operators to the language: as and is. as is an unchecked typecast that gets optimized away in runtime, and is checks if a variable has a given runtime type.

The main use for is is pattern-matching union types:

local function f(x: Enum)
    -- Doesn't work, says `x` has type `Variant1 | Variant2` even inside `if` branches.
    if x.type == "variant1" then
        local y: Variant1 = x
    else
        local y: Variant2 = x
    end

    -- Works correctly, substitutes the predicate in `where`.
    if x is Variant1 then
        local y: Variant1 = x
    else
        local y: Variant2 = x
    end
end

Its semantics don’t seem to be thought-out well, though. is T resolves the type to T in any context, not just for union types:

local function f(x: any)
    if x is Variant1 then
        local y: Variant1 = x
    end
end

But in this context, x is not even guaranteed to be a table, so the access x.type == "variant1" substituted from where may fail. While I would expect the result of where to be trusted blindly, I wouldn’t expect the calculation itself to be trusted, since it’s type-checked – but, apparently, not strictly enough.

Furthermore, in this context, is works even on types without where – those get an implicit where type(self) == "table", meaning that is itself is effectively unsound:

local record Record
end

local function f(x: any)
    if x is Record then
        local y: Record = x
        -- Assumes `y` is `Record` even though the runtime check only verifies it's a table.
    end
end

In addition, x is any lowers to type(x) == "table" (which is strange because any is supposed to represent any value), and x is nil lowers to x == nil, despite the fact that comparison operators can be overloaded.

While each issue can be solved individually (and I reported some of them), their multitude makes me question the design decisions that allowed them to manifest.

Metatables

Lua implements OOP with prototypes. To create an object, you need to do two things:

-- 1. Create a data-only table with the right fields.
local obj = {
    a = 1,
    b = 2,
    ...
}

-- 2. Set its metatable.
setmetatable(obj, {
    __index = Type,
})

The metatable controls many properties of the object, including how attribute accesses to absent keys are resolved. This allows us to share method definitions across object instances:

-- Define methods once.
local Person = {
    say_hello = function(self) print("Hello", self.name) end,
}

-- Create multiple objects whose attribute accesses fall back to `Person`.
local alice = { name = "Alice" }
setmetatable(alice, { __index = Person })
local bob = { name = "Bob" }
setmetatable(bob, { __index = Person })

-- `alice` doesn't have a `say_hello` field, but `alice` inherits from `Person`, which does, so
-- that's what `alice.say_hello` resolves to.
alice.say_hello(alice)
-- A colon instead of a dot is syntax sugar for passing the receiver as the first argument.
bob:say_hello()

The states of the object after setmetatable and before setmetatable are fundamentally different: one enables method calls, overloaded operators, and static properties, while the other one doesn’t. But Teal doesn’t see the difference:

local record Person
    name: string
end

function Person:say_hello()
    print("Hello", self.name)
end

local alice: Person = { name = "alice" }
-- Oops, forgot `setmetatable`!
alice:say_hello() -- compiles just fine, errors in runtime

This is quite a surprising omission. While setmetatable(obj, metatable) mutates obj, it also returns obj, so you could plausibly imagine Teal introducing a signature like

setmetatable: function<T>(onlyfields<T>, metatable<T>): T

…and inferring the type of { name = "alice" } as onlyfields<Person>, not Person. In fact, Teal already supports the metatable<T> type, so this feels like an honest omission – but I can’t help but get surprised that such a basic thing got forgotten.

Effects

Lua offers coroutines, which can be used to implement iterators and async functions. A coroutine is created by coroutine.create(closure), and any call coroutine.yield(...) nested arbitrarily deep in closure sends a value upward to the coroutine owner (e.g. a for loop or an async runtime). The owner can then resume the coroutine from after the yield point.

As every other time someone “solves” the coloring problem, it turns out that static typing brings it back, but worse. Let’s look at a concrete example:

local function f()
    coroutine.yield(1)
end

local function g()
    f()
end

local function h()
    local coro = coroutine.create(g)
    local ok, value = coroutine.resume(coro)
    assert(ok and value == 1)
end

f yields a number, but that fact is not represented in its signature: f takes no parameters and returns no values, so there’s nothing to annotate. By the time we get to h, we’ve already passed through g, which has forgotten that it yields anything at all (the body of g doesn’t mention coroutine once, and f’s signature doesn’t mention it yields either), so we’ve lost all type information. Then how do we type value in h?

We need an annotation that survives nested calls. That’s called an effect. We could imagine an effect yields <type> and a requirement that every call to a function with yields <type> must itself occur in a function annotated with yields <type>:

local function f() yields number
    coroutine.yield(1) -- requires `f` to have `yields number`
end

local function g() yields number
    f() -- requires `g` to have `yields number`
end

local function h()
    local coro = coroutine.create(g) -- takes `function() yields T` for a generic `T`
    local ok, value = coroutine.resume(coro) -- returns `boolean, T`
    assert(ok and value == 1)
end

Teal doesn’t have an effect system: it just marks coroutine.resume and coroutine.yield as taking/returning any values (TypeScript’s unknown) and requires the user to insert type casts. That’s quite unfortunate, since even this rudimentary form of effects would make coroutine-based iterators sound.

This also means that coroutine.wrap has to be annotated as function<F>(F): F, unsoundly ignoring yielded values. function(function(): R yields T): (function(): R | T) would be a better signature if Teal supported effects (still not quite correct due to arguments, but that requires typestate).

While coroutines are rare in computational Lua, they are used extensively in GUIs and server logic. I wouldn’t call supporting it a must, but event-driven software is an important niche, and it would be great to see this limitation fixed.

Numbers

Teal has a strange relationship with numbers, though it’s arguably Lua’s fault.

Modern Lua has two numeric types: integer and float. They behave completely differently, e.g. integers wrap-around on overflow and floats work like floats do. Teal exposes integer as a subtype of number, but doesn’t offer float, meaning that this function cannot be annotated safely:

local function f(a: number, b: number)
    if a > 0 and b > 0 then
        -- Valid for `float`s, invalid for `integer`s.
        assert(a + b > 0)
    end
end

It quickly becomes clear that Teal inconsistently treats integer as meaning “any number that is whole” or “integer according to Lua rules” in different places:

stdlib

On a positive note, there is a surprisingly small amount of issues in the standard library annotations. We’ve already seen issues with coroutine methods, setmetatable, and integers, but the remaining problems are very minor:

Modules

Using a type defined in a different file requires a manual import. The design is similar to TypeScript, but there are subtle differences between JavaScript and Lua that make this choice questionable.

In TypeScript, you can export a named type and import it in one of two ways:

// a.ts
export interface Test {
    field: string
}

// b.ts
import { Test } from "./a.ts"
// or
import type { Test } from "./a.ts"

import compiles to a runtime import and is useful for types with runtime presence, such as classes. import type is zero-cost and erased during transpilation.

Lua doesn’t have named exports: each module returns only one value, so named returns are simulated by returning a table. But Teal’s equivalent of import type doesn’t recognize the idiomatic way to construct it!

-- a.tl
local interface Test end
return {
    Test = Test,
}

-- b.tl
-- Error: 'require' did not return a type, got record (Test: Test)
local type Test = require("a").Test

Instead, you have to define a temporary record and return that:

-- a.tl
local record A
    interface Test end
end

return A

records are just tables in disguise, so you can still populate them with runtime values via function A.f() and A.x = ... – but you can’t use a table literal directly.

But the fact that records can be returned from modules means that some types imported with local type exist in runtime, which is completely unlike TypeScript:

-- b.tl
local type Test = require("a").Test
return Test

-- compiles to:
local Test = require("a").Test
return Test

This matters because the compiler and the runtime resolve modules differently: when importing external libraries, Teal will use the .d.tl type annotations, and Lua will use the untyped .lua code. If the .d.tl file contains helper type definitions that don’t exist in .lua, you may get errors in runtime despite using local type, without a warning.

So local type = require is less powerful than local = require, but at the same time it’s not guaranteed to be compile-time-only. Then why does it exist? Optimization aside, there seems to be just one reason: interfaces don’t have a runtime representation, so local = require fails for them, forcing the use of local type = require.

None of this screams “great design” to me. If materializing types is considered cheap, interfaces with nested types should be materialized as well, so that nil accesses on reexports are avoided and local type becomes unnecessary. And if it’s expensive, local type should never materialize automatically.

Syntax

This section is a little subjective, but hopefully convincing.

Lua has a simple grammar: Lua code can be parsed pretty much without lookahead and is easy to generate. There are many compatible Lua parsers and runtimes.

Teal grammar looks similar to Lua, but it’s much subtler, and the docs don’t make that obvious. Here’s one example.

record, interface, global, etc. are contextual keywords in Teal, meaning that they can also be used as variable names. This is necessary for compatibility with pre-existing Lua code. So a Teal parser needs to disambiguate between local record and local record Name ... end, but in general that’s impossible:

-- Consider this snippet:
local record global interface X end end

-- Interpretation #1: a local named `record`, followed by an empty global interface named `X`, and
-- then the parent scope is closed.
    local record
    global interface X end
end

-- Interpretation #2: a local record named `global`, containing an empty interface `X`.
local record global
    interface X end
end

-- Ambiguous example -- which line closes the `do` scope?
do
local record global interface X end end
local record global interface X end end

The official compiler uses subtle heuristics, so writing a Teal parser requires copying them precisely. This is the reason why my partial reimplementation of Teal is so long.

A simpler difference is that Teal is whitespace-sensitive, unlike Lua. The same unannotated code, transpiled with the same Teal compiler, can behave differently depending on whether the file is named test.tl or test.lua:

f()
(g)()

Teal assumes lines beginning with ( start a new statement, Lua treats them as the continuation of a previous expression. Leading ( is rare in Lua, so this seldom causes trouble, but in Teal it’s common to see (obj as T).f(), so they had to adjust it. I wish they changed the syntax of as instead, perhaps to cast<T>(obj).f() or obj.as<T>.f().

Yet this whitespace sensitivity is not applied in a different context where it would help. Both function and function() are valid types in Teal, but function( is always parsed greedily without consideration for whitespace:

local x: function 
(f)() -- syntax error, because this is parsed as `function(f)()`

Other than that, it’s just wild how inconsistent the syntax is compared to Lua:

Focus

Given such glaring issues in the core parts of the language – tl, I assume, initially stood for Typed Lua, and there are quite a few problems with both the “typed” part and the “Lua” part – one would hope that the maintenance efforts were focused on strengthening the foundation.

In practice, though, Teal tries to do so much more, with varying results:

Teal seems to be more of an experimentation ground than a reliable tool – it’s a kitchen sink of ideas, some better than others. It’s more of a general-purpose, all-around Lua preprocessor than a typed superset.

And that’s fine! Despite quite a few problems and limitations, I learned a lot from studying Teal, and I’m glad that it exists. It’s clearly useful at least to some people, and it removes a fair share of footguns. Maybe it’s not a great typing language, but it’s versatile enough that you can (mis)use it for some wild stuff.

But I wish I didn’t have to learn this lesson the hard way. Teal should advertise its purpose and capabilities better – in light of the above, the website and the documentation are frankly misleading:

Teal is a statically-typed dialect of Lua. It extends Lua with type annotations, allowing you to specify arrays, maps and records, as well as interfaces, union types and generics.

Yes, but: it also adds a ton of stuff unrelated to type annotations, and the listed type system features are underbaked.

It aims to fill a niche similar to that of TypeScript in the JavaScript world, but adhering to Lua’s spirit of minimalism, portability and embeddability.

Yes, but: transpilation requiring type checking is not TypeScript-like, a language with two macro systems is not minimalistic, and the bolted-upon compatibility module doesn’t enable portable semantics.

Here is a quick taste of what Teal code looks like:

local function add(a: number, b: number): number
   return a + b
end

local s = add(1, 2)
print(s)

Yes, but: realistic Teal code has type definitions and OOP patterns, which quickly get uglier than the example hints at.

Conclusion

I think Teal is having a bit of an identity crisis, but it refuses to admit that. We would all be better off if it made a decision on what it’s trying to grow into. That’s difficult, and perhaps not a priority for development – but I think it would do wonders for adoption and production readiness.