Squiggle 0.10.0

After a six-month development period, we’ve released Squiggle 0.10.0.

This version introduces important architectural improvements like more robust support for multi-model projects and two new kinds of compile-time type checks. These improvements will be particularly beneficial as laying a foundation for future updates.

This release also includes UI improvements, as well as several new functions and bugfixes.

Note: During this period, our development time was split between Squiggle and Squiggle AI (a separate project on top of Squiggle programming language). You can find out more about Squiggle AI in our recent EA forum post.

New project architecture

SqProject subsystem — the part of Squiggle that's responsible for orchestrating model runs, which can be quite complicated in case of multi-module programs — got a complete rewrite.

Pre-0.10, SqProject APIs were imperative, which caused issues at the boundary between Squiggle language and Squiggle React components. Sometimes this led to bugs and playground crashes, and sometimes it was preventing new features: it worked in simple cases, but wouldn't behave correctly if we tried to do something more advanced.

The new rewrite is quite radical in its functional approach, and is inspired by Git architecture. The new SqProject stores every version of Squiggle module on edits, and every version of Squiggle module outputs, as content-addressable immutable objects in a tree of dependencies. When the data is no longer needed, it gets garbage-collected.

If you're interested in more technical details on this, check out Multi-module projects in Squiggle documentation. In Squiggle playground, you can check out "Dependency Graph" viewer mode to see how the tree evolves when you do edits or add imports.

Web Worker runners by default

In Squiggle v0.9.4 and v0.9.5, we had an experimental "Web Worker" runner that you could enable in Playground settings.

In v0.10, it's enabled by default. SqProject rewrite is one change that allowed us to stabilize this, and the implementation of a reliable serializer for DAGs is another piece of the puzzle.

With Web Workers, all Squiggle code runs in a separate Web Worker thread, and the results are marshaled back to the main JS thread asynchronously. So, the UI should freeze much less often now.

More detailed documentation on runners can be found here.

Type inference

In v0.10, we for did some groundwork on compile-time type inference and semantic analysis.

Previously, Squiggle code execution went through this pipeline:

  1. Parse source code to AST
  2. Compile AST to Intermediate Representation
  3. Run the IR

Now we have one additional step between steps 1 and 2: transforming AST to typed AST. This step does type checks and gradual type inference.

Several consequences of this feature you might notice:

  1. In the playground, you'll see inferred types on hover. For now, this only works on top-level variables, but types for local variables in blocks and function definitions are all inferred too.
  2. If you define a function that's not correct semantically (e.g. f() = 1 + ""), that's never called, your program would fail now even if you never call the function in your code.
  3. Generally, semantically incorrect code will fail faster, because these checks are done at compile time.

For now, this feature has a few significant limitations, which we hope to improve in the future releases:

  1. There is no way to annotate value or variable types explicitly yet. All types are inferred implicitly. This is convenient — you'll never have to do extra work to satisfy the type system — but might be not enough in some situations, especially in case of functions.
  2. While we do have function parameter annotations, which we use for deciding how to render function charts and validate function parameters, this older feature is implemented separately from the type system. Parameter annotations don't get uplifted to the compile-time parameter types yet. In the future, function parameter annotations and compile-time types will be unified.
  3. Another big limitation is that while the type system we use supports generics (e.g., x = [1,2,3] will be assigned List(Number) type), we don't infer types correctly when a generic function is applied to a generic argument. For example, x = [1,2,3] -> map({|a|a}) will get the type List('B).

Playground improvements

On the React components side, the most noticeable change in Squiggle 0.10 is the new UI for the right side of the playground, output viewer.

Before (Squiggle 0.9.5)
After (Squiggle 0.10.0)

In addition to the visual changes, the variables in the viewer now default to the collapsed state. If you want to expand the variable by default, you can use @startOpen decorator.

Unit type annotations (Experimental)

Another feature, contributed to Squiggle by Michael Dickens, is unit type annotations.

Unit type annotations allow users to annotate variables with unit types, such as kilograms or dollars, or combinations of those, such as m/s^2.

Try in playground

Attempts to use incompatible types will fail:

Try in playground

Unit type annotations are documented here.

This feature is independent from the type system described above.

Note that this feature is experimental and might be deprecated or removed in the future in favor of some other syntax. Preliminary plans can be found in this GitHub issue.

Other changes

New standard library functions: List.sample, List.sampleN, Number.mod.

New standard library constants: Number.minValue and Number.maxValue.

Bugfixes and improvements:

Comments