Squiggle AI: Better Annotation, Customization

We've been hard at work improving Squiggle AI since our last announcement. Our latest update introduces a style guide, flexible documentation steps, and improved workflows that make it easier to create clear, well-documented models.

New Style Guide & Step

One major challenge with the original version of Squiggle AI was that it produced models that were often difficult to read or understand. We've spent a while improving this.

We've created a new Style Guide for Squiggle code that's now part of the LLM prompts. You're also welcome to use this directly. The guide provides detailed recommendations for making estimates, assigning annotations, writing documentation, and more.

Adding this guide to our main prompts wasn't enough to ensure consistent execution, so we developed a new explicit step, shortly called documentation. We recommend running 2-4 steps to ensure proper variable annotation and documentation.

Customizable Steps

You can now customize the number of "numeric" and "documentation" steps:

  • Numeric steps: Review the model output, identify unreasonable elements, and adjust the model accordingly
  • Documentation steps: Apply the style guide

If either step produces broken code, automatic fixes will be implemented.

One trade-off of these steps is increased cost. If you request 2-3 steps of each, the total workflow could easily get to $0.20 to $0.30 or so.

If you have a bit of time, I recommend starting at zero for both. Once you are happy with a generation, run these steps on it afterwards.

Enhanced Workflows and Features

Previously, Squiggle AI had a "Fix" feature for Squiggle code. This has been replaced with an "Improve" workflow. This new workflow not only fixes code but also allows you to add numeric and documentation steps, making it easy to automatically enhance existing code.

If you have some Squiggle models that you'd like to see cleaned up, we suggest trying this.

We've also made numerous other improvements:

  • More polished UI
  • Smoother overall experience
  • String formatting for summarizing key results in explanatory notebooks
  • Various minor refinements

Examples

We've been saving a bunch of examples of Squiggle AI generated code here. The recent ones haven't been manually edited, and that often shows.

Going Forward

I think this is a good step towards automated modeling. At the same time, there's clearly a long way to go.

Squiggle AI still doesn't fetch online information, and it does a poor job at estimating individual parameters. It's not the best at coming up with great crucial considerations. Visualizations have proven quite fragile. We recommend using other AI tools, and your brain, for these things now.

Get Started

If you haven't tried Squiggle AI, this is a good time. We actively encourage people to experiment with it for various use cases. Run queries 2-5 times to get a variety of responses. Go crazy.

Comments