Summary & Key Takeaways ​

• Simon Willison launched llm-typesafe, a new plugin for his llm CLI tool.
• The plugin integrates with TypeSafe AI's Jev model for structured outputs.
• It enables asking yes/no ('noul') questions with confidence scores.
• Users can define choice questions with specific criteria for categorization.
• Scoring questions allow for evaluating responses against a defined scale.
• This tool aims to make LLM interactions more reliable and programmatic.

Our Commentary ​

This is exactly the kind of tooling we need to make LLMs truly useful in applications. Moving beyond free-form text to structured, type-safe outputs for yes/no, choices, and scores is a huge leap for reliability and developer experience. I've been waiting for something like this to make LLM integration less 'fuzzy.' Simon Willison continues to deliver incredibly practical tools for the AI ecosystem. I'm already thinking about how I can use this.