Summary & Key Takeaways ​

• Introduces "Chartography," a new benchmark for AI.
• Focuses on professional chart understanding and reasoning.
• Includes complex chart types like Kaplan-Meier curves and candlesticks.
• Benchmarks are written and graded by domain experts.
• Aims to evaluate multimodal AI models' visual comprehension.

Our Commentary ​

This is a much-needed benchmark. AI's ability to understand complex charts, especially domain-specific ones, is a huge gap. We've all seen models struggle with simple graphs, let alone Kaplan-Meier curves. Having domain experts grade these is the right approach. It's about real-world utility, not just synthetic scores.