Autofy: Automated Synthesis of Formally Verified Code
Although large language models (LLMs) are becoming increasingly capable at coding, ensuring the formal correctness of their output remains a major challenge. We present Autofy, a system that orchestrates multiple LLM roles to synthesize SPARK/Ada programs that (i) compile, (ii) pass formal verification and (iii) satisfy functional tests. Autofy integrates Alire for project bootstrap, GNAT/GCC and gnatpp for style/syntax/semantics checks and compilation, GNATprove for formal verification, as well as functional tests to ensure guard-railed results. Two complementary flows are supported: a \emph{Multi-Version} approach that cross-combines independent contract and implementation variants, and a \emph{Director-Feedback} approach in which a Director LLM reads compiler/prover/test logs and emits targeted edits. In evaluations across multiple datasets and models, Autofy solved up to 31% (\emph{Multi-Version}) / 71% (\emph{Director-Feedback}) of SPARK coding tasks - substantially outperforming naive prompting (5%). We discuss system design, empirical results, and open challenges for combining LLMs with sound software engineering and formal methods.
Sun 12 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | Session 2: Verification, Synthesis & Automated ReasoningResearch Track / FormaliSE Program at Oceania VIII | ||
14:00 30mTalk | Autofy: Automated Synthesis of Formally Verified Code Research Track Nils Purschke Technical University of Munich, André Schamschurko Technical University of Munich, Sven Kirchner Technical University of Munich, Chengdong Wu Technical University of Munich, Alois Knoll Technical University of Munich | ||
14:30 30mTalk | Bounded Synthesis of Synchronized Distributed Models from Lightweight Specifications Research Track Pablo Castro Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Luciano Putruele National University of Rio Cuarto, Renzo Degiovanni Luxembourg Institute of Science and Technology, Nazareno Aguirre University of Rio Cuarto/CONICET, Argentina, and Guangdong Technion-Israel Institute of Technology, China | ||
15:00 15mTalk | TensorEgg: Formally Verifying Rewrite Rules and Applying Them to Tensor Programs Research Track Akash Kothari University of Illinois at Urbana-Champaign, Dhruv Baronia University of Illinois at Urbana-Champaign | ||
15:15 15mTalk | VeriROS: Verifiable ROS2 Navigation Execution Framework Research Track | ||