Save the Date!

September 4th, 2025 7pm CEST – Munich🥨NLP Discord Server.

About this Event

LLMs have shown to be capable of solving a wide range of tasks, yet their performance is surprisingly sensitive to prompt formulation. Manually tuning prompts can be time-consuming, and we lack clear guidelines on what works well. This makes automated prompt optimization an important tool for improving performance in a systematic and efficient way. In this talk, we introduce CAPO (Cost-Aware Prompt Optimization), a method that finds prompts achieving high performance while keeping costs low. CAPO has been shown to outperform existing prompt optimization techniques across various benchmarks, achieving substantial improvements such as a 25%p accuracy boost on GSM8K using a Llama-3.3-70B. To make these advances accessible to the broader community, we have developed Promptolution, an open-source framework for implementing and experimenting with prompt optimization approaches.

[PAPER]

Speakers

Tom Zehle is a Master’s student in Statistics & Data Science at LMU Munich and currently works as a Data Scientist at Airbus. He will begin his PhD this October at the ELLIS Institute in Tübingen, where his doctoral research will focus on specialization techniques for generalist foundation models, with a particular emphasis on prompt optimization, as well as finetuning and in-context learning. Together with Timo Heiß and Moritz Schlager, he co-developed CAPO and Promptolution, exploring practical and cost-efficient approaches to automated prompt optimization.