International Conference on Software Engineering for Adaptive and Self-Managing Systems

• ASTL: An Adaptive Serving Stack for LLMs to Balance Utilization and Tail Latency