Running warmup..
Timing with prompt_tokens=1024, generation_tokens=512, batch_size=1.
Trial 1:  prompt_tps=466.746, generation_tps=17.777, peak_memory=16.991, total_time=31.008
Trial 2:  prompt_tps=469.500, generation_tps=17.784, peak_memory=16.991, total_time=30.986
Trial 3:  prompt_tps=468.553, generation_tps=17.600, peak_memory=16.992, total_time=31.291
Trial 4:  prompt_tps=447.236, generation_tps=17.379, peak_memory=16.992, total_time=31.765
Trial 5:  prompt_tps=444.815, generation_tps=17.255, peak_memory=16.993, total_time=31.988
Averages: prompt_tps=459.370, generation_tps=17.559, peak_memory=16.992