Paper

Twice as Fast, Until It Isn't: Quantized Serving of Mid-Size Open LLMs · ByteTrail