The team behind continuous batching says your idle GPUs should be running inference, not sitting dark

FriendliAI — founded by the researcher behind continuous batching, the technique at the core of vLLM — is launching InferenceSense, a platform that fills idle neocloud GPU capacity with paid AI ...

Qwen3.5 family: Fireworks of new LLMs from Alibaba

In the last few days, Qwen then set off real fireworks with new models. Qwen started with the large models Qwen3.5-122B-A10B, ...
GitHub

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration [

AWQ search for accurate quantization. Pre-computed AWQ model zoo for LLMs (LLaMA-1&2, OPT, Vicuna, LLaVA; load to generate quantized weights). Memory-efficient 4-bit Linear in PyTorch. Efficient CUDA ...
GitHub

[AAAI 2024 (Oral)] OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models

This is the code for the paper [OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models](https://arxiv.org/abs/2306. ...
IEEE

GausiQ: Generalized Automatic Hybrid-Precision Quantization for MIMO Detection

Abstract: Automatic quantization generates efficient hybrid precision quantization schemes without manual effort, offering a promising approach for developing hardware-friendly MIMO detectors. However ...
IEEE

Vector Quantization With Error Uniformly Distributed Over an Arbitrary Set

Abstract: For uniform scalar quantization, the error distribution is approximately a uniform distribution over an interval (which is also a 1-dimensional ball ...