☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 30 days agoA SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformersgithub.comexternal-linkmessage-square0linkfedilinkarrow-up18arrow-down11cross-posted to: Aii@programming.devtechnology@lemmy.mlhackernews@lemmy.bestiver.selocalllama@sh.itjust.works
arrow-up17arrow-down1external-linkA SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformersgithub.com☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 30 days agomessage-square0linkfedilinkcross-posted to: Aii@programming.devtechnology@lemmy.mlhackernews@lemmy.bestiver.selocalllama@sh.itjust.works
