moe

SGLang是面向大语言模型与视觉语言模型的高速推理框架。

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

使用PEFT或全参数方法对500余个大语言模型和200余个多模态大语言模型进行CPT/SFT/DPO/GRPO训练

FlashInfer：大语言模型服务的核心计算库

一款开源简洁高颜值的酷狗第三方客户端 An open-source, concise, and aesthetically pleasing third-party client for KuGou that supports Windows / macOS / Linux / Web

反向图像搜索工具（SauceNao、IQDB、Ascii2D、trace.moe 等）

面向大语言模型的高吞吐量与内存优化型推理服务引擎