LLMs

Large language models (LLMs), APIs, agents, and harnesses.

Models and recipes

• Hugging face
• Unsloth
• Club 3090 : recipes for 3090/4090/5090 owners.
• MiaAI-Lab : various recipes for popular models.
• exllamav3 : ExLlamaV3 is an inference library for running local LLMs on modern consumer GPUs.

Qwen3.8-27B

• Unsloth
• ISTA: GSQ-RCO : small quants
• MiaAI-Lab (SPARK) on one DGX Spark.
• MiaAI-Lab (16GB) for one 16-24 GB GPU.
• OrcaSAQ-2-27B : High-fidelity 3-bit Qwen3.8 for long-horizon agents.
• Swift-1.5-Qwen3.8-27b : UkisAI’s reasoning-efficient derivative of Qwen3.8-27B.

Qwen3.8-Flash-Next

• Unsloth
• MiaAI-Lab (SPARK) on one DGX Spark. (NVFP4 quant)

Gemma4

• Gemma4-31B-QAT
• Gemma4-26B-A4B-QAT

Speech recognition and text-to-speech

• VoxCPM

Document processing

• docling

Biology

• evo2

Runtime

• FreeToken : Optimized for MoE models.
• Unsloth desktop
• Lemonade for AMD GPUs.
• Lucebox
• ninfer : C++/CUDA inference engine for explicitly registered Qwen checkpoints on a single NVIDIA GeForce RTX 5090.

llama.cpp

• llama.cpp GitHub repo.
• beellama.cpp : llama.cpp fork supporting KVarN KV cache format.
• ik_llama.cpp : llama.cpp fork with new quants and improved performance for MoE models.

vLLM

• vllm GitHub repo.
• vllm-radiance docker image for dual AMD R9700’s.
• spark-vllm-docker : vLLM docker images for DGX sparks.

SGLang

• SGLang GitHub repo.
• SGLang Cookbook

Agents and harnesses

• Codewhale
• Deepseek harness (DSH)
• deepseek reasonix : cache-friendly harness for DeepSeek.
• Hermes agent
• Minimax code
• OpenCode
• pi coding agent

MCP servers

Model Context Protocol (MCP)

• DuckDuckGo
• Julia mcp
• Kaimon.jl : MCP server giving AI agents full access to Julia’s runtime via a live Gate.