MiniMax-2.5 Expert Offload and website generation

Measurements and generated sites

MiniMax-2.5 (229B MoE, 8 active experts out of 256) ran on 96GB VRAM with -ot exps=CPU. IQ5_K decoded at 34-37 tok/s. I also generated a React landing page with IQ4_NL and a dental clinic static site with IQ3_S.

Test Environment

Model Specifications

Part 1: Expert Offload Benchmark (IQ5_K)

Execution Command

  podman run --rm -it \
  --device nvidia.com/gpu=all \
  -p 8081:8080 \
  --shm-size 16g \
  --cap-add=SYS_NICE \
  -v "$MO":/models:ro,Z \
  $IMG \
  --host 0.0.0.0 --port 8080 \
  -m "$MODEL" \
  --no-mmap --jinja \
  -c 65536 \
  --threads 13 --threads-batch 25 \
  -b 2048 -ub 2048 \
  -ngl 99 \
  -ot exps=CPU \
  -ctk f16 -ctv f16 \
  --warmup-batch \
  -fa on
  

Key parameters:

• -ot exps=CPU: Force Expert weights (bulk of 157GB) to CPU memory
• -ngl 99: Shows 63/63 layers GPU-offloaded, but Experts override via -ot
• --no-mmap: Pin 157GB in physical RAM, eliminate page faults
• --warmup-batch: Run warmup on startup

Memory Layout

Actual GPU VRAM usage is ~21GB. The remaining 75GB is available for KV cache expansion.

Results: 8 Consecutive Runs (65K Context)

Extended Session (131K Context)

TG was 33.7-37.7 tok/s, without a sharp decline as context grew.

What Expert Offload showed

• Speed: TG varied about 10% across 8 runs, which I interpret as consistent PCIe rate-limiting.
• Offload display: “offloaded 63/63 layers to GPU” still leaves Expert weights on CPU through -ot.
• Prompt cache: 6,029 tokens used 1,460 MiB; 23,104 tokens used 5,596 MiB. Repeated runs shortened TTFT.
• Tokenizer warning: special_eos_id is not in special_eog_ids leaves a risk of unstable EOG / EOS handling.

One-Prompt Generation Live Demonstration

The video records website generation from one prompt.

Video link: https://www.youtube.com/watch?v=zisR4hPfT2c

It shows AGENTS.md input, token-by-token speed, and the generated code.

Part 2: React LP One-Shot Generation (IQ4_NL)

Configuration

• Quantization: IQ4_NL (lighter than IQ5_K, quality nearly maintained)
• Context: 262,144 (256K)
• Input: AGENTS.md (detailed site specification for loFT LLC)
• Stack: Vite + React + TypeScript + Tailwind CSS v4

Results

The model generated the loFT LLC corporate site from AGENTS.md.

Generated component structure (Atomic Design):

• Atoms: Button, Heading, Text, Icon, Input, TextArea, Badge
• Molecules: ServiceCard, CaseStudyCard, TestimonialCard, NavItem
• Organisms: NavBar, HeroSection, ServiceGrid, ProcessTimeline, ContactForm, Footer

Implemented features:

1. Hero section (animated background with function curves)
2. Contact form (react-hook-form + zod validation)
3. SEO optimization (JSON-LD structured data, dynamic meta tags, favicon generation)
4. SPA routing (react-router-dom)

Tailwind v4 repair After npx tailwindcss init failed, the model switched from the old tailwind.config.js approach to CSS-first @import "tailwindcss";.

TypeScript error handling: The first build attempt produced 18 TypeScript errors falling into predictable groups:

• Prop and routing mismatch (LegalPage designed one way, consumed another)
• import type required under verbatimModuleSyntax
• Callback typing inconsistencies (onClick?: () => void vs (e: any) => void)
• React.cloneElement typing issues
• Unused imports after refactoring

One repair cycle from the build logs made the build pass.

  vite v7.3.1 building client environment for production...
✓ 147 modules transformed.
dist/index.html                   0.46 kB │ gzip:   0.29 kB
dist/assets/index-BeGmd-zS.css   30.76 kB │ gzip:   5.99 kB
dist/assets/index-CCc5Lf0B.js   357.41 kB │ gzip: 111.66 kB
✓ built in 797ms
  

The session checked the dev server with lsof -i :5173, obtained HTML with curl, and updated title, description, OGP, and favicon. The generated web development artifact was ready for browser inspection.

Part 3: Dental Clinic Static Site (IQ3_S GPU Full Load)

Why IQ3_S

I traded some precision for more GPU-resident Expert weights and less -ot exps=CPU latency.

Requirements

• 6-page static HTML site (index, services, doctors, info, visit, access)
• No build step, Tailwind CSS (CDN), Alpine.js (CDN)

Results: 6 Pages in ~18 Minutes

• Total time: 18 minutes 2 seconds (design, implementation, diagnostics)
• Diagnostics: Zero errors and warnings
• Quality: Even at IQ3_S, accessibility (ESC key close, focus management) and semantic HTML structure were maintained

Generated Site Screenshots

Quantization-Level Operating Guidelines

These tasks also used lower quantization to place more weights on GPU.

Impressions from site generation

A 229B model ran locally at 37 tok/s. Independent experts make CPU placement an option when VRAM is insufficient. Both generated sites kept their multi-file structure consistent in this test.

Reproduction

Model Download

  huggingface-cli download <quantizer>/MiniMax-M2.5-GGUF \
  --include "MiniMax-M2.5-IQ5_K.gguf" \
  --local-dir /mnt/data/hf/hub/models--MiniMax-2.5-GGUF
  

Benchmark

Use the Part 1 command with ik_llama.cpp Expert Offload support.

Measurement

  curl -s http://localhost:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"minimax","messages":[{"role":"user","content":"Explain MoE architecture"}],"max_tokens":512}'
  

Technical Notes

Why –no-mmap

Loading 157GB with mmap produced intermittent page faults. --no-mmap reads it into RAM to avoid those inference delays, at the cost of several minutes of startup.

Expert count and Decode speed

MiniMax-2.5 selects 8 of 256 experts, versus GLM-4.7-Flash at 4 of 64. Selection costs more, but each expert is small (FFN 1,536 dimensions). Expert count alone does not determine Decode speed.