OneRange

Small & Efficient Models

This course covers the economics and technique of small models: what they can genuinely do, distillation concepts, on-device deployment, and routing between small and frontier models by task.

What this course covers

The course runs across 10 topics, each one a short adaptive session rather than a recorded lecture. The tutor explains the idea, works an example, checks understanding, and adjusts the next step to the answer given.

  • The Landscape and Economics of Small Models — Examine the cost, latency, and capability trade-offs of modern small models (e.g., Llama-3-8B, Phi-3) compared to frontier models, focusing on advanced parameter-efficient architectures.
  • Evaluating Capabilities: Small vs. Frontier Models — Analyze task complexity to determine if a small model can perform the job. Learn to design custom evaluation benchmarks to test capability boundaries.
  • Core Concepts of Knowledge Distillation — Deep dive into teacher-student frameworks, understanding Kullback-Leibler (KL) divergence, logit-based distillation, and feature-based distillation techniques.
  • Hands-on Distillation: Training a Student Model — Set up and run a distillation pipeline using PyTorch and Hugging Face, fine-tuning a small student model using outputs from a frontier teacher model.
  • Model Compression: Quantization and Pruning — Explore post-training quantization (PTQ) techniques (AWQ, GPTQ) and structured pruning to dramatically reduce memory footprint without losing accuracy.
  • Optimizing and Deploying On-Device — Package and deploy optimized models to edge/on-device environments using execution engines like llama.cpp, ONNX Runtime, and TensorRT-LLM.
  • Architecting Hybrid Routing Systems — Design routing architectures that dynamically direct user queries to either a fast, cheap small model or a powerful, expensive frontier model.
  • Implementing a Dynamic Router — Write a lightweight routing classifier using semantic embeddings and historical query performance data to route requests in real-time.
  • Scenario: Managing Cost vs. Accuracy in Production — Navigate a real-world scenario balancing strict latency budgets, API cost constraints, and accuracy requirements by tuning router thresholds and model fallbacks.
  • Observability and Drift in Routing Systems — Establish monitoring pipelines to track routing accuracy, detect task drift, and continuously evaluate when to trigger model re-distillation.

Skills you build

Results are measured against named skills in the OneRange taxonomy of more than 10,000 skills, so a manager sees proficiency per skill rather than a completion tick.

Who it is for

ML engineers, cost-conscious builders. The material is pitched at advanced level, and takes roughly 120 minutes at a typical pace. Because every session adapts, someone who already knows a topic moves through it quickly instead of sitting through an explanation they do not need.

It sits in the Category: LLMs track of the Vero AI Catalog, and can be assigned to one person, a team, or the whole company.

How it is delivered and assessed

Delivery is conversational and interactive, including branching scenario, code lab, guided lab exercises. There is no video to sit through and no slide deck to click past.

Understanding is checked with a lab assessment of 5 items, with a pass mark of 70%.

Administrators can copy this course into their own library and adapt it — edit the outline, change the duration, swap the assessment format, or ground it in internal documentation so answers cite the company's own source material.

Also in this track

  • Reasoning Models in Practice — A practical course on reasoning-class models: what extended thinking buys you, when it is worth the cost and latency, and how to prompt them well. Learners compare standard and reasoning models on real tasks.
  • Large Language Models Deep Dive — This course gives a deeper understanding of how large language models work so learners can use and choose them well. It covers architecture and tokens, training and alignment, key parameters, and model selection tradeoffs. Participants finish able to make informed decisions about which model to use and why.
  • Model Engineering & Customization Pathway — Choose, run, and customize models beyond prompting.

Related

Frequently asked questions

How long does this course take?

It runs roughly 120 minutes at a typical pace. Because every session adapts, someone who already knows a topic moves through it quickly instead of sitting through an explanation they do not need.

How is it assigned to a team?

An administrator assigns it from the Vero dashboard to one person, a team, a department or the whole company, and sees progress and results per person and per skill.

Can we customise it with our own documents and terminology?

Yes. Administrators can copy this course into their own library and adapt it — edit the outline, change the duration, swap the assessment format, or ground it in internal documentation so answers cite the company's own source material.

Course code CAT-SLM.