Local & On-Device AI Tools

Learners run capable AI models entirely on their own hardware with Ollama and LM Studio. The course covers model selection, performance realities, and when local AI's privacy advantages justify its limits.

What this course covers

The course runs across 9 topics, each one a short adaptive session rather than a recorded lecture. The tutor explains the idea, works an example, checks understanding, and adjusts the next step to the answer given.

The Architecture of Local LLMs and Hardware Demands — Examine CPU vs. GPU execution, VRAM constraints, and hardware architectures necessary for running multi-billion parameter models locally.
Privacy, Compliance, and Cost Tradeoffs — Analyze the strategic decisions of self-hosting vs. cloud APIs, focusing on data governance, compliance (HIPAA/GDPR), and long-term cost modeling.
Deploying Models with Ollama CLI — Install Ollama, manage model lifecycles via CLI, and run your first open-weight model entirely offline.
Customizing Models with Ollama Modelfiles — Write and build custom Modelfiles to define system prompts, adjust temperature parameters, and set stop tokens for specialized local assistants.
Visual Model Management with LM Studio — Navigate the LM Studio interface to search Hugging Face, download GGUF models, configure hardware settings, and use the local playground.
Model Selection: Quantization and Formats — Evaluate quantization methods (e.g., Q4_K_M vs. Q8) and formats like GGUF, balancing model accuracy against memory footprint.
Integrating Local Models via OpenAI-Compatible APIs — Write Python scripts to query your locally running Ollama and LM Studio instances using standardized API endpoints.
Performance Tuning and Benchmarking — Measure tokens-per-second performance, configure GPU offloading layers, and optimize context window sizes to prevent system bottlenecks.
Building a Fully Private Local RAG Pipeline — Connect a local embedding model and a local LLM to query a private document database without external API calls.

Skills you build

Results are measured against named skills in the OneRange taxonomy of more than 10,000 skills, so a manager sees proficiency per skill rather than a completion tick.

Who it is for

Technical users, IT, privacy-sensitive teams. The material is pitched at advanced level, and takes roughly 120 minutes at a typical pace. Because every session adapts, someone who already knows a topic moves through it quickly instead of sitting through an explanation they do not need.

It sits in the Tool: Local AI track of the Vero AI Catalog, and can be assigned to one person, a team, or the whole company.

How it is delivered and assessed

Delivery is conversational and interactive, including guided lab exercises. There is no video to sit through and no slide deck to click past.

Understanding is checked with a tool assessment of 5 items, with a pass mark of 70%.

Administrators can copy this course into their own library and adapt it — edit the outline, change the duration, swap the assessment format, or ground it in internal documentation so answers cite the company's own source material.

Also in this track

Model Engineering & Customization Pathway — Choose, run, and customize models beyond prompting.

Frequently asked questions

How long does this course take?

It runs roughly 120 minutes at a typical pace. Because every session adapts, someone who already knows a topic moves through it quickly instead of sitting through an explanation they do not need.

How is it assigned to a team?

An administrator assigns it from the Vero dashboard to one person, a team, a department or the whole company, and sees progress and results per person and per skill.

Can we customise it with our own documents and terminology?

Yes. Administrators can copy this course into their own library and adapt it — edit the outline, change the duration, swap the assessment format, or ground it in internal documentation so answers cite the company's own source material.

Course code TUL-LOC-101.