Open Models & Local Inference

Open Models & Local Inference | 7 modules | ~14-20 hours

Overview

This section teaches the practical open-model path for learners who want to move beyond hosted chat tools.

The goal is not to turn every learner into an infrastructure engineer on day one.

The goal is to help learners understand:

• where open models come from
• how to evaluate them responsibly
• how to run them on Apple Silicon or Linux boxes
• how quantization changes what is realistic on real hardware
• how to choose between runtimes without turning local inference into cargo cult

This is still part of the top-level AI track.

It sits between AI Building and AI for Kubernetes & Platform Work, before the deeper AI/ML Engineering sections.

Modules

Module	Topic
1.1	Open Models and Model Hubs
1.2	Hugging Face for Learners
1.3	Quantization and Model Formats
1.4	MLX on Apple Silicon
1.5	Running Open Models on Linux Boxes
1.6	Choosing Between Ollama, MLX, Transformers, and vLLM
1.7	Gemma 4 and the Open-Model Landscape

Outcome

By the end of this section, you should be able to:

• explain the difference between open-model access and closed API use
• read a model card without treating it like marketing copy
• understand why quantization changes hardware requirements and quality tradeoffs
• choose a sane local runtime for Apple Silicon or Linux
• know when local inference is enough and when deeper AI/ML engineering is needed

What This Section Covers vs What It Hands Off

This section covers:

• learner-facing model hubs
• model cards and practical model choice
• quantization basics
• local runtime choice
• Apple Silicon and Linux local paths

It does not try to duplicate the deeper engineering modules for:

• reproducible CUDA/ROCm environment management
• production inference architecture
• serious model serving internals
• fine-tuning workflows

Those move to:

• AI/ML Engineering: Prerequisites
• AI/ML Engineering: AI Infrastructure
• AI/ML Engineering: Advanced GenAI & Safety

Recommended Route

AI Foundations
   |
AI-Native Work
   |
AI Building
   |
Open Models & Local Inference
   |
AI for Kubernetes & Platform Work
   |
AI/ML Engineering (optional deeper path)

After This Section

Choose the next route based on your goal:

Goal	Next Step
use AI safely in Kubernetes and platform workflows	AI for Kubernetes & Platform Work
build practical local-first AI apps	AI/ML Engineering: AI-Native Development
build retrieval-backed applications	AI/ML Engineering: Vector Search & RAG
study model behavior more deeply	AI/ML Engineering: Generative AI
operate local or private serving seriously	AI/ML Engineering: AI Infrastructure