CI/CD Pipelines Toolkit
Toolkit Track | 3 Modules | ~2.5 hours total
Overview
Section titled “Overview”The CI/CD Pipelines Toolkit covers modern pipeline orchestration beyond traditional CI systems. These tools represent the next generation of build and deployment automation—programmable, Kubernetes-native, and designed for complex workflows.
This toolkit builds on concepts from DevSecOps Discipline and complements the GitOps & Deployments Toolkit.
Prerequisites
Section titled “Prerequisites”Before starting this toolkit:
- DevSecOps Discipline — CI/CD concepts
- Container fundamentals
- Kubernetes basics
- Programming experience (for Dagger)
Modules
Section titled “Modules”| # | Module | Complexity | Time |
|---|---|---|---|
| 3.1 | Dagger | [COMPLEX] | 45-50 min |
| 3.2 | Tekton | [COMPLEX] | 45-50 min |
| 3.3 | Argo Workflows | [COMPLEX] | 40-45 min |
Learning Outcomes
Section titled “Learning Outcomes”After completing this toolkit, you will be able to:
- Write Dagger pipelines — Programmable, portable CI in Go/Python/TypeScript
- Build Tekton pipelines — Kubernetes-native Tasks and Pipelines
- Orchestrate Argo Workflows — DAG-based parallel job execution
- Choose the right tool — Understand trade-offs between approaches
Tool Selection Guide
Section titled “Tool Selection Guide”WHICH PIPELINE TOOL?─────────────────────────────────────────────────────────────────
"I want pipelines as code, testable and debuggable locally"└──▶ Dagger • Write in Go/Python/TypeScript • Run locally or in any CI • IDE support, type safety
"I need Kubernetes-native, YAML-based pipelines"└──▶ Tekton • Tasks and Pipelines as CRDs • Integrates with OpenShift • Catalog of reusable tasks
"I need complex DAGs and parallel data processing"└──▶ Argo Workflows • Sophisticated dependency graphs • ML/data pipeline focus • Handles thousands of pods
COMPARISON:───────────────────────────────────────────────────────────────── Dagger Tekton Argo Workflows─────────────────────────────────────────────────────────────────Language Code YAML YAMLRuns locally ✓ ✗ ✗K8s native ✗ ✓ ✓DAG support Basic Steps only Full DAGParallel loops ✓ Limited ✓✓ML workflows ✗ ✗ ✓✓Catalog/Hub Modules Hub ✗Learning curve Medium Medium HighThe Modern CI/CD Stack
Section titled “The Modern CI/CD Stack”┌─────────────────────────────────────────────────────────────────┐│ MODERN CI/CD STACK │├─────────────────────────────────────────────────────────────────┤│ ││ DEVELOPER PUSHES CODE ││ │ ││ ▼ ││ ┌──────────────────────────────────────────────────────────┐ ││ │ CI PIPELINE │ ││ │ │ ││ │ Option 1: DAGGER │ ││ │ • Portable, runs anywhere │ ││ │ • Write in real programming language │ ││ │ • Test locally before pushing │ ││ │ │ ││ │ Option 2: TEKTON │ ││ │ • Kubernetes-native │ ││ │ • Tasks as pods │ ││ │ • Triggers for webhooks │ ││ │ │ ││ │ Option 3: ARGO WORKFLOWS │ ││ │ • Complex DAGs │ ││ │ • Data processing │ ││ │ • ML pipelines │ ││ └────────────────────────────┬─────────────────────────────┘ ││ │ ││ │ Build artifact (container) ││ ▼ ││ ┌──────────────────────────────────────────────────────────┐ ││ │ CD PIPELINE │ ││ │ │ ││ │ ArgoCD / Flux │ ││ │ • Syncs from Git │ ││ │ • Progressive delivery │ ││ │ • GitOps workflow │ ││ └────────────────────────────┬─────────────────────────────┘ ││ │ ││ ▼ ││ ┌──────────────────────────────────────────────────────────┐ ││ │ KUBERNETES │ ││ └──────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────┘Study Path
Section titled “Study Path”Module 3.1: Dagger │ │ Programmable, portable pipelines ▼Module 3.2: Tekton │ │ Kubernetes-native CI/CD ▼Module 3.3: Argo Workflows │ │ DAG-based orchestration ▼[Toolkit Complete] → Security Tools ToolkitKey Concepts
Section titled “Key Concepts”Pipeline Paradigms
Section titled “Pipeline Paradigms”| Paradigm | Example | Best For |
|---|---|---|
| Declarative YAML | Tekton, GitHub Actions | Simple, standard pipelines |
| Programmable | Dagger | Complex logic, testing locally |
| DAG-based | Argo Workflows | Parallel processing, ML |
Common Patterns
Section titled “Common Patterns”PATTERN: Fan-out / Fan-in
┌──▶ Process 1 ──┐Input ─┼──▶ Process 2 ──┼──▶ Aggregate └──▶ Process 3 ──┘
Use: Parallel testing, data processingTool: Argo Workflows excels here
─────────────────────────────────────────────────────────────────
PATTERN: Sequential with Gates
Lint ──▶ Test ──▶ [Approval] ──▶ Build ──▶ Deploy
Use: Standard CI/CDTool: All three support this
─────────────────────────────────────────────────────────────────
PATTERN: Matrix Builds
┌──▶ Go 1.20 + LinuxSource ──┼──▶ Go 1.21 + Linux └──▶ Go 1.21 + Windows
Use: Cross-platform testingTool: Dagger and Argo WorkflowsRelated Tracks
Section titled “Related Tracks”- Before: DevSecOps Discipline — CI/CD concepts
- Related: IaC Tools Toolkit — Terraform for pipeline infrastructure
- Related: GitOps & Deployments Toolkit — Deploy what CI builds
- After: Security Tools Toolkit — Secure the pipeline
“The best pipeline is invisible—developers push code, users get features. These tools make that magic happen.”