Module 4.1: DevSecOps Fundamentals

Discipline Module | Complexity: [MEDIUM] | Time: 30-35 min

Prerequisites

Before starting this module:

Required: Security Principles Track — Defense in depth, least privilege
Required: Basic CI/CD concepts (build, test, deploy pipelines)
Recommended: GitOps Track — Modern deployment practices
Helpful: Experience with security scanning tools

What You’ll Be Able to Do

After completing this module, you will be able to:

Evaluate your organization’s security posture to identify gaps in the DevSecOps lifecycle
Design a DevSecOps strategy that integrates security into every phase of software delivery
Implement security champion programs that distribute security knowledge across development teams
Analyze the cost of security remediation at each lifecycle stage to justify shift-left investments

Why This Module Matters

Your team ships code fast. Daily. Maybe multiple times a day.

Then the security team shows up. They’ve been auditing for three weeks. They have 47 findings. Your velocity just hit a wall.

This is the old world.

DevSecOps exists because security at the end doesn’t scale. When you find a vulnerability two weeks after the code was written, the developer has moved on. Context is lost. Fixing is expensive. Arguing begins.

What if security was continuous, automated, and embedded in how you already work?

After this module, you’ll understand:

Why DevSecOps emerged and what problem it solves
The shift-left philosophy and its practical implications
How security integrates into CI/CD pipelines
The DevSecOps culture shift beyond tooling

The Evolution: Security’s Journey

The Waterfall Era (1970s-1990s)

Security was a phase. Late in the project.

Requirements → Design → Development → Testing → SECURITY REVIEW → Deployment
                                                     │
                                              "Here are 200 issues.
                                               Good luck."

Problems:

Findings came too late to fix properly
Security team seen as blockers
Developers didn’t learn security thinking
Compliance was a yearly audit nightmare

The Agile Era (2000s-2010s)

Agile sped up development. Security didn’t keep up.

Sprint 1 → Sprint 2 → Sprint 3 → ... → Sprint N → SECURITY REVIEW
                                                        │
                                                 "You have 3 weeks
                                                  of security debt
                                                  per sprint."

What changed:

Faster delivery, same security process
Security debt accumulated rapidly
Penetration tests found the same issues repeatedly
“Security can’t keep up” became normal

The DevOps Era (2010s)

DevOps merged development and operations. Security watched from outside.

┌─────────────────────────────────────────┐
│           DevOps Infinity Loop           │
│                                          │
│     Plan → Code → Build → Test →        │
│      ↑                        ↓          │
│     ← Monitor ← Operate ← Deploy ←      │
│                                          │
└─────────────────────────────────────────┘
                     │
              Where's security?
                     │
                     ▼
              Standing outside,
              waving the compliance
              checklist

The result:

Speed to production: excellent
Security posture: questionable
Compliance: scrambling
Trust between teams: strained

The DevSecOps Era (2015-Present)

Security moves into the loop, not after it.

┌─────────────────────────────────────────┐
│         DevSecOps Infinity Loop          │
│                                          │
│   Plan → Code → Build → Test →          │
│    ↑    [SEC]  [SEC]   [SEC]   ↓        │
│    ← Monitor ← Operate ← Deploy ←       │
│      [SEC]     [SEC]     [SEC]           │
│                                          │
└─────────────────────────────────────────┘

SEC = Security checks embedded at every stage

The transformation:

Security is everyone’s responsibility
Automated checks replace manual reviews
Vulnerabilities found in minutes, not months
Security team becomes enablers, not blockers

The Shift-Left Philosophy

What “Shift Left” Means

Imagine your development pipeline as a timeline:

LEFT                                                            RIGHT
─────────────────────────────────────────────────────────────────▶
 │         │           │          │            │           │
 │         │           │          │            │           │
Code     Build       Test      Deploy      Runtime    Production
Written  Artifacts  Verified   Staged     Monitoring   Incidents

Shift left means moving security activities earlier (left) in this timeline.

Why Left Is Better

The cost of fixing a security issue increases exponentially as it moves right:

                                                    ┌─────────────┐
                                                    │ Production  │
                                              ┌─────┤    $1M+     │
                                              │     └─────────────┘
                                              │
                                        ┌─────┴───┐
                                        │ Runtime │
                                  ┌─────┤  $100K  │
                                  │     └─────────┘
                                  │
                            ┌─────┴───┐
                            │ Deploy  │
                      ┌─────┤  $10K   │
                      │     └─────────┘
                      │
                ┌─────┴───┐
                │  Test   │
          ┌─────┤  $1000  │
          │     └─────────┘
          │
    ┌─────┴───┐
    │  Build  │
    │   $100  │
    └────┬────┘
         │
   ┌─────┴───┐
   │  Code   │
   │   $10   │
   └─────────┘

Real numbers from IBM’s System Sciences Institute:

Bug found during design: $1 to fix
Bug found during coding: $6.5 to fix
Bug found during testing: $15 to fix
Bug found in production: $100 to fix

For security vulnerabilities, the multipliers are even higher due to:

Breach costs
Regulatory fines
Reputation damage
Incident response

Shift Left in Practice

Stage	Traditional Security	Shift-Left Security
Code	No checks	IDE plugins, pre-commit hooks, secrets detection
Build	Maybe SAST	SAST, SCA, container scanning, IaC scanning
Test	Pen test at end	DAST in pipeline, fuzzing, API testing
Deploy	Manual review	Policy as code, admission control
Runtime	Hope for the best	Threat detection, runtime protection

Did You Know?

The term “DevSecOps” appeared around 2012, but gained momentum after the 2017 Equifax breach. That breach—caused by an unpatched Apache Struts vulnerability—exposed 147 million people’s data and cost over $1.4 billion. The vulnerability had a patch available for two months before the breach.
The average time to detect a breach is 207 days (IBM Cost of a Data Breach Report 2023). DevSecOps aims to find vulnerabilities in minutes, not months. Organizations with fully deployed security AI and automation identified breaches 108 days faster.
“Rugged Software” was an early name for what became DevSecOps. The Rugged Manifesto (2010) declared: “I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic and national security.”
Netflix pioneered “Security Monkey” in 2014—one of the first tools to automatically detect security misconfigurations in AWS. They open-sourced it, influencing the entire DevSecOps movement toward automated security monitoring.

The DevSecOps Pipeline

Anatomy of a Secure Pipeline

┌─────────────────────────────────────────────────────────────────┐
│                     DEVSECOPS PIPELINE                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  PRE-COMMIT          BUILD              TEST                    │
│  ┌──────────┐       ┌──────────┐       ┌──────────┐            │
│  │ Secrets  │       │  SAST    │       │  DAST    │            │
│  │ scanning │       │ scanning │       │ scanning │            │
│  ├──────────┤       ├──────────┤       ├──────────┤            │
│  │ Linting  │       │   SCA    │       │   IAST   │            │
│  │ (security│       │ (deps)   │       │          │            │
│  │  rules)  │       ├──────────┤       └──────────┘            │
│  └──────────┘       │  Image   │                               │
│                     │ scanning │                               │
│                     └──────────┘                               │
│                                                                 │
│  DEPLOY              RUNTIME            CONTINUOUS              │
│  ┌──────────┐       ┌──────────┐       ┌──────────┐            │
│  │ Config   │       │ Runtime  │       │Compliance│            │
│  │ scanning │       │ security │       │ scanning │            │
│  ├──────────┤       ├──────────┤       ├──────────┤            │
│  │ Policy   │       │  Threat  │       │ Audit    │            │
│  │ checks   │       │ detection│       │ logging  │            │
│  └──────────┘       └──────────┘       └──────────┘            │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Security Testing Types

Type	Full Name	What It Does	When
SAST	Static Application Security Testing	Analyzes source code without running it	Build
SCA	Software Composition Analysis	Checks dependencies for known vulnerabilities	Build
DAST	Dynamic Application Security Testing	Tests running application from outside	Test
IAST	Interactive Application Security Testing	Monitors application during testing	Test
RASP	Runtime Application Self-Protection	Protects running application	Runtime

How They Complement Each Other

┌─────────────────────────────────────────────────────────────┐
│                   SECURITY TESTING COVERAGE                  │
│                                                              │
│  YOUR CODE            DEPENDENCIES         RUNNING APP       │
│  ┌──────────┐        ┌──────────┐        ┌──────────┐       │
│  │          │        │          │        │          │       │
│  │   SAST   │        │   SCA    │        │   DAST   │       │
│  │          │        │          │        │   IAST   │       │
│  │ SQL injection     │ Log4Shell │        │   RASP   │       │
│  │ XSS patterns │    │ known CVEs│        │          │       │
│  │ Hardcoded     │   │ outdated  │        │ Auth bypass│     │
│  │ secrets       │   │ licenses  │        │ Business   │     │
│  │               │   │           │        │ logic flaws│     │
│  └──────────┘        └──────────┘        └──────────┘       │
│                                                              │
│        ◀── Static (code) ──▶   ◀── Dynamic (running) ──▶    │
└─────────────────────────────────────────────────────────────┘

None alone is sufficient. Each catches different vulnerability types:

SAST finds code issues but can’t see runtime behavior
DAST finds runtime issues but can’t see code
SCA finds known vulnerabilities but not zero-days
Use them together for defense in depth

War Story: The $0 Breach Prevention

A fintech company I consulted for was about to ship a major release. Typical story: tight deadline, pressure from the top, “we’ll fix security later.”

The Before:

Their pipeline:

Developer pushes code
Unit tests run
Deploy to staging
Manual QA
Deploy to production
(Quarterly security review, maybe)

The Incident That Almost Was:

Three days before launch, someone finally suggested adding Trivy to scan their container images. “Just in case.”

The scan found:

2 critical vulnerabilities in their base image
1 high severity in a dependency
Their Redis container was running as root
An exposed debug endpoint in the Java app

The Fix (hours, not weeks):

Updated base image: 30 minutes
Updated dependency: 15 minutes
Fixed Dockerfile: 10 minutes
Removed debug endpoint: 5 minutes

Total cost: A few hours of developer time.

If found in production: Potential breach, incident response, disclosure, regulatory scrutiny.

The After:

They implemented:

Pre-commit hooks (secrets scanning)
SAST in build stage
Container scanning before push
Policy checks before deploy

Three months later:

Pipeline catches 5-10 issues per week
Average fix time: 20 minutes (developer still has context)
Zero security incidents
Security team now reviews design, not hunting for SQL injections

The Culture Shift

DevSecOps isn’t just tools. It’s a fundamental shift in who owns security.

The Old Model: Security as Gatekeeper

Developers ──▶ Build Features ──▶ Throw Over Wall ──▶ Security Reviews
                                          │
                                   "Not my problem
                                    until they find
                                    something"

Characteristics:

Security = separate team
Review = end of cycle
Finding = blame game
Knowledge = siloed
Developers = don’t learn security
Security team = overwhelmed

The New Model: Security as Enabler

┌─────────────────────────────────────────────────────────┐
│                    SHARED OWNERSHIP                      │
│                                                          │
│  Developers        Security         Operations          │
│  ┌──────────┐     ┌──────────┐     ┌──────────┐        │
│  │ Write    │     │ Build    │     │ Deploy   │        │
│  │ secure   │ ←── │ tooling  │ ──▶ │ securely │        │
│  │ code     │     │ & guides │     │          │        │
│  └──────────┘     └──────────┘     └──────────┘        │
│       │                │                │               │
│       └────────────────┴────────────────┘               │
│                        │                                │
│               Shared responsibility                     │
│               Shared knowledge                          │
│               Shared success                            │
└─────────────────────────────────────────────────────────┘

Characteristics:

Security = everyone’s responsibility
Review = continuous, automated
Finding = learning opportunity
Knowledge = shared
Developers = security-aware
Security team = consultants, architects, enablers

Making the Shift

From	To
”Security said no"	"Let’s find a secure way"
"They’ll scan it later"	"Scan before I push"
"That’s the security team’s job"	"Security is my job too"
"We’ll fix it in the next release"	"Fix it now, it’s cheaper"
"Good enough for staging"	"Production-ready from the start”

Principles for Success

Principle 1: Automate Everything

Manual security reviews don’t scale. Automation does.

Manual                              Automated
──────                              ─────────
- Review once (end)                 - Check every commit
- Inconsistent                      - Consistent
- Slow                              - Fast
- Expensive                         - Cheap (after setup)
- Limited coverage                  - Comprehensive
- Security team bottleneck          - Security team enabled

Automate:

Secret detection (pre-commit)
Static analysis (build)
Dependency scanning (build)
Container scanning (build)
IaC scanning (build)
Policy enforcement (deploy)

Principle 2: Fail Fast, Fix Fast

Don’t accumulate security debt. Address issues immediately.

Issue Age vs. Fix Difficulty
────────────────────────────

Time since introduced    │                            ╱
                         │                          ╱
                         │                        ╱
  Difficulty to fix      │                    ╱
                         │                ╱
                         │            ╱
                         │        ╱
                         │    ╱
                         │╱
                         └─────────────────────────────────
                          Minutes  Hours  Days  Weeks  Months

Break the build on critical issues.
Don't let issues age.

Principle 3: Security as Code

Treat security the same way you treat application code.

# Security policies in code (OPA/Rego example)
package kubernetes.admission

deny[msg] {
    input.request.kind.kind == "Pod"
    not input.request.object.spec.securityContext.runAsNonRoot
    msg := "Pods must not run as root"
}

Benefits:

Version controlled (history, rollback)
Peer reviewed (PRs for policy changes)
Tested (policy unit tests)
Documented (self-documenting code)
Auditable (Git history)

Principle 4: Feedback Loops

Developers need immediate, actionable feedback.

Bad feedback:

Security Scan Report
Page 1 of 47
...

Good feedback:

❌ Build Failed: High Severity Vulnerability

File: src/auth/login.go:42
Issue: SQL Injection vulnerability
Code:  query := "SELECT * FROM users WHERE id = " + userId

Fix: Use parameterized queries
Example:
  query := "SELECT * FROM users WHERE id = ?"
  db.Query(query, userId)

Docs: https://security.company.com/sql-injection

Common Mistakes

Mistake	Problem	Solution
Tool overload	Too many tools, alert fatigue	Start with 2-3 essential tools, add gradually
Breaking builds for everything	Developers bypass security	Critical = break, High = warn, tune over time
No developer training	Tools find issues, devs don’t understand	Invest in security champions, training
Security theater	Check boxes without value	Measure what matters (MTTR, escape rate)
Ignoring false positives	Trust erodes	Tune aggressively, suppress known good
One-time implementation	Tools become outdated	Treat like product: maintain, improve

Quiz: Check Your Understanding

Question 1

A developer finds a SQL injection vulnerability in their code. The code was written 3 weeks ago and is now in production. Why is this more expensive to fix than if it was caught during development?

Show Answer

Cost multipliers for production vulnerabilities:

Context loss — Developer has moved on to other work, needs to reload context
Potential exposure — May already have been exploited
Incident response — Need to assess if exploitation occurred
Emergency change — Hotfix requires expedited testing, deployment
Disclosure — May need to notify customers if data exposed
Regulatory — GDPR, PCI, etc. may require notification
Reputation — Trust impact with customers

If caught at development:

Developer has full context
No exposure
Normal change process
No disclosure needed
No reputation impact

This is why shift-left is so valuable: $10 at code time vs $1000+ at runtime.

Question 2

Your SAST tool reports 200 findings on a legacy codebase. The team is overwhelmed and wants to disable the tool. What should you do?

Show Answer

Don’t disable — tune and prioritize:

Triage by severity:
- Critical/High: Must fix, block new code with these patterns
- Medium: Track, address over time
- Low: Consider suppressing
Focus on new code:
- Establish baseline of existing issues
- Block only new issues from being introduced
- Legacy becomes “accepted risk” to address over time
Suppress known false positives:
- Create suppression rules with justification
- Review suppressions periodically
Enable incrementally:
- Start with a few high-value rules
- Add more as team capacity allows
Create security debt backlog:
- Track legacy issues like tech debt
- Address alongside feature work

The goal: 100% of new code scanned, 0 new critical issues, legacy addressed incrementally.

Question 3

Why is SCA (Software Composition Analysis) particularly important for modern applications?

Show Answer

Modern apps are mostly dependencies:

According to studies:

80-90% of code in applications is from dependencies
Average app has 200-500 direct dependencies
Each dependency has transitive dependencies

Why SCA matters:

You didn’t write it — Can’t review it like your own code
Known vulnerabilities exist — CVE databases track them
Attackers know too — Log4Shell was exploited within hours
Transitive risk — Your dependency’s dependency’s vulnerability is your vulnerability
License risk — GPL in your commercial product?
Supply chain attacks — Malicious packages uploaded to registries

Real examples:

Log4Shell (CVE-2021-44228): Affected millions of applications
Event-Stream: Popular npm package hijacked
ua-parser-js: Supply chain attack on 8M weekly downloads

SCA catches what SAST can’t: known vulnerabilities in code you didn’t write.

Question 4

What’s the difference between DevSecOps and just “adding security tools to CI/CD”?

Show Answer

Tools alone aren’t DevSecOps. The difference is culture and outcomes.

Just adding tools:

Security tools run in pipeline
Security team configures and reviews
Developers ignore/bypass when blocking
Findings accumulate as backlog
“We have security scanning” (checkbox)

True DevSecOps:

Security is shared responsibility
Developers own and understand findings
Fast feedback, fast fixes
Continuous improvement
Security team enables, doesn’t gatekeep
Metrics show improvement over time

Key indicators of true DevSecOps:

Developers fix issues without security team
Mean time to remediate (MTTR) decreasing
New vulnerabilities caught before prod
Security team consults on design, not hunting bugs
Security included in sprint planning
Champions in every team

DevSecOps is a culture shift with tools. Tools without culture is just expensive compliance.

Hands-On Exercise: Your First Security Scan

Experience the DevSecOps pipeline firsthand by scanning a vulnerable application.

Part 1: Setup Vulnerable Application

# Create a project with intentional vulnerabilities
mkdir devsecops-intro && cd devsecops-intro

# Create a Python app with security issues
cat > app.py << 'EOF'
import os
import sqlite3

# VULNERABILITY: Hardcoded credentials
DB_PASSWORD = "super_secret_123"
API_KEY = "sk_live_1234567890abcdef"

def get_user(user_id):
    conn = sqlite3.connect('users.db')
    # VULNERABILITY: SQL Injection
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return conn.execute(query).fetchone()

def run_command(cmd):
    # VULNERABILITY: Command Injection
    os.system(f"echo {cmd}")

if __name__ == "__main__":
    print(get_user(input("Enter user ID: ")))
EOF

# Create requirements with vulnerable dependencies
cat > requirements.txt << 'EOF'
requests==2.25.0
pyyaml==5.3.1
urllib3==1.26.4
EOF

Part 2: Run SAST Scan with Bandit

# Install Bandit (Python SAST tool)
pip install bandit

# Scan the application
bandit -r . -f txt

# Expected output should show:
# - Hardcoded password (B105)
# - SQL injection (B608)
# - Shell injection (B605)

Part 3: Run SCA Scan with pip-audit

# Install pip-audit
pip install pip-audit

# Scan dependencies
pip-audit -r requirements.txt

# You should see CVEs for:
# - requests (CVE-2023-32681)
# - pyyaml (CVE-2020-14343)
# - urllib3 (multiple CVEs)

Part 4: Run All-in-One Scan with Trivy

# Install Trivy (if not installed)
# macOS: brew install trivy
# Linux: see https://aquasecurity.github.io/trivy

# Scan filesystem for all vulnerability types
trivy fs --severity HIGH,CRITICAL .

# Trivy finds:
# - Vulnerable dependencies (SCA)
# - Secrets in code
# - Misconfigurations

Part 5: Fix the Issues

# Fix the code issues
cat > app_fixed.py << 'EOF'
import os
import sqlite3

def get_user(user_id):
    conn = sqlite3.connect('users.db')
    # FIXED: Parameterized query
    query = "SELECT * FROM users WHERE id = ?"
    return conn.execute(query, (user_id,)).fetchone()

def run_command(cmd):
    # FIXED: Use subprocess with list args
    import subprocess
    # Only allow specific commands
    allowed = ['date', 'whoami']
    if cmd in allowed:
        subprocess.run([cmd], check=True)

if __name__ == "__main__":
    # FIXED: Get credentials from environment
    db_password = os.getenv("DB_PASSWORD")
    api_key = os.getenv("API_KEY")
    print(get_user(input("Enter user ID: ")))
EOF

# Fix dependencies
cat > requirements_fixed.txt << 'EOF'
requests>=2.31.0
pyyaml>=6.0.1
urllib3>=2.0.7
EOF

# Re-run scans to verify fixes
bandit -r app_fixed.py -f txt
pip-audit -r requirements_fixed.txt

Success Criteria

Ran Bandit and identified 3+ code vulnerabilities
Ran pip-audit and found vulnerable dependencies
Ran Trivy for comprehensive scanning
Fixed code vulnerabilities (parameterized queries, no hardcoded secrets)
Updated dependencies to patched versions
Re-ran scans and verified fixes

Key Takeaways

DevSecOps shifts security left — Find issues early when they’re cheap to fix
Automation is essential — Manual reviews don’t scale with modern delivery velocity
Culture before tools — Shared responsibility matters more than tool selection
Multiple testing types — SAST, SCA, DAST each catch different vulnerabilities
Fast feedback, fast fixes — Developers fix what they just wrote, not what they forgot

Summary

DevSecOps integrates security into every stage of software delivery. Instead of security as a gate at the end, it’s embedded throughout:

Pre-commit: Catch secrets and obvious issues
Build: SAST, SCA, container scanning
Test: DAST, IAST
Deploy: Policy enforcement
Runtime: Threat detection

The shift from “security reviews code” to “developers own security” requires culture change, not just tools. Organizations that embrace DevSecOps ship faster and more securely because finding issues early is faster and cheaper than finding them in production.

Next Module

Continue to Module 4.2: Shift-Left Security to learn specific techniques for catching vulnerabilities early in development.

“Security is not a feature. It’s a property of the system — and building it in is cheaper than bolting it on.”

Module 4.1: DevSecOps Fundamentals

Prerequisites

What You’ll Be Able to Do

Why This Module Matters

The Evolution: Security’s Journey

The Waterfall Era (1970s-1990s)

The Agile Era (2000s-2010s)

The DevOps Era (2010s)

The DevSecOps Era (2015-Present)

The Shift-Left Philosophy

What “Shift Left” Means

Why Left Is Better

Shift Left in Practice

Did You Know?

The DevSecOps Pipeline

Anatomy of a Secure Pipeline

Security Testing Types

How They Complement Each Other

War Story: The $0 Breach Prevention

The Culture Shift

The Old Model: Security as Gatekeeper

The New Model: Security as Enabler

Making the Shift

Principles for Success

Principle 1: Automate Everything

Principle 2: Fail Fast, Fix Fast

Principle 3: Security as Code

Principle 4: Feedback Loops

Common Mistakes

Quiz: Check Your Understanding

Question 1

Question 2

Question 3

Question 4

Hands-On Exercise: Your First Security Scan

Part 1: Setup Vulnerable Application

Part 2: Run SAST Scan with Bandit

Part 3: Run SCA Scan with pip-audit

Part 4: Run All-in-One Scan with Trivy

Part 5: Fix the Issues

Success Criteria

Key Takeaways

Further Reading

Summary

Next Module