How to Build a CI/CD Pipeline From Scratch: Beginner Guide

Build Your First CI/CD Pipeline From Zero

Building your first CI/CD pipeline can feel like a daunting task, especially when workplace systems are complex and risky to experiment with. However, the core principles are remarkably straightforward: a pipeline is simply an automated process that installs dependencies, runs tests, and builds your application in a clean, reproducible environment . This guide will walk you through the fundamental concepts and practical steps you need to how to build a ci/cd pipeline from scratch, transforming manual, error-prone deployments into a reliable, automated workflow.

You don't need Kubernetes or a production-scale system to practice CI/CD. Start with a small project, a Git repository, and a CI tool like GitHub Actions, GitLab CI, or Jenkins. The most critical step is ensuring your build is reproducible locally before you write a single line of pipeline configuration.

The Foundation: Reproducibility Is Key

Before you write your first pipeline file, your project must be able to build cleanly from scratch. If this fails on a fresh clone of your repository, it will fail in CI .

Step 1: Make Your Local Build Reproducible

The golden rule is to ensure your application builds successfully from a clean slate on your local machine. For example:

• Node.js: npm ci && npm test && npm run build
• Python: pip install -r requirements.txt && pytest
• Go: go test ./... && go build

Step 2: Commit Lock Files

Lock files (like package-lock.json, go.sum, or Pipfile.lock) are essential for deterministic builds. Without them, your CI pipeline might install different dependency versions than you used locally, leading to unpredictable test failures. In Node.js, npm ci enforces the lock file exactly, preventing this "dependency drift" .

Writing Your First Pipeline

With a reproducible build, you can now automate it. A basic pipeline has three stages: Build, Test, and Deploy .

Step 3: Choose Your CI/CD Tool

Your choice often depends on where your code lives. Here are three common options:

Tool	Best For	Pipeline Definition	Self-Hosted Option
GitHub Actions	Projects already on GitHub, public repos (free).	.github/workflows/*.yml	Yes (Self-hosted runners)
GitLab CI	All-in-one DevOps platform with Git hosting.	.gitlab-ci.yml	Yes (GitLab CE)
Jenkins	Highly customizable, mature ecosystem, complex build logic.	Jenkinsfile (Groovy)	Yes (via Java, plugin-based)

Step 4: Create a Minimal Pipeline Configuration

The pipeline definition is a YAML or Groovy file in your repository root that tells the CI runner what to do.

Example: .gitlab-ci.yml (GitLab CI) This defines a pipeline that builds a Go application, saves the binary as an artifact, and then executes it in a later stage .

default:
  image: golang

stages:
  - build
  - run

build go:
  stage: build
  script:
    - go build
  artifacts:
    paths: 
      - array

run go:
  stage: run
  script:
    - ./array

Example: Jenkinsfile (Declarative Pipeline) This does the same thing, using a Docker image to run the Node.js app .

pipeline {
    agent { docker { image 'node:20-alpine' } }
    stages {
        stage('Build') {
            steps { sh 'npm ci && npm run build' }
        }
        stage('Test') {
            steps { sh 'npm run test' }
        }
    }
}

⚠️ Security Warning: Never hardcode environment variables like database URLs or API keys in your repository. Use your CI tool's built-in secrets manager (GitLab CI Variables, Jenkins Credentials, or GitHub Actions Secrets) to inject them into your jobs .

Sharing Data Between Stages

A key feature of modern pipelines is the ability to share data between jobs. Without this, you would have to rebuild your application in every stage, wasting time and resources.

Step 5: Use Artifacts

Artifacts are files or directories saved from one job and made available to subsequent jobs. In the GitLab CI example, the build go job saves the array binary. The run go job then downloads that binary automatically, using it without needing to recompile .

Automating Deployments (Optional)

Once CI is working, you can add continuous deployment. The principle is simple: deployments should only happen after tests pass .

Step 6: Implement a Basic Deployment Strategy

For simple applications, an SSH command can copy files to your server.

**Example: Deploy with rsync **

deploy:
  stage: deploy
  image: alpine:latest
  only:
    - main
  script:
    - apk add --no-cache openssh-client rsync
    - rsync -avz --delete dist/ deploy@yourserver:/var/www/app/

If you're using containers, the deploy stage typically builds a Docker image, pushes it to a registry, and then uses SSH to trigger a docker compose pull && docker compose up -d on the production server .

Practicing and Troubleshooting

Reading about CI/CD is not enough. To truly learn, you must run pipelines and deliberately break them .

Step 7: Intentionally Break Things to Learn

This is where real understanding happens. Try these experiments:

1. Remove a dependency declaration. Does CI fail? Why?
2. Delete the lock file. Do version drifts break the build?
3. Add a flaky test. Does it pass locally but fail in CI? This often exposes differences in the clean CI environment .

Step 8: Use Test Reports and Logs

CI should provide visibility into which tests failed, which are slow, and which are flaky . Use the logs not just to find errors, but to understand the entire process: how long does dependency installation take? Where exactly does the build fail?

Frequently Asked Questions

How long does it take to build my first CI/CD pipeline? For a simple project, you can have a basic pipeline running in under an hour. The GitLab CI hands-on lab estimates 15 minutes for a foundational pipeline . The time is usually spent writing the configuration file and troubleshooting reproducibility issues.

Should I choose Jenkins, GitHub Actions, or GitLab CI for a beginner? If your code is on GitHub, GitHub Actions is a natural choice. For an all-in-one platform, GitLab CI is excellent. Jenkins is incredibly powerful but has a steeper learning curve and requires more maintenance, so it's often better for larger, more complex setups .

What is the difference between an artifact and a cache in CI/CD? Artifacts are used to pass files between stages (e.g., passing a compiled binary from the build stage to the deploy stage). Caches are used to speed up builds by reusing files like dependencies (node_modules) across different pipeline runs .

Why do my tests pass locally but fail in the CI pipeline? This is usually a sign of environment drift. Your local machine may have different software versions or environment variables. This is why commit lock files are crucial and why running your project from a clean clone locally is a necessary first step .

Do I need Docker to build a CI/CD pipeline? Not strictly, but it is highly recommended. Running jobs inside Docker containers ensures isolation and reproducibility. It guarantees that each job runs in a clean, known environment, preventing jobs from interfering with each other or polluting the host server .

— Editorial Team