Pipeline 设计与规范
本节介绍企业级 CI/CD 流水线的设计原则、架构模式、编码规范和最佳实践，帮助团队构建可靠、可维护、可扩展的持续交付系统。
设计原则
| 原则 |
说明 |
实践 |
| Declarative（声明式） |
描述期望状态而非步骤 |
使用 YAML/DSL 定义 Pipeline |
| Idempotent（幂等性） |
多次执行结果一致 |
独立 Stage，支持重跑 |
| Atomic（原子性） |
每个 Stage 独立成功或失败 |
不保存中间状态 |
| Fast Feedback |
快速发现问题 |
分层测试，优先快速检查 |
| Traceable（可追溯） |
记录每次变更的来龙去脉 |
版本化 Artifacts，元数据关联 |
| Secure（安全） |
最小权限，Secret 管理 |
IRSA/Vault，不明文存储密钥 |
流水线架构
经典三层流水线
┌──────────────────────────────────────────────────────────┐
│ CI Pipeline │
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │Checkout│──▶│ Build │──▶│ Test │──▶│ Scan │ │
│ └────────┘ └────────┘ └────────┘ └────────┘ │
│ │ │
│ ┌────────┐ ┌────────┐ ┌────────┐ │ │
│ │ Package│◀──│ Package│◀──│Package │ │ │
│ │ Image │ │ Chart │ │Bundle │ │ │
│ └────────┘ └────────┘ └────────┘ │ │
└────────────────────────────────────────────│──────────────┘
▼
┌──────────────────────────────────────────────────────────┐
│ CD Pipeline │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Staging │──▶│ PrePRD │──▶│ Production │ │
│ │ Auto-Deploy│ │ Gate/Auto │ │ Canary/% │ │
│ └────────────┘ └────────────┘ └────────────┘ │
└──────────────────────────────────────────────────────────┘
组件职责
| 组件 |
职责 |
技术选型 |
| Source |
代码拉取、依赖解析 |
GitLab CI/Tekton/Jenkins |
| Build |
编译、打包、镜像构建 |
Docker/Buildpacks |
| Test |
单元、集成、E2E 测试 |
JUnit/Pytest/Cypress |
| Security |
SAST/DAST/依赖扫描 |
Trivy/Snyk/Semgrep |
| Publish |
制品库、Helm Chart、Registry |
Harbor/ChartMuseum |
| Deploy |
环境部署、配置变更 |
ArgoCD/Flagger |
| Monitor |
质量门禁、性能监控 |
Prometheus/Grafana |
GitOps 流水线模式
模式一：应用流水线触发 GitOps
Code Commit → CI Pipeline → Build Image → Update Git Manifest → ArgoCD Sync
# CI Pipeline: 构建并更新 GitOps 仓库
stages:
- build
- push
- update-manifest
update-manifest:
stage: update-manifest
image: alpine/git
before_script:
- git clone https://github.com/example/gitops-repo.git
script:
- cd gitops-repo
- yq e '.spec.source.helm.parameters[0].value = "myapp:new-tag"' -i values/production.yaml
- git config user.email "[email protected]"
- git config user.name "CI Robot"
- git add values/production.yaml
- git commit -m "Update myapp to new-tag [skip ci]"
- git push
模式二：Trunk-based + GitOps
# Feature Branch
feature-xxx → build → test → (merge to main)
# Main Branch (Trunk)
main → build → test → integration-test → deploy-staging → deploy-production
Stage 设计规范
快速失败设计
stages:
- lint # 最快，< 1min
- unit-test # 快，< 5min
- build # 中等，< 10min
- security # 中等，< 5min
- e2e-test # 慢，< 30min
- deploy # 可变
条件执行
# 仅在特定分支执行
deploy:production:
stage: deploy
only:
- main
- tags
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: manual
# 仅在变更时执行（增量构建）
build:image:
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_COMMIT_BRANCH
changes:
- src/**
- Dockerfile
制品管理
版本号规范（SemVer）
MAJOR.MINOR.PATCH[-prerelease]+buildmetadata
1.0.0
1.0.0-alpha.1
1.0.0+20240101.1
Docker 镜像 Tag 策略
| 环境 |
Tag 策略 |
示例 |
| Dev |
latest |
myapp:latest |
| Feature |
Branch name |
myapp:feature-xxx |
| Staging |
Commit SHA |
myapp:a1b2c3d4 |
| Production |
Semantic Tag |
myapp:v1.2.3 |
Helm Chart 版本管理
# Chart.yaml
apiVersion: v2
name: myapp
version: 1.2.3
appVersion: "1.2.3"
环境策略
环境类型
| 环境 |
用途 |
触发方式 |
回滚 |
| Dev |
开发调试 |
Push 触发 |
自动 |
| Test |
功能测试 |
PR 触发 |
自动 |
| Staging |
预生产验证 |
Merge 触发 |
自动/手动 |
| Pre-Production |
灰度验证 |
Tag 触发 |
手动 |
| Production |
最终发布 |
审批后 |
手动 |
环境隔离策略
# Kubernetes Namespace 隔离
---
apiVersion: v1
kind: Namespace
metadata:
name: dev
labels:
environment: dev
---
apiVersion: v1
kind: Namespace
metadata:
name: staging
labels:
environment: staging
---
apiVersion: v1
kind: Namespace
metadata:
name: production
labels:
environment: production
测试策略
测试金字塔
┌─────────┐
│ E2E │ ← 少量，30min
┌─────────┐┌─────────┐
│Integration│ ← 中等，1h
┌─────────┐┌─────────┐┌─────────┐
│ Unit │ ← 大量，5min
└─────────┘└─────────┘└─────────┘
测试配置示例
test:unit:
stage: test
script:
- make test:unit
coverage: '/Coverage: \d+\.\d+%/'
artifacts:
reports:
junit: reports/unit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura.xml
test:integration:
stage: test
services:
- postgres:15
- redis:7
- elasticsearch:8
script:
- make test:integration
artifacts:
reports:
junit: reports/integration.xml
test:e2e:
stage: test
image: cypress/base:18
script:
- npm install
- npm run cypress:run
artifacts:
when: always
paths:
- cypress/videos/
- cypress/screenshots/
安全门禁
扫描矩阵
| 扫描类型 |
工具 |
检查内容 |
阶段 |
| SAST |
Semgrep |
代码安全 |
Build |
| DAST |
OWASP ZAP |
运行时漏洞 |
Deploy |
| SCA |
Trivy/Snyk |
依赖漏洞 |
Build |
| Container |
Trivy/Hadolint |
镜像漏洞 |
Build |
| IaC |
Checkov/Tfsec |
基础设施安全 |
Build |
质量门禁配置
quality-gate:
stage: verify
image: curlimages/curl:latest
script: |
# 质量门禁检查
COVERAGE=$(cat coverage-summary.json | jq '.total.lines.percent')
if (( $(echo "$COVERAGE < 80" | bc -l) )); then
echo "Coverage $COVERAGE% is below threshold 80%"
exit 1
fi
VULNERABILITIES=$(cat trivy-report.json | jq '.Summary.Critical + .Summary.High')
if (( VULNERABILITIES > 0 )); then
echo "Found $VULNERABILITIES critical/high vulnerabilities"
exit 1
fi
allow_failure: false
通知与集成
Slack 通知
notify:slack:
stage: notify
image: curlimages/curl:latest
script: |
if [ "$CI_COMMIT_BRANCH" == "main" ]; then
curl -X POST -H 'Content-Type: application/json' -d "{"text":"✅ Build passed: ${CI_PROJECT_NAME} ${CI_COMMIT_SHA:0:8}"}" $SLACK_WEBHOOK_URL
fi
only:
- main
when: on_success
质量数据上报
quality-report:
stage: report
image: node:20-alpine
before_script:
- npm install -g sonar-scanner
script:
- sonar-scanner \
-Dsonar.projectKey=${CI_PROJECT_NAME} \
-Dsonar.host.url=$SONAR_HOST \
-Dsonar.token=$SONAR_TOKEN \
-Dsonar.branch.name=$CI_COMMIT_REF_NAME
only:
- main
- merge_requests
故障排查
# 本地调试（GitLab CI）
gitlab-runner exec docker --docker-privileged build
# 本地调试（Jenkins）
jenkins-cli -s http://localhost:8080 build <job-name> -s
# Tekton 本地调试
tkn pipelinerun logs <pipelinerun-name> -n default -f
# 查看流水线历史
kubectl get pipelinerun -n default
kubectl describe pipelinerun <pipelinerun-name> -n default
下一步