316 lines
6.9 KiB
Markdown
316 lines
6.9 KiB
Markdown
# Build Server Setup Guide
|
|
|
|
Complete guide for setting up a dedicated build VPS for MotoVaultPro CI/CD pipeline.
|
|
|
|
## Overview
|
|
|
|
The build server isolates resource-intensive Docker builds from the production server, ensuring deployments don't impact application performance.
|
|
|
|
```
|
|
+-------------------+ +--------------------+
|
|
| GitLab Server | | Production Server |
|
|
| (CI/CD + Registry)| | (Shell Runner) |
|
|
+--------+----------+ +----------+---------+
|
|
| |
|
|
v v
|
|
+--------+----------+ +----------+---------+
|
|
| Build VPS | | Blue-Green Stacks |
|
|
| (Docker Runner) |---->| + Shared Data |
|
|
+-------------------+ +--------------------+
|
|
```
|
|
|
|
## Server Requirements
|
|
|
|
### Minimum Specifications
|
|
|
|
| Resource | Requirement |
|
|
|----------|-------------|
|
|
| CPU | 2 cores |
|
|
| RAM | 4GB |
|
|
| Storage | 50GB SSD |
|
|
| Network | 100Mbps+ |
|
|
| OS | Ubuntu 22.04 LTS / Debian 12 |
|
|
|
|
### Network Requirements
|
|
|
|
- Outbound HTTPS to GitLab instance
|
|
- Outbound HTTPS to Docker registries (for fallback)
|
|
- SSH access for administration
|
|
|
|
---
|
|
|
|
## Installation Steps
|
|
|
|
### 1. Update System
|
|
|
|
```bash
|
|
sudo apt update && sudo apt upgrade -y
|
|
sudo apt install -y curl git ca-certificates gnupg
|
|
```
|
|
|
|
### 2. Install Docker Engine
|
|
|
|
```bash
|
|
# Add Docker's official GPG key
|
|
sudo install -m 0755 -d /etc/apt/keyrings
|
|
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
|
sudo chmod a+r /etc/apt/keyrings/docker.gpg
|
|
|
|
# Add the repository to Apt sources
|
|
echo \
|
|
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
|
|
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
|
|
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
|
|
|
# Install Docker
|
|
sudo apt update
|
|
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
|
|
|
# Verify installation
|
|
docker --version
|
|
docker compose version
|
|
```
|
|
|
|
### 3. Install GitLab Runner
|
|
|
|
```bash
|
|
# Add GitLab Runner repository
|
|
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
|
|
|
|
# Install GitLab Runner
|
|
sudo apt install gitlab-runner
|
|
|
|
# Verify installation
|
|
gitlab-runner --version
|
|
```
|
|
|
|
### 4. Register Runner with Shell Executor
|
|
|
|
```bash
|
|
sudo gitlab-runner register \
|
|
--non-interactive \
|
|
--url "https://git.motovaultpro.com" \
|
|
--registration-token "YOUR_REGISTRATION_TOKEN" \
|
|
--executor "shell" \
|
|
--description "Build Server - Shell Executor" \
|
|
--tag-list "build" \
|
|
--run-untagged="false" \
|
|
--locked="true"
|
|
```
|
|
|
|
**Notes:**
|
|
- Replace `YOUR_REGISTRATION_TOKEN` with the token from GitLab Admin > CI/CD > Runners
|
|
- Shell executor runs jobs directly on the host with access to Docker
|
|
- Tag `build` is used in `.gitlab-ci.yml` to route build jobs to this server
|
|
|
|
### 5. Add gitlab-runner to Docker Group
|
|
|
|
The gitlab-runner user needs access to Docker:
|
|
|
|
```bash
|
|
sudo usermod -aG docker gitlab-runner
|
|
|
|
# Verify access
|
|
sudo -u gitlab-runner docker info
|
|
sudo -u gitlab-runner docker compose version
|
|
```
|
|
|
|
### 6. Configure Docker Registry Authentication
|
|
|
|
Create credentials file for GitLab Container Registry:
|
|
|
|
```bash
|
|
# Login to GitLab Container Registry (creates ~/.docker/config.json)
|
|
docker login registry.motovaultpro.com -u <deploy-token-username> -p <deploy-token>
|
|
```
|
|
|
|
**Creating Deploy Token:**
|
|
1. Go to GitLab Project > Settings > Repository > Deploy Tokens
|
|
2. Create token with `read_registry` and `write_registry` scopes
|
|
3. Use the token username/password for Docker login
|
|
|
|
---
|
|
|
|
## Verification
|
|
|
|
### Test Runner Registration
|
|
|
|
```bash
|
|
sudo gitlab-runner verify
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
Verifying runner... is alive runner=XXXXXX
|
|
```
|
|
|
|
### Test Docker Access
|
|
|
|
```bash
|
|
sudo gitlab-runner exec docker --docker-privileged test-job
|
|
```
|
|
|
|
### Test Registry Push
|
|
|
|
```bash
|
|
# Build and push a test image
|
|
docker build -t registry.motovaultpro.com/motovaultpro/test:latest -f- . <<EOF
|
|
FROM alpine:latest
|
|
RUN echo "test"
|
|
EOF
|
|
|
|
docker push registry.motovaultpro.com/motovaultpro/test:latest
|
|
```
|
|
|
|
---
|
|
|
|
## Maintenance
|
|
|
|
### Disk Cleanup
|
|
|
|
Docker builds accumulate disk space. Set up automated cleanup:
|
|
|
|
```bash
|
|
# Create cleanup script
|
|
sudo tee /usr/local/bin/docker-cleanup.sh > /dev/null <<'EOF'
|
|
#!/bin/bash
|
|
# Remove unused Docker resources older than 7 days
|
|
docker system prune -af --filter "until=168h"
|
|
docker volume prune -f
|
|
EOF
|
|
|
|
sudo chmod +x /usr/local/bin/docker-cleanup.sh
|
|
|
|
# Add to crontab (run daily at 3 AM)
|
|
echo "0 3 * * * /usr/local/bin/docker-cleanup.sh >> /var/log/docker-cleanup.log 2>&1" | sudo crontab -
|
|
```
|
|
|
|
### Log Rotation
|
|
|
|
Configure log rotation for GitLab Runner:
|
|
|
|
```bash
|
|
sudo tee /etc/logrotate.d/gitlab-runner > /dev/null <<EOF
|
|
/var/log/gitlab-runner/*.log {
|
|
daily
|
|
rotate 7
|
|
compress
|
|
missingok
|
|
notifempty
|
|
}
|
|
EOF
|
|
```
|
|
|
|
### Update Runner
|
|
|
|
```bash
|
|
# Update GitLab Runner
|
|
sudo apt update
|
|
sudo apt upgrade gitlab-runner
|
|
|
|
# Restart runner
|
|
sudo gitlab-runner restart
|
|
```
|
|
|
|
---
|
|
|
|
## Security Considerations
|
|
|
|
### Firewall Configuration
|
|
|
|
```bash
|
|
# Allow only necessary outbound traffic
|
|
sudo ufw default deny incoming
|
|
sudo ufw default allow outgoing
|
|
sudo ufw allow ssh
|
|
sudo ufw enable
|
|
```
|
|
|
|
### Runner Security
|
|
|
|
- **Locked runner**: Only accepts jobs from the specific project
|
|
- **Protected tags**: Only runs on protected branches (main)
|
|
- **Docker socket**: Mounted read-only where possible
|
|
|
|
### Secrets Management
|
|
|
|
The build server does NOT store application secrets. All secrets are:
|
|
- Stored in GitLab CI/CD Variables
|
|
- Injected at runtime on the production server
|
|
- Never cached in Docker layers
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Runner Not Picking Up Jobs
|
|
|
|
```bash
|
|
# Check runner status
|
|
sudo gitlab-runner status
|
|
|
|
# View runner logs
|
|
sudo journalctl -u gitlab-runner -f
|
|
|
|
# Re-register runner if needed
|
|
sudo gitlab-runner unregister --all-runners
|
|
sudo gitlab-runner register
|
|
```
|
|
|
|
### Docker Build Failures
|
|
|
|
```bash
|
|
# Check Docker daemon
|
|
sudo systemctl status docker
|
|
|
|
# Check available disk space
|
|
df -h
|
|
|
|
# Clear Docker cache
|
|
docker system prune -af
|
|
```
|
|
|
|
### Registry Push Failures
|
|
|
|
```bash
|
|
# Verify registry login
|
|
docker login registry.motovaultpro.com
|
|
|
|
# Check network connectivity
|
|
curl -v https://registry.motovaultpro.com/v2/
|
|
|
|
# Verify image exists
|
|
docker images | grep motovaultpro
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### Important Paths
|
|
|
|
| Path | Description |
|
|
|------|-------------|
|
|
| `/etc/gitlab-runner/config.toml` | Runner configuration |
|
|
| `/var/log/gitlab-runner/` | Runner logs |
|
|
| `~/.docker/config.json` | Docker registry credentials |
|
|
| `/var/lib/docker/` | Docker data |
|
|
|
|
### Common Commands
|
|
|
|
```bash
|
|
# Runner management
|
|
sudo gitlab-runner status
|
|
sudo gitlab-runner restart
|
|
sudo gitlab-runner verify
|
|
|
|
# Docker management
|
|
docker system df # Check disk usage
|
|
docker system prune -af # Clean all unused resources
|
|
docker images # List images
|
|
docker ps -a # List containers
|
|
|
|
# View build logs
|
|
sudo journalctl -u gitlab-runner --since "1 hour ago"
|
|
```
|