egullickson/motovaultpro
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

CI/CD Gitea v1.0
Some checks failed
Deploy to Staging / Build Images (push) Failing after 7s
Details
Deploy to Staging / Deploy to Staging (push) Has been skipped
Details
Deploy to Staging / Verify Staging (push) Has been skipped
Details
Deploy to Staging / Notify Staging Ready (push) Has been skipped
Details
Deploy to Staging / Notify Staging Failure (push) Failing after 6s
Details

Browse Source
This commit is contained in:
Eric Gullickson
2025-12-29 18:51:41 -06:00
parent 9b0de6a5b8
commit 83d79da3aa
15 changed files with 1101 additions and 929 deletions
Download Patch File Download Diff File Expand all files Collapse all files

335
docs/BUILD-SERVER-SETUP.md
View File

@@ -1,22 +1,28 @@
# Build Server Setup Guide
# Build/Staging Server Setup Guide
Complete guide for setting up a dedicated build VPS for MotoVaultPro CI/CD pipeline.
Complete guide for setting up the build and staging server for MotoVaultPro CI/CD with Gitea Actions.
## Overview
The build server isolates resource-intensive Docker builds from the production server, ensuring deployments don't impact application performance.
The build server serves dual purposes:
1. **Build Server**: Builds Docker images and pushes to Gitea Package Registry
2. **Staging Server**: Runs full application stack at staging.motovaultpro.com
```
+-------------------+ +--------------------+
| GitLab Server | | Production Server |
| (CI/CD + Registry)| | (Shell Runner) |
+--------+----------+ +----------+---------+
| |
v v
+--------+----------+ +----------+---------+
| Build VPS | | Blue-Green Stacks |
| (Docker Runner) |---->| + Shared Data |
+-------------------+ +--------------------+
| Gitea Server | | Production Server |
| git.motovaultpro | | (mvp-prod runner) |
| + Package Registry| +----------+---------+
+--------+----------+ |
| v
v motovaultpro.com
+--------+----------+
| Build/Staging VPS |
| (mvp-build runner)|
+-------------------+
|
v
staging.motovaultpro.com
```
## Server Requirements
@@ -25,16 +31,16 @@ The build server isolates resource-intensive Docker builds from the production s
| Resource | Requirement |
|----------|-------------|
| CPU | 2 cores |
| RAM | 4GB |
| Storage | 50GB SSD |
| CPU | 4 cores |
| RAM | 8GB |
| Storage | 100GB SSD |
| Network | 100Mbps+ |
| OS | Ubuntu 22.04 LTS / Debian 12 |
### Network Requirements
- Outbound HTTPS to GitLab instance
- Outbound HTTPS to Docker registries (for fallback)
- Port 80/443 open (for staging.motovaultpro.com)
- Outbound HTTPS to git.motovaultpro.com
- SSH access for administration
---
@@ -45,7 +51,7 @@ The build server isolates resource-intensive Docker builds from the production s
```bash
sudo apt update && sudo apt upgrade -y
sudo apt install -y curl git ca-certificates gnupg
sudo apt install -y curl git ca-certificates gnupg jq
```
### 2. Install Docker Engine
@@ -56,7 +62,7 @@ sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the repository to Apt sources
# Add the repository
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
@@ -71,95 +77,162 @@ docker --version
docker compose version
```
### 3. Install GitLab Runner
### 3. Install act_runner
```bash
# Add GitLab Runner repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
# Install GitLab Runner
sudo apt install gitlab-runner
# Download act_runner binary
curl -L https://gitea.com/gitea/act_runner/releases/download/v0.2.12/act_runner-0.2.12-linux-amd64 -o /tmp/act_runner
sudo mv /tmp/act_runner /usr/local/bin/act_runner
sudo chmod +x /usr/local/bin/act_runner
# Verify installation
gitlab-runner --version
act_runner --version
```
### 4. Register Runner with Shell Executor
### 4. Create act_runner User
```bash
sudo gitlab-runner register \
--non-interactive \
--url "https://git.motovaultpro.com" \
--registration-token "YOUR_REGISTRATION_TOKEN" \
--executor "shell" \
--description "Build Server - Shell Executor" \
--tag-list "build" \
--run-untagged="false" \
--locked="true"
# Create user for running act_runner
sudo useradd -r -m -s /bin/bash act_runner
sudo usermod -aG docker act_runner
# Create config directory
sudo mkdir -p /etc/act_runner
sudo chown act_runner:act_runner /etc/act_runner
```
**Notes:**
- Replace `YOUR_REGISTRATION_TOKEN` with the token from GitLab Admin > CI/CD > Runners
- Shell executor runs jobs directly on the host with access to Docker
- Tag `build` is used in `.gitlab-ci.yml` to route build jobs to this server
### 5. Register Runner with Gitea
### 5. Add gitlab-runner to Docker Group
The gitlab-runner user needs access to Docker:
Get a registration token from: `git.motovaultpro.com/egullickson/motovaultpro/settings/actions/runners`
```bash
sudo usermod -aG docker gitlab-runner
# Generate config
sudo -u act_runner act_runner generate-config > /etc/act_runner/config.yaml
# Verify access
sudo -u gitlab-runner docker info
sudo -u gitlab-runner docker compose version
# Register runner with staging/build label
sudo -u act_runner act_runner register --no-interactive \
--instance https://git.motovaultpro.com \
--token <REGISTRATION_TOKEN> \
--name "Build/Staging Server" \
--labels "mvp-build:host"
```
### 6. Configure Docker Registry Authentication
Create credentials file for GitLab Container Registry:
### 6. Create Systemd Service
```bash
# Login to GitLab Container Registry (creates ~/.docker/config.json)
docker login registry.motovaultpro.com -u <deploy-token-username> -p <deploy-token>
cat << 'EOF' | sudo tee /etc/systemd/system/act_runner.service
[Unit]
Description=Gitea Actions Runner
After=docker.service network.target
[Service]
ExecStart=/usr/local/bin/act_runner daemon --config /etc/act_runner/config.yaml
WorkingDirectory=/home/act_runner
User=act_runner
Group=act_runner
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable act_runner --now
sudo systemctl status act_runner
```
**Creating Deploy Token:**
1. Go to GitLab Project > Settings > Repository > Deploy Tokens
2. Create token with `read_registry` and `write_registry` scopes
3. Use the token username/password for Docker login
---
## Staging Environment Setup
### 1. Clone Repository
```bash
sudo mkdir -p /opt/motovaultpro
sudo chown act_runner:act_runner /opt/motovaultpro
sudo -u act_runner git clone https://git.motovaultpro.com/egullickson/motovaultpro.git /opt/motovaultpro
```
### 2. Create Staging Secrets Directory
```bash
sudo mkdir -p /opt/motovaultpro/secrets/staging
sudo chown -R act_runner:act_runner /opt/motovaultpro/secrets
sudo chmod 700 /opt/motovaultpro/secrets/staging
```
### 3. Configure DNS
Add DNS A record:
```
staging.motovaultpro.com -> <build-server-ip>
```
### 4. Configure Cloudflare (if using)
Ensure `staging.motovaultpro.com` is proxied through Cloudflare or has a valid SSL certificate configured.
### 5. Initialize Data Directories
```bash
cd /opt/motovaultpro
sudo mkdir -p data/backups data/documents
sudo chown -R 1001:1001 data/backups data/documents
```
---
## Docker Registry Authentication
### Login to Gitea Package Registry
```bash
# Login as act_runner user
sudo -u act_runner docker login git.motovaultpro.com -u egullickson
# Enter your Gitea access token when prompted
```
### Create Access Token
1. Go to `git.motovaultpro.com/user/settings/applications`
2. Create new token with scopes:
- `read:packages`
- `write:packages`
3. Save token securely
---
## Verification
### Test Runner Registration
### Check Runner Status
```bash
sudo gitlab-runner verify
sudo systemctl status act_runner
```
Expected output:
```
Verifying runner... is alive runner=XXXXXX
```
### Check Runner Registration
Go to `git.motovaultpro.com/egullickson/motovaultpro/settings/actions/runners` and verify the runner appears as "Online".
### Test Docker Access
```bash
sudo gitlab-runner exec docker --docker-privileged test-job
sudo -u act_runner docker info
sudo -u act_runner docker compose version
```
### Test Registry Push
```bash
# Build and push a test image
docker build -t registry.motovaultpro.com/motovaultpro/test:latest -f- . <<EOF
sudo -u act_runner docker build -t git.motovaultpro.com/egullickson/test:latest -f- . <<EOF
FROM alpine:latest
RUN echo "test"
EOF
docker push registry.motovaultpro.com/motovaultpro/test:latest
sudo -u act_runner docker push git.motovaultpro.com/egullickson/test:latest
```
---
@@ -168,8 +241,6 @@ docker push registry.motovaultpro.com/motovaultpro/test:latest
### Disk Cleanup
Docker builds accumulate disk space. Set up automated cleanup:
```bash
# Create cleanup script
sudo tee /usr/local/bin/docker-cleanup.sh > /dev/null <<'EOF'
@@ -185,102 +256,64 @@ sudo chmod +x /usr/local/bin/docker-cleanup.sh
echo "0 3 * * * /usr/local/bin/docker-cleanup.sh >> /var/log/docker-cleanup.log 2>&1" | sudo crontab -
```
### Log Rotation
Configure log rotation for GitLab Runner:
```bash
sudo tee /etc/logrotate.d/gitlab-runner > /dev/null <<EOF
/var/log/gitlab-runner/*.log {
daily
rotate 7
compress
missingok
notifempty
}
EOF
```
### Update Runner
```bash
# Update GitLab Runner
sudo apt update
sudo apt upgrade gitlab-runner
# Download new version
curl -L https://gitea.com/gitea/act_runner/releases/download/v0.2.12/act_runner-0.2.12-linux-amd64 -o /tmp/act_runner
sudo mv /tmp/act_runner /usr/local/bin/act_runner
sudo chmod +x /usr/local/bin/act_runner
# Restart runner
sudo gitlab-runner restart
# Restart service
sudo systemctl restart act_runner
```
---
## Security Considerations
### Firewall Configuration
```bash
# Allow only necessary outbound traffic
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw enable
```
### Runner Security
- **Locked runner**: Only accepts jobs from the specific project
- **Protected tags**: Only runs on protected branches (main)
- **Docker socket**: Mounted read-only where possible
### Secrets Management
The build server does NOT store application secrets. All secrets are:
- Stored in GitLab CI/CD Variables
- Injected at runtime on the production server
- Never cached in Docker layers
---
## Troubleshooting
### Runner Not Picking Up Jobs
```bash
# Check runner status
sudo gitlab-runner status
# Check service status
sudo systemctl status act_runner
# View runner logs
sudo journalctl -u gitlab-runner -f
# View logs
sudo journalctl -u act_runner -f
# Re-register runner if needed
sudo gitlab-runner unregister --all-runners
sudo gitlab-runner register
# Check registration
sudo -u act_runner act_runner list
```
### Docker Build Failures
### Docker Permission Issues
```bash
# Check Docker daemon
sudo systemctl status docker
# Ensure act_runner is in docker group
sudo usermod -aG docker act_runner
# Check available disk space
df -h
# Clear Docker cache
docker system prune -af
# Restart service
sudo systemctl restart act_runner
```
### Registry Push Failures
### Registry Authentication Failures
```bash
# Verify registry login
docker login registry.motovaultpro.com
# Re-login to registry
sudo -u act_runner docker logout git.motovaultpro.com
sudo -u act_runner docker login git.motovaultpro.com -u egullickson
```
# Check network connectivity
curl -v https://registry.motovaultpro.com/v2/
### Staging Not Accessible
# Verify image exists
docker images | grep motovaultpro
```bash
# Check containers
docker ps
# Check Traefik logs
docker logs mvp-traefik-staging
# Check SSL certificate
curl -vI https://staging.motovaultpro.com
```
---
@@ -291,25 +324,27 @@ docker images | grep motovaultpro
| Path | Description |
|------|-------------|
| `/etc/gitlab-runner/config.toml` | Runner configuration |
| `/var/log/gitlab-runner/` | Runner logs |
| `~/.docker/config.json` | Docker registry credentials |
| `/var/lib/docker/` | Docker data |
| `/opt/motovaultpro` | Application root |
| `/opt/motovaultpro/secrets/staging` | Staging secrets |
| `/etc/act_runner/config.yaml` | Runner configuration |
| `/home/act_runner/.docker/config.json` | Registry credentials |
### Common Commands
```bash
# Runner management
sudo gitlab-runner status
sudo gitlab-runner restart
sudo gitlab-runner verify
sudo systemctl status act_runner
sudo systemctl restart act_runner
sudo journalctl -u act_runner -f
# Docker management
docker system df # Check disk usage
docker system prune -af # Clean all unused resources
docker images # List images
docker ps -a # List containers
docker system df
docker system prune -af
docker ps
docker logs -f mvp-backend-staging
# View build logs
sudo journalctl -u gitlab-runner --since "1 hour ago"
# Staging stack
cd /opt/motovaultpro
docker compose -f docker-compose.yml -f docker-compose.staging.yml ps
docker compose -f docker-compose.yml -f docker-compose.staging.yml logs -f
```