mirror of
https://github.com/terraphim/gitea-infrastructure.git
synced 2026-07-16 00:00:32 +02:00
feat: Gitea deployment with 1Password and Caddy
Complete infrastructure deployment including: - Docker Compose templates with 1Password integration (op://) - Caddy reverse proxy configuration for git.terraphim.cloud - Automated deployment script (deploy-to-bigbox.sh) - Comprehensive documentation and deployment article - Network isolation: Public (Gitea) + Tailscale (Prometheus/S3) - Security: No hardcoded secrets, all via 1Password Services deployed: - Gitea 1.22.6 (Git hosting) - PostgreSQL 15 (Database) - SeaweedFS (S3-compatible storage) - Prometheus (Monitoring) Excludes: - docker-compose.yml (contains resolved secrets) - s3_config.json (contains resolved secrets) - Data directories (gitea/, postgres/, seaweedfs/) Refs: Private deployment on bigbox with TerraphimPlatform vault
This commit is contained in:
Alex
@@ -0,0 +1,869 @@
|
||||
# Implementation Plan: Deploy Gitea Stack to Bigbox with 1Password and Caddy
|
||||
|
||||
**Status**: Draft
|
||||
**Research Doc**: .docs/RESEARCH-gitea-bigbox-deployment.md
|
||||
**Author**: Claude
|
||||
**Date**: 2026-02-16
|
||||
**Estimated Effort**: 3-4 hours
|
||||
|
||||
## Overview
|
||||
|
||||
### Summary
|
||||
Deploy the Gitea development infrastructure stack to bigbox using 1Password for secrets management (`op inject`) and Caddy for reverse proxy/SSL termination. The deployment sources 1Password credentials via `op_zesticai_non_prod.sh` and uses template files for secure secret injection.
|
||||
|
||||
### Approach
|
||||
1. Create 1Password vault items for all required secrets
|
||||
2. Create template files (`.template`) with 1Password references
|
||||
3. Configure Caddy as reverse proxy with automatic HTTPS
|
||||
4. Deploy using `op inject` to populate secrets at runtime
|
||||
5. Validate all services through Caddy endpoints
|
||||
|
||||
### Scope
|
||||
|
||||
**In Scope:**
|
||||
- Create 1Password vault items for all credentials
|
||||
- Create `docker-compose.yml.template` with 1Password references
|
||||
- Create `s3_config.json.template` with 1Password references
|
||||
- Configure Caddy reverse proxy with SSL
|
||||
- Deploy with `op inject` workflow
|
||||
- Update Caddy instance on bigbox
|
||||
- Validate service accessibility via HTTPS
|
||||
|
||||
**Out of Scope:**
|
||||
- Gitea runner registration (future work)
|
||||
- Earthly Buildkit integration (future work)
|
||||
- Automated backup configuration
|
||||
- CI/CD pipeline setup
|
||||
- Custom SSL certificates (Caddy handles automatically)
|
||||
|
||||
**Avoid At All Cost** (from 5/25 analysis):
|
||||
- Hardcoded credentials in any deployed files
|
||||
- Manual secret management
|
||||
- Complex deployment automation
|
||||
- Multiple Caddy instances
|
||||
- Kubernetes migration
|
||||
|
||||
## Architecture
|
||||
|
||||
### Component Diagram
|
||||
```
|
||||
Public Internet Tailscale Network (100.106.66.0/8)
|
||||
│ │
|
||||
▼ ▼
|
||||
┌──────────────┐ ┌─────────────────────┐
|
||||
│ Caddy │ │ bigbox │
|
||||
│ (existing) │ │ │
|
||||
│ │ │ ┌─────────────────┐ │
|
||||
│ │──443/HTTPS──────────────▶│ │ Gitea │ │
|
||||
│ │ git.terraphim.cloud │ │ localhost:3000 │ │
|
||||
└──────────────┘ │ └─────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Prometheus │ │
|
||||
│ │ 100.106.66.7 │ │
|
||||
│ │ :9000 │ │
|
||||
│ └─────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ S3 │ │
|
||||
│ │ 100.106.66.7 │ │
|
||||
│ │ :8333 │ │
|
||||
│ └─────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ SeaweedFS │ │
|
||||
│ │ (internal) │ │
|
||||
│ └─────────────────┘ │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
### Secrets Management Flow
|
||||
```
|
||||
Developer Machine
|
||||
│
|
||||
├── Create/Update 1Password vault items
|
||||
│
|
||||
├── Create .template files with op:// references
|
||||
│
|
||||
└── Transfer files to bigbox
|
||||
│
|
||||
▼
|
||||
bigbox Server
|
||||
│
|
||||
├── source ~/op_zesticai_non_prod.sh
|
||||
│ └── Export OP_SERVICE_ACCOUNT_TOKEN
|
||||
│
|
||||
├── op inject --in-file docker-compose.yml.template \
|
||||
│ --out-file docker-compose.yml
|
||||
│ └── Resolves op:// references to actual secrets
|
||||
│
|
||||
└── docker compose up -d
|
||||
└── Containers receive secrets as env vars
|
||||
```
|
||||
|
||||
### Service Exposure
|
||||
```
|
||||
Public (via Caddy on terraphim.cloud):
|
||||
git.terraphim.cloud ───────▶ localhost:3000 (Gitea Web)
|
||||
|
||||
Tailscale Network Only (100.106.66.7):
|
||||
100.106.66.7:9000 ─────────▶ Prometheus
|
||||
100.106.66.7:8333 ─────────▶ SeaweedFS S3 API
|
||||
|
||||
Internal Docker Network Only:
|
||||
SeaweedFS Master, Volume, Filer (no external ports)
|
||||
PostgreSQL (no external ports)
|
||||
|
||||
SSH (Port 222):
|
||||
All interfaces - for Git over SSH
|
||||
```
|
||||
|
||||
### Network Security
|
||||
- **Public**: Only Gitea web interface via Caddy HTTPS
|
||||
- **Tailscale**: Prometheus and S3 accessible only within VPN
|
||||
- **Internal**: All other services on Docker network only
|
||||
- **SSH**: Port 222 exposed for Git operations
|
||||
|
||||
### Key Design Decisions
|
||||
| Decision | Rationale | Alternatives Rejected |
|
||||
|----------|-----------|----------------------|
|
||||
| 1Password + op inject | Enterprise-grade secrets management | Hardcoded secrets (security risk) |
|
||||
| Caddy reverse proxy | Automatic HTTPS, simple config | Nginx (more complex SSL) |
|
||||
| Service account auth | Non-interactive automation | Interactive CLI login |
|
||||
| Template pattern | Clear separation of templates and rendered files | Direct file editing |
|
||||
|
||||
### Eliminated Options (Essentialism)
|
||||
| Option Rejected | Why Rejected | Risk of Including |
|
||||
|-----------------|--------------|-------------------|
|
||||
| Hardcoded credentials | Security vulnerability | Credential exposure |
|
||||
| Manual secret management | Error-prone, not auditable | Compliance issues |
|
||||
| Kubernetes Secrets | Overkill for single host | Complexity explosion |
|
||||
| HashiCorp Vault | Additional infrastructure | Operational burden |
|
||||
|
||||
### Simplicity Check
|
||||
|
||||
> "Minimum code that solves the problem. Nothing speculative."
|
||||
|
||||
**What if this could be easy?**
|
||||
The simplest secure deployment is: create templates with `op://` references, source the 1Password script, run `op inject`, then `docker compose up`. Caddy handles SSL automatically. No complex automation, no hardcoded secrets, just secure defaults.
|
||||
|
||||
**Senior Engineer Test**: Would a senior engineer call this overcomplicated? No - this is production-grade security with minimal complexity.
|
||||
|
||||
**Nothing Speculative Checklist**:
|
||||
- [x] No features the user didn't request
|
||||
- [x] No abstractions "in case we need them later"
|
||||
- [x] No flexibility "just in case"
|
||||
- [x] No premature optimization
|
||||
- [x] Security-first approach
|
||||
|
||||
## File Changes
|
||||
|
||||
### New Files
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `.docs/RESEARCH-gitea-bigbox-deployment.md` | Research findings (Phase 1) |
|
||||
| `.docs/IMPLEMENTATION-gitea-bigbox-deployment.md` | This implementation plan (Phase 2) |
|
||||
| `docker-compose.yml.template` | Template with 1Password references |
|
||||
| `s3_config.json.template` | Template with 1Password references |
|
||||
| `Caddyfile` | Caddy reverse proxy configuration |
|
||||
| `.env.template` | Environment variables template |
|
||||
|
||||
### Modified Files
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `.gitignore` | Add `docker-compose.yml`, `s3_config.json`, `.env` (no hardcoded secrets in git) |
|
||||
|
||||
### Files to Exclude from Git
|
||||
| File | Reason |
|
||||
|------|--------|
|
||||
| `docker-compose.yml` | Contains resolved secrets after op inject |
|
||||
| `s3_config.json` | Contains resolved secrets after op inject |
|
||||
| `.env` | Contains resolved secrets after op inject |
|
||||
| `*.decrypted` | Any decrypted files |
|
||||
|
||||
## 1Password Vault Structure
|
||||
|
||||
### Required Vault Items
|
||||
Create these items in the `zesticai-non-prod` vault (or update existing):
|
||||
|
||||
#### Item: gitea-postgres
|
||||
```
|
||||
Title: gitea-postgres
|
||||
Username: gitea
|
||||
Password: [generate strong password]
|
||||
```
|
||||
|
||||
#### Item: gitea-s3
|
||||
```
|
||||
Title: gitea-s3
|
||||
access-key: [generate random key]
|
||||
secret-key: [generate random secret]
|
||||
```
|
||||
|
||||
#### Item: gitea-server (optional)
|
||||
```
|
||||
Title: gitea-server
|
||||
jwt-secret: [generate random string]
|
||||
internal-token: [generate random string]
|
||||
```
|
||||
|
||||
### 1Password References in Templates
|
||||
```yaml
|
||||
# docker-compose.yml.template
|
||||
environment:
|
||||
- GITEA__database__PASSWD=op://zesticai-non-prod/gitea-postgres/password
|
||||
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://zesticai-non-prod/gitea-s3/access-key
|
||||
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://zesticai-non-prod/gitea-s3/secret-key
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Pre-Deployment Verification
|
||||
**Description:** Verify bigbox is ready and 1Password is configured
|
||||
**Tests:** SSH connectivity, Docker availability, 1Password CLI, service account
|
||||
**Estimated:** 15 minutes
|
||||
|
||||
```bash
|
||||
# Verify SSH access
|
||||
ssh bigbox "whoami"
|
||||
|
||||
# Verify Docker
|
||||
ssh bigbox "docker --version && docker compose version"
|
||||
|
||||
# Verify 1Password CLI
|
||||
ssh bigbox "op --version"
|
||||
|
||||
# Verify service account script
|
||||
ssh bigbox "cat ~/op_zesticai_non_prod.sh | head -5"
|
||||
|
||||
# Test 1Password authentication
|
||||
ssh bigbox "source ~/op_zesticai_non_prod.sh && op vault list"
|
||||
|
||||
# Check Caddy
|
||||
ssh bigbox "caddy version"
|
||||
|
||||
# Check available ports
|
||||
ssh bigbox "sudo netstat -tlnp | grep -E ':80|:443' || echo 'Ports 80/443 available'"
|
||||
```
|
||||
|
||||
### Step 2: Create 1Password Vault Items
|
||||
**Description:** Create or update vault items with required secrets
|
||||
**Tests:** All items exist with correct fields
|
||||
**Dependencies:** Step 1
|
||||
**Estimated:** 20 minutes
|
||||
|
||||
```bash
|
||||
# Source 1Password credentials locally (if you have access)
|
||||
source ~/op_zesticai_non_prod.sh
|
||||
|
||||
# Create gitea-postgres item (or update existing)
|
||||
op item create \
|
||||
--vault="zesticai-non-prod" \
|
||||
--category=login \
|
||||
--title="gitea-postgres" \
|
||||
--generate-password=20,letters,digits \
|
||||
username=gitea
|
||||
|
||||
# Create gitea-s3 item with access credentials
|
||||
op item create \
|
||||
--vault="zesticai-non-prod" \
|
||||
--category=custom \
|
||||
--title="gitea-s3" \
|
||||
access-key="$(openssl rand -hex 20)" \
|
||||
secret-key="$(openssl rand -hex 40)"
|
||||
|
||||
# Verify items created
|
||||
op item list --vault="zesticai-non-prod" | grep gitea
|
||||
```
|
||||
|
||||
### Step 3: Create Template Files
|
||||
**Description:** Create `.template` files with 1Password references
|
||||
**Tests:** Templates contain valid op:// references
|
||||
**Dependencies:** Step 2
|
||||
**Estimated:** 30 minutes
|
||||
|
||||
#### Create docker-compose.yml.template
|
||||
```yaml
|
||||
version: "3"
|
||||
# Template file - use 'op inject' to generate docker-compose.yml
|
||||
networks:
|
||||
gitea:
|
||||
external: false
|
||||
|
||||
services:
|
||||
server:
|
||||
image: gitea/gitea:1.22.6
|
||||
container_name: gitea
|
||||
environment:
|
||||
- USER_UID=1000
|
||||
- USER_GID=1000
|
||||
- GITEA__database__DB_TYPE=postgres
|
||||
- GITEA__database__HOST=db:5432
|
||||
- GITEA__database__NAME=gitea
|
||||
- GITEA__database__USER=gitea
|
||||
- GITEA__database__PASSWD=op://zesticai-non-prod/gitea-postgres/password
|
||||
- GITEA__actions__ENABLE=true
|
||||
- GITEA__repository__LFS_START_SERVER=true
|
||||
- GITEA__repository__LFS_CONTENT_PATH=/data/lfs
|
||||
- GITEA__storage__type=minio
|
||||
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://zesticai-non-prod/gitea-s3/access-key
|
||||
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://zesticai-non-prod/gitea-s3/secret-key
|
||||
- GITEA__storage__MINIO_BUCKET=gitea
|
||||
- GITEA__storage__MINIO_LOCATION=us-east-1
|
||||
- GITEA__storage__MINIO_ENDPOINT=http://s3:8333
|
||||
- GITEA__storage__MINIO_INSECURE_SKIP_VERIFY=false
|
||||
- GITEA__storage__MINIO_USE_SSL=false
|
||||
- GITEA__storage__SERVE_DIRECT=true
|
||||
user: "1000:1000"
|
||||
restart: always
|
||||
networks:
|
||||
- gitea
|
||||
volumes:
|
||||
- ./gitea:/data
|
||||
- /etc/timezone:/etc/timezone:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
ports:
|
||||
- "3000:3000"
|
||||
- "222:22"
|
||||
depends_on:
|
||||
- db
|
||||
- s3
|
||||
|
||||
db:
|
||||
image: postgres:15
|
||||
restart: always
|
||||
environment:
|
||||
- POSTGRES_USER=gitea
|
||||
- POSTGRES_PASSWORD=op://zesticai-non-prod/gitea-postgres/password
|
||||
- POSTGRES_DB=gitea
|
||||
networks:
|
||||
- gitea
|
||||
volumes:
|
||||
- ./postgres:/var/lib/postgresql/data
|
||||
|
||||
master:
|
||||
image: chrislusf/seaweedfs
|
||||
ports:
|
||||
- 9333:9333
|
||||
- 19333:19333
|
||||
- 9324:9324
|
||||
command: "master -ip=master -ip.bind=0.0.0.0 -metricsPort=9324"
|
||||
|
||||
volume:
|
||||
image: chrislusf/seaweedfs
|
||||
ports:
|
||||
- 8080:8080
|
||||
- 18080:18080
|
||||
- 9325:9325
|
||||
command: 'volume -mserver="master:9333" -ip.bind=0.0.0.0 -port=8080 -metricsPort=9325'
|
||||
depends_on:
|
||||
- master
|
||||
volumes:
|
||||
- ./seaweedfs:/data
|
||||
|
||||
filer:
|
||||
image: chrislusf/seaweedfs
|
||||
ports:
|
||||
- 8888:8888
|
||||
- 18888:18888
|
||||
- 9326:9326
|
||||
command: 'filer -master="master:9333" -ip.bind=0.0.0.0 -metricsPort=9326'
|
||||
tty: true
|
||||
stdin_open: true
|
||||
depends_on:
|
||||
- master
|
||||
- volume
|
||||
volumes:
|
||||
- ./seaweedfs_filter:/data
|
||||
|
||||
s3:
|
||||
image: chrislusf/seaweedfs
|
||||
container_name: s3storage
|
||||
hostname: s3storage
|
||||
ports:
|
||||
# Bind to Tailscale interface only (100.106.66.7)
|
||||
# Accessible only within Tailscale network, not public internet
|
||||
- "100.106.66.7:8333:8333"
|
||||
- "100.106.66.7:9327:9327"
|
||||
command: 's3 -config=/etc/seaweedfs/s3.json -filer="filer:8888" -ip.bind=0.0.0.0 -metricsPort=9327'
|
||||
depends_on:
|
||||
- master
|
||||
- volume
|
||||
- filer
|
||||
volumes:
|
||||
- ./s3_config.json:/etc/seaweedfs/s3.json
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus:v2.21.0
|
||||
ports:
|
||||
# Bind to Tailscale interface only (100.106.66.7)
|
||||
# Accessible only within Tailscale network, not public internet
|
||||
- "100.106.66.7:9000:9090"
|
||||
volumes:
|
||||
- ./prometheus:/etc/prometheus
|
||||
command: --web.enable-lifecycle --config.file=/etc/prometheus/prometheus.yml
|
||||
depends_on:
|
||||
- s3
|
||||
```
|
||||
|
||||
#### Create s3_config.json.template
|
||||
```json
|
||||
{
|
||||
"identities": [
|
||||
{
|
||||
"name": "anonymous",
|
||||
"actions": [
|
||||
"Read"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "gitea",
|
||||
"credentials": [
|
||||
{
|
||||
"accessKey": "op://zesticai-non-prod/gitea-s3/access-key",
|
||||
"secretKey": "op://zesticai-non-prod/gitea-s3/secret-key"
|
||||
}
|
||||
],
|
||||
"actions": [
|
||||
"Admin",
|
||||
"Read",
|
||||
"List",
|
||||
"Tagging",
|
||||
"Write"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Create Caddy Configuration
|
||||
**Description:** Add Gitea route to existing Caddy instance on bigbox
|
||||
**Tests:** Caddy configuration is valid and can be reloaded
|
||||
**Dependencies:** None
|
||||
**Estimated:** 20 minutes
|
||||
|
||||
**Important:** Bigbox already has Caddy running with existing routes (terraphim.cloud, privacy1st.org). We will add Gitea to the existing configuration using Caddy's admin API or by reloading the config.
|
||||
|
||||
#### Create Caddyfile Snippet
|
||||
```caddyfile
|
||||
# Caddyfile snippet for Gitea on bigbox
|
||||
# This adds Gitea to the existing Caddy instance
|
||||
|
||||
# Gitea Web Interface - Public via terraphim.cloud domain
|
||||
git.terraphim.cloud {
|
||||
reverse_proxy localhost:3000
|
||||
|
||||
# Logging
|
||||
log {
|
||||
output file /var/log/caddy/gitea-access.log
|
||||
format json
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Deployment Options
|
||||
|
||||
**Option A: Add to existing Caddy via API (Recommended)**
|
||||
```bash
|
||||
# Add Gitea route to running Caddy instance via admin API
|
||||
curl -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"match": [{"host": ["git.terraphim.cloud"]}],
|
||||
"handle": [{
|
||||
"handler": "subroute",
|
||||
"routes": [{
|
||||
"handle": [{
|
||||
"handler": "reverse_proxy",
|
||||
"upstreams": [{"dial": "localhost:3000"}]
|
||||
}]
|
||||
}]
|
||||
}],
|
||||
"terminal": true
|
||||
}'
|
||||
```
|
||||
|
||||
**Option B: Update Caddyfile and reload**
|
||||
```bash
|
||||
# Add the snippet to existing Caddyfile
|
||||
sudo cat >> /etc/caddy/Caddyfile << 'EOF'
|
||||
|
||||
# Gitea Web Interface
|
||||
git.terraphim.cloud {
|
||||
reverse_proxy localhost:3000
|
||||
log {
|
||||
output file /var/log/caddy/gitea-access.log
|
||||
format json
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Validate and reload
|
||||
sudo caddy validate --config /etc/caddy/Caddyfile
|
||||
sudo caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
### Step 5: Update .gitignore
|
||||
**Description:** Prevent committed secrets
|
||||
**Tests:** .gitignore properly excludes sensitive files
|
||||
**Dependencies:** Step 3
|
||||
**Estimated:** 5 minutes
|
||||
|
||||
```bash
|
||||
# Add to .gitignore
|
||||
cat >> .gitignore << 'EOF'
|
||||
|
||||
# Secrets - Never commit these
|
||||
docker-compose.yml
|
||||
s3_config.json
|
||||
.env
|
||||
*.decrypted
|
||||
*.secret
|
||||
EOF
|
||||
```
|
||||
|
||||
### Step 6: Transfer Files to Bigbox
|
||||
**Description:** Transfer templates and configuration to bigbox
|
||||
**Tests:** All files transferred successfully
|
||||
**Dependencies:** Step 3, 4, 5
|
||||
**Estimated:** 10 minutes
|
||||
|
||||
```bash
|
||||
# Create remote directory
|
||||
ssh bigbox "mkdir -p ~/gitea-stack"
|
||||
|
||||
# Transfer template files (not rendered files)
|
||||
rsync -avz --progress \
|
||||
--exclude='.git' \
|
||||
--exclude='gitea' \
|
||||
--exclude='postgres' \
|
||||
--exclude='seaweedfs' \
|
||||
--exclude='seaweedfs_filter' \
|
||||
--exclude='docker-compose.yml' \
|
||||
--exclude='s3_config.json' \
|
||||
--exclude='.env' \
|
||||
. bigbox:~/gitea-stack/
|
||||
|
||||
# Transfer Caddyfile separately
|
||||
rsync -avz Caddyfile bigbox:~/gitea-stack/
|
||||
|
||||
# Verify transfer
|
||||
ssh bigbox "ls -la ~/gitea-stack/"
|
||||
```
|
||||
|
||||
### Step 7: Inject Secrets and Start Services
|
||||
**Description:** Use `op inject` to populate secrets and start services
|
||||
**Tests:** All containers start with correct environment variables
|
||||
**Dependencies:** Step 6
|
||||
**Estimated:** 15 minutes
|
||||
|
||||
```bash
|
||||
# SSH to bigbox and execute deployment
|
||||
ssh bigbox << 'REMOTESCRIPT'
|
||||
|
||||
# Change to project directory
|
||||
cd ~/gitea-stack
|
||||
|
||||
# Source 1Password service account
|
||||
echo "Loading 1Password credentials..."
|
||||
source ~/op_zesticai_non_prod.sh
|
||||
|
||||
# Verify 1Password is working
|
||||
echo "Verifying 1Password connection..."
|
||||
op vault list
|
||||
|
||||
# Inject secrets into docker-compose.yml
|
||||
echo "Injecting secrets into docker-compose.yml..."
|
||||
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
|
||||
|
||||
# Inject secrets into s3_config.json
|
||||
echo "Injecting secrets into s3_config.json..."
|
||||
op inject --in-file s3_config.json.template --out-file s3_config.json
|
||||
|
||||
# Create data directories
|
||||
echo "Creating data directories..."
|
||||
mkdir -p gitea postgres seaweedfs seaweedfs_filter prometheus
|
||||
|
||||
# Set permissions for Gitea (runs as UID 1000)
|
||||
chmod -R 1000:1000 gitea 2>/dev/null || sudo chown -R 1000:1000 gitea
|
||||
|
||||
# Start services
|
||||
echo "Starting services..."
|
||||
docker compose up -d
|
||||
|
||||
# Wait for startup
|
||||
sleep 30
|
||||
|
||||
# Check status
|
||||
echo "Checking service status..."
|
||||
docker compose ps
|
||||
|
||||
REMOTESCRIPT
|
||||
```
|
||||
|
||||
### Step 8: Configure Caddy (Add Gitea Route)
|
||||
**Description:** Add Gitea route to existing Caddy instance on bigbox
|
||||
**Tests:** Gitea route added and accessible via HTTPS
|
||||
**Dependencies:** Step 7
|
||||
**Estimated:** 15 minutes
|
||||
|
||||
**Important:** Bigbox already has Caddy running with existing services. We'll add the Gitea route without disrupting existing configuration.
|
||||
|
||||
```bash
|
||||
ssh bigbox << 'REMOTESCRIPT'
|
||||
|
||||
# Create log directory
|
||||
echo "Creating log directory..."
|
||||
sudo mkdir -p /var/log/caddy
|
||||
|
||||
# Check if Caddy is running
|
||||
echo "Checking Caddy status..."
|
||||
if ! pgrep -x caddy > /dev/null; then
|
||||
echo "ERROR: Caddy is not running on bigbox"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Option 1: Add route via Caddy Admin API (recommended - no reload needed)
|
||||
echo "Adding Gitea route to Caddy via Admin API..."
|
||||
curl -s -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"@id": "gitea_route",
|
||||
"match": [{"host": ["git.terraphim.cloud"]}],
|
||||
"handle": [{
|
||||
"handler": "subroute",
|
||||
"routes": [{
|
||||
"handle": [{
|
||||
"handler": "reverse_proxy",
|
||||
"upstreams": [{"dial": "localhost:3000"}]
|
||||
}]
|
||||
}]
|
||||
}],
|
||||
"terminal": true
|
||||
}'
|
||||
|
||||
# Verify route was added
|
||||
echo "Verifying Caddy configuration..."
|
||||
curl -s http://localhost:2019/config/apps/http/servers/srv0/routes | grep -q "git.terraphim.cloud" && echo "✓ Gitea route added successfully" || echo "✗ Failed to add Gitea route"
|
||||
|
||||
# Alternative: If API fails, backup to updating Caddyfile and reloading
|
||||
# echo "Adding Gitea configuration to Caddyfile..."
|
||||
# sudo tee -a /etc/caddy/Caddyfile > /dev/null << 'EOF'
|
||||
#
|
||||
# # Gitea Web Interface
|
||||
# git.terraphim.cloud {
|
||||
# reverse_proxy localhost:3000
|
||||
# log {
|
||||
# output file /var/log/caddy/gitea-access.log
|
||||
# format json
|
||||
# }
|
||||
# }
|
||||
# EOF
|
||||
#
|
||||
# # Validate and reload
|
||||
# sudo caddy validate --config /etc/caddy/Caddyfile && sudo caddy reload --config /etc/caddy/Caddyfile
|
||||
|
||||
echo "Caddy configuration updated"
|
||||
|
||||
REMOTESCRIPT
|
||||
```
|
||||
|
||||
### Step 9: Verify Deployment
|
||||
**Description:** Validate all services are operational
|
||||
**Tests:** Health checks pass for all services
|
||||
**Dependencies:** Step 8
|
||||
**Estimated:** 15 minutes
|
||||
|
||||
```bash
|
||||
ssh bigbox << 'REMOTESCRIPT'
|
||||
|
||||
echo "=== Verifying Service Health ==="
|
||||
|
||||
# Test Gitea (via Caddy - public HTTPS)
|
||||
echo "Testing Gitea via Caddy..."
|
||||
curl -s -o /dev/null -w "Gitea HTTPS Status: %{http_code}\n" https://git.terraphim.cloud/api/healthz 2>/dev/null || echo "Gitea HTTPS not ready (DNS may need propagation)"
|
||||
|
||||
# Test Gitea (direct - localhost)
|
||||
echo "Testing Gitea directly..."
|
||||
curl -s -o /dev/null -w "Gitea HTTP Status: %{http_code}\n" http://localhost:3000/api/healthz || echo "Gitea not ready"
|
||||
|
||||
# Test S3 API (via Tailscale IP)
|
||||
echo "Testing SeaweedFS S3 (Tailscale)..."
|
||||
curl -s http://100.106.66.7:8333 > /dev/null && echo "S3 API (Tailscale): OK" || echo "S3 not ready"
|
||||
|
||||
# Test Prometheus (via Tailscale IP)
|
||||
echo "Testing Prometheus (Tailscale)..."
|
||||
curl -s http://100.106.66.7:9000/-/healthy > /dev/null && echo "Prometheus (Tailscale): OK" || echo "Prometheus not ready"
|
||||
|
||||
# Test SSH port
|
||||
echo "Testing SSH port..."
|
||||
nc -zv localhost 222 2>&1 | grep -q succeeded && echo "SSH port: OK" || echo "SSH port not responding"
|
||||
|
||||
# Check container logs for errors
|
||||
echo ""
|
||||
echo "=== Checking for errors in logs ==="
|
||||
cd ~/gitea-stack
|
||||
docker compose logs --tail=20 | grep -i error || echo "No errors found"
|
||||
|
||||
echo ""
|
||||
echo "=== Container Status ==="
|
||||
docker compose ps
|
||||
|
||||
REMOTESCRIPT
|
||||
```
|
||||
|
||||
### Step 10: Document Access Endpoints
|
||||
**Description:** Record access endpoints for the user
|
||||
**Tests:** Documentation complete with all URLs
|
||||
**Dependencies:** Step 9
|
||||
**Estimated:** 10 minutes
|
||||
|
||||
**Public Access (via Caddy HTTPS):**
|
||||
- Gitea Web: https://git.terraphim.cloud
|
||||
- Gitea SSH: ssh://bigbox:222 (direct, not through Caddy)
|
||||
|
||||
**Tailscale Network Only (100.106.66.7):**
|
||||
- Prometheus: http://100.106.66.7:9000
|
||||
- SeaweedFS S3: http://100.106.66.7:8333
|
||||
- SeaweedFS S3 Metrics: http://100.106.66.7:9327
|
||||
|
||||
**Internal Docker Network (no external access):**
|
||||
- PostgreSQL: db:5432 (internal only)
|
||||
- SeaweedFS Master: master:9333
|
||||
- SeaweedFS Volume: volume:8080
|
||||
- SeaweedFS Filer: filer:8888
|
||||
|
||||
**Local Development Access:**
|
||||
- Gitea (direct): http://localhost:3000
|
||||
|
||||
**Important Notes:**
|
||||
- Prometheus and S3 are NOT accessible from the public internet
|
||||
- They are only accessible within the Tailscale VPN network
|
||||
- Connect to Tailscale first: `tailscale up`
|
||||
|
||||
## Test Strategy
|
||||
|
||||
### Deployment Tests
|
||||
| Test | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| SSH connectivity | Step 1 | Verify remote access |
|
||||
| 1Password auth | Step 1 | Verify service account |
|
||||
| Caddy validation | Step 8 | Config syntax correct |
|
||||
| Secret injection | Step 7 | op:// references resolve |
|
||||
| Service startup | Step 7 | All containers start |
|
||||
| Caddy routing | Step 9 | HTTP requests routed correctly |
|
||||
|
||||
### Validation Commands
|
||||
```bash
|
||||
# Verify containers running
|
||||
ssh bigbox "docker compose ps"
|
||||
|
||||
# Test Caddy routing
|
||||
ssh bigbox "curl -s -o /dev/null -w '%{http_code}' http://localhost:3000"
|
||||
|
||||
# Verify no hardcoded secrets in templates
|
||||
ssh bigbox "grep -r 'op://' ~/gitea-stack/*.template"
|
||||
|
||||
# Verify secrets resolved in compose file
|
||||
ssh bigbox "grep -E 'PASSWORD|SECRET|KEY' ~/gitea-stack/docker-compose.yml | head -5"
|
||||
|
||||
# Test 1Password connectivity
|
||||
ssh bigbox "source ~/op_zesticai_non_prod.sh && op item list"
|
||||
```
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If deployment fails:
|
||||
1. Stop services: `ssh bigbox "cd ~/gitea-stack && docker compose down"`
|
||||
2. Stop Caddy: `ssh bigbox "sudo pkill caddy || sudo systemctl stop caddy"`
|
||||
3. Remove data directories: `ssh bigbox "rm -rf ~/gitea-stack"`
|
||||
4. Revert Caddy config: `ssh bigbox "sudo rm /etc/caddy/Caddyfile"`
|
||||
5. Debug and retry
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Immediate Actions Post-Deployment
|
||||
1. Change Gitea admin password after first login
|
||||
2. Verify no secrets in git repository
|
||||
3. Review Caddy access logs regularly
|
||||
4. Set up firewall rules if needed
|
||||
|
||||
### Security Checklist
|
||||
- [ ] No hardcoded secrets in git
|
||||
- [ ] All secrets injected via 1Password
|
||||
- [ ] Caddy automatically manages SSL certificates
|
||||
- [ ] Direct service ports not exposed externally (except 222 for SSH)
|
||||
- [ ] 1Password service account token secured on bigbox
|
||||
|
||||
### 1Password Security
|
||||
- Service account token stored in `~/op_zesticai_non_prod.sh`
|
||||
- Token has limited scope (zesticai-non-prod vault)
|
||||
- File permissions should be restricted: `chmod 600 ~/op_zesticai_non_prod.sh`
|
||||
|
||||
## Open Items
|
||||
|
||||
| Item | Status | Owner |
|
||||
|------|--------|-------|
|
||||
| Create 1Password vault items | Pending | Deployer |
|
||||
| Determine Caddy domain names | Pending | Deployer |
|
||||
| Verify 80/443 port availability | Pending | Deployer |
|
||||
| Set up DNS records | Pending | Deployer (if using real domain) |
|
||||
|
||||
## Deployment Commands Reference
|
||||
|
||||
### Quick Deploy (Full)
|
||||
```bash
|
||||
# 1. Create 1Password items locally
|
||||
source ~/op_zesticai_non_prod.sh
|
||||
op item create --vault="zesticai-non-prod" --category=login --title="gitea-postgres" --generate-password=20 username=gitea
|
||||
op item create --vault="zesticai-non-prod" --category=custom --title="gitea-s3" access-key="$(openssl rand -hex 20)" secret-key="$(openssl rand -hex 40)"
|
||||
|
||||
# 2. Transfer files
|
||||
rsync -avz --exclude='.git' --exclude='gitea' --exclude='postgres' \
|
||||
--exclude='seaweedfs' --exclude='seaweedfs_filter' \
|
||||
--exclude='docker-compose.yml' --exclude='s3_config.json' \
|
||||
. bigbox:~/gitea-stack/
|
||||
|
||||
# 3. Deploy on bigbox
|
||||
ssh bigbox "cd ~/gitea-stack && source ~/op_zesticai_non_prod.sh && op inject --in-file docker-compose.yml.template --out-file docker-compose.yml && op inject --in-file s3_config.json.template --out-file s3_config.json && docker compose up -d"
|
||||
|
||||
# 4. Configure Caddy
|
||||
ssh bigbox "sudo cp ~/gitea-stack/Caddyfile /etc/caddy/Caddyfile && sudo caddy validate --config /etc/caddy/Caddyfile && sudo caddy run --config /etc/caddy/Caddyfile &"
|
||||
```
|
||||
|
||||
### Status Check
|
||||
```bash
|
||||
ssh bigbox "cd ~/gitea-stack && docker compose ps"
|
||||
ssh bigbox "curl -s http://localhost:3000/api/healthz && echo 'Gitea OK'"
|
||||
ssh bigbox "sudo caddy list-modules 2>&1 | head -5"
|
||||
```
|
||||
|
||||
### Stop Services
|
||||
```bash
|
||||
ssh bigbox "cd ~/gitea-stack && docker compose down"
|
||||
ssh bigbox "sudo pkill caddy || sudo systemctl stop caddy"
|
||||
```
|
||||
|
||||
### View Logs
|
||||
```bash
|
||||
ssh bigbox "cd ~/gitea-stack && docker compose logs -f"
|
||||
ssh bigbox "sudo tail -f /var/log/caddy/gitea-access.log"
|
||||
```
|
||||
|
||||
## Approval
|
||||
|
||||
- [ ] Research document reviewed
|
||||
- [ ] Implementation plan reviewed
|
||||
- [ ] 1Password vault structure defined
|
||||
- [ ] Security considerations acknowledged
|
||||
- [ ] Rollback plan understood
|
||||
- [ ] Ready to proceed with deployment
|
||||
|
||||
---
|
||||
|
||||
**Next Steps After Approval:**
|
||||
1. Create 1Password vault items
|
||||
2. Create template files locally
|
||||
3. Transfer to bigbox
|
||||
4. Execute deployment
|
||||
5. Validate all services via Caddy
|
||||
Reference in New Issue
Block a user