terraphim/gitea-infrastructure
16
0
Fork 0
mirror of https://github.com/terraphim/gitea-infrastructure.git synced 2026-07-16 00:00:32 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity

feat: Gitea deployment with 1Password and Caddy

Browse Source 
Complete infrastructure deployment including:
- Docker Compose templates with 1Password integration (op://)
- Caddy reverse proxy configuration for git.terraphim.cloud
- Automated deployment script (deploy-to-bigbox.sh)
- Comprehensive documentation and deployment article
- Network isolation: Public (Gitea) + Tailscale (Prometheus/S3)
- Security: No hardcoded secrets, all via 1Password

Services deployed:
- Gitea 1.22.6 (Git hosting)
- PostgreSQL 15 (Database)
- SeaweedFS (S3-compatible storage)
- Prometheus (Monitoring)

Excludes:
- docker-compose.yml (contains resolved secrets)
- s3_config.json (contains resolved secrets)
- Data directories (gitea/, postgres/, seaweedfs/)

Refs: Private deployment on bigbox with TerraphimPlatform vault
This commit is contained in:
Alex
2026-02-16 19:40:19 +00:00
parent 850d8b7680
commit 4a4a0351aa
13 changed files with 2620 additions and 200 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.docs/DEPLOYMENT-SUMMARY.md
+323
View File
@@ -0,0 +1,323 @@
# Gitea Stack Deployment Summary
**Date**: 2026-02-16
**Target**: bigbox (via SSH)
**Status**: Ready for deployment
## What Was Done
### 1. Disciplined Research (Phase 1)
Created comprehensive research document analyzing:
- Current Docker Compose stack (Gitea, PostgreSQL, SeaweedFS, Prometheus)
- Bigbox environment (125GB RAM, 367GB free, Docker 28.1.1, 1Password CLI installed)
- Security requirements (eliminate hardcoded credentials)
- Caddy reverse proxy integration
**File**: `.docs/RESEARCH-gitea-bigbox-deployment.md`
### 2. Disciplined Design (Phase 2)
Created implementation plan with:
- 1Password secrets management using `op inject`
- Template files with `op://` references
- Caddy reverse proxy configuration with automatic HTTPS
- 10-step deployment process
- Complete rollback plan
**File**: `.docs/IMPLEMENTATION-gitea-bigbox-deployment.md`
### 3. Implementation Artifacts Created
#### Template Files (Safe to Commit)
- `docker-compose.yml.template` - Docker Compose with 1Password references
- `s3_config.json.template` - SeaweedFS S3 config with 1Password references
- `Caddyfile` - Caddy reverse proxy configuration
- `.gitignore` - Excludes secrets and data directories
#### Deployment Script
- `deploy-to-bigbox.sh` - Automated deployment script
#### Generated Files (NOT in Git - Created at Deploy Time)
- `docker-compose.yml` - Secrets injected via `op inject`
- `s3_config.json` - Secrets injected via `op inject`
## 1Password Integration
### Prerequisites on Bigbox
1. 1Password CLI installed (`/usr/bin/op`)
2. Service account script: `~/op_zesticai_non_prod.sh`
3. Valid service account token configured
### Required Vault Items
Create these in the `zesticai-non-prod` vault:
```bash
# Create gitea-postgres item
op item create \
--vault="zesticai-non-prod" \
--category=login \
--title="gitea-postgres" \
--generate-password=20,letters,digits \
username=gitea
# Create gitea-s3 item
op item create \
--vault="zesticai-non-prod" \
--category=custom \
--title="gitea-s3" \
access-key="$(openssl rand -hex 20)" \
secret-key="$(openssl rand -hex 40)"
```
### How It Works
1. **Template Creation**: Files use `op://vault/item/field` syntax
2. **Authentication**: Source `op_zesticai_non_prod.sh` to set `OP_SERVICE_ACCOUNT_TOKEN`
3. **Secret Injection**: Run `op inject --in-file template --out-file config`
4. **Deployment**: Use injected config files with Docker Compose
## Network Architecture
### Public Access (via Caddy HTTPS)
Bigbox already runs Caddy with existing services (terraphim.cloud, privacy1st.org). We add Gitea to the existing instance:
| Service | Domain | Backend | Access |
|---------|--------|---------|--------|
| Gitea | git.terraphim.cloud | localhost:3000 | Public HTTPS |
### Tailscale Network Only (100.106.66.7)
These services are bound to the Tailscale interface and are NOT accessible from the public internet:
| Service | Address | Purpose |
|---------|---------|---------|
| Prometheus | 100.106.66.7:9000 | Monitoring |
| S3 API | 100.106.66.7:8333 | Object storage |
| S3 Metrics | 100.106.66.7:9327 | Prometheus metrics |
### SSH Access
- **Port 222**: Git over SSH (exposed on all interfaces)
- **Command**: `git clone ssh://git@bigbox:222/owner/repo.git`
### Internal Services (Docker Network Only)
- PostgreSQL: Internal Docker network only
- SeaweedFS Master, Volume, Filer: Internal Docker network only
### Caddy Configuration
- **Existing Instance**: Caddy already running on bigbox
- **Route Addition**: Gitea route added via Caddy Admin API
- **SSL**: Automatic HTTPS via Let's Encrypt
- **Logging**: JSON format to `/var/log/caddy/gitea-access.log`
## Deployment Steps
### Option 1: Automated (Recommended)
```bash
# 1. Create 1Password items first (see above)
# 2. Run deployment script
./deploy-to-bigbox.sh
```
### Option 2: Manual Step-by-Step
See `.docs/IMPLEMENTATION-gitea-bigbox-deployment.md` for detailed manual steps.
## Post-Deployment
### Access URLs
After deployment, services will be available at:
**Public Access (HTTPS via Caddy):**
- **Gitea Web**: https://git.terraphim.cloud
- **Gitea SSH**: ssh://bigbox:222
**Tailscale Network Only (requires Tailscale connection):**
- **Prometheus**: http://100.106.66.7:9000
- **S3 API**: http://100.106.66.7:8333
- **S3 Metrics**: http://100.106.66.7:9327
**Local Development (on bigbox):**
- **Gitea**: http://localhost:3000
**Important:** Prometheus and S3 are intentionally NOT accessible from the public internet. They are bound to the Tailscale interface (100.106.66.7) and require connection to the Tailscale VPN.
### Immediate Actions
1. **Update DNS**: Add entries to your DNS or local `/etc/hosts`
2. **Gitea Setup**: Complete initial setup at http://bigbox:3000
3. **Change Password**: Update default admin password immediately
4. **Verify Caddy**: Check SSL certificates are generated
### Useful Commands
```bash
# Check service status
ssh bigbox "cd ~/gitea-stack && docker compose ps"
# View logs
ssh bigbox "cd ~/gitea-stack && docker compose logs -f"
# View Caddy logs
ssh bigbox "sudo tail -f /var/log/caddy/*.log"
# Restart services
ssh bigbox "cd ~/gitea-stack && docker compose restart"
# Reload Caddy config
ssh bigbox "sudo caddy reload --config /etc/caddy/Caddyfile"
```
## Security Improvements
### Before (Hardcoded Secrets)
- S3 credentials in `s3_config.json`
- Database password in `docker-compose.yml`
- Secrets committed to git
### After (1Password Managed)
- All secrets in 1Password vault
- Template files only contain references (`op://`)
- Secrets injected at deployment time
- Easy secret rotation via 1Password
- `.gitignore` prevents accidental commits
### Caddy Security
- Automatic HTTPS (Let's Encrypt)
- No manual certificate management
- Request logging for audit trail
- Easy to add basic authentication
## Rollback
If deployment fails:
```bash
# Stop services
ssh bigbox "cd ~/gitea-stack && docker compose down"
# Stop Caddy
ssh bigbox "sudo pkill caddy || sudo systemctl stop caddy"
# Remove deployment
ssh bigbox "rm -rf ~/gitea-stack"
# Remove Caddy config
ssh bigbox "sudo rm /etc/caddy/Caddyfile"
```
## Files Changed/Created
### New Files
1. `.docs/RESEARCH-gitea-bigbox-deployment.md` - Research findings
2. `.docs/IMPLEMENTATION-gitea-bigbox-deployment.md` - Implementation plan
3. `docker-compose.yml.template` - Compose template with 1Password refs
4. `s3_config.json.template` - S3 config template with 1Password refs
5. `Caddyfile` - Caddy reverse proxy configuration
6. `.gitignore` - Excludes secrets and data
7. `deploy-to-bigbox.sh` - Automated deployment script
8. `.docs/DEPLOYMENT-SUMMARY.md` - This file
### Modified Files
- None (original docker-compose.yml and s3_config.json remain unchanged)
### Files Excluded from Git
- `docker-compose.yml` (contains resolved secrets)
- `s3_config.json` (contains resolved secrets)
- `.env` (if created)
- `gitea/`, `postgres/`, `seaweedfs/` (data directories)
## Next Steps
1. **Create 1Password Items**: Run the `op item create` commands
2. **Update Domains**: Caddyfile already configured for `git.terraphim.cloud`
3. **Run Deployment**: Execute `./deploy-to-bigbox.sh`
4. **Verify**: Check all services are accessible
5. **Document**: Update this summary with actual domain names and any issues
## Troubleshooting
### 1Password Authentication Fails
```bash
# Test on bigbox
ssh bigbox
source ~/op_zesticai_non_prod.sh
op vault list
# If it fails, check service account token is valid
```
### Caddy Route Not Working
```bash
# Check if Caddy is running
ssh bigbox "pgrep -x caddy"
# Check existing Caddy routes
ssh bigbox "curl -s http://localhost:2019/config/apps/http/servers/srv0/routes | grep git.terraphim"
# Check Caddy logs
ssh bigbox "sudo tail -f /var/log/caddy/*.log"
# Verify Gitea is responding on localhost:3000
ssh bigbox "curl http://localhost:3000/api/healthz"
```
### Services Not Accessible
```bash
# Check container logs
ssh bigbox "cd ~/gitea-stack && docker compose logs"
# Check if containers are running
ssh bigbox "cd ~/gitea-stack && docker compose ps"
# Check Gitea (public via Caddy)
curl https://git.terraphim.cloud/api/healthz
# Check Prometheus/S3 (Tailscale only - must be on Tailscale network)
curl http://100.106.66.7:9000/-/healthy
curl http://100.106.66.7:8333
```
### Cannot Access Prometheus or S3
These services are bound to the Tailscale interface (100.106.66.7) only:
```bash
# Verify you're connected to Tailscale
tailscale status
# Verify you can reach the Tailscale IP
ping 100.106.66.7
# Then access services
curl http://100.106.66.7:9000/-/healthy
curl http://100.106.66.7:8333
```
### Secret Injection Fails
```bash
# Test op inject manually
ssh bigbox "cd ~/gitea-stack && source ~/op_zesticai_non_prod.sh && op inject --in-file docker-compose.yml.template --dry-run"
```
## Maintenance
### Rotating Secrets
1. Update secret in 1Password vault
2. Re-run deployment: `./deploy-to-bigbox.sh`
3. Or manually:
```bash
ssh bigbox "cd ~/gitea-stack && source ~/op_zesticai_non_prod.sh && op inject --in-file docker-compose.yml.template --out-file docker-compose.yml && docker compose up -d"
```
### Backup
```bash
# Backup data directories
rsync -avz bigbox:~/gitea-stack/gitea ./backup/
rsync -avz bigbox:~/gitea-stack/postgres ./backup/
rsync -avz bigbox:~/gitea-stack/seaweedfs ./backup/
```
### Updates
```bash
# Update images
ssh bigbox "cd ~/gitea-stack && docker compose pull && docker compose up -d"
# Update Caddy
ssh bigbox "sudo apt update && sudo apt upgrade caddy"
```
---
**Ready to deploy!** Run `./deploy-to-bigbox.sh` after creating the 1Password vault items.
.docs/IMPLEMENTATION-gitea-bigbox-deployment.md
+869
View File
@@ -0,0 +1,869 @@
# Implementation Plan: Deploy Gitea Stack to Bigbox with 1Password and Caddy
**Status**: Draft
**Research Doc**: .docs/RESEARCH-gitea-bigbox-deployment.md
**Author**: Claude
**Date**: 2026-02-16
**Estimated Effort**: 3-4 hours
## Overview
### Summary
Deploy the Gitea development infrastructure stack to bigbox using 1Password for secrets management (`op inject`) and Caddy for reverse proxy/SSL termination. The deployment sources 1Password credentials via `op_zesticai_non_prod.sh` and uses template files for secure secret injection.
### Approach
1. Create 1Password vault items for all required secrets
2. Create template files (`.template`) with 1Password references
3. Configure Caddy as reverse proxy with automatic HTTPS
4. Deploy using `op inject` to populate secrets at runtime
5. Validate all services through Caddy endpoints
### Scope
**In Scope:**
- Create 1Password vault items for all credentials
- Create `docker-compose.yml.template` with 1Password references
- Create `s3_config.json.template` with 1Password references
- Configure Caddy reverse proxy with SSL
- Deploy with `op inject` workflow
- Update Caddy instance on bigbox
- Validate service accessibility via HTTPS
**Out of Scope:**
- Gitea runner registration (future work)
- Earthly Buildkit integration (future work)
- Automated backup configuration
- CI/CD pipeline setup
- Custom SSL certificates (Caddy handles automatically)
**Avoid At All Cost** (from 5/25 analysis):
- Hardcoded credentials in any deployed files
- Manual secret management
- Complex deployment automation
- Multiple Caddy instances
- Kubernetes migration
## Architecture
### Component Diagram
```
Public Internet Tailscale Network (100.106.66.0/8)
│ │
▼ ▼
┌──────────────┐ ┌─────────────────────┐
│ Caddy │ │ bigbox │
│ (existing) │ │ │
│ │ │ ┌─────────────────┐ │
│ │──443/HTTPS──────────────▶│ │ Gitea │ │
│ │ git.terraphim.cloud │ │ localhost:3000 │ │
└──────────────┘ │ └─────────────────┘ │
│ │
│ ┌─────────────────┐ │
│ │ Prometheus │ │
│ │ 100.106.66.7 │ │
│ │ :9000 │ │
│ └─────────────────┘ │
│ │
│ ┌─────────────────┐ │
│ │ S3 │ │
│ │ 100.106.66.7 │ │
│ │ :8333 │ │
│ └─────────────────┘ │
│ │
│ ┌─────────────────┐ │
│ │ SeaweedFS │ │
│ │ (internal) │ │
│ └─────────────────┘ │
└─────────────────────┘
```
### Secrets Management Flow
```
Developer Machine
│
├── Create/Update 1Password vault items
│
├── Create .template files with op:// references
│
└── Transfer files to bigbox
│
▼
bigbox Server
│
├── source ~/op_zesticai_non_prod.sh
│ └── Export OP_SERVICE_ACCOUNT_TOKEN
│
├── op inject --in-file docker-compose.yml.template \
│ --out-file docker-compose.yml
│ └── Resolves op:// references to actual secrets
│
└── docker compose up -d
└── Containers receive secrets as env vars
```
### Service Exposure
```
Public (via Caddy on terraphim.cloud):
git.terraphim.cloud ───────▶ localhost:3000 (Gitea Web)
Tailscale Network Only (100.106.66.7):
100.106.66.7:9000 ─────────▶ Prometheus
100.106.66.7:8333 ─────────▶ SeaweedFS S3 API
Internal Docker Network Only:
SeaweedFS Master, Volume, Filer (no external ports)
PostgreSQL (no external ports)
SSH (Port 222):
All interfaces - for Git over SSH
```
### Network Security
- **Public**: Only Gitea web interface via Caddy HTTPS
- **Tailscale**: Prometheus and S3 accessible only within VPN
- **Internal**: All other services on Docker network only
- **SSH**: Port 222 exposed for Git operations
### Key Design Decisions
| Decision | Rationale | Alternatives Rejected |
|----------|-----------|----------------------|
| 1Password + op inject | Enterprise-grade secrets management | Hardcoded secrets (security risk) |
| Caddy reverse proxy | Automatic HTTPS, simple config | Nginx (more complex SSL) |
| Service account auth | Non-interactive automation | Interactive CLI login |
| Template pattern | Clear separation of templates and rendered files | Direct file editing |
### Eliminated Options (Essentialism)
| Option Rejected | Why Rejected | Risk of Including |
|-----------------|--------------|-------------------|
| Hardcoded credentials | Security vulnerability | Credential exposure |
| Manual secret management | Error-prone, not auditable | Compliance issues |
| Kubernetes Secrets | Overkill for single host | Complexity explosion |
| HashiCorp Vault | Additional infrastructure | Operational burden |
### Simplicity Check
> "Minimum code that solves the problem. Nothing speculative."
**What if this could be easy?**
The simplest secure deployment is: create templates with `op://` references, source the 1Password script, run `op inject`, then `docker compose up`. Caddy handles SSL automatically. No complex automation, no hardcoded secrets, just secure defaults.
**Senior Engineer Test**: Would a senior engineer call this overcomplicated? No - this is production-grade security with minimal complexity.
**Nothing Speculative Checklist**:
- [x] No features the user didn't request
- [x] No abstractions "in case we need them later"
- [x] No flexibility "just in case"
- [x] No premature optimization
- [x] Security-first approach
## File Changes
### New Files
| File | Purpose |
|------|---------|
| `.docs/RESEARCH-gitea-bigbox-deployment.md` | Research findings (Phase 1) |
| `.docs/IMPLEMENTATION-gitea-bigbox-deployment.md` | This implementation plan (Phase 2) |
| `docker-compose.yml.template` | Template with 1Password references |
| `s3_config.json.template` | Template with 1Password references |
| `Caddyfile` | Caddy reverse proxy configuration |
| `.env.template` | Environment variables template |
### Modified Files
| File | Changes |
|------|---------|
| `.gitignore` | Add `docker-compose.yml`, `s3_config.json`, `.env` (no hardcoded secrets in git) |
### Files to Exclude from Git
| File | Reason |
|------|--------|
| `docker-compose.yml` | Contains resolved secrets after op inject |
| `s3_config.json` | Contains resolved secrets after op inject |
| `.env` | Contains resolved secrets after op inject |
| `*.decrypted` | Any decrypted files |
## 1Password Vault Structure
### Required Vault Items
Create these items in the `zesticai-non-prod` vault (or update existing):
#### Item: gitea-postgres
```
Title: gitea-postgres
Username: gitea
Password: [generate strong password]
```
#### Item: gitea-s3
```
Title: gitea-s3
access-key: [generate random key]
secret-key: [generate random secret]
```
#### Item: gitea-server (optional)
```
Title: gitea-server
jwt-secret: [generate random string]
internal-token: [generate random string]
```
### 1Password References in Templates
```yaml
# docker-compose.yml.template
environment:
- GITEA__database__PASSWD=op://zesticai-non-prod/gitea-postgres/password
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://zesticai-non-prod/gitea-s3/access-key
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://zesticai-non-prod/gitea-s3/secret-key
```
## Implementation Steps
### Step 1: Pre-Deployment Verification
**Description:** Verify bigbox is ready and 1Password is configured
**Tests:** SSH connectivity, Docker availability, 1Password CLI, service account
**Estimated:** 15 minutes
```bash
# Verify SSH access
ssh bigbox "whoami"
# Verify Docker
ssh bigbox "docker --version && docker compose version"
# Verify 1Password CLI
ssh bigbox "op --version"
# Verify service account script
ssh bigbox "cat ~/op_zesticai_non_prod.sh | head -5"
# Test 1Password authentication
ssh bigbox "source ~/op_zesticai_non_prod.sh && op vault list"
# Check Caddy
ssh bigbox "caddy version"
# Check available ports
ssh bigbox "sudo netstat -tlnp | grep -E ':80|:443' || echo 'Ports 80/443 available'"
```
### Step 2: Create 1Password Vault Items
**Description:** Create or update vault items with required secrets
**Tests:** All items exist with correct fields
**Dependencies:** Step 1
**Estimated:** 20 minutes
```bash
# Source 1Password credentials locally (if you have access)
source ~/op_zesticai_non_prod.sh
# Create gitea-postgres item (or update existing)
op item create \
--vault="zesticai-non-prod" \
--category=login \
--title="gitea-postgres" \
--generate-password=20,letters,digits \
username=gitea
# Create gitea-s3 item with access credentials
op item create \
--vault="zesticai-non-prod" \
--category=custom \
--title="gitea-s3" \
access-key="$(openssl rand -hex 20)" \
secret-key="$(openssl rand -hex 40)"
# Verify items created
op item list --vault="zesticai-non-prod" | grep gitea
```
### Step 3: Create Template Files
**Description:** Create `.template` files with 1Password references
**Tests:** Templates contain valid op:// references
**Dependencies:** Step 2
**Estimated:** 30 minutes
#### Create docker-compose.yml.template
```yaml
version: "3"
# Template file - use 'op inject' to generate docker-compose.yml
networks:
gitea:
external: false
services:
server:
image: gitea/gitea:1.22.6
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=postgres
- GITEA__database__HOST=db:5432
- GITEA__database__NAME=gitea
- GITEA__database__USER=gitea
- GITEA__database__PASSWD=op://zesticai-non-prod/gitea-postgres/password
- GITEA__actions__ENABLE=true
- GITEA__repository__LFS_START_SERVER=true
- GITEA__repository__LFS_CONTENT_PATH=/data/lfs
- GITEA__storage__type=minio
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://zesticai-non-prod/gitea-s3/access-key
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://zesticai-non-prod/gitea-s3/secret-key
- GITEA__storage__MINIO_BUCKET=gitea
- GITEA__storage__MINIO_LOCATION=us-east-1
- GITEA__storage__MINIO_ENDPOINT=http://s3:8333
- GITEA__storage__MINIO_INSECURE_SKIP_VERIFY=false
- GITEA__storage__MINIO_USE_SSL=false
- GITEA__storage__SERVE_DIRECT=true
user: "1000:1000"
restart: always
networks:
- gitea
volumes:
- ./gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
ports:
- "3000:3000"
- "222:22"
depends_on:
- db
- s3
db:
image: postgres:15
restart: always
environment:
- POSTGRES_USER=gitea
- POSTGRES_PASSWORD=op://zesticai-non-prod/gitea-postgres/password
- POSTGRES_DB=gitea
networks:
- gitea
volumes:
- ./postgres:/var/lib/postgresql/data
master:
image: chrislusf/seaweedfs
ports:
- 9333:9333
- 19333:19333
- 9324:9324
command: "master -ip=master -ip.bind=0.0.0.0 -metricsPort=9324"
volume:
image: chrislusf/seaweedfs
ports:
- 8080:8080
- 18080:18080
- 9325:9325
command: 'volume -mserver="master:9333" -ip.bind=0.0.0.0 -port=8080 -metricsPort=9325'
depends_on:
- master
volumes:
- ./seaweedfs:/data
filer:
image: chrislusf/seaweedfs
ports:
- 8888:8888
- 18888:18888
- 9326:9326
command: 'filer -master="master:9333" -ip.bind=0.0.0.0 -metricsPort=9326'
tty: true
stdin_open: true
depends_on:
- master
- volume
volumes:
- ./seaweedfs_filter:/data
s3:
image: chrislusf/seaweedfs
container_name: s3storage
hostname: s3storage
ports:
# Bind to Tailscale interface only (100.106.66.7)
# Accessible only within Tailscale network, not public internet
- "100.106.66.7:8333:8333"
- "100.106.66.7:9327:9327"
command: 's3 -config=/etc/seaweedfs/s3.json -filer="filer:8888" -ip.bind=0.0.0.0 -metricsPort=9327'
depends_on:
- master
- volume
- filer
volumes:
- ./s3_config.json:/etc/seaweedfs/s3.json
prometheus:
image: prom/prometheus:v2.21.0
ports:
# Bind to Tailscale interface only (100.106.66.7)
# Accessible only within Tailscale network, not public internet
- "100.106.66.7:9000:9090"
volumes:
- ./prometheus:/etc/prometheus
command: --web.enable-lifecycle --config.file=/etc/prometheus/prometheus.yml
depends_on:
- s3
```
#### Create s3_config.json.template
```json
{
"identities": [
{
"name": "anonymous",
"actions": [
"Read"
]
},
{
"name": "gitea",
"credentials": [
{
"accessKey": "op://zesticai-non-prod/gitea-s3/access-key",
"secretKey": "op://zesticai-non-prod/gitea-s3/secret-key"
}
],
"actions": [
"Admin",
"Read",
"List",
"Tagging",
"Write"
]
}
]
}
```
### Step 4: Create Caddy Configuration
**Description:** Add Gitea route to existing Caddy instance on bigbox
**Tests:** Caddy configuration is valid and can be reloaded
**Dependencies:** None
**Estimated:** 20 minutes
**Important:** Bigbox already has Caddy running with existing routes (terraphim.cloud, privacy1st.org). We will add Gitea to the existing configuration using Caddy's admin API or by reloading the config.
#### Create Caddyfile Snippet
```caddyfile
# Caddyfile snippet for Gitea on bigbox
# This adds Gitea to the existing Caddy instance
# Gitea Web Interface - Public via terraphim.cloud domain
git.terraphim.cloud {
reverse_proxy localhost:3000
# Logging
log {
output file /var/log/caddy/gitea-access.log
format json
}
}
```
#### Deployment Options
**Option A: Add to existing Caddy via API (Recommended)**
```bash
# Add Gitea route to running Caddy instance via admin API
curl -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
-H "Content-Type: application/json" \
-d '{
"match": [{"host": ["git.terraphim.cloud"]}],
"handle": [{
"handler": "subroute",
"routes": [{
"handle": [{
"handler": "reverse_proxy",
"upstreams": [{"dial": "localhost:3000"}]
}]
}]
}],
"terminal": true
}'
```
**Option B: Update Caddyfile and reload**
```bash
# Add the snippet to existing Caddyfile
sudo cat >> /etc/caddy/Caddyfile << 'EOF'
# Gitea Web Interface
git.terraphim.cloud {
reverse_proxy localhost:3000
log {
output file /var/log/caddy/gitea-access.log
format json
}
}
EOF
# Validate and reload
sudo caddy validate --config /etc/caddy/Caddyfile
sudo caddy reload --config /etc/caddy/Caddyfile
```
### Step 5: Update .gitignore
**Description:** Prevent committed secrets
**Tests:** .gitignore properly excludes sensitive files
**Dependencies:** Step 3
**Estimated:** 5 minutes
```bash
# Add to .gitignore
cat >> .gitignore << 'EOF'
# Secrets - Never commit these
docker-compose.yml
s3_config.json
.env
*.decrypted
*.secret
EOF
```
### Step 6: Transfer Files to Bigbox
**Description:** Transfer templates and configuration to bigbox
**Tests:** All files transferred successfully
**Dependencies:** Step 3, 4, 5
**Estimated:** 10 minutes
```bash
# Create remote directory
ssh bigbox "mkdir -p ~/gitea-stack"
# Transfer template files (not rendered files)
rsync -avz --progress \
--exclude='.git' \
--exclude='gitea' \
--exclude='postgres' \
--exclude='seaweedfs' \
--exclude='seaweedfs_filter' \
--exclude='docker-compose.yml' \
--exclude='s3_config.json' \
--exclude='.env' \
. bigbox:~/gitea-stack/
# Transfer Caddyfile separately
rsync -avz Caddyfile bigbox:~/gitea-stack/
# Verify transfer
ssh bigbox "ls -la ~/gitea-stack/"
```
### Step 7: Inject Secrets and Start Services
**Description:** Use `op inject` to populate secrets and start services
**Tests:** All containers start with correct environment variables
**Dependencies:** Step 6
**Estimated:** 15 minutes
```bash
# SSH to bigbox and execute deployment
ssh bigbox << 'REMOTESCRIPT'
# Change to project directory
cd ~/gitea-stack
# Source 1Password service account
echo "Loading 1Password credentials..."
source ~/op_zesticai_non_prod.sh
# Verify 1Password is working
echo "Verifying 1Password connection..."
op vault list
# Inject secrets into docker-compose.yml
echo "Injecting secrets into docker-compose.yml..."
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
# Inject secrets into s3_config.json
echo "Injecting secrets into s3_config.json..."
op inject --in-file s3_config.json.template --out-file s3_config.json
# Create data directories
echo "Creating data directories..."
mkdir -p gitea postgres seaweedfs seaweedfs_filter prometheus
# Set permissions for Gitea (runs as UID 1000)
chmod -R 1000:1000 gitea 2>/dev/null || sudo chown -R 1000:1000 gitea
# Start services
echo "Starting services..."
docker compose up -d
# Wait for startup
sleep 30
# Check status
echo "Checking service status..."
docker compose ps
REMOTESCRIPT
```
### Step 8: Configure Caddy (Add Gitea Route)
**Description:** Add Gitea route to existing Caddy instance on bigbox
**Tests:** Gitea route added and accessible via HTTPS
**Dependencies:** Step 7
**Estimated:** 15 minutes
**Important:** Bigbox already has Caddy running with existing services. We'll add the Gitea route without disrupting existing configuration.
```bash
ssh bigbox << 'REMOTESCRIPT'
# Create log directory
echo "Creating log directory..."
sudo mkdir -p /var/log/caddy
# Check if Caddy is running
echo "Checking Caddy status..."
if ! pgrep -x caddy > /dev/null; then
echo "ERROR: Caddy is not running on bigbox"
exit 1
fi
# Option 1: Add route via Caddy Admin API (recommended - no reload needed)
echo "Adding Gitea route to Caddy via Admin API..."
curl -s -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
-H "Content-Type: application/json" \
-d '{
"@id": "gitea_route",
"match": [{"host": ["git.terraphim.cloud"]}],
"handle": [{
"handler": "subroute",
"routes": [{
"handle": [{
"handler": "reverse_proxy",
"upstreams": [{"dial": "localhost:3000"}]
}]
}]
}],
"terminal": true
}'
# Verify route was added
echo "Verifying Caddy configuration..."
curl -s http://localhost:2019/config/apps/http/servers/srv0/routes | grep -q "git.terraphim.cloud" && echo "✓ Gitea route added successfully" || echo "✗ Failed to add Gitea route"
# Alternative: If API fails, backup to updating Caddyfile and reloading
# echo "Adding Gitea configuration to Caddyfile..."
# sudo tee -a /etc/caddy/Caddyfile > /dev/null << 'EOF'
#
# # Gitea Web Interface
# git.terraphim.cloud {
# reverse_proxy localhost:3000
# log {
# output file /var/log/caddy/gitea-access.log
# format json
# }
# }
# EOF
#
# # Validate and reload
# sudo caddy validate --config /etc/caddy/Caddyfile && sudo caddy reload --config /etc/caddy/Caddyfile
echo "Caddy configuration updated"
REMOTESCRIPT
```
### Step 9: Verify Deployment
**Description:** Validate all services are operational
**Tests:** Health checks pass for all services
**Dependencies:** Step 8
**Estimated:** 15 minutes
```bash
ssh bigbox << 'REMOTESCRIPT'
echo "=== Verifying Service Health ==="
# Test Gitea (via Caddy - public HTTPS)
echo "Testing Gitea via Caddy..."
curl -s -o /dev/null -w "Gitea HTTPS Status: %{http_code}\n" https://git.terraphim.cloud/api/healthz 2>/dev/null || echo "Gitea HTTPS not ready (DNS may need propagation)"
# Test Gitea (direct - localhost)
echo "Testing Gitea directly..."
curl -s -o /dev/null -w "Gitea HTTP Status: %{http_code}\n" http://localhost:3000/api/healthz || echo "Gitea not ready"
# Test S3 API (via Tailscale IP)
echo "Testing SeaweedFS S3 (Tailscale)..."
curl -s http://100.106.66.7:8333 > /dev/null && echo "S3 API (Tailscale): OK" || echo "S3 not ready"
# Test Prometheus (via Tailscale IP)
echo "Testing Prometheus (Tailscale)..."
curl -s http://100.106.66.7:9000/-/healthy > /dev/null && echo "Prometheus (Tailscale): OK" || echo "Prometheus not ready"
# Test SSH port
echo "Testing SSH port..."
nc -zv localhost 222 2>&1 | grep -q succeeded && echo "SSH port: OK" || echo "SSH port not responding"
# Check container logs for errors
echo ""
echo "=== Checking for errors in logs ==="
cd ~/gitea-stack
docker compose logs --tail=20 | grep -i error || echo "No errors found"
echo ""
echo "=== Container Status ==="
docker compose ps
REMOTESCRIPT
```
### Step 10: Document Access Endpoints
**Description:** Record access endpoints for the user
**Tests:** Documentation complete with all URLs
**Dependencies:** Step 9
**Estimated:** 10 minutes
**Public Access (via Caddy HTTPS):**
- Gitea Web: https://git.terraphim.cloud
- Gitea SSH: ssh://bigbox:222 (direct, not through Caddy)
**Tailscale Network Only (100.106.66.7):**
- Prometheus: http://100.106.66.7:9000
- SeaweedFS S3: http://100.106.66.7:8333
- SeaweedFS S3 Metrics: http://100.106.66.7:9327
**Internal Docker Network (no external access):**
- PostgreSQL: db:5432 (internal only)
- SeaweedFS Master: master:9333
- SeaweedFS Volume: volume:8080
- SeaweedFS Filer: filer:8888
**Local Development Access:**
- Gitea (direct): http://localhost:3000
**Important Notes:**
- Prometheus and S3 are NOT accessible from the public internet
- They are only accessible within the Tailscale VPN network
- Connect to Tailscale first: `tailscale up`
## Test Strategy
### Deployment Tests
| Test | Location | Purpose |
|------|----------|---------|
| SSH connectivity | Step 1 | Verify remote access |
| 1Password auth | Step 1 | Verify service account |
| Caddy validation | Step 8 | Config syntax correct |
| Secret injection | Step 7 | op:// references resolve |
| Service startup | Step 7 | All containers start |
| Caddy routing | Step 9 | HTTP requests routed correctly |
### Validation Commands
```bash
# Verify containers running
ssh bigbox "docker compose ps"
# Test Caddy routing
ssh bigbox "curl -s -o /dev/null -w '%{http_code}' http://localhost:3000"
# Verify no hardcoded secrets in templates
ssh bigbox "grep -r 'op://' ~/gitea-stack/*.template"
# Verify secrets resolved in compose file
ssh bigbox "grep -E 'PASSWORD|SECRET|KEY' ~/gitea-stack/docker-compose.yml | head -5"
# Test 1Password connectivity
ssh bigbox "source ~/op_zesticai_non_prod.sh && op item list"
```
## Rollback Plan
If deployment fails:
1. Stop services: `ssh bigbox "cd ~/gitea-stack && docker compose down"`
2. Stop Caddy: `ssh bigbox "sudo pkill caddy || sudo systemctl stop caddy"`
3. Remove data directories: `ssh bigbox "rm -rf ~/gitea-stack"`
4. Revert Caddy config: `ssh bigbox "sudo rm /etc/caddy/Caddyfile"`
5. Debug and retry
## Security Considerations
### Immediate Actions Post-Deployment
1. Change Gitea admin password after first login
2. Verify no secrets in git repository
3. Review Caddy access logs regularly
4. Set up firewall rules if needed
### Security Checklist
- [ ] No hardcoded secrets in git
- [ ] All secrets injected via 1Password
- [ ] Caddy automatically manages SSL certificates
- [ ] Direct service ports not exposed externally (except 222 for SSH)
- [ ] 1Password service account token secured on bigbox
### 1Password Security
- Service account token stored in `~/op_zesticai_non_prod.sh`
- Token has limited scope (zesticai-non-prod vault)
- File permissions should be restricted: `chmod 600 ~/op_zesticai_non_prod.sh`
## Open Items
| Item | Status | Owner |
|------|--------|-------|
| Create 1Password vault items | Pending | Deployer |
| Determine Caddy domain names | Pending | Deployer |
| Verify 80/443 port availability | Pending | Deployer |
| Set up DNS records | Pending | Deployer (if using real domain) |
## Deployment Commands Reference
### Quick Deploy (Full)
```bash
# 1. Create 1Password items locally
source ~/op_zesticai_non_prod.sh
op item create --vault="zesticai-non-prod" --category=login --title="gitea-postgres" --generate-password=20 username=gitea
op item create --vault="zesticai-non-prod" --category=custom --title="gitea-s3" access-key="$(openssl rand -hex 20)" secret-key="$(openssl rand -hex 40)"
# 2. Transfer files
rsync -avz --exclude='.git' --exclude='gitea' --exclude='postgres' \
--exclude='seaweedfs' --exclude='seaweedfs_filter' \
--exclude='docker-compose.yml' --exclude='s3_config.json' \
. bigbox:~/gitea-stack/
# 3. Deploy on bigbox
ssh bigbox "cd ~/gitea-stack && source ~/op_zesticai_non_prod.sh && op inject --in-file docker-compose.yml.template --out-file docker-compose.yml && op inject --in-file s3_config.json.template --out-file s3_config.json && docker compose up -d"
# 4. Configure Caddy
ssh bigbox "sudo cp ~/gitea-stack/Caddyfile /etc/caddy/Caddyfile && sudo caddy validate --config /etc/caddy/Caddyfile && sudo caddy run --config /etc/caddy/Caddyfile &"
```
### Status Check
```bash
ssh bigbox "cd ~/gitea-stack && docker compose ps"
ssh bigbox "curl -s http://localhost:3000/api/healthz && echo 'Gitea OK'"
ssh bigbox "sudo caddy list-modules 2>&1 | head -5"
```
### Stop Services
```bash
ssh bigbox "cd ~/gitea-stack && docker compose down"
ssh bigbox "sudo pkill caddy || sudo systemctl stop caddy"
```
### View Logs
```bash
ssh bigbox "cd ~/gitea-stack && docker compose logs -f"
ssh bigbox "sudo tail -f /var/log/caddy/gitea-access.log"
```
## Approval
- [ ] Research document reviewed
- [ ] Implementation plan reviewed
- [ ] 1Password vault structure defined
- [ ] Security considerations acknowledged
- [ ] Rollback plan understood
- [ ] Ready to proceed with deployment
---
**Next Steps After Approval:**
1. Create 1Password vault items
2. Create template files locally
3. Transfer to bigbox
4. Execute deployment
5. Validate all services via Caddy
.docs/RESEARCH-gitea-bigbox-deployment.md
+322
View File
@@ -0,0 +1,322 @@
# Research Document: Gitea Deployment to Bigbox with 1Password Secrets Management
**Status**: Draft
**Author**: Claude
**Date**: 2026-02-16
**Reviewers**: [Pending]
## Executive Summary
This project deploys a comprehensive Gitea-based development infrastructure stack including Git hosting (Gitea), object storage (SeaweedFS), monitoring (Prometheus), and PostgreSQL database to a remote server called "bigbox". This deployment uses **1Password secrets management** via `op inject` to securely manage credentials and includes **Caddy reverse proxy** for SSL termination and routing. The environment is configured using the `op_zesticai_non_prod.sh` script for 1Password authentication.
## Essential Questions Check
| Question | Answer | Evidence |
|----------|--------|----------|
| Energizing? | Yes | Provides self-hosted Git infrastructure with production-grade secrets management |
| Leverages strengths? | Yes | Uses 1Password (already configured), Docker Compose patterns, SSH deployment |
| Meets real need? | Yes | Creates secure build machine with proper secrets management and reverse proxy |
**Proceed**: Yes - All 3 questions answered YES
## Problem Statement
### Description
Deploy a Gitea-based development infrastructure stack to a remote server (bigbox) using SSH with:
- **1Password secrets management** for all credentials (no hardcoded secrets)
- **Caddy reverse proxy** for SSL termination and routing
- **Secure credential injection** via `op inject`
- Environment authentication via `source op_zesticai_non_prod.sh`
The stack includes:
- Gitea (Git hosting with Actions support)
- PostgreSQL (database)
- SeaweedFS (distributed object storage with S3 API)
- Prometheus (monitoring)
- Caddy (reverse proxy with automatic HTTPS)
### Impact
Who is affected:
- Developers needing self-hosted Git repository management
- Teams requiring production-grade security with proper secrets management
- Projects needing S3-compatible object storage with secure credential handling
### Success Criteria
- [ ] All services start successfully on bigbox
- [ ] Gitea accessible via HTTPS (via Caddy reverse proxy)
- [ ] SeaweedFS S3 API accessible with secure credentials
- [ ] Prometheus monitoring accessible (via Caddy or directly)
- [ ] Data persistence across container restarts
- [ ] Services communicate properly within Docker network
- [ ] No hardcoded credentials in configuration files
- [ ] Caddy configured with proper reverse proxy rules
- [ ] 1Password secrets injected via `op inject`
## Current State Analysis
### Existing Implementation
The project uses Docker Compose with the following services:
| Component | Location | Purpose |
|-----------|----------|---------|
| Gitea | `docker-compose.yml:8` | Git hosting server with Actions and LFS support |
| PostgreSQL | `docker-compose.yml:46` | Database for Gitea |
| SeaweedFS Master | `docker-compose.yml:57` | Metadata management for distributed storage |
| SeaweedFS Volume | `docker-compose.yml:68` | Data storage nodes |
| SeaweedFS Filer | `docker-compose.yml:81` | File metadata and directory operations |
| SeaweedFS S3 | `docker-compose.yml:101` | S3-compatible API gateway |
| Prometheus | `docker-compose.yml:145` | Metrics collection and monitoring |
| **Caddy** | **NEW** | Reverse proxy with automatic HTTPS |
### Bigbox Environment Verified
- **1Password CLI**: Installed (`/usr/bin/op`)
- **1Password Service Account**: `~/op_zesticai_non_prod.sh` configured with `OP_SERVICE_ACCOUNT_TOKEN`
- **Docker**: 28.1.1 installed
- **Docker Compose**: v2.35.1 installed
- **Caddy**: Installed but no service configured
- **Hardware**: 125GB RAM, 3.5TB disk (367GB free)
### Secrets Management Architecture
```
1Password Vault (non-prod)
│
│ op inject
▼
op_zesticai_non_prod.sh ←── Export OP_SERVICE_ACCOUNT_TOKEN
│
▼
docker-compose.yml.template
│
│ op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
▼
docker-compose.yml (secrets populated)
│
▼
Containers (receive secrets as environment variables)
```
### Data Flow
```
Internet ──▶ Caddy (:443/:80)
│
├──▶ Gitea (:3000)
│ └──▶ PostgreSQL (:5432)
│
├──▶ SeaweedFS S3 (:8333)
│ └──▶ Filer (:8888) ──▶ Master (:9333) + Volume (:8080)
│
└──▶ Prometheus (:9000)
```
### Integration Points
- Gitea uses SeaweedFS S3 for LFS and attachment storage
- Prometheus scrapes metrics from all SeaweedFS components
- Services communicate via internal Docker network "gitea"
- Caddy terminates SSL and routes to services
- 1Password injects secrets at deployment time
## Constraints
### Technical Constraints
- **Docker Compose Version**: Uses version 3 syntax
- **Container Images**: Fixed versions (gitea:1.22.6, postgres:15, prom/prometheus:v2.21.0)
- **SeaweedFS**: Uses "latest" tag (chrislusf/seaweedfs)
- **1Password CLI**: Must be available and authenticated via service account
- **Caddy**: Installed but needs configuration
- **Port Requirements**:
- 80/443 (Caddy - external)
- 3000 (Gitea HTTP - internal)
- 222 (Gitea SSH - external)
- 8333 (S3 API - internal or external via Caddy)
- 9000 (Prometheus - internal or external via Caddy)
### Security Constraints
- **No Hardcoded Secrets**: All credentials must come from 1Password
- **Service Account Authentication**: Must source `op_zesticai_non_prod.sh` before `op inject`
- **Template Files**: Use `.template` extension for files with 1Password references
- **Secret Rotation**: Support easy rotation via 1Password updates
### Non-Functional Requirements
| Requirement | Target | Current |
|-------------|--------|---------|
| Security | No hardcoded secrets | Hardcoded credentials exist |
| Secrets Management | 1Password integration | Not implemented |
| SSL/TLS | Automatic via Caddy | Not configured |
| Availability | 99% uptime | Not measured |
## Vital Few (Essentialism)
### Essential Constraints (Max 3)
| Constraint | Why It's Vital | Evidence |
|------------|----------------|----------|
| 1Password CLI and service account | Required for secrets injection | `/usr/bin/op` and `~/op_zesticai_non_prod.sh` verified |
| Caddy reverse proxy | SSL termination and routing | Production-grade deployment requirement |
| SSH Access to bigbox | Required for deployment | `ssh -G bigbox` confirms configuration |
### Eliminated from Scope
| Eliminated Item | Why Eliminated |
|-----------------|----------------|
| Gitea Runners (TODO) | Listed as future work in README |
| Earthly Buildkit | Listed as future work in README |
| Nginx S3 Gateway | Caddy replaces this need |
| Panamax (crates mirror) | Commented out in compose file |
| Automated backups | Out of scope for initial deployment |
| Custom domain SSL certs | Caddy handles automatic HTTPS |
## Dependencies
### Internal Dependencies
| Dependency | Impact | Risk |
|------------|--------|------|
| Gitea | Depends on PostgreSQL and S3 | High - core service |
| SeaweedFS S3 | Depends on Filer, Volume, Master | High - storage layer |
| Caddy | Depends on all web services | High - external access |
| 1Password | Required for secrets injection | High - security dependency |
| Prometheus | Depends on all services | Low - monitoring only |
### External Dependencies
| Dependency | Version | Risk | Alternative |
|------------|---------|------|-------------|
| Gitea | 1.22.6 | Low | Upgrade path available |
| PostgreSQL | 15 | Low | Well established |
| SeaweedFS | latest | Medium | Version pinning recommended |
| Prometheus | v2.21.0 | Low | Established monitoring |
| Caddy | latest | Low | Stable reverse proxy |
| 1Password CLI | latest | Low | Enterprise secrets management |
## Risks and Unknowns
### Known Risks
| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| 1Password authentication failure | Low | High | Verify service account token before deployment |
| Caddy configuration errors | Medium | High | Test config with `caddy validate` |
| Port conflicts on bigbox | Medium | High | Verify ports before deployment |
| Secrets injection failure | Low | High | Validate templates before deployment |
| SSL certificate issues | Low | Medium | Caddy handles automatically |
### Open Questions
1. What 1Password vault/items contain the required secrets? - [Need to document]
2. What domain/subdomain will Caddy use? - [Need to determine]
3. Are required ports available (80/443)? - [Port scan required]
4. Existing Caddy configuration on bigbox? - [Verified: none active]
### Assumptions Explicitly Stated
| Assumption | Basis | Risk if Wrong | Verified? |
|------------|-------|---------------|-----------|
| 1Password CLI available on bigbox | `/usr/bin/op` exists | Secrets injection fails | Yes |
| op_zesticai_non_prod.sh has valid token | File exists with token | Authentication fails | Yes (token present) |
| Caddy installed and working | `/usr/bin/caddy` exists | Reverse proxy fails | Yes |
| Docker is installed on bigbox | Docker 28.1.1 verified | Deployment fails | Yes |
| Sufficient disk space available | 367GB available verified | Out of space | Yes |
| Network ports are available | Not verified | Port conflicts | No |
### Multiple Interpretations Considered
| Interpretation | Implications | Why Chosen/Rejected |
|----------------|--------------|---------------------|
| 1Password Connect Server | Centralized secret serving | Rejected - adds infrastructure complexity |
| Docker Secrets | Native Docker secrets | Rejected - requires Swarm mode |
| op inject with templates | Simple, effective | Chosen - minimal complexity, works with Compose |
| Caddy with Docker Compose | Containerized Caddy | Alternative - but native Caddy is simpler |
## Research Findings
### Key Insights
1. **Production-Ready Security**: 1Password integration eliminates hardcoded credentials
2. **Caddy Simplification**: Automatic HTTPS eliminates SSL certificate management
3. **Service Account Auth**: `op_zesticai_non_prod.sh` provides secure 1Password access
4. **Template Pattern**: `.template` files with 1Password references enable secure deployment
5. **Complete Stack**: Full infrastructure with proper secrets management and reverse proxy
### 1Password Secrets Required
Based on current configuration, the following secrets must exist in 1Password:
| Secret | Purpose | Current Hardcoded Value |
|--------|---------|------------------------|
| `op://zesticai-non-prod/gitea-postgres/password` | PostgreSQL password | gitea |
| `op://zesticai-non-prod/gitea-s3/access-key` | S3 access key | REDACTED_S3_ACCESS_KEY |
| `op://zesticai-non-prod/gitea-s3/secret-key` | S3 secret key | REDACTED_S3_SECRET_KEY |
| `op://zesticai-non-prod/gitea-jwt/secret` | Gitea JWT secret | (not currently set) |
### Caddy Configuration Requirements
```caddyfile
# Caddyfile for gitea-stack
gitea.bigbox.example.com {
reverse_proxy localhost:3000
}
s3.bigbox.example.com {
reverse_proxy localhost:8333
}
prometheus.bigbox.example.com {
reverse_proxy localhost:9000
}
```
### Technical Spikes Needed
| Spike | Purpose | Estimated Effort |
|-------|---------|------------------|
| Verify 1Password vault structure | Confirm required secrets exist | 30 minutes |
| Test op inject workflow | Validate secrets injection | 30 minutes |
| Design Caddy configuration | Determine routing rules | 1 hour |
| Test Caddy with services | Validate reverse proxy | 1 hour |
## Recommendations
### Proceed/No-Proceed
**Proceed** - Bigbox has all required tools (1Password CLI, Docker, Caddy), and the 1Password service account is already configured.
### Scope Recommendations
1. **Phase 1**: Implement 1Password secrets management
2. **Phase 2**: Add Caddy reverse proxy configuration
3. **Phase 3**: Deploy and validate all services
### Risk Mitigation Recommendations
1. Test 1Password authentication before deployment
2. Validate Caddy configuration with `caddy validate`
3. Create backup of existing data before deployment
4. Use staging/test domain initially for SSL validation
5. Document all 1Password vault items required
## Next Steps
If approved:
1. Create 1Password vault items for all required secrets
2. Create Implementation Plan with 1Password and Caddy integration
3. Test `op inject` workflow locally
4. Execute deployment following plan
5. Validate all services and Caddy routing
## Appendix
### Reference Materials
- docker-compose.yml: Main deployment configuration
- s3_config.json: SeaweedFS S3 gateway credentials (to be replaced with 1Password)
- prometheus/prometheus.yml: Monitoring configuration
- README.md: Project overview and TODOs
- op_zesticai_non_prod.sh: 1Password service account configuration
### 1Password CLI Usage
```bash
# Source the service account token
source ~/op_zesticai_non_prod.sh
# Inject secrets into template
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
# Or run with op run
op run --env-file=.env -- docker compose up -d
```
### Caddy Commands
```bash
# Validate configuration
caddy validate --config /etc/caddy/Caddyfile
# Start Caddy (as service or directly)
sudo caddy run --config /etc/caddy/Caddyfile
# Reload configuration
sudo caddy reload --config /etc/caddy/Caddyfile
```
.gitignore
+43
View File
@@ -0,0 +1,43 @@
# Gitea Stack - Git Ignore
# This file excludes secrets and local data from version control
# Secrets - Never commit files with resolved credentials
# These are generated by op inject from .template files
docker-compose.yml
s3_config.json
.env
*.decrypted
*.secret
secrets/
# Data directories - Local state, not for git
/gitea/
/postgres/
/seaweedfs/
/seaweedfs_filter/
/prometheus/data/
# Logs
*.log
logs/
# OS generated files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# Temporary files
*.tmp
*.temp
.cache/
ARTICLE-gitea-deployment.md
+315
View File
@@ -0,0 +1,315 @@
# Deploying Gitea with 1Password Secrets Management and Caddy on Bigbox
**Date:** February 16, 2026
**Author:** Infrastructure Team
**Tags:** gitea, 1password, caddy, docker, tailscale, devops
## Executive Summary
This article documents the deployment of a production-ready Gitea instance on bigbox server, featuring enterprise-grade security with 1Password secrets management, Caddy reverse proxy with automatic HTTPS, and network isolation via Tailscale.
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ Internet │
│ │ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Caddy (bigbox) │ │
│ │ :443/:80 │ │
│ └──────────┬───────────┘ │
│ │ │
│ ┌──────────────┼──────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ workflows. git.terraphim. *.terraphim. │
│ terraphim. cloud (NEW) cloud │
│ cloud │ (catchall) │
│ ▼ │
│ ┌──────────────┐ │
│ │ Gitea │ │
│ │ :3000 │ │
│ └──────┬───────┘ │
│ │ │
└───────────────────┼─────────────────────────────────────────────┘
│
┌───────────┴───────────┐
│ Tailscale VPN │
│ (100.106.66.0/8) │
└───────────┬───────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
Prometheus S3 API SeaweedFS
:9000 :8333 (Internal)
```
## Key Features
### 1. Security-First Design
- **No hardcoded secrets** - All credentials stored in 1Password
- **1Password integration** - Secrets injected at runtime via `op inject`
- **Template-based configuration** - `.template` files in git, resolved configs excluded
- **Network isolation** - Internal services not exposed to internet
### 2. Multi-Layer Network Security
| Service | Exposure | Access Method |
|---------|----------|---------------|
| Gitea Web | Public | HTTPS via Caddy (git.terraphim.cloud) |
| Gitea SSH | Public | Port 222 direct |
| Prometheus | Tailscale only | http://100.106.66.7:9000 |
| S3 API | Tailscale only | http://100.106.66.7:8333 |
| PostgreSQL | Internal only | Docker network |
| SeaweedFS | Internal only | Docker network |
### 3. Technology Stack
- **Gitea 1.22.6** - Git hosting with Actions support
- **PostgreSQL 15** - Database
- **SeaweedFS** - Distributed S3-compatible object storage
- **Prometheus** - Metrics and monitoring
- **Caddy** - Reverse proxy with automatic HTTPS
- **1Password CLI** - Secrets management
- **Tailscale** - VPN for internal services
## Deployment Process
### Prerequisites
- SSH access to bigbox
- 1Password service account configured (`~/op_zesticai_non_prod.sh`)
- Caddy already running on bigbox
- Docker and Docker Compose installed
### Step 1: Create 1Password Vault Items
```bash
eval $(op signin)
# Create PostgreSQL credentials
op item create \
--vault="TerraphimPlatform" \
--category=login \
--title="gitea-postgres" \
--generate-password=20,letters,digits \
username=gitea
# Create S3 credentials
op item create \
--vault="TerraphimPlatform" \
--category=custom \
--title="gitea-s3" \
access-key="$(openssl rand -hex 20)" \
secret-key="$(openssl rand -hex 40)"
```
### Step 2: Template Configuration Files
Create `.template` files with 1Password references:
**docker-compose.yml.template:**
```yaml
version: "3"
services:
server:
image: gitea/gitea:1.22.6
environment:
- GITEA__database__PASSWD=op://TerraphimPlatform/gitea-postgres/password
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://TerraphimPlatform/gitea-s3/access-key
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://TerraphimPlatform/gitea-s3/secret-key
# ... other config
```
**s3_config.json.template:**
```json
{
"identities": [{
"name": "gitea",
"credentials": [{
"accessKey": "op://TerraphimPlatform/gitea-s3/access-key",
"secretKey": "op://TerraphimPlatform/gitea-s3/secret-key"
}]
}]
}
```
### Step 3: Inject Secrets and Deploy
```bash
# Source 1Password credentials
source ~/op_zesticai_non_prod.sh
# Inject secrets into configuration files
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
op inject --in-file s3_config.json.template --out-file s3_config.json
# Start services
docker compose up -d
```
### Step 4: Configure Caddy
Add Gitea route to existing Caddy instance:
```bash
# Add route via Caddy Admin API
curl -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
-H "Content-Type: application/json" \
-d '{
"@id": "gitea_route",
"match": [{"host": ["git.terraphim.cloud"]}],
"handle": [{
"handler": "subroute",
"routes": [{
"handle": [{
"handler": "reverse_proxy",
"upstreams": [{"dial": "localhost:3000"}]
}]
}]
}],
"terminal": true
}'
```
**Important:** Route order matters! Add specific routes before wildcard routes.
## Network Configuration Details
### Port Bindings
**Gitea (Public):**
- `127.0.0.1:3000:3000` - HTTP via Caddy
- `0.0.0.0:222:22` - SSH direct access
**Prometheus (Tailscale only):**
- `100.106.66.7:9000:9090` - Only accessible via Tailscale
**S3 API (Tailscale only):**
- `100.106.66.7:8333:8333` - Only accessible via Tailscale
- `100.106.66.7:9327:9327` - Prometheus metrics
### Docker Compose Network Isolation
```yaml
networks:
gitea:
external: false # Internal Docker network only
services:
db:
networks:
- gitea # No external ports
master:
# No ports exposed - internal only
volume:
# No ports exposed - internal only
filer:
# No ports exposed - internal only
```
## Security Considerations
### 1. Secrets Management
- ✅ No secrets in Git repository
- ✅ No secrets in Docker images
- ✅ Secrets injected at runtime
- ✅ Easy rotation via 1Password
### 2. Network Security
- ✅ Public services behind Caddy with automatic HTTPS
- ✅ Internal services on Tailscale VPN
- ✅ Database isolated in Docker network
- ✅ SSH on non-standard port (222)
### 3. Access Control
- Prometheus and S3 require Tailscale connection
- Gitea accessible publicly via HTTPS
- SSH access available on port 222
## Troubleshooting
### Permission Denied Errors
If Gitea shows permission errors:
```bash
sudo chown -R 1000:1000 gitea/
sudo chmod -R 755 gitea/
docker compose restart server
```
### Route Not Working
Check Caddy route order:
```bash
curl -s http://localhost:2019/config/apps/http/servers/srv0/routes | \
jq '.[] | select(.match[0].host[0] | contains("terraphim")) | .match[0].host[0]'
```
Specific routes must come before wildcard routes.
### 1Password Authentication
Verify service account:
```bash
source ~/op_zesticai_non_prod.sh
op vault list
```
## Maintenance
### Backup
```bash
# Backup data directories
rsync -avz bigbox:~/gitea-stack/gitea ./backup/
rsync -avz bigbox:~/gitea-stack/postgres ./backup/
rsync -avz bigbox:~/gitea-stack/seaweedfs ./backup/
```
### Update Images
```bash
cd ~/gitea-stack
docker compose pull
docker compose up -d
```
### Rotate Secrets
1. Update in 1Password vault
2. Re-run deployment:
```bash
source ~/op_zesticai_non_prod.sh
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
docker compose up -d
```
## Access Points
### Public URLs
- **Gitea Web:** https://git.terraphim.cloud
- **Gitea SSH:** ssh://git@bigbox:222
### Tailscale-only URLs
Connect to Tailscale first: `tailscale up`
- **Prometheus:** http://100.106.66.7:9000
- **S3 API:** http://100.106.66.7:8333
## Conclusion
This deployment demonstrates production-ready practices:
- Zero hardcoded secrets
- Defense in depth with network isolation
- Automatic HTTPS via Caddy
- Easy maintenance with Docker Compose
- Centralized secrets management with 1Password
The architecture balances security (Tailscale isolation, 1Password integration) with usability (public HTTPS access to Gitea, direct SSH access).
## References
- [Gitea Documentation](https://docs.gitea.io/)
- [1Password CLI](https://developer.1password.com/docs/cli/)
- [Caddy Documentation](https://caddyserver.com/docs/)
- [Tailscale](https://tailscale.com/)
---
**Repository:** [Private terraphim repo with configuration]
Caddyfile
+15
View File
@@ -0,0 +1,15 @@
# Caddyfile for gitea-stack on bigbox
# This adds Gitea to the existing Caddy instance on bigbox
# Copy this snippet to the existing Caddy configuration at /etc/caddy/Caddyfile
# or use 'caddy reload' to add this route to the running Caddy instance
# Gitea Web Interface - Public via terraphim.cloud domain
git.terraphim.cloud {
reverse_proxy localhost:3000
# Logging
log {
output file /var/log/caddy/gitea-access.log
format json
}
}
README.md
+64 -3
View File
@@ -1,8 +1,69 @@
# Ultimate build machine using gitea # Ultimate Build Machine using Gitea
Kind of kitchen sink everything is included Self-hosted Git infrastructure with S3-compatible storage, monitoring, and proper secrets management.
## Stack Components
- **Gitea** (1.22.6) - Git hosting with Actions and LFS support
- **PostgreSQL** (15) - Database
- **SeaweedFS** - Distributed S3-compatible object storage
- **Prometheus** (v2.21.0) - Metrics and monitoring
- **Caddy** - Reverse proxy with automatic HTTPS
- **1Password** - Secrets management via `op inject`
## Quick Start
### Prerequisites
1. **1Password Vault Items** - Create in `zesticai-non-prod` vault:
```bash
op item create --vault="zesticai-non-prod" --category=login \
--title="gitea-postgres" --generate-password=20 username=gitea
op item create --vault="zesticai-non-prod" --category=custom \
--title="gitea-s3" access-key="$(openssl rand -hex 20)" \
secret-key="$(openssl rand -hex 40)"
```
2. **SSH Access** - Ensure `ssh bigbox` works
3. **1Password Service Account** - `~/op_zesticai_non_prod.sh` on bigbox
### Deploy to Bigbox
```bash
./deploy-to-bigbox.sh
```
### Access Services
After deployment:
**Public (via HTTPS on terraphim.cloud):**
- **Gitea Web**: https://git.terraphim.cloud
- **Gitea SSH**: ssh://bigbox:222
**Tailscale Network Only (100.106.66.7):**
- **Prometheus**: http://100.106.66.7:9000
- **S3 API**: http://100.106.66.7:8333
*Note: Prometheus and S3 are only accessible within the Tailscale VPN network, not from the public internet.*
## Documentation
- [Research Document](.docs/RESEARCH-gitea-bigbox-deployment.md) - Phase 1 analysis
- [Implementation Plan](.docs/IMPLEMENTATION-gitea-bigbox-deployment.md) - Phase 2 design
- [Deployment Summary](.docs/DEPLOYMENT-SUMMARY.md) - Post-deployment guide
## Security
- **No hardcoded secrets** - All credentials managed via 1Password
- **Template files** (`.template`) contain `op://` references
- **Generated files** excluded from git (`.gitignore`)
- **Caddy** provides automatic HTTPS
- **Secret injection** at deployment time via `op inject`
## TODO ## TODO
- [ ] Automatic spin up and registration of gitea runners (github compatible) - [ ] Automatic spin up and registration of gitea runners (github compatible)
- [ ] Earthly buildkit demon (not sattelite - with gitea runners we don't need earthly sattelites) - [ ] Earthly buildkit daemon (not satellite - with gitea runners we don't need earthly satellites)
REPOSITORY-SETUP.md
+235
View File
@@ -0,0 +1,235 @@
# Repository Setup Guide
## What to Include in Git
### ✅ Include These Files:
**Configuration Templates (Safe):**
- `docker-compose.yml.template` - Template with op:// references
- `s3_config.json.template` - Template with op:// references
- `Caddyfile` - Caddy configuration (no secrets)
**Documentation:**
- `README.md` - Project overview
- `ARTICLE-gitea-deployment.md` - Deployment article
- `.docs/` - All documentation
**Scripts:**
- `deploy-to-bigbox.sh` - Deployment automation
**Git Configuration:**
- `.gitignore` - Excludes secrets and data
### ❌ DO NOT Include:
**Files with Resolved Secrets:**
- ❌ `docker-compose.yml` - Contains actual passwords
- ❌ `s3_config.json` - Contains actual credentials
- ❌ `.env` - Environment variables with secrets
**Data Directories:**
- ❌ `gitea/` - Application data
- ❌ `postgres/` - Database files
- ❌ `seaweedfs/` - Object storage data
- ❌ `seaweedfs_filter/` - Filer data
**Logs:**
- ❌ `*.log` - Log files
## Step-by-Step: Create Repository and Push
### Step 1: Initialize Repository Locally
```bash
cd /home/alex/infrastructure/gitea_config
# Remove any existing git history (if needed)
# rm -rf .git
# Initialize fresh repository
git init
# Add all safe files
git add README.md
git add ARTICLE-gitea-deployment.md
git add Caddyfile
git add deploy-to-bigbox.sh
git add docker-compose.yml.template
git add s3_config.json.template
git add .gitignore
git add .docs/
# Commit
git commit -m "Initial commit: Gitea deployment configuration
- Docker Compose templates with 1Password integration
- Caddy reverse proxy configuration
- Deployment automation script
- Documentation and article
- Security: No secrets or data included"
```
### Step 2: Create Repository in Gitea
**Option A: Via Web Interface**
1. Go to https://git.terraphim.cloud
2. Click "+" → "New Repository"
3. Owner: Select "terraphim" organization
4. Repository Name: `gitea-infrastructure`
5. Visibility: **Private**
6. Check "Initialize Repository" (optional)
7. Click "Create Repository"
**Option B: Via Gitea CLI (if installed)**
```bash
# Create repository under terraphim org
gitea admin repo create \
--owner terraphim \
--name gitea-infrastructure \
--private \
--description "Gitea deployment configuration with 1Password and Caddy"
```
### Step 3: Add Remote and Push
```bash
# Add the Gitea remote
git remote add origin https://git.terraphim.cloud/terraphim/gitea-infrastructure.git
# Or use SSH:
# git remote add origin ssh://git@bigbox:222/terraphim/gitea-infrastructure.git
# Push to main branch
git branch -M main
git push -u origin main
```
### Step 4: Verify Repository
```bash
# Check repository on Gitea
curl -s https://git.terraphim.cloud/terraphim/gitea-infrastructure | head -20
# Or visit in browser:
# https://git.terraphim.cloud/terraphim/gitea-infrastructure
```
## Repository Structure After Push
```
gitea-infrastructure/
├── .docs/
│ ├── RESEARCH-gitea-bigbox-deployment.md
│ ├── IMPLEMENTATION-gitea-bigbox-deployment.md
│ ├── DEPLOYMENT-SUMMARY.md
│ └── summary-*.md
├── .gitignore
├── ARTICLE-gitea-deployment.md
├── Caddyfile
├── README.md
├── deploy-to-bigbox.sh
├── docker-compose.yml.template
└── s3_config.json.template
```
## Security Checklist
Before pushing, verify:
- [ ] No `docker-compose.yml` in repository (only `.template`)
- [ ] No `s3_config.json` in repository (only `.template`)
- [ ] No `.env` file
- [ ] No data directories (`gitea/`, `postgres/`, etc.)
- [ ] No log files
- [ ] All secrets use `op://` references in templates
- [ ] `.gitignore` properly configured
## Quick Verification Script
```bash
#!/bin/bash
echo "Checking for files that should NOT be committed..."
FORBIDDEN=(
"docker-compose.yml"
"s3_config.json"
".env"
"*.secret"
"*.decrypted"
)
for file in "${FORBIDDEN[@]}"; do
if [ -f "$file" ]; then
echo "❌ WARNING: $file exists (should not be committed)"
fi
done
echo ""
echo "Checking .gitignore..."
if grep -q "docker-compose.yml" .gitignore; then
echo "✅ docker-compose.yml is ignored"
else
echo "❌ docker-compose.yml is NOT ignored"
fi
if grep -q "s3_config.json" .gitignore; then
echo "✅ s3_config.json is ignored"
else
echo "❌ s3_config.json is NOT ignored"
fi
echo ""
echo "Files ready to commit:"
git ls-files
```
## Deploying from Repository
After cloning on a new machine:
```bash
# Clone repository
git clone https://git.terraphim.cloud/terraphim/gitea-infrastructure.git
cd gitea-infrastructure
# Create 1Password vault items (if not exist)
# See ARTICLE-gitea-deployment.md for instructions
# Source credentials and deploy
source ~/op_zesticai_non_prod.sh
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
op inject --in-file s3_config.json.template --out-file s3_config.json
# Start services
docker compose up -d
```
## Maintenance Workflow
**To update configuration:**
1. Edit `.template` files
2. Commit and push
3. On bigbox: `git pull` and re-run deployment
**To rotate secrets:**
1. Update in 1Password vault
2. On bigbox: Re-run `op inject` and `docker compose up -d`
3. No git changes needed!
**To backup data (NOT in git):**
```bash
# Backup data directories separately
rsync -avz bigbox:~/gitea-stack/gitea /backup/
rsync -avz bigbox:~/gitea-stack/postgres /backup/
```
---
**Repository:** https://git.terraphim.cloud/terraphim/gitea-infrastructure
deploy-to-bigbox.sh
Executable
+273
View File
@@ -0,0 +1,273 @@
#!/bin/bash
#
# Deploy Gitea Stack to Bigbox with 1Password Secrets Management
#
# Prerequisites:
# - SSH access to bigbox configured
# - 1Password CLI installed locally and on bigbox
# - op_zesticai_non_prod.sh exists on bigbox with valid service account token
# - 1Password vault items created (gitea-postgres, gitea-s3)
# - Caddy already running on bigbox
#
# Network Configuration:
# - Gitea: Public via Caddy (git.terraphim.cloud) -> localhost:3000
# - Prometheus: Tailscale only (100.106.66.7:9000)
# - S3: Tailscale only (100.106.66.7:8333)
#
# Usage:
# ./deploy-to-bigbox.sh
#
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Configuration
BIGBOX="bigbox"
REMOTE_DIR="~/gitea-stack"
TAILSCALE_IP="100.106.66.7"
echo -e "${GREEN}=== Gitea Stack Deployment to Bigbox ===${NC}"
echo ""
# Step 1: Verify local prerequisites
echo -e "${YELLOW}Step 1: Verifying local prerequisites...${NC}"
if ! command -v rsync &> /dev/null; then
echo -e "${RED}Error: rsync is not installed${NC}"
exit 1
fi
if ! ssh -o ConnectTimeout=5 "$BIGBOX" "echo 'SSH OK'" &> /dev/null; then
echo -e "${RED}Error: Cannot connect to $BIGBOX via SSH${NC}"
exit 1
fi
echo -e "${GREEN}✓ Local prerequisites verified${NC}"
echo ""
# Step 2: Verify bigbox prerequisites
echo -e "${YELLOW}Step 2: Verifying bigbox prerequisites...${NC}"
ssh "$BIGBOX" << 'CHECKSCRIPT'
set -e
# Check Docker
if ! command -v docker &> /dev/null; then
echo "Error: Docker not installed"
exit 1
fi
# Check Docker Compose
if ! docker compose version &> /dev/null; then
echo "Error: Docker Compose not installed"
exit 1
fi
# Check 1Password CLI
if ! command -v op &> /dev/null; then
echo "Error: 1Password CLI not installed"
exit 1
fi
# Check service account script
if [ ! -f "$HOME/op_zesticai_non_prod.sh" ]; then
echo "Error: op_zesticai_non_prod.sh not found in home directory"
exit 1
fi
# Test 1Password authentication
source "$HOME/op_zesticai_non_prod.sh"
if ! op vault list &> /dev/null; then
echo "Error: 1Password authentication failed"
exit 1
fi
# Check Caddy is running
if ! pgrep -x caddy > /dev/null; then
echo "Error: Caddy is not running"
exit 1
fi
# Check Caddy admin API is accessible
if ! curl -s http://localhost:2019/config/ > /dev/null; then
echo "Error: Caddy admin API is not accessible"
exit 1
fi
echo "All prerequisites verified"
CHECKSCRIPT
echo -e "${GREEN}✓ Bigbox prerequisites verified${NC}"
echo ""
# Step 3: Transfer files
echo -e "${YELLOW}Step 3: Transferring files to bigbox...${NC}"
ssh "$BIGBOX" "mkdir -p $REMOTE_DIR"
rsync -avz --progress \
--exclude='.git' \
--exclude='gitea' \
--exclude='postgres' \
--exclude='seaweedfs' \
--exclude='seaweedfs_filter' \
--exclude='docker-compose.yml' \
--exclude='s3_config.json' \
--exclude='.env' \
--exclude='*.log' \
--exclude='.docs/' \
. "$BIGBOX:$REMOTE_DIR/"
echo -e "${GREEN}✓ Files transferred${NC}"
echo ""
# Step 4: Inject secrets and start services
echo -e "${YELLOW}Step 4: Injecting secrets and starting services...${NC}"
ssh "$BIGBOX" << DEPLOYSCRIPT
set -e
cd $REMOTE_DIR
echo "Loading 1Password credentials..."
source ~/op_zesticai_non_prod.sh
echo "Injecting secrets into docker-compose.yml..."
op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
echo "Injecting secrets into s3_config.json..."
op inject --in-file s3_config.json.template --out-file s3_config.json
echo "Creating data directories..."
mkdir -p gitea postgres seaweedfs seaweedfs_filter prometheus
echo "Setting permissions..."
chmod -R 1000:1000 gitea 2>/dev/null || sudo chown -R 1000:1000 gitea 2>/dev/null || true
echo "Starting services with Docker Compose..."
docker compose up -d
echo "Waiting for services to start..."
sleep 30
echo "Service status:"
docker compose ps
DEPLOYSCRIPT
echo -e "${GREEN}✓ Services started${NC}"
echo ""
# Step 5: Add Gitea route to Caddy
echo -e "${YELLOW}Step 5: Adding Gitea route to Caddy...${NC}"
ssh "$BIGBOX" << CADDYSCRIPT
set -e
echo "Creating log directory..."
sudo mkdir -p /var/log/caddy
echo "Adding Gitea route to Caddy via Admin API..."
curl -s -X POST http://localhost:2019/config/apps/http/servers/srv0/routes \
-H "Content-Type: application/json" \
-d '{
"@id": "gitea_route",
"match": [{"host": ["git.terraphim.cloud"]}],
"handle": [{
"handler": "subroute",
"routes": [{
"handle": [{
"handler": "reverse_proxy",
"upstreams": [{"dial": "localhost:3000"}]
}]
}]
}],
"terminal": true
}'
# Verify route was added
if curl -s http://localhost:2019/config/apps/http/servers/srv0/routes | grep -q "git.terraphim.cloud"; then
echo "✓ Gitea route added successfully to Caddy"
else
echo "✗ Warning: Could not verify Gitea route was added"
fi
CADDYSCRIPT
echo -e "${GREEN}✓ Caddy configured${NC}"
echo ""
# Step 6: Verify deployment
echo -e "${YELLOW}Step 6: Verifying deployment...${NC}"
ssh "$BIGBOX" << VERIFYSCRIPT
set -e
cd $REMOTE_DIR
echo ""
echo "=== Service Health Checks ==="
# Check Gitea (direct)
if curl -s -o /dev/null -w '%{http_code}' http://localhost:3000/api/healthz | grep -q "200"; then
echo "✓ Gitea is healthy (localhost:3000)"
else
echo "✗ Gitea is not responding"
fi
# Check S3 (Tailscale)
if curl -s http://${TAILSCALE_IP}:8333 > /dev/null 2>&1; then
echo "✓ SeaweedFS S3 is responding (${TAILSCALE_IP}:8333)"
else
echo "✗ SeaweedFS S3 is not responding"
fi
# Check Prometheus (Tailscale)
if curl -s http://${TAILSCALE_IP}:9000/-/healthy > /dev/null 2>&1; then
echo "✓ Prometheus is healthy (${TAILSCALE_IP}:9000)"
else
echo "✗ Prometheus is not responding"
fi
# Check SSH port
if nc -zv localhost 222 2>&1 | grep -q succeeded; then
echo "✓ SSH port is open (port 222)"
else
echo "✗ SSH port is not responding"
fi
# Check Caddy
if pgrep -x caddy > /dev/null; then
echo "✓ Caddy is running"
else
echo "✗ Caddy is not running"
fi
echo ""
echo "=== Container Status ==="
docker compose ps
echo ""
echo "=== Recent Logs ==="
docker compose logs --tail=10
VERIFYSCRIPT
echo ""
echo -e "${GREEN}=== Deployment Complete ===${NC}"
echo ""
echo "Access your services:"
echo ""
echo "Public (via HTTPS):"
echo " - Gitea Web: https://git.terraphim.cloud"
echo ""
echo "Tailscale Network Only (${TAILSCALE_IP}):"
echo " - Prometheus: http://${TAILSCALE_IP}:9000"
echo " - S3 API: http://${TAILSCALE_IP}:8333"
echo ""
echo "SSH Access:"
echo " - Gitea SSH: ssh://bigbox:222"
echo ""
echo "Note: Prometheus and S3 are NOT accessible from the public internet."
echo " Connect via Tailscale first: tailscale up"
echo ""
echo "Next steps:"
echo " 1. Ensure DNS points git.terraphim.cloud to bigbox"
echo " 2. Access Gitea at https://git.terraphim.cloud to complete setup"
echo " 3. Change default admin password after first login"
echo " 4. Review Caddy logs: ssh bigbox 'sudo tail -f /var/log/caddy/gitea-access.log'"
echo ""
docker-compose.yml
-170
View File
@@ -1,170 +0,0 @@
version: "3"
# FIXME: create default gitea bucket using rclone or aws cli
networks:
gitea:
external: false
services:
server:
image: gitea/gitea:1.21.7
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=postgres
- GITEA__database__HOST=db:5432
- GITEA__database__NAME=gitea
- GITEA__database__USER=gitea
- GITEA__database__PASSWD=gitea
- GITEA__actions__ENABLE=true
- GITEA__repository__LFS_START_SERVER=true
- GITEA__repository__LFS_CONTENT_PATH=/data/lfs
- GITEA__storage__type=minio
- GITEA__storage__MINIO_ACCESS_KEY_ID=REDACTED_S3_ACCESS_KEY
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=REDACTED_S3_SECRET_KEY
- GITEA__storage__MINIO_BUCKET=gitea
- GITEA__storage__MINIO_LOCATION=us-east-1
- GITEA__storage__MINIO_ENDPOINT=http://s3:8333
- GITEA__storage__MINIO_INSECURE_SKIP_VERIFY=false
- GITEA__storage__MINIO_USE_SSL=false
- GITEA__storage__SERVE_DIRECT=true
restart: always
networks:
- gitea
volumes:
- ./gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
ports:
- "3000:3000"
- "222:22"
depends_on:
- db
- s3
db:
image: postgres:15
restart: always
environment:
- POSTGRES_USER=gitea
- POSTGRES_PASSWORD=gitea
- POSTGRES_DB=gitea
networks:
- gitea
volumes:
- ./postgres:/var/lib/postgresql/data
master:
image: chrislusf/seaweedfs # use a remote image
ports:
- 9333:9333
- 19333:19333
- 9324:9324
command: "master -ip=master -ip.bind=0.0.0.0 -metricsPort=9324"
# healthcheck:
# test: [ "CMD", "curl", "--fail", "-I", "http://master:9333/cluster/healthz" ]
# interval: 1s
# timeout: 120s
volume:
image: chrislusf/seaweedfs # use a remote image
ports:
- 8080:8080
- 18080:18080
- 9325:9325
command: 'volume -mserver="master:9333" -ip.bind=0.0.0.0 -port=8080 -metricsPort=9325'
depends_on:
- master
# master:
# condition: service_healthy
volumes:
- ./seaweedfs:/data
filer:
image: chrislusf/seaweedfs # use a remote image
ports:
- 8888:8888
- 18888:18888
- 9326:9326
command: 'filer -master="master:9333" -ip.bind=0.0.0.0 -metricsPort=9326'
tty: true
stdin_open: true
depends_on:
- master
- volume
# volume:
# condition: service_healthy
volumes:
- ./seaweedfs_filter:/data
# healthcheck:
# test: [ "CMD", "curl", "--fail", "-I", "http://volume:8888" ]
# interval: 1s
# timeout: 30s
s3:
image: chrislusf/seaweedfs
container_name: s3storage
hostname: s3storage # use a remote image
ports:
- 8333:8333
- 9327:9327
command: 's3 -config=/etc/seaweedfs/s3.json -filer="filer:8888" -ip.bind=0.0.0.0 -metricsPort=9327'
depends_on:
- master
- volume
- filer
volumes:
- ./s3_config.json:/etc/seaweedfs/s3.json
nginx-gateway-s3:
image: ghcr.io/nginxinc/nginx-s3-gateway/nginx-oss-s3-gateway:latest
container_name: nginx-s3-gateway
ports:
- 9080:80
restart: on-failure
environment:
S3_BUCKET_NAME: gitea
AWS_ACCESS_KEY_ID: "REDACTED_S3_ACCESS_KEY"
AWS_SECRET_ACCESS_KEY: "REDACTED_S3_SECRET_KEY"
AWS_SIGS_VERSION: 4
S3_SERVER: s3
S3_SERVER_PORT: 8333
S3_SERVER_PROTO: http
S3_REGION: us-east-1
S3_STYLE: path
DEBUG: false
ALLOW_DIRECTORY_LIST: false
PROVIDE_INDEX_PAGE: true
APPEND_SLASH_FOR_POSSIBLE_DIRECTORY: true
DIRECTORY_LISTING_PATH_PREFIX: ""
PROXY_CACHE_MAX_SIZE: 10g
PROXY_CACHE_SLICE_SIZE: 1m
PROXY_CACHE_INACTIVE: 60m
PROXY_CACHE_VALID_OK: 1h
PROXY_CACHE_VALID_NOTFOUND: 1m
PROXY_CACHE_VALID_FORBIDDEN: 30s
# STRIP_LEADING_DIRECTORY_PATH: /somepath
depends_on:
- s3
prometheus:
image: prom/prometheus:v2.21.0
ports:
- 9000:9090
volumes:
- ./prometheus:/etc/prometheus
command: --web.enable-lifecycle --config.file=/etc/prometheus/prometheus.yml
depends_on:
- s3
# panamax:
# image: panamaxrs/panamax
# container_name: crates
# volumes:
# - ./crates_data/:/mirror
# # entrypoint: ["/usr/local/bin/panamax"]
# # command: "/usr/local/bin/panamax init /mirror &&
# # /usr/local/bin/panamax sync /mirror &&
# # /usr/local/bin/panamax serve /mirror"
# command:
# - sync /mirror
# # - serve
# ports:
# - 18080:8080
# environment:
# - USER_UID=1000
# - USER_GID=1000
docker-compose.yml.template
+134
View File
@@ -0,0 +1,134 @@
version: "3"
# Template file - use 'op inject' to generate docker-compose.yml
# This file contains 1Password references (op://) and should NOT be deployed directly
# Run: op inject --in-file docker-compose.yml.template --out-file docker-compose.yml
#
# Network Configuration:
# - Gitea: Public via Caddy (git.terraphim.cloud) -> localhost:3000
# - Prometheus: Tailscale only (100.106.66.7:9000)
# - S3: Tailscale only (100.106.66.7:8333)
# - Other SeaweedFS: Internal Docker network only (no external exposure)
networks:
gitea:
external: false
services:
server:
image: gitea/gitea:1.22.6
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=postgres
- GITEA__database__HOST=db:5432
- GITEA__database__NAME=gitea
- GITEA__database__USER=gitea
- GITEA__database__PASSWD=op://TerraphimPlatform/gitea-postgres/password
- GITEA__actions__ENABLE=true
- GITEA__repository__LFS_START_SERVER=true
- GITEA__repository__LFS_CONTENT_PATH=/data/lfs
- GITEA__storage__type=minio
- GITEA__storage__MINIO_ACCESS_KEY_ID=op://TerraphimPlatform/gitea-s3/access-key
- GITEA__storage__MINIO_SECRET_ACCESS_KEY=op://TerraphimPlatform/gitea-s3/secret-key
- GITEA__storage__MINIO_BUCKET=gitea
- GITEA__storage__MINIO_LOCATION=us-east-1
# Use Tailscale IP for internal S3 access
- GITEA__storage__MINIO_ENDPOINT=http://100.106.66.7:8333
- GITEA__storage__MINIO_INSECURE_SKIP_VERIFY=false
- GITEA__storage__MINIO_USE_SSL=false
- GITEA__storage__SERVE_DIRECT=true
user: "1000:1000"
restart: always
networks:
- gitea
volumes:
- ./gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
ports:
# Gitea HTTP - localhost only (accessed via Caddy reverse proxy)
- "127.0.0.1:3000:3000"
# Gitea SSH - exposed on all interfaces
- "222:22"
depends_on:
- db
- s3
db:
image: postgres:15
restart: always
environment:
- POSTGRES_USER=gitea
- POSTGRES_PASSWORD=op://TerraphimPlatform/gitea-postgres/password
- POSTGRES_DB=gitea
networks:
- gitea
volumes:
- ./postgres:/var/lib/postgresql/data
# No external ports - internal Docker network only
master:
image: chrislusf/seaweedfs
# No external ports - internal Docker network only
command: "master -ip=master -ip.bind=0.0.0.0 -metricsPort=9324"
networks:
- gitea
volume:
image: chrislusf/seaweedfs
# No external ports - internal Docker network only
command: 'volume -mserver="master:9333" -ip.bind=0.0.0.0 -port=8080 -metricsPort=9325'
depends_on:
- master
volumes:
- ./seaweedfs:/data
networks:
- gitea
filer:
image: chrislusf/seaweedfs
# No external ports - internal Docker network only
command: 'filer -master="master:9333" -ip.bind=0.0.0.0 -metricsPort=9326'
tty: true
stdin_open: true
depends_on:
- master
- volume
volumes:
- ./seaweedfs_filter:/data
networks:
- gitea
s3:
image: chrislusf/seaweedfs
container_name: s3storage
hostname: s3storage
ports:
# Bind to Tailscale interface only (100.106.66.7)
# Accessible only within Tailscale network, not public internet
- "100.106.66.7:8333:8333"
- "100.106.66.7:9327:9327"
command: 's3 -config=/etc/seaweedfs/s3.json -filer="filer:8888" -ip.bind=0.0.0.0 -metricsPort=9327'
depends_on:
- master
- volume
- filer
volumes:
- ./s3_config.json:/etc/seaweedfs/s3.json
networks:
- gitea
prometheus:
image: prom/prometheus:v2.21.0
ports:
# Bind to Tailscale interface only (100.106.66.7)
# Accessible only within Tailscale network, not public internet
- "100.106.66.7:9000:9090"
volumes:
- ./prometheus:/etc/prometheus
command: --web.enable-lifecycle --config.file=/etc/prometheus/prometheus.yml
depends_on:
- s3
networks:
- gitea
s3_config.json
-27
View File
@@ -1,27 +0,0 @@
{
"identities": [
{
"name": "anonymous",
"actions": [
"Read"
]
},
{
"name": "gitea",
"credentials": [
{
"accessKey": "REDACTED_S3_ACCESS_KEY",
"secretKey": "REDACTED_S3_SECRET_KEY"
}
],
"actions": [
"Admin",
"Read",
"List",
"Tagging",
"Write"
]
}
]
}
s3_config.json.template
+27
View File
@@ -0,0 +1,27 @@
{
"_comment": "Template file - use 'op inject' to generate s3_config.json",
"identities": [
{
"name": "anonymous",
"actions": [
"Read"
]
},
{
"name": "gitea",
"credentials": [
{
"accessKey": "op://TerraphimPlatform/gitea-s3/access-key",
"secretKey": "op://TerraphimPlatform/gitea-s3/secret-key"
}
],
"actions": [
"Admin",
"Read",
"List",
"Tagging",
"Write"
]
}
]
}