mirror of
https://github.com/terraphim/gitea-infrastructure.git
synced 2026-07-16 00:00:32 +02:00
fix: Update S3 config and add Gitea URL settings
- Change MINIO_ENDPOINT from IP to Docker hostname (s3storage) - Add ROOT_URL and DOMAIN settings for proper Gitea configuration - Give anonymous users Admin permission for bucket creation - Document SeaweedFS bucket creation process
This commit is contained in:
Alex
@@ -0,0 +1,336 @@
|
||||
# Gitea Storage Architecture: Local vs S3
|
||||
|
||||
## Overview
|
||||
|
||||
Gitea uses a **hybrid storage model** with both local filesystem and S3 (SeaweedFS) storage. This document explains what is stored where and how to verify the configuration.
|
||||
|
||||
## Storage Types in Gitea
|
||||
|
||||
### 1. Git Repositories (Local Only)
|
||||
|
||||
**Location:** `/data/git/` inside the container
|
||||
**Storage Type:** Local filesystem only
|
||||
**S3 Backup:** Possible via external tools
|
||||
|
||||
```
|
||||
/data/git/repositories/
|
||||
├── terraphim/
|
||||
│ └── gitea-infrastructure.git/
|
||||
│ ├── HEAD
|
||||
│ ├── config
|
||||
│ ├── description
|
||||
│ ├── hooks/
|
||||
│ ├── info/
|
||||
│ ├── objects/
|
||||
│ └── refs/
|
||||
```
|
||||
|
||||
**Why local?**
|
||||
- Git operations require fast filesystem access
|
||||
- Object storage is too slow for git's object database
|
||||
- S3 is used for auxiliary data only
|
||||
|
||||
**Verify:**
|
||||
```bash
|
||||
ssh bigbox
|
||||
cd ~/gitea-stack
|
||||
docker compose exec server ls -la /data/git/repositories/
|
||||
```
|
||||
|
||||
### 2. Git LFS (Large File Storage)
|
||||
|
||||
**Current Configuration:** Local storage
|
||||
**Location:** `/data/lfs/`
|
||||
**Can be moved to:** S3 (SeaweedFS)
|
||||
|
||||
**Current Setting:**
|
||||
```ini
|
||||
[repository]
|
||||
LFS_START_SERVER = true
|
||||
LFS_CONTENT_PATH = /data/lfs # Local path
|
||||
```
|
||||
|
||||
**Verify:**
|
||||
```bash
|
||||
docker compose exec server ls -la /data/lfs/
|
||||
```
|
||||
|
||||
**Status:** ❌ **Not configured for S3** (using local path despite MinIO settings)
|
||||
|
||||
### 3. Attachments & Files (S3 via MinIO)
|
||||
|
||||
**Location:** S3 (SeaweedFS)
|
||||
**Bucket:** `gitea`
|
||||
**Endpoint:** `http://100.106.66.7:8333`
|
||||
|
||||
**What's stored in S3:**
|
||||
- Issue attachments
|
||||
- Release attachments
|
||||
- Repository avatars
|
||||
- User avatars
|
||||
- Package files
|
||||
- Actions artifacts
|
||||
|
||||
**Configuration:**
|
||||
```ini
|
||||
[storage]
|
||||
MINIO_USE_SSL = false
|
||||
MINIO_LOCATION = us-east-1
|
||||
MINIO_ENDPOINT = http://100.106.66.7:8333
|
||||
MINIO_BUCKET = gitea
|
||||
MINIO_ACCESS_KEY_ID = [from 1Password]
|
||||
MINIO_SECRET_ACCESS_KEY = [from 1Password]
|
||||
type = minio
|
||||
```
|
||||
|
||||
**Verify S3 connection:**
|
||||
```bash
|
||||
# Check S3 bucket
|
||||
curl http://100.106.66.7:8333/gitea
|
||||
|
||||
# Or using AWS CLI
|
||||
aws --endpoint-url http://100.106.66.7:8333 s3 ls s3://gitea/
|
||||
```
|
||||
|
||||
### 4. Container Registry
|
||||
|
||||
**Location:** Local by default
|
||||
**Can be configured for:** S3 storage
|
||||
|
||||
**Configuration needed:**
|
||||
```ini
|
||||
[storage.packages]
|
||||
type = minio
|
||||
MINIO_ENDPOINT = http://100.106.66.7:8333
|
||||
MINIO_BUCKET = gitea-registry
|
||||
```
|
||||
|
||||
**Status:** ⚠️ **Not configured yet**
|
||||
|
||||
## Current Configuration Analysis
|
||||
|
||||
### What's Actually Configured
|
||||
|
||||
| Data Type | Location | S3 Configured | Notes |
|
||||
|-----------|----------|---------------|-------|
|
||||
| Git repos | Local (`/data/git/`) | ❌ No | Must be local |
|
||||
| LFS files | Local (`/data/lfs/`) | ⚠️ Partial | Config in env but LFS path is local |
|
||||
| Attachments | S3 (planned) | ✅ Yes | Config present |
|
||||
| Avatars | S3 (planned) | ✅ Yes | Config present |
|
||||
| Packages | Local | ❌ No | Not configured |
|
||||
| Registry | Local | ❌ No | Not configured |
|
||||
| Actions logs | Local | ✅ Yes | Uses storage config |
|
||||
|
||||
### Configuration Issue Found
|
||||
|
||||
**Problem:** The app.ini shows placeholders instead of actual secrets:
|
||||
|
||||
```ini
|
||||
MINIO_ACCESS_KEY_ID = VAULT_REFTerraphimPlatform/gitea-s3/access-key
|
||||
```
|
||||
|
||||
**Should be:**
|
||||
```ini
|
||||
MINIO_ACCESS_KEY_ID = actual_access_key_value
|
||||
```
|
||||
|
||||
**Impact:** S3 storage won't work until secrets are properly injected.
|
||||
|
||||
## Storage Architecture Diagram
|
||||
|
||||
```
|
||||
Gitea Server
|
||||
│
|
||||
├── /data/git/ ← Git repositories (LOCAL)
|
||||
│ └── repositories/
|
||||
│ └── terraphim/
|
||||
│ └── gitea-infrastructure.git/
|
||||
│ └── [git objects]
|
||||
│
|
||||
├── /data/lfs/ ← LFS files (LOCAL)
|
||||
│ └── [large files]
|
||||
│
|
||||
├── /data/gitea/attachments/ ← Attachments (LOCAL - not S3!)
|
||||
│
|
||||
└── [S3 via MinIO] ← Object storage (SeaweedFS)
|
||||
└── Bucket: gitea
|
||||
└── [attachments, avatars, packages]
|
||||
↑
|
||||
│
|
||||
SeaweedFS S3
|
||||
100.106.66.7:8333
|
||||
```
|
||||
|
||||
## Fixing the Configuration
|
||||
|
||||
### Step 1: Verify Secrets Were Injected
|
||||
|
||||
```bash
|
||||
ssh bigbox
|
||||
cd ~/gitea-stack
|
||||
|
||||
# Check if secrets are resolved
|
||||
grep "MINIO_ACCESS_KEY_ID" docker-compose.yml
|
||||
# Should show actual value, not op://
|
||||
```
|
||||
|
||||
### Step 2: Re-inject Secrets if Needed
|
||||
|
||||
```bash
|
||||
source ~/op_zesticai_non_prod.sh
|
||||
cat docker-compose.yml.template | grep -v '^#' | op inject --out-file docker-compose.yml
|
||||
cat s3_config.json.template | grep -v '^ *//' | grep -v '_comment' | op inject --out-file s3_config.json
|
||||
```
|
||||
|
||||
### Step 3: Restart Gitea
|
||||
|
||||
```bash
|
||||
docker compose restart server
|
||||
```
|
||||
|
||||
### Step 4: Verify S3 Connectivity
|
||||
|
||||
```bash
|
||||
# Test S3 connection from Gitea container
|
||||
docker compose exec server wget -qO- http://100.106.66.7:8333/gitea || echo "S3 not accessible"
|
||||
```
|
||||
|
||||
## Moving LFS to S3 (Optional)
|
||||
|
||||
To store LFS files in S3 instead of local:
|
||||
|
||||
### Option 1: Update Configuration
|
||||
|
||||
Edit Gitea config in UI:
|
||||
1. Admin Panel → Configuration → "Repository - LFS"
|
||||
2. Change storage type to "minio"
|
||||
3. Set endpoint: `100.106.66.7:8333`
|
||||
4. Set bucket: `gitea-lfs`
|
||||
5. Restart Gitea
|
||||
|
||||
### Option 2: Edit app.ini
|
||||
|
||||
```bash
|
||||
# Backup current config
|
||||
docker compose exec server cp /data/gitea/conf/app.ini /data/gitea/conf/app.ini.bak
|
||||
|
||||
# Edit config
|
||||
docker compose exec server sed -i 's|^LFS_CONTENT_PATH.*|# LFS_CONTENT_PATH moved to S3|' /data/gitea/conf/app.ini
|
||||
|
||||
# Add LFS storage section
|
||||
docker compose exec server sh -c 'cat >> /data/gitea/conf/app.ini << EOF
|
||||
|
||||
[lfs]
|
||||
STORAGE_TYPE = minio
|
||||
MINIO_ENDPOINT = 100.106.66.7:8333
|
||||
MINIO_BUCKET = gitea-lfs
|
||||
EOF'
|
||||
|
||||
# Restart
|
||||
docker compose restart server
|
||||
```
|
||||
|
||||
### Step 3: Migrate Existing LFS Files
|
||||
|
||||
```bash
|
||||
# If you have existing LFS files to migrate:
|
||||
docker compose exec server gitea admin lfs-migrate
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### What to Back Up
|
||||
|
||||
**Critical - Must Back Up:**
|
||||
```bash
|
||||
# Git repositories
|
||||
cd ~/gitea-stack && tar czf backup-git-$(date +%Y%m%d).tar.gz gitea/git/
|
||||
|
||||
# Database
|
||||
cd ~/gitea-stack && docker compose exec -T db pg_dump -U gitea gitea > backup-db-$(date +%Y%m%d).sql
|
||||
|
||||
# Configuration
|
||||
cd ~/gitea-stack && tar czf backup-config-$(date +%Y%m%d).tar.gz gitea/gitea/conf/
|
||||
```
|
||||
|
||||
**Optional - Can Be Regenerated:**
|
||||
- LFS files (if stored in S3, backup S3 bucket)
|
||||
- Attachments (if stored in S3, backup S3 bucket)
|
||||
- Cache files
|
||||
|
||||
**S3 Backup:**
|
||||
```bash
|
||||
# Backup entire S3 bucket
|
||||
aws --endpoint-url http://100.106.66.7:8333 s3 sync s3://gitea ./backup-s3/
|
||||
```
|
||||
|
||||
## Monitoring Storage
|
||||
|
||||
### Check Disk Usage
|
||||
|
||||
```bash
|
||||
# Git repositories
|
||||
docker compose exec server du -sh /data/git/
|
||||
|
||||
# LFS files
|
||||
docker compose exec server du -sh /data/lfs/
|
||||
|
||||
# Local attachments
|
||||
docker compose exec server du -sh /data/gitea/attachments/
|
||||
|
||||
# Everything
|
||||
docker compose exec server du -sh /data/
|
||||
```
|
||||
|
||||
### Check S3 Usage
|
||||
|
||||
```bash
|
||||
# List S3 buckets
|
||||
curl http://100.106.66.7:8333/
|
||||
|
||||
# Check bucket contents
|
||||
aws --endpoint-url http://100.106.66.7:8333 s3 ls s3://gitea/ --recursive
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
### What's Stored Where?
|
||||
|
||||
| Component | Local Path | S3 Path | Recommendation |
|
||||
|-----------|------------|---------|----------------|
|
||||
| **Git repos** | `/data/git/` | N/A | Keep local, backup externally |
|
||||
| **LFS files** | `/data/lfs/` | Not configured | Configure for S3 if large files |
|
||||
| **Attachments** | `/data/gitea/attachments/` | `s3://gitea/attachments/` | Move to S3 |
|
||||
| **Avatars** | `/data/gitea/avatars/` | `s3://gitea/avatars/` | Move to S3 |
|
||||
| **Packages** | `/data/gitea/packages/` | Not configured | Configure if using registry |
|
||||
| **Actions logs** | `/data/gitea/actions_log/` | Configured for S3 | Verify working |
|
||||
|
||||
### Current Status
|
||||
|
||||
✅ **Working:**
|
||||
- Git repositories (local)
|
||||
- Gitea application (running)
|
||||
- S3/MinIO endpoint accessible
|
||||
|
||||
⚠️ **Needs Attention:**
|
||||
- Secrets not properly injected (showing VAULT_REF)
|
||||
- LFS using local storage
|
||||
- Attachments may still be local despite config
|
||||
|
||||
❌ **Not Configured:**
|
||||
- Container registry (not using S3)
|
||||
|
||||
### Next Steps
|
||||
|
||||
1. Fix secret injection in docker-compose.yml
|
||||
2. Restart Gitea with proper S3 configuration
|
||||
3. Verify attachments are being stored in S3
|
||||
4. (Optional) Configure LFS for S3
|
||||
5. (Optional) Configure container registry for S3
|
||||
|
||||
---
|
||||
|
||||
**Questions?** Check the Gitea logs:
|
||||
```bash
|
||||
docker compose logs server -f | grep -i storage
|
||||
```
|
||||
Reference in New Issue
Block a user