terraphim/gitea-infrastructure
16
0
Fork 0
mirror of https://github.com/terraphim/gitea-infrastructure.git synced 2026-07-16 00:00:32 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity

refactor: Replace GitHub Actions mirroring with Gitea native pull mirror

Browse Source 
- Remove sync-to-gitea.yml and mirror-check.yml GitHub Actions workflows
- Set up Gitea as pull mirror from GitHub (1h interval, via POST /repos/migrate)
- Rewrite MIRRORING-GUIDE.md with pull mirror setup and gotchas
- Update HANDOVER.md and lessons-learned.md with new approach

Gitea now automatically pulls from GitHub. No Actions, no dual-push,
no billing dependency. Push to GitHub only; Gitea syncs within 1 hour.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Alex
2026-02-18 15:24:37 +01:00
parent a98941d74d
commit 1c8ab5939e
5 changed files with 253 additions and 511 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MIRRORING-GUIDE.md
+127 -430
View File
@@ -1,467 +1,164 @@
# Repository Mirroring: GitHub ↔ Gitea
Complete guide for setting up mirroring between GitHub and self-hosted Gitea.
# Repository Mirroring: GitHub -> Gitea Pull Mirror
## Overview
**Mirroring** keeps two repositories in sync automatically. Choose your direction:
GitHub is the primary repository. Gitea automatically pulls from GitHub every hour using its native pull mirror feature. No GitHub Actions, no manual dual-push, no cron jobs.
| Direction | Use Case |
|-----------|----------|
| **GitHub → Gitea** | GitHub as primary, Gitea as backup/mirror |
| **Gitea → GitHub** | Gitea as primary, GitHub for visibility/CI |
| **Bidirectional** | Both accept commits (advanced) |
```
Developer -> git push -> GitHub (primary)
|
v (automatic, every 1h)
Gitea (read-only mirror)
```
---
## Current Configuration
## Method 1: Gitea as Mirror (GitHub → Gitea)
| Setting | Value |
|---------|-------|
| Primary | https://github.com/terraphim/gitea-infrastructure (private) |
| Mirror | https://git.terraphim.cloud/terraphim/gitea-infrastructure |
| Direction | GitHub -> Gitea (one-way pull) |
| Interval | 1 hour |
| Service | `github` (uses GitHub API for private repo access) |
| Auth | GitHub token stored in Gitea migration config |
Gitea periodically pulls from GitHub. Best for backup scenarios.
## Daily Workflow
### Step 1: Create Mirror in Gitea
**Via Web Interface:**
1. Go to https://git.terraphim.cloud
2. Click **"+"** → **"New Migration"**
3. Select **"GitHub"**
4. Enter URL: `https://github.com/terraphim/gitea-infrastructure`
5. **Authentication:**
- If private repo: Enter GitHub Personal Access Token
- If public: Leave empty
6. Check **"This repository will be a mirror"**
7. Click **"Migrate Repository"**
**Via Gitea API:**
Push to GitHub only. Gitea syncs automatically.
```bash
# Get Gitea API token (Settings → Applications → Generate Token)
GITEA_TOKEN="your_gitea_api_token"
git push origin main
```
# Create mirror
curl -X POST \
"https://git.terraphim.cloud/api/v1/repos/migrate" \
-H "Authorization: token ${GITEA_TOKEN}" \
That is all. Gitea pulls within 1 hour.
## Manual Sync
Trigger an immediate sync without waiting for the interval:
```bash
source ~/op_zesticai_non_prod.sh
GITEA_TOKEN=$(op read "op://TerraphimPlatform/git.terraphim.cloud-admin-token/credential")
curl -s -X POST \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure/mirror-sync" \
-H "Authorization: token $GITEA_TOKEN"
```
## Check Mirror Status
```bash
source ~/op_zesticai_non_prod.sh
GITEA_TOKEN=$(op read "op://TerraphimPlatform/git.terraphim.cloud-admin-token/credential")
curl -s "https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure" \
-H "Authorization: token $GITEA_TOKEN" | \
jq '{mirror, mirror_interval, mirror_updated, empty, size}'
```
## Change Sync Interval
```bash
curl -s -X PATCH \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure" \
-H "Authorization: token $GITEA_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"clone_addr": "https://github.com/terraphim/gitea-infrastructure",
"repo_name": "gitea-infrastructure-mirror",
"repo_owner": "terraphim",
"mirror": true,
"auth_token": "your_github_token_if_private"
}'
-d '{"mirror_interval": "30m"}'
```
### Step 2: Configure Mirror Settings
Minimum interval is 10 minutes (enforced by Gitea server config).
1. Go to repository **Settings** → **Repository**
2. Scroll to **Mirror Settings**
3. Set **Synchronization Interval**: `8h` (or your preference)
4. Click **"Update Mirror"**
## Recreating the Mirror
### Step 3: Manual Sync (Optional)
If the mirror breaks (e.g., GitHub token expires, repo gets corrupted):
```bash
# Trigger immediate sync
curl -X POST \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure-mirror/mirror-sync" \
-H "Authorization: token ${GITEA_TOKEN}"
source ~/op_zesticai_non_prod.sh
GITEA_TOKEN=$(op read "op://TerraphimPlatform/git.terraphim.cloud-admin-token/credential")
GH_TOKEN=$(gh auth token)
# Step 1: Delete existing mirror
curl -s -X DELETE \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure" \
-H "Authorization: token $GITEA_TOKEN"
# Step 2: Wait for deletion to complete
sleep 10
# Step 3: Verify deleted
curl -s "https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure" \
-H "Authorization: token $GITEA_TOKEN" -w "\nHTTP %{http_code}\n"
# Should return HTTP 404
# Step 4: Recreate as pull mirror
curl -s -X POST "https://git.terraphim.cloud/api/v1/repos/migrate" \
-H "Authorization: token $GITEA_TOKEN" \
-H "Content-Type: application/json" \
-d "{
\"clone_addr\": \"https://github.com/terraphim/gitea-infrastructure\",
\"repo_name\": \"gitea-infrastructure\",
\"repo_owner\": \"terraphim\",
\"service\": \"github\",
\"auth_token\": \"$GH_TOKEN\",
\"mirror\": true,
\"mirror_interval\": \"1h\"
}"
# Step 5: Verify (wait a few seconds for async clone)
sleep 15
curl -s "https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure" \
-H "Authorization: token $GITEA_TOKEN" | \
jq '{id, mirror, mirror_interval, mirror_updated, empty, size}'
# Should show: mirror: true, empty: false
```
---
## Critical Gotchas
## Method 2: GitHub as Mirror (Gitea → GitHub)
### 1. Existing repos cannot be converted to mirrors
Push from Gitea to GitHub. Best when Gitea is your primary.
Gitea only creates mirrors via the `POST /repos/migrate` endpoint. There is no way to convert an existing repo to a mirror. You must delete and recreate.
### Step 1: Add GitHub Remote to Local Repository
### 2. The `service` field matters for private repos
```bash
cd /home/alex/infrastructure/gitea_config
| Service | Behavior |
|---------|----------|
| `github` | Uses GitHub API. Requires `auth_token` for private repos. |
| `git` | Uses `git clone`. Fails with "terminal prompts disabled" for private repos. |
# Add GitHub as mirror remote
git remote add github https://github.com/terraphim/gitea-infrastructure.git
Always use `service: "github"` with a valid GitHub token for private repositories.
# Or use SSH (recommended for automation)
git remote add github git@github.com:terraphim/gitea-infrastructure.git
```
### 3. Failed migrations leave empty repos
### Step 2: Configure Push Mirroring in Gitea
If the async clone fails (bad token, network error), Gitea still creates the repo entry with `mirror: true, empty: true`. The `mirror-sync` endpoint returns "Repository is not a mirror" on these broken repos. Delete and recreate.
**Option A: Gitea Push Mirror (Native)**
### 4. The `github.personal.token` in 1Password is expired
1. In Gitea, go to repository **Settings** → **Repository**
2. Scroll to **Mirror Settings**
3. Check **"Push Mirror"**
4. Set **Git Remote URL**: `https://github.com/terraphim/gitea-infrastructure.git`
5. Add authentication:
- **Username**: Your GitHub username
- **Password/Token**: GitHub Personal Access Token
6. Set **Interval**: `8h`
7. Click **"Add Push Mirror"**
As of 2026-02-18, the `op://TerraphimPlatform/github.personal.token/token` returns a token that GitHub rejects with HTTP 401. Use `gh auth token` instead for a working token.
**Option B: Using Git Hooks (More Control)**
### 5. Wait between delete and create
Create a post-receive hook in Gitea:
Gitea needs time to fully delete a repo. Without a `sleep 10` between delete and create, you get `409 Conflict: The repository with the same name already exists`.
1. Go to repository **Settings** → **Git Hooks**
2. Find **post-receive** hook
3. Edit and add:
## Why Not GitHub Actions?
```bash
#!/bin/bash
# Push to GitHub after receiving commits
The previous approach used two GitHub Actions workflows:
- `sync-to-gitea.yml` -- pushed to Gitea on every GitHub push
- `mirror-check.yml` -- checked sync status every 6 hours
# Use deploy key or HTTPS with token
export GIT_SSH_COMMAND='ssh -i /path/to/deploy_key -o StrictHostKeyChecking=no'
Problems:
1. GitHub Actions billing issues caused silent failures
2. Required storing a `GITEA_TOKEN` secret in GitHub
3. Consumed GitHub Actions minutes
4. Added unnecessary complexity
git push github main --mirror
The Gitea pull mirror replaces both workflows with zero configuration on the GitHub side.
echo "Mirrored to GitHub"
```
## API Reference
### Step 3: Test Push Mirroring
```bash
# Make a test commit
echo "# Test" >> README.md
git add README.md
git commit -m "test: Mirror sync"
# Push to Gitea
git push origin main
# Check GitHub (should appear within seconds)
```
---
## Method 3: Bidirectional Mirroring
Both repositories accept commits. **Warning:** Can cause conflicts!
### Architecture
```
Developer A → Gitea ←→ GitHub ← Developer B
↑___________↓
Bidirectional sync
```
### Setup
1. **Set up both mirrors:**
- GitHub → Gitea (Method 1)
- Gitea → GitHub (Method 2)
2. **Conflict Resolution Strategy:**
- Always use `git merge` instead of `git push --force`
- Set up webhook for immediate sync
- Have clear contribution guidelines
3. **Webhook-based Immediate Sync:**
**In Gitea:**
- Settings → Webhooks → Add Webhook → Gitea
- Target URL: Your sync service
- Events: Push
**GitHub → Gitea sync service:**
```python
# sync-service.py (simplified)
from flask import Flask, request
import subprocess
app = Flask(__name__)
@app.route('/github-webhook', methods=['POST'])
def github_hook():
# Trigger Gitea pull
subprocess.run(['curl', '-X', 'POST',
'https://git.terraphim.cloud/api/v1/repos/terraphim/repo/mirror-sync'])
return 'OK'
@app.route('/gitea-webhook', methods=['POST'])
def gitea_hook():
# Trigger GitHub push
subprocess.run(['git', 'push', 'github', 'main'])
return 'OK'
```
---
## Method 4: Local Git Configuration (Simplest)
Push to both remotes simultaneously from your local machine.
### Setup
```bash
cd /home/alex/infrastructure/gitea_config
# Check current remotes
git remote -v
# origin: should be Gitea
# Add GitHub as additional remote
git remote add github https://github.com/terraphim/gitea-infrastructure.git
# Or use SSH:
git remote add github git@github.com:terraphim/gitea-infrastructure.git
```
### Push to Both
```bash
# Push to Gitea (origin)
git push origin main
# Push to GitHub
git push github main
```
### Automate with Git Alias
Add to `~/.gitconfig`:
```ini
[alias]
pushall = !git push origin $(git branch --show-current) && git push github $(git branch --show-current)
pa = pushall
```
Usage:
```bash
git pa # Pushes to both remotes
```
### Automate with Bash Function
Add to `~/.bashrc` or `~/.zshrc`:
```bash
git-push-all() {
local current_branch=$(git branch --show-current)
echo "Pushing to all remotes..."
git push origin "$current_branch"
git push github "$current_branch"
echo "✅ Pushed to Gitea and GitHub"
}
alias gpa='git-push-all'
```
---
## Authentication Setup
### GitHub Personal Access Token
1. Go to GitHub → Settings → Developer settings → Personal access tokens
2. Generate new token (classic)
3. Scopes needed:
- `repo` (full control of private repositories)
- `workflow` (if using GitHub Actions)
4. Copy token
### Use Token in Gitea
When setting up push mirror:
- **Username**: Your GitHub username
- **Password**: Your Personal Access Token (not your password!)
### SSH Key Setup (Recommended)
**For automated pushes:**
1. Generate deploy key:
```bash
ssh-keygen -t ed25519 -f ~/.ssh/gitea_github_deploy -C "gitea-mirror"
```
2. Add public key to GitHub:
- Repository Settings → Deploy Keys → Add deploy key
- Paste contents of `~/.ssh/gitea_github_deploy.pub`
- Allow write access
3. Add private key to Gitea server:
```bash
# Copy to Gitea server
scp ~/.ssh/gitea_github_deploy bigbox:~/.ssh/
ssh bigbox "chmod 600 ~/.ssh/gitea_github_deploy"
```
4. Configure SSH for GitHub in Gitea:
```bash
# Create SSH config
ssh bigbox "cat > ~/.ssh/config << 'EOF'
Host github-mirror
HostName github.com
User git
IdentityFile ~/.ssh/gitea_github_deploy
StrictHostKeyChecking no
EOF"
```
5. Update remote to use SSH:
```bash
ssh bigbox "cd ~/gitea-stack && git remote set-url github git@github-mirror:terraphim/gitea-infrastructure.git"
```
---
## Verification and Monitoring
### Check Mirror Status
**In Gitea:**
- Repository → Settings → Repository → Mirror Settings
- Shows last sync time and status
**Via API:**
```bash
# Check mirror status
curl -s \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure-mirror" \
-H "Authorization: token ${GITEA_TOKEN}" | \
jq '.mirror_updated, .mirror_interval'
```
### Monitoring Script
Create `check-mirror.sh`:
```bash
#!/bin/bash
# Check if mirrors are in sync
echo "Checking repository sync status..."
# Get latest commit from GitHub
GITHUB_SHA=$(curl -s \
"https://api.github.com/repos/terraphim/gitea-infrastructure/commits/main" | \
jq -r '.sha')
# Get latest commit from Gitea
GITEA_SHA=$(curl -s \
"https://git.terraphim.cloud/api/v1/repos/terraphim/gitea-infrastructure/commits/main" | \
jq -r '.sha')
if [ "$GITHUB_SHA" == "$GITEA_SHA" ]; then
echo "✅ Repositories are in sync"
echo "Latest commit: ${GITHUB_SHA:0:7}"
else
echo "⚠️ Repositories are OUT OF SYNC"
echo "GitHub: ${GITHUB_SHA:0:7}"
echo "Gitea: ${GITEA_SHA:0:7}"
fi
```
---
## Troubleshooting
### Mirror Not Syncing
**Check Gitea logs:**
```bash
ssh bigbox "docker logs gitea | grep -i mirror"
```
**Test manually:**
```bash
ssh bigbox
cd ~/gitea-stack
git fetch github
git log HEAD..github/main # Check if behind
```
### Authentication Failures
**GitHub token expired:**
- Generate new token in GitHub
- Update in Gitea mirror settings
**SSH key issues:**
```bash
ssh bigbox "ssh -T github-mirror"
# Should show: Hi terraphim/gitea-infrastructure! You've successfully authenticated
```
### Conflict Resolution
If repositories diverge:
```bash
# On local machine
git fetch origin # Gitea
git fetch github # GitHub
# Check differences
git log --left-right --graph --cherry-pick --oneline origin/main...github/main
# Merge changes
git checkout main
git merge origin/main
git merge github/main
# Push to both
git push origin main
git push github main
```
---
## Best Practices
### 1. One-Way vs Two-Way
- **Use one-way** for most cases (clear primary/secondary)
- **Use two-way** only for specific workflows with clear contribution guidelines
### 2. Sync Frequency
- **Backup mirrors**: 24h interval
- **Active development**: 1h interval or webhook-triggered
- **CI/CD**: Immediate webhook sync
### 3. Notifications
Set up alerts for sync failures:
- Gitea webhook on sync failure
- GitHub Actions monitoring
- Email notifications
### 4. Documentation
Document your mirroring strategy:
- Which repository is primary?
- Where should PRs be opened?
- How to handle conflicts?
---
## Quick Reference
| Task | Command |
|------|---------|
| Check remotes | `git remote -v` |
| Add GitHub remote | `git remote add github https://github.com/terraphim/gitea-infrastructure.git` |
| Push to all | `git push origin main && git push github main` |
| Force sync | `curl -X POST "https://git.terraphim.cloud/api/v1/repos/terraphim/repo/mirror-sync"` |
| Check status | `git log origin/main..github/main` |
---
## Recommended Setup for Your Use Case
**Primary: Gitea (self-hosted, private)**
**Mirror: GitHub (backup, CI/CD)**
1. Use **Method 2 (Push Mirror)** from Gitea to GitHub
2. Set sync interval: 1 hour
3. Use SSH authentication with deploy key
4. Enable GitHub Actions for CI/CD on GitHub side
5. Keep Gitea as single source of truth for commits
This gives you:
- ✅ Full control on Gitea
- ✅ GitHub as backup
- ✅ Can use GitHub Actions for CI/CD
- ✅ Simple one-way sync (less conflicts)
| Method | Endpoint | Purpose |
|--------|----------|---------|
| `POST` | `/api/v1/repos/migrate` | Create repo as pull mirror |
| `POST` | `/api/v1/repos/{owner}/{repo}/mirror-sync` | Trigger immediate sync |
| `PATCH` | `/api/v1/repos/{owner}/{repo}` | Change `mirror_interval` |
| `GET` | `/api/v1/repos/{owner}/{repo}` | Check mirror status |
| `DELETE` | `/api/v1/repos/{owner}/{repo}` | Delete (before recreating) |