Final Push

2026-05-13 21:08:27 +01:00
parent e4f4992a1b
commit 659e4a72c5
32 changed files with 2173 additions and 3500 deletions
--- a/docs/06-deployment-openstack.md
+++ b/docs/06-deployment-openstack.md
@@ -0,0 +1,611 @@
+# 06 — Deployment Guide: OpenStack VPS
+
+This guide covers the complete deployment of LexiChain on a **private OpenStack VPS** running Ubuntu 22.04, using Docker, Nginx as a reverse proxy, and Ethereum Sepolia as the blockchain network.
+
+**Estimated setup time:** 60–90 minutes (first time)
+
+---
+
+## Architecture Overview
+
+```
+Internet
+   │
+   ▼
+[Nginx] ── TLS termination ── port 443/80
+   │
+   ▼
+[Docker: lexichain-app] ── port 3000 (internal)
+   │
+   ├──► [PostgreSQL] ── port 5432 (internal or external)
+   ├──► [Clerk API]   ── external HTTPS
+   ├──► [UploadThing] ── external HTTPS
+   ├──► [Gemini API]  ── external HTTPS
+   ├──► [Mistral API] ── external HTTPS
+   └──► [Ethereum Sepolia RPC] ── external HTTPS
+```
+
+---
+
+## Prerequisites
+
+Before starting, ensure you have:
+
+- [ ] OpenStack VPS with **Ubuntu 22.04 LTS** (minimum: 2 vCPU, 4 GB RAM, 40 GB disk)
+- [ ] SSH access to the VPS
+- [ ] A **PostgreSQL** database (on the same VPS or a managed DB service)
+- [ ] All external service accounts set up (Clerk, UploadThing, Gemini, Mistral)
+- [ ] A Sepolia ETH wallet with some test ETH (see [Smart Contract docs](./05-smart-contract.md))
+- [ ] The `DocumentRegistry` contract deployed to Sepolia (see [Smart Contract docs](./05-smart-contract.md))
+
+---
+
+## Part 1: VPS Initial Setup
+
+### 1.1 Connect and update the VPS
+
+```bash
+ssh ubuntu@<VPS_IP>
+
+sudo apt update && sudo apt upgrade -y
+sudo apt install -y curl wget git unzip ufw
+```
+
+### 1.2 Configure the firewall (UFW)
+
+```bash
+# Allow SSH (important: do this FIRST to avoid locking yourself out)
+sudo ufw allow OpenSSH
+
+# Allow HTTP and HTTPS
+sudo ufw allow 80/tcp
+sudo ufw allow 443/tcp
+
+# Enable firewall
+sudo ufw enable
+
+# Verify
+sudo ufw status
+```
+
+> If you are using OpenStack Security Groups, also open ports 80, 443, and 22 in the Security Group rules on the OpenStack dashboard.
+
+---
+
+## Part 2: Install Docker
+
+```bash
+# Install Docker
+curl -fsSL https://get.docker.com | sudo sh
+
+# Add your user to the docker group (avoids needing sudo for docker commands)
+sudo usermod -aG docker $USER
+
+# Log out and back in for the group change to take effect
+exit
+# ... reconnect via SSH ...
+
+# Verify
+docker --version
+docker compose version
+```
+
+---
+
+## Part 3: Set Up PostgreSQL
+
+**Option A: PostgreSQL on the same VPS (recommended for simplicity)**
+
+```bash
+sudo apt install -y postgresql postgresql-contrib
+
+# Start and enable PostgreSQL
+sudo systemctl start postgresql
+sudo systemctl enable postgresql
+
+# Create database and user
+sudo -u postgres psql <<EOF
+CREATE USER lexichain WITH PASSWORD 'your_strong_password_here';
+CREATE DATABASE lexichain OWNER lexichain;
+GRANT ALL PRIVILEGES ON DATABASE lexichain TO lexichain;
+EOF
+```
+
+Test the connection:
+```bash
+psql -U lexichain -d lexichain -h localhost -c "SELECT version();"
+```
+
+Your `DATABASE_URL` will be:
+```
+postgresql://lexichain:your_strong_password_here@localhost:5432/lexichain
+```
+
+**Option B: Use a managed PostgreSQL service**  
+If your organization provides a managed DB, use the connection string provided by that service.
+
+---
+
+## Part 4: Build the Docker Image
+
+### 4.1 Clone the repository on the VPS
+
+```bash
+git clone https://github.com/your-org/lexichain.git /opt/lexichain
+cd /opt/lexichain
+```
+
+Or transfer the code via `scp` / `rsync`:
+```bash
+# From your local machine:
+rsync -avz --exclude='.git' --exclude='node_modules' --exclude='.next' \
+  ./ ubuntu@<VPS_IP>:/opt/lexichain/
+```
+
+### 4.2 Build the Docker image
+
+```bash
+cd /opt/lexichain
+docker build -t lexichain:latest .
+```
+
+This will:
+1. Install npm dependencies
+2. Generate Prisma client
+3. Build the Next.js production bundle
+4. Create a minimal runner image with a non-root user
+
+> The build takes **3–8 minutes** on first run. Subsequent builds use Docker layer cache and are faster.
+
+---
+
+## Part 5: Configure Environment Variables
+
+Create the production environment file:
+
+```bash
+sudo mkdir -p /etc/lexichain
+sudo nano /etc/lexichain/production.env
+```
+
+Fill in all values:
+
+```bash
+# ── Application ──────────────────────────────────────
+NODE_ENV=production
+APP_URL=https://yourdomain.com
+# If you don't have a domain yet, use http://<VPS_IP>
+# APP_URL=http://<VPS_IP>
+
+# ── Clerk Authentication ──────────────────────────────
+NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_live_...
+CLERK_SECRET_KEY=sk_live_...
+CLERK_WEBHOOK_SECRET=whsec_...
+NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in
+NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up
+NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL=/dashboard
+NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL=/dashboard
+
+# ── Database ──────────────────────────────────────────
+DATABASE_URL=postgresql://lexichain:your_strong_password_here@localhost:5432/lexichain
+
+# ── File Storage (UploadThing) ────────────────────────
+UPLOADTHING_TOKEN=your_uploadthing_token
+UPLOADTHING_APP_ID=your_uploadthing_app_id
+
+# ── AI — Gemini (Primary) ─────────────────────────────
+AI_API_KEY1=AIza...
+AI_API_KEY2=AIza...   # optional second key for rotation
+AI_MODEL_PRIMARY=gemini-2.5-flash-preview-04-17
+AI_EMBEDDING_MODEL=text-embedding-004
+
+# ── AI — Mistral (Fallback) ───────────────────────────
+MISTRAL_API_KEY=your_mistral_api_key
+AI_MODEL_FALLBACK=mistral-large-latest
+AI_MODEL_MISTRAL_VISION=pixtral-large-latest
+AI_MODEL_MISTRAL_OCR=mistral-ocr-latest
+
+# ── Blockchain (Sepolia Testnet) ──────────────────────
+BLOCKCHAIN_NETWORK=sepolia
+BLOCKCHAIN_RPC_URL=https://rpc.sepolia.org
+BLOCKCHAIN_PRIVATE_KEY=0x...
+BLOCKCHAIN_CONTRACT_ADDRESS=0x...
+
+# ── Email (SMTP) ──────────────────────────────────────
+EMAIL_HOST=smtp-relay.brevo.com
+EMAIL_PORT=587
+EMAIL_SECURE=false
+EMAIL_USER=your_brevo_smtp_user
+EMAIL_PASS=your_brevo_smtp_password
+MAIL_FROM=LexiChain <no-reply@yourdomain.com>
+```
+
+Secure the file:
+```bash
+sudo chmod 600 /etc/lexichain/production.env
+```
+
+---
+
+## Part 6: Run Database Migrations
+
+Before starting the app, run Prisma migrations:
+
+```bash
+docker run --rm \
+  --env-file /etc/lexichain/production.env \
+  --network host \
+  lexichain:latest \
+  npx prisma migrate deploy
+```
+
+> `--network host` allows the container to reach PostgreSQL on `localhost`.  
+> This command is safe to run on every deployment — it only applies pending migrations.
+
+---
+
+## Part 7: Start the Application
+
+```bash
+docker run -d \
+  --name lexichain \
+  --restart unless-stopped \
+  --env-file /etc/lexichain/production.env \
+  --network host \
+  -p 3000:3000 \
+  lexichain:latest
+```
+
+Verify the container is running:
+```bash
+docker ps
+docker logs lexichain --tail 50
+```
+
+Test that the app responds:
+```bash
+curl http://localhost:3000/api/health
+# Expected: {"status":"ok"}
+```
+
+---
+
+## Part 8: Set Up Nginx Reverse Proxy
+
+### 8.1 Install Nginx
+
+```bash
+sudo apt install -y nginx
+sudo systemctl start nginx
+sudo systemctl enable nginx
+```
+
+### 8.2 Configure Nginx
+
+```bash
+sudo nano /etc/nginx/sites-available/lexichain
+```
+
+**Without TLS (temporary — use if you don't have a domain yet):**
+
+```nginx
+server {
+    listen 80;
+    server_name _;   # Matches any hostname or IP
+
+    # Increase upload limits for contract PDFs
+    client_max_body_size 35M;
+
+    location / {
+        proxy_pass http://localhost:3000;
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection 'upgrade';
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+        proxy_cache_bypass $http_upgrade;
+        proxy_read_timeout 120s;
+        proxy_send_timeout 120s;
+    }
+}
+```
+
+**With TLS (recommended — requires a domain):**
+
+```nginx
+server {
+    listen 80;
+    server_name yourdomain.com www.yourdomain.com;
+    # Redirect all HTTP to HTTPS
+    return 301 https://$host$request_uri;
+}
+
+server {
+    listen 443 ssl http2;
+    server_name yourdomain.com www.yourdomain.com;
+
+    ssl_certificate     /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
+    ssl_protocols TLSv1.2 TLSv1.3;
+    ssl_ciphers HIGH:!aNULL:!MD5;
+
+    # Increase upload limits for contract PDFs
+    client_max_body_size 35M;
+
+    location / {
+        proxy_pass http://localhost:3000;
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection 'upgrade';
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_cache_bypass $http_upgrade;
+        proxy_read_timeout 120s;
+        proxy_send_timeout 120s;
+    }
+}
+```
+
+Enable the site:
+```bash
+sudo ln -s /etc/nginx/sites-available/lexichain /etc/nginx/sites-enabled/
+sudo nginx -t  # Test configuration
+sudo systemctl reload nginx
+```
+
+### 8.3 Get TLS Certificate (if you have a domain)
+
+```bash
+sudo apt install -y certbot python3-certbot-nginx
+sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com
+# Follow the prompts
+
+# Auto-renew (runs twice daily)
+sudo systemctl enable certbot.timer
+```
+
+---
+
+## Part 9: Configure Clerk Webhooks
+
+For user sync to work in production, configure your Clerk webhook:
+
+1. Go to [Clerk Dashboard](https://dashboard.clerk.com) → your app → **Webhooks**
+2. Add a new endpoint:
+   - **URL**: `https://yourdomain.com/api/webhooks/clerk`
+   - **Events**: `user.created`, `user.updated`, `user.deleted`
+3. Copy the **Signing Secret** → set as `CLERK_WEBHOOK_SECRET` in your env file
+4. Restart the container after updating the env:
+   ```bash
+   docker stop lexichain && docker rm lexichain
+   # Re-run the docker run command from Part 7
+   ```
+
+---
+
+## Part 10: Verify the Deployment
+
+Run through this checklist:
+
+```bash
+# 1. App responds
+curl https://yourdomain.com/api/health
+
+# 2. View app logs
+docker logs lexichain -f
+
+# 3. Check Nginx logs
+sudo tail -f /var/log/nginx/access.log
+sudo tail -f /var/log/nginx/error.log
+```
+
+**Manual UI verification:**
+- [ ] Sign-up / sign-in flow works
+- [ ] Upload a PDF contract → analysis runs
+- [ ] Blockchain proof registered (check `/blockchain` page)
+- [ ] Email notification received after analysis
+- [ ] Dashboard stats populated
+
+---
+
+## Part 11: Updates & Rollbacks
+
+### Deploy a new version
+
+```bash
+cd /opt/lexichain
+
+# Pull latest code
+git pull origin main
+
+# Build new image
+docker build -t lexichain:latest .
+
+# Run migrations (safe — only applies new ones)
+docker run --rm \
+  --env-file /etc/lexichain/production.env \
+  --network host \
+  lexichain:latest \
+  npx prisma migrate deploy
+
+# Replace running container
+docker stop lexichain && docker rm lexichain
+
+docker run -d \
+  --name lexichain \
+  --restart unless-stopped \
+  --env-file /etc/lexichain/production.env \
+  --network host \
+  -p 3000:3000 \
+  lexichain:latest
+```
+
+### Rollback to a previous version
+
+Tag your images before deploying to enable rollback:
+
+```bash
+# Before deploying new version, tag current as backup
+docker tag lexichain:latest lexichain:backup
+
+# If the new deployment fails, rollback:
+docker stop lexichain && docker rm lexichain
+docker run -d --name lexichain --restart unless-stopped \
+  --env-file /etc/lexichain/production.env \
+  --network host -p 3000:3000 \
+  lexichain:backup
+```
+
+---
+
+## Part 12: Monitoring & Maintenance
+
+### View application logs
+
+```bash
+# Live logs
+docker logs lexichain -f
+
+# Last 100 lines
+docker logs lexichain --tail 100
+
+# Filter for errors
+docker logs lexichain 2>&1 | grep -i error
+```
+
+### Monitor container health
+
+```bash
+docker inspect --format='{{json .State.Health}}' lexichain | python3 -m json.tool
+```
+
+### Database maintenance
+
+```bash
+# Connect to PostgreSQL
+sudo -u postgres psql -d lexichain
+
+# Check database size
+\l+
+
+# Vacuum analyze (optimize queries)
+VACUUM ANALYZE;
+```
+
+### Disk space management
+
+```bash
+# Remove unused Docker images
+docker image prune -f
+
+# Check disk usage
+df -h
+du -sh /var/lib/docker/
+```
+
+---
+
+## Part 13: Environment Variable Reference
+
+Complete reference for all variables used in production:
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `NODE_ENV` | ✅ | Must be `production` |
+| `APP_URL` | ✅ | Full public URL of the app |
+| `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` | ✅ | Clerk publishable key |
+| `CLERK_SECRET_KEY` | ✅ | Clerk secret key |
+| `CLERK_WEBHOOK_SECRET` | ✅ | Clerk webhook signing secret |
+| `NEXT_PUBLIC_CLERK_SIGN_IN_URL` | ✅ | `/sign-in` |
+| `NEXT_PUBLIC_CLERK_SIGN_UP_URL` | ✅ | `/sign-up` |
+| `NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL` | ✅ | `/dashboard` |
+| `NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL` | ✅ | `/dashboard` |
+| `DATABASE_URL` | ✅ | PostgreSQL connection string |
+| `UPLOADTHING_TOKEN` | ✅ | UploadThing auth token |
+| `UPLOADTHING_APP_ID` | ✅ | UploadThing app ID |
+| `AI_API_KEY1` | ✅ | Gemini API key (primary) |
+| `AI_API_KEY2` | ❌ | Gemini API key (rotation) |
+| `AI_API_KEY3` | ❌ | Gemini API key (rotation) |
+| `AI_MODEL_PRIMARY` | ✅ | Gemini model name |
+| `AI_EMBEDDING_MODEL` | ✅ | `text-embedding-004` |
+| `MISTRAL_API_KEY` | ❌ | Mistral key (activates fallback) |
+| `AI_MODEL_FALLBACK` | ❌ | `mistral-large-latest` |
+| `AI_MODEL_MISTRAL_VISION` | ❌ | `pixtral-large-latest` |
+| `AI_MODEL_MISTRAL_OCR` | ❌ | `mistral-ocr-latest` |
+| `BLOCKCHAIN_NETWORK` | ✅ | `sepolia` |
+| `BLOCKCHAIN_RPC_URL` | ✅ | Sepolia RPC endpoint |
+| `BLOCKCHAIN_PRIVATE_KEY` | ✅ | Platform wallet private key |
+| `BLOCKCHAIN_CONTRACT_ADDRESS` | ✅ | DocumentRegistry address |
+| `EMAIL_HOST` | ❌ | SMTP server hostname |
+| `EMAIL_PORT` | ❌ | SMTP port (587 or 465) |
+| `EMAIL_SECURE` | ❌ | `false` for STARTTLS |
+| `EMAIL_USER` | ❌ | SMTP username |
+| `EMAIL_PASS` | ❌ | SMTP password |
+| `MAIL_FROM` | ❌ | From address for emails |
+
+---
+
+## Troubleshooting
+
+### Container won't start
+
+```bash
+docker logs lexichain
+# Look for: DATABASE_URL connection refused, missing env vars, etc.
+```
+
+Common causes:
+- `DATABASE_URL` is wrong or PostgreSQL is not running
+- A required env var is missing
+- Port 3000 is already in use: `sudo lsof -i :3000`
+
+### 502 Bad Gateway from Nginx
+
+The app is not running or not listening:
+```bash
+docker ps           # Is the container running?
+curl localhost:3000  # Does it respond locally?
+docker logs lexichain --tail 20
+```
+
+### Blockchain not working
+
+```bash
+# Test RPC connectivity
+curl -X POST https://rpc.sepolia.org \
+  -H "Content-Type: application/json" \
+  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'
+
+# Verify your wallet has Sepolia ETH
+# Check: https://sepolia.etherscan.io/address/<YOUR_WALLET_ADDRESS>
+```
+
+### AI analysis failing
+
+- Verify `AI_API_KEY1` is valid at https://aistudio.google.com
+- Check if Gemini model name is correct (model names change — check Google AI docs)
+- If all Gemini keys are exhausted, verify `MISTRAL_API_KEY` is set for fallback
+
+### Email not sending
+
+- Without `EMAIL_HOST`, emails are silently dropped in production
+- Test SMTP credentials with: `npx nodemailer-test` or Brevo's SMTP tester
+- Check Brevo sending quota
+
+---
+
+## Security Hardening Checklist
+
+- [ ] Production env file has `chmod 600`
+- [ ] PostgreSQL password is strong (20+ random characters)
+- [ ] Blockchain wallet is dedicated (not holding real ETH)
+- [ ] Clerk webhooks endpoint is HTTPS only
+- [ ] UFW firewall enabled with only required ports
+- [ ] TLS certificate installed and auto-renewing
+- [ ] `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` starts with `pk_live_` (not `pk_test_`)
+- [ ] `NODE_ENV=production` set
+- [ ] No `.env` file committed to git