Architecture

System architecture, components, and data flow documentation for EnvCat.

System Overview

EnvCat is a production-ready, multi-tenant secrets management platform consisting of four containerized services orchestrated via Docker Compose.

┌──────────────────────────────────────────────────────────────┐
│  Docker Network: constants                                   │
│                                                               │
│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌────────────┐  │
│  │  Redis   │  │  API      │  │  Web     │  │  Docs      │  │
│  │  :6379   │  │  :5000    │  │  :8888   │  │  :8889     │  │
│  │(internal)│  │(internal) │  │(public)  │  │(public)    │  │
│  └────┬─────┘  └─────┬─────┘  └────┬─────┘  └────────────┘  │
│       │              │              │                         │
│   Ephemeral      Bundles        BFF Layer                    │
│   Requests      (SQLite)    (Clerk → Flask)                  │
│                Multi-tenant   HMAC signed                     │
└──────────────────────────────────────────────────────────────┘
          ▲                           ▲
          │                           │
     CLI requests                Browser approves
     (Go binary)                (https://env.cat)
          │                           │
          └── HTTPS (production) ─────┘

Design Principles:

Production-first: Clerk authentication, multi-tenancy, HMAC signing
Security-first: Zero-trust BFF layer, encrypted at-rest and end-to-end
Modular: Swap components (crypto, storage, connectors)
Developer-friendly: Intuitive CLI and web interface
Self-hostable: Can be deployed on any Docker-capable infrastructure

Components

1. Redis (Ephemeral Storage)

Purpose: Store short-lived approval request state.

Technology: Redis 7 (in-memory key-value store)

Port: 6379 (internal only, not exposed to host)

Data Stored:

Request metadata (req:<id> hash)
- status: pending or ready
- cli_pub: CLI public key (base64)
- ciphertext: Encrypted secrets (base64)
- created: Timestamp

TTL: 5 minutes (configurable via REQUEST_TTL)

Security:

No plaintext secrets (only ciphertext)
Single-use (keys deleted after retrieval)
Automatic expiration (TTL)

Persistence: None (ephemeral only, appendonly no)

2. API (Flask Backend)

Purpose: Core backend for approval flow and bundle management.

Technology: Flask 3.0 (Python web framework)

Port: 5000 (internal only, not exposed to host)

Responsibilities:

Request management (create, context, wait, encrypt-from-bundle)
Bundle/key CRUD with multi-tenancy (create, read, update, delete)
Import/export (.env, JSON)
At-rest encryption/decryption (AES-256-GCM via Fernet)
End-to-end encryption to CLI (NaCl sealed boxes)
User management (JIT provisioning from Clerk webhooks)

Key Libraries:

Flask - Web framework
SQLAlchemy - ORM for SQLite
PyNaCl - Cryptography (libsodium bindings)
cryptography - Fernet (AES-256-GCM)
redis - Redis client
python-dotenv - .env parsing

Storage:

SQLite database (/data/app.db in container)
Mounted as Docker volume (api_data:/data)
Multi-tenant schema with tenant_id on all resources

Security:

Not exposed to host (internal Docker network only)
HMAC authentication from BFF (shared secret)
Tenant isolation enforced in queries
Encryption key in environment variable (BUNDLE_KMS_KEY)

3. Web (Next.js UI + BFF)

Purpose: User interface for bundle management and approval flow + zero-trust BFF layer.

Technology: Next.js 14 (React framework with App Router)

Port: 8888 (public, exposed to host in local dev; HTTPS in production)

Components:

Frontend (React):

/ - Landing page
/sign-in, /sign-up - Clerk authentication
/approve/[id] - Approval page (select bundle + keys)
/bundles - Bundle list and creation
/bundles/[id] - Bundle detail and key management
/keys - Key library (reusable keys across bundles)

Backend (Next.js API Routes - BFF):

/api/v1/* - Proxy to Flask API with HMAC signing
Server-side only (no client exposure)
Uses internal Docker network (http://api:5000)
Authentication: Clerk middleware protects all routes
Authorization: HMAC signs requests with user/tenant context

Key Libraries:

next - Framework
react - UI library
typescript - Type safety
@clerk/nextjs - Authentication (production)
shadcn/ui - UI components

Why BFF?

Zero-trust architecture: Flask never trusts requests directly
Authentication gateway: Clerk verifies users before forwarding
HMAC signing: BFF signs all requests with shared secret
Tenant isolation: Extracts tenant from Clerk, sends to Flask
Security: Flask API not exposed to host

4. Docs (Docusaurus)

Purpose: Public documentation site (this site!).

Technology: Docusaurus 3 (documentation framework)

Port: 8889 (public, exposed to host)

Content: Markdown files in web-docs/docs/

Features:

Auto-generated sidebar
Search (Algolia DocSearch or local)
Versioning support
Dark mode

Independence: Can be deployed separately from application.

Port Configuration

Service	Internal Port	Production URL	Notes
Redis	6379	Internal only	Ephemeral state storage
API (Flask)	5000	Internal only	Not exposed to internet
Web (Next.js)	3000	https://env.cat	Public gateway (BFF + UI)
Docs (Docusaurus)	3000	https://docs.env.cat	Public documentation

Why this architecture?

Security: Flask not exposed to host; all access via BFF
Simplicity: Clean port mapping (8888 = app, 8889 = docs)
Isolation: Services communicate only via Docker network
Zero-trust: BFF verifies auth before signing requests to Flask

Data Flow

Complete Approval Flow (with Authentication)

1. CLI: Generate Keypair
   ├─ Private key (memory only)
   └─ Public key (base64)

2. CLI → Web BFF: POST /api/v1/requests (public endpoint, no auth)
   ├─ Body: {client_pubkey, bundle_id, template_keys}
   ├─ Web BFF: No Clerk auth required (public endpoint)
   └─ Web BFF → API: Proxy request (no HMAC, public endpoint)

3. API: Create Request
   ├─ Generate request ID
   ├─ Store in Redis: req:<id> {status: pending, cli_pub: ...}
   ├─ Set TTL: 5 minutes
   └─ Response: {id, verification_uri_complete, interval, expires_in}

4. CLI: Display & Poll
   ├─ Show QR code + URL
   ├─ Print: https://env.cat/approve/<id>
   └─ Poll: GET /api/v1/requests/<id>/wait (every 3 sec, public)

5. Browser: Open Approval Page
   ├─ Navigate to /approve/<id>
   ├─ Clerk middleware: Check authentication
   ├─ If not logged in → Redirect to /sign-in?redirect_url=/approve/<id>
   ├─ User signs in via Clerk (email/password, Google, GitHub)
   └─ Clerk redirects back to /approve/<id>

6. Authenticated Page Load
   ├─ Fetch: GET /api/v1/requests/<id>/context (public, no auth)
   ├─ Fetch: GET /api/v1/bundles (authenticated)
   │   ├─ BFF: Extract userId + tenantId from Clerk session
   │   ├─ BFF: Sign request with HMAC (userId|tenantId|timestamp)
   │   ├─ BFF → API: Forward with X-User-Id, X-Tenant-Id, X-Signature
   │   └─ API: Verify HMAC, return bundles for this tenant
   └─ Display: Bundle picker + key selection

7. User: Approve
   ├─ Select bundle (e.g., production/api)
   ├─ Check keys to send (e.g., STRIPE_KEY, DATABASE_URL)
   └─ Click "Approve"

8. Web UI → Web BFF: POST /api/v1/requests/<id>/encrypt-from-bundle
   ├─ Body: {bundle_id, include_keys: [...]}
   ├─ BFF: Verify Clerk session (requireSession())
   ├─ BFF: Extract userId + tenantId
   ├─ BFF: Sign request with HMAC
   ├─ BFF → API: Forward with signed headers
   └─ API: Verify HMAC signature

9. API: Server-Side Re-Encryption
   ├─ Verify HMAC (shared secret between BFF and Flask)
   ├─ Extract tenant_id from headers
   ├─ Load bundle from SQLite (WHERE tenant_id = ...)
   ├─ Decrypt values at-rest (Fernet/AES-256-GCM, using BUNDLE_KMS_KEY)
   ├─ Build payload: {KEY1: value1, KEY2: value2}
   ├─ Encrypt to CLI public key (NaCl sealed box)
   ├─ Store ciphertext in Redis: req:<id> {status: ready, ciphertext: ...}
   └─ Response: {ok: true}

10. CLI: Receive & Decrypt
    ├─ Poll returns: {status: ready, ciphertext_base64: ...}
    ├─ Base64 decode ciphertext
    ├─ Decrypt with private key (sealed box open)
    ├─ Parse JSON: {KEY1: value1, KEY2: value2}
    └─ Output shell-safe exports

11. CLI: Output
    ├─ Format sh: export KEY1=value1
    ├─ Format fish: set -x KEY1 value1
    └─ Format .env: KEY1=value1 (if --write flag)

12. Cleanup
    ├─ Redis: Delete req:<id> (single-use)
    └─ CLI: Destroy private key (memory cleared)

Security layers:

Public endpoints (CLI): /api/v1/requests (create, poll) - no auth
Authenticated endpoints (Browser): /api/v1/bundles, /encrypt-from-bundle - Clerk + HMAC
Zero-trust: Flask never trusts requests without valid HMAC signature
Tenant isolation: All queries filtered by tenant_id from signed headers

Database Schema (Multi-Tenant)

tenants table:

CREATE TABLE tenants (
    id TEXT PRIMARY KEY,           -- Clerk user ID (user_xxx)
    email TEXT UNIQUE,             -- User email
    name TEXT,                     -- Display name
    created_at DATETIME,
    updated_at DATETIME
);

bundles table:

CREATE TABLE bundles (
    id TEXT PRIMARY KEY,           -- UUID
    tenant_id TEXT NOT NULL,       -- Foreign key to tenants
    name TEXT NOT NULL,            -- e.g., "production/api"
    description TEXT,              -- Optional description
    tags TEXT,                     -- Comma-separated tags
    created_at DATETIME,
    updated_at DATETIME,
    UNIQUE(tenant_id, name),       -- Unique per tenant
    FOREIGN KEY(tenant_id) REFERENCES tenants(id) ON DELETE CASCADE
);
CREATE INDEX idx_bundles_tenant ON bundles(tenant_id);

keys table (reusable key library):

CREATE TABLE keys (
    id TEXT PRIMARY KEY,           -- UUID
    tenant_id TEXT NOT NULL,       -- Foreign key to tenants
    name TEXT NOT NULL,            -- Variable name (e.g., "STRIPE_KEY")
    value TEXT NOT NULL,           -- Encrypted value (Fernet/AES-256-GCM)
    is_secret BOOLEAN DEFAULT TRUE,
    description TEXT,              -- Optional description
    created_at DATETIME,
    updated_at DATETIME,
    UNIQUE(tenant_id, name),       -- Unique per tenant
    FOREIGN KEY(tenant_id) REFERENCES tenants(id) ON DELETE CASCADE
);
CREATE INDEX idx_keys_tenant ON keys(tenant_id);
CREATE INDEX idx_keys_name ON keys(tenant_id, name);

bundle_keys table (many-to-many):

CREATE TABLE bundle_keys (
    bundle_id TEXT NOT NULL,       -- Foreign key to bundles
    key_id TEXT NOT NULL,          -- Foreign key to keys
    order INTEGER DEFAULT 0,       -- Display order
    required BOOLEAN DEFAULT FALSE,-- Required for approval?
    created_at DATETIME,
    PRIMARY KEY(bundle_id, key_id),
    FOREIGN KEY(bundle_id) REFERENCES bundles(id) ON DELETE CASCADE,
    FOREIGN KEY(key_id) REFERENCES keys(id) ON DELETE CASCADE
);
CREATE INDEX idx_bundle_keys_bundle ON bundle_keys(bundle_id);
CREATE INDEX idx_bundle_keys_key ON bundle_keys(key_id);

Key design decisions:

Tenant isolation: All resources have tenant_id foreign key
Reusable keys: Keys are first-class entities, attached to bundles via bundle_keys
Cascade deletes: Deleting a tenant/bundle/key automatically cleans up associations
Unique constraints: Bundle names and key names unique per tenant (allows same name across tenants)

Redis Schema

Request entry:

Key: req:<id>
Type: Hash
TTL: 300 seconds (5 minutes)

Fields:
  status: "pending" | "ready"
  cli_pub: "<base64_public_key>"
  ciphertext: "<base64_ciphertext>" (only when ready)
  created: "<timestamp>"

Encryption Architecture

At-Rest (Key Storage)

Algorithm: Fernet (AES-128-CBC + HMAC-SHA256) via Python cryptography library

Why Fernet?

Audited implementation: Part of cryptography library (widely used)
Authenticated encryption: Built-in HMAC prevents tampering
Simple API: Encrypt/decrypt with automatic nonce/timestamp handling
Versioned: Format includes version byte for future algorithm upgrades

Process:

Plaintext: "sk_live_51abc..."
    ↓
Key: BUNDLE_KMS_KEY (base64-encoded Fernet key from env)
    ↓
Encrypt: Fernet(key).encrypt(plaintext.encode())
    ↓
Ciphertext: version|timestamp|iv|ciphertext|hmac (all base64)
    ↓
Store in SQLite: keys.value

Decryption:

Load from SQLite: keys.value
    ↓
Key: BUNDLE_KMS_KEY
    ↓
Decrypt: Fernet(key).decrypt(ciphertext)
    ↓
Verify: HMAC check (automatic)
    ↓
Plaintext: "sk_live_51abc..."

Key rotation (future):

Fernet supports MultiFernet for key rotation
Old keys decrypt old values, new keys encrypt new values
Migrate values in background

End-to-End (CLI Communication)

Algorithm: libsodium sealed box (X25519 + XSalsa20-Poly1305)

Process:

CLI: Generate keypair
    ├─ Private key: 32 bytes (memory only)
    └─ Public key: 32 bytes (sent to server)

Server: Encrypt
    ├─ Load plaintext from bundle (decrypted from at-rest)
    ├─ Serialize JSON: {"API_KEY": "sk_test_123"}
    ├─ Encrypt to CLI public key: sealed_box(json, cli_pubkey)
    └─ Store ciphertext in Redis

CLI: Decrypt
    ├─ Receive ciphertext from server
    ├─ Decrypt with private key: sealed_box_open(ciphertext, privkey)
    ├─ Parse JSON: {"API_KEY": "sk_test_123"}
    └─ Output: export API_KEY=sk_test_123

Security property: Only the CLI can decrypt. Server encrypts but cannot decrypt.

Service Interconnections

Network Topology

Docker Network: constants

┌─────────────┐
│  Redis      │ ← API reads/writes ephemeral state
│  :6379      │
└─────────────┘
       ↑
       │
┌─────────────┐
│  API        │ ← Web BFF proxies all requests here
│  :8080      │ ← Not exposed to host (internal only)
└─────────────┘
       ↑
       │ (internal network: http://api:8080)
       │
┌─────────────┐
│  Web (BFF)  │ ← Public gateway (port 8888)
│  :3000→8888 │ ← Browser + CLI access this
└─────────────┘
       ↑
       │ (external: http://localhost:8888)
       │
   Browser / CLI

API Contract: Web BFF → Flask API

All authenticated BFF routes use HMAC signing:

// web/app/api/v1/bundles/route.ts (actual implementation)
import { flaskFetch } from '@/lib/flask-client';
import { requireSession } from '@/lib/clerk-session';

export async function GET(request: NextRequest) {
  // 1. Verify Clerk session
  const { userId, tenantId } = await requireSession();
  
  // 2. Forward with HMAC signed headers
  const response = await flaskFetch('/api/v1/bundles', {
    method: 'GET',
    userId,    // From Clerk: user_2xxx
    tenantId,  // From Clerk metadata or default "personal"
  });
  
  return response;
}

HMAC signing process (web/lib/flask-auth.ts):

export function signRequest(userId: string, tenantId: string) {
  const timestamp = Math.floor(Date.now() / 1000).toString();
  const payload = `${userId}|${tenantId}|${timestamp}`;
  const signature = crypto.createHmac('sha256', BFF_SHARED_SECRET)
    .update(payload)
    .digest('hex');
  
  return {
    'X-User-Id': userId,
    'X-Tenant-Id': tenantId,
    'X-Timestamp': timestamp,
    'X-Signature': signature,
  };
}

Flask verification (api/utils/bff_auth.py):

@verify_bff_signature
def get_bundles():
    # g.user_id and g.tenant_id now available (from HMAC headers)
    bundles = Bundle.query.filter_by(tenant_id=g.tenant_id).all()
    return jsonify([b.to_dict() for b in bundles])

Security properties:

Shared secret: BFF and Flask share BFF_SHARED_SECRET (env variable)
Replay protection: Timestamp must be within 60 seconds
Tampering protection: HMAC signature verifies userId|tenantId|timestamp
Zero-trust: Flask never trusts requests without valid signature

Deployment Patterns

Local Development (Current)

# docker-compose.yml
services:
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]  # Exposed for debugging
  
  api:
    build: ./api
    ports: []  # NOT exposed to host
  
  web:
    build: ./web
    ports: ["8888:3000"]  # Public gateway
  
  web-docs:
    build: ./web-docs
    ports: ["8889:3000"]  # Public docs

Self-hosting: For self-hosting instructions, see our GitHub repository.

Production Deployment (Current - Fly.io)

Live at: https://env.cat (Web UI) + https://docs.env.cat (Docs)

Infrastructure:

Platform: Fly.io (global edge deployment)
Services:
- web (Next.js) - fly.toml in web/
- api (Flask) - fly.toml in api/
- web-docs (Docusaurus) - fly.toml in web-docs/
Database: Fly.io persistent volume (SQLite in /data)
Redis: Fly.io Redis (managed service)
Auth: Clerk (production authentication)

Environment variables (production):

# Clerk (authentication)
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_live_xxx
CLERK_SECRET_KEY=sk_live_xxx
NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in
NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up

# BFF → Flask HMAC signing
BFF_SHARED_SECRET=<random-256-bit-hex>

# Flask encryption
BUNDLE_KMS_KEY=<base64-encoded-fernet-key>

# Flask API
REDIS_URL=redis://fly-redis.internal:6379
DATABASE_URL=sqlite:////data/app.db
PORT=5000

# Next.js BFF
FLASK_API_BASE=http://api.internal:5000

Security checklist:

✅ HTTPS via Fly.io (automatic TLS)
✅ Clerk authentication (email/password, Google, GitHub)
✅ HMAC signed requests (BFF → Flask)
✅ Tenant isolation (multi-tenant database schema)
✅ Encrypted at-rest (Fernet/AES-256)
✅ Encrypted in-transit (NaCl sealed boxes to CLI)
✅ Flask not exposed to internet (internal network only)

Scaling considerations:

Horizontal: Multiple Flask instances behind internal load balancer
Vertical: Increase machine size for high-traffic periods
Database: Migrate from SQLite to PostgreSQL for >1M keys
Redis: Fly.io Redis cluster for high availability

Performance Characteristics

Latency

Local development:

Request creation: < 10ms
Poll wait (pending): 0-3s (polling interval)
Encrypt-from-bundle: 10-50ms (depends on bundle size)
Total flow: 3-10s (user approval time dominates)

Bottlenecks:

User approval time (3-10s typical)
SQLite queries (< 5ms for bundles < 1000 items)
Encryption (< 1ms per value)

Throughput

SQLite:

Reads: 1000+ req/s (bundles cached in memory)
Writes: 100+ req/s (sequential writes)

Redis:

Reads/writes: 10,000+ req/s (in-memory)

Bottleneck: User approval (one at a time)

Storage

SQLite database size:

Empty: ~100 KB
10 bundles × 20 items: ~200 KB
100 bundles × 100 items: ~5 MB
Encrypted values: +30% overhead (nonce + MAC)

Redis memory:

Empty: ~1 MB
Active requests: ~1 KB per request
Max concurrent: 100 requests = ~100 KB

Future Enhancements

Planned for M8-M12:

M8: Secret Connectors

Fetch secrets from AWS Secrets Manager, GCP Secret Manager
Sync bundles with external sources
Two-way sync (read/write)

M9: Authentication & RBAC

User management (login, logout)
Role-based access control (admin, user, viewer)
Per-bundle permissions

M10: Audit Logs

Track all access (who, what, when)
Immutable audit trail (append-only table)
Compliance reports

M11: GitHub Integration

Use in GitHub Actions (CI/CD)
Secrets as code (bundle definitions in git)
Automated secret scanning

M12: Secret Scanning

Scan git repos for leaked secrets
Detect patterns (API keys, tokens)
Alert on leaks

Additional Architecture Documentation:

Services Documentation - See internal docs/SERVICES.md for detailed service specs
Data Flow - Request lifecycle deep dive (coming soon)
Security - Encryption and threat model

System Overview​

Components​

1. Redis (Ephemeral Storage)​

2. API (Flask Backend)​

3. Web (Next.js UI + BFF)​

4. Docs (Docusaurus)​

Port Configuration​

Data Flow​

Complete Approval Flow (with Authentication)​

Database Schema (Multi-Tenant)​

Redis Schema​

Encryption Architecture​

At-Rest (Key Storage)​

End-to-End (CLI Communication)​

Service Interconnections​

Network Topology​

API Contract: Web BFF → Flask API​

Deployment Patterns​

Local Development (Current)​

Production Deployment (Current - Fly.io)​

Performance Characteristics​

Latency​

Throughput​

Storage​

Future Enhancements​

M8: Secret Connectors​

M9: Authentication & RBAC​

M10: Audit Logs​

M11: GitHub Integration​

M12: Secret Scanning​