yiekheng
a42fdf54b0
Two wins, one root cause: every API request was opening TWO fresh MySQL
connections plus four wasted round-trips before the real query.
Old per-request shape (GET /acc/):
1. DB() constructor → open conn, SHOW TABLES LIKE 'acc',
SHOW TABLES LIKE 'user', close
2. db.query() → open conn, run SELECT, close
That's ~4 round-trips for ~10 ms of useful work. With the dashboard's
30 s auto-refresh and two open tabs (accounts + users), the api-server
churned through ~10 fresh MySQL connections every minute even when
nothing changed.
Changes:
- app/db.py: introduce a process-wide MySQLConnectionPool (size 8 by
default, override with DB_POOL_SIZE). DB() now just touches the cached
pool — no schema check, no fresh handshake. query()/execute() rent a
connection from the pool and return it via conn.close().
- app/db.py: extract the schema check into verify_tables_once() — runs
once at WSGI boot inside create_app() so a misconfigured DB still
fails fast at startup.
- app/cm_api.py: _close_database_connection() removed; the finally
blocks that wrapped every route are gone too. Pool reclamation lives
inside DB now.
- app/cm_api.py: create_app() and run() invoke verify_tables_once()
once at startup instead of CM_API.__init__ doing nothing useful.
Net: ~4× round-trip reduction per request, no MySQL handshake on the
hot path. With two gunicorn workers × pool_size 8 = 16 max in-flight
connections, well under MySQL's default max_connections=151.
(The user asked about 'batching the queries' — but the queries already
return the full row set in one shot. The bottleneck was connection
churn, not query shape. If row count grows past the comfortable single-
fetch range later, swap to LIMIT/OFFSET pagination at the API + table
component layer.)
CM Bot v2 – Portainer Setup (Gitea Registry)
Brief, copy/paste-ready steps to run the published images from gitea.04080616.xyz using Portainer.
What gets deployed
cm-api(port 3000, internal-only),cm-web(Next.js dashboard, container port 3000 → hostCM_WEB_HOST_PORT),cm-telegram,cm-transfer- Container names prefixed with
CM_DEPLOY_NAME(e.g.rex-cm-telegram-bot) - Docker network:
${CM_DEPLOY_NAME}-network(bridge) - Named volume:
${CM_DEPLOY_NAME}-web-auth-datafor/data/auth(passkey JSON store)
Environment configs
Per-deployment templates live in envs/<name>/.env.example (committed). Each operator copies the example to a sibling .env (gitignored — never committed) and fills in the real secrets:
envs/
├── dev/.env.example # Local development tier (port 8010)
├── rex/.env.example # Rex deployment (port 8011)
└── siong/.env.example # Siong deployment (port 8012)
For Portainer-hosted deployments (rex/siong):
cp envs/rex/.env.example envs/rex/.env
# Fill in DB_PASSWORD, CM_AGENT_*, CM_SECURITY_PIN, TELEGRAM_BOT_TOKEN, etc.
# Then load the variables into the Portainer stack environment.
For local development, see the dev tier flow:
cp envs/dev/.env.example .env
bash scripts/dev.sh up
Key variables
| Variable | Description |
|---|---|
CM_DEPLOY_NAME |
Unique prefix for containers/network (e.g. rex-cm, siong-cm) |
CM_WEB_HOST_PORT |
Host port for the Next.js dashboard (unique per deployment; e.g. 8010/8011/8012) |
CM_AUTH_SECRET |
64-hex session signing secret (bash scripts/gen_auth_secret.sh --write) |
TELEGRAM_BOT_TOKEN |
Your Telegram bot token |
DB_HOST / DB_USER / DB_PASSWORD / DB_NAME |
Database connection |
CM_PREFIX_PATTERN |
Username prefix pattern |
CM_AGENT_ID / CM_AGENT_PASSWORD / CM_SECURITY_PIN |
Agent credentials (also used as the dashboard sign-in identity) |
CM_BOT_BASE_URL |
Bot API base URL |
One-time: add the registry in Portainer
- Portainer → Registries → Add registry → Custom.
- Name:
gitea-prod(any) - Registry URL:
gitea.04080616.xyz - Username: your Gitea username; Password: the PAT. Save.
Deploy the stack (fast path)
- Portainer → Stacks → Add stack → Web editor.
- Paste the contents of
docker-compose.ymlfrom this repo (not the override). - Load all variables from the appropriate
envs/<name>/.envinto the stack environment variables. Make sureCM_AUTH_SECRETis present (generate withbash scripts/gen_auth_secret.sh). - Click Deploy the stack. Portainer will pull
cm-<service>:<tag>fromgitea.04080616.xyz/yiekhengand start all four containers.
Migrating an existing pre-B4 stack
The Flask web (port 8000-range) was retired and replaced by the Next.js dashboard. To upgrade:
- In your stack
.env, dropCM_WEB_NEXT_HOST_PORT. SetCM_WEB_HOST_PORTto whatCM_WEB_NEXT_HOST_PORTwas (e.g. 8011/8012). AddCM_AUTH_SECRET=$(openssl rand -hex 32). - Update aaPanel
proxy_passif it pointed to the old Flask port (8001/8005) — switch it to the new one (8011/8012). - Redeploy the stack. The old
${CM_DEPLOY_NAME}-web-viewand${CM_DEPLOY_NAME}-web-nextcontainers go away; a single${CM_DEPLOY_NAME}-webtakes over.
Updating to a new image tag
- Edit the stack → change
DOCKER_IMAGE_TAG→ Update the stack. - Portainer re-pulls and recreates the services with the new tag.
Running multiple deployments on same host
Each deployment needs unique values for:
CM_DEPLOY_NAME– avoids container/network name conflictsCM_WEB_HOST_PORT– avoids port conflicts
Common issues
- Pull denied: PAT missing
read:packageor wrong username/PAT in the registry entry. - Port already allocated: check
CM_WEB_HOST_PORTis unique across deployments. - No port bindings applied: ensure network driver stays
bridge(nothostormacvlan).