For 3k+ row deployments, returning the full table in one shot is the
bottleneck — the JSON payload alone is hundreds of KB and the client
mounts thousands of EditableCell instances on every visit. Pagination
with auto-fetch on scroll shrinks both the wire payload and the initial
render to a single page (200 rows).
Server (app/cm_api.py):
- /acc/ and /user/ accept ?limit, ?offset, ?prefix, ?dir, plus ?sort on
/user/ (f_username | last_update_time). Defaults: limit=200 (capped at
1000), offset=0, dir=desc.
- ORDER BY done in SQL with prefix-priority: rows whose username starts
with the configured CM_PREFIX_PATTERN come first, then asc/desc by the
sort column. The 'dir' value is whitelisted to ASC|DESC before string
interpolation; everything else goes through parameterised binding.
- Schema verification (verify_tables_once) deferred to first request via
a Flask before_request hook — keeps create_app() free of MySQL touches
so unit tests + gunicorn preload still work without a live DB.
Web client:
- web/lib/api.ts: getAccountsPage / getUsersPage return { rows, hasMore }.
hasMore = (rows.length === PAGE_SIZE), so the client knows when to
stop fetching. Each page is its own Next.js cache entry (the URL is
the cache key) — caching from the previous commit still applies.
- web/app/actions.ts: loadMoreAccounts / loadMoreUsers Server Actions
for next-page requests; refreshAccounts / refreshUsers force-evict the
cache via revalidateTag before refetching page 1.
- web/app/page.tsx + users/page.tsx: only fetch the first page now.
- web/components/{accounts,users}-table.tsx: rewrote state model. Rows
accumulate as the user scrolls. An IntersectionObserver on a sentinel
div near the bottom triggers loadMore when it enters the viewport
(300px rootMargin so the next page starts loading before the user
reaches the end). useOptimistic wraps the accumulated rows for in-
flight edits; on success the row is committed locally so the change
survives even though we no longer router.refresh.
- Sort toggle now refetches from page 1 with the new dir/sort param.
Local sort over a partial set would be inconsistent.
- Mutations: delete filters from local state; create + refresh both
reset to page 1 so the row appears in its sorted position.
- Header count shows '<loaded>+' when more pages exist so the operator
knows what they're seeing isn't the full table.
Removed AutoRefresh:
- web/app/layout.tsx no longer mounts AutoRefresh.
- web/components/auto-refresh.tsx deleted.
- Reason: router.refresh every 30s would yank the user back to page 1
every time, losing scroll position and accumulated rows. Manual
Refresh button replaces it (now wired to refreshAccounts/refreshUsers
which evict cache + refetch).
Tests: deferred verify_tables_once() means tests.test_bot_cli's
CreateAppFactoryTests pass without DB env vars again. All 38 existing
tests pass.
CM Bot v2 – Portainer Setup (Gitea Registry)
Brief, copy/paste-ready steps to run the published images from gitea.04080616.xyz using Portainer.
What gets deployed
cm-api(port 3000, internal-only),cm-web(Next.js dashboard, container port 3000 → hostCM_WEB_HOST_PORT),cm-telegram,cm-transfer- Container names prefixed with
CM_DEPLOY_NAME(e.g.rex-cm-telegram-bot) - Docker network:
${CM_DEPLOY_NAME}-network(bridge) - Named volume:
${CM_DEPLOY_NAME}-web-auth-datafor/data/auth(passkey JSON store)
Environment configs
Per-deployment templates live in envs/<name>/.env.example (committed). Each operator copies the example to a sibling .env (gitignored — never committed) and fills in the real secrets:
envs/
├── dev/.env.example # Local development tier (port 8010)
├── rex/.env.example # Rex deployment (port 8011)
└── siong/.env.example # Siong deployment (port 8012)
For Portainer-hosted deployments (rex/siong):
cp envs/rex/.env.example envs/rex/.env
# Fill in DB_PASSWORD, CM_AGENT_*, CM_SECURITY_PIN, TELEGRAM_BOT_TOKEN, etc.
# Then load the variables into the Portainer stack environment.
For local development, see the dev tier flow:
cp envs/dev/.env.example .env
bash scripts/dev.sh up
Key variables
| Variable | Description |
|---|---|
CM_DEPLOY_NAME |
Unique prefix for containers/network (e.g. rex-cm, siong-cm) |
CM_WEB_HOST_PORT |
Host port for the Next.js dashboard (unique per deployment; e.g. 8010/8011/8012) |
CM_AUTH_SECRET |
64-hex session signing secret (bash scripts/gen_auth_secret.sh --write) |
TELEGRAM_BOT_TOKEN |
Your Telegram bot token |
DB_HOST / DB_USER / DB_PASSWORD / DB_NAME |
Database connection |
CM_PREFIX_PATTERN |
Username prefix pattern |
CM_AGENT_ID / CM_AGENT_PASSWORD / CM_SECURITY_PIN |
Agent credentials (also used as the dashboard sign-in identity) |
CM_BOT_BASE_URL |
Bot API base URL |
One-time: add the registry in Portainer
- Portainer → Registries → Add registry → Custom.
- Name:
gitea-prod(any) - Registry URL:
gitea.04080616.xyz - Username: your Gitea username; Password: the PAT. Save.
Deploy the stack (fast path)
- Portainer → Stacks → Add stack → Web editor.
- Paste the contents of
docker-compose.ymlfrom this repo (not the override). - Load all variables from the appropriate
envs/<name>/.envinto the stack environment variables. Make sureCM_AUTH_SECRETis present (generate withbash scripts/gen_auth_secret.sh). - Click Deploy the stack. Portainer will pull
cm-<service>:<tag>fromgitea.04080616.xyz/yiekhengand start all four containers.
Migrating an existing pre-B4 stack
The Flask web (port 8000-range) was retired and replaced by the Next.js dashboard. To upgrade:
- In your stack
.env, dropCM_WEB_NEXT_HOST_PORT. SetCM_WEB_HOST_PORTto whatCM_WEB_NEXT_HOST_PORTwas (e.g. 8011/8012). AddCM_AUTH_SECRET=$(openssl rand -hex 32). - Update aaPanel
proxy_passif it pointed to the old Flask port (8001/8005) — switch it to the new one (8011/8012). - Redeploy the stack. The old
${CM_DEPLOY_NAME}-web-viewand${CM_DEPLOY_NAME}-web-nextcontainers go away; a single${CM_DEPLOY_NAME}-webtakes over.
Updating to a new image tag
- Edit the stack → change
DOCKER_IMAGE_TAG→ Update the stack. - Portainer re-pulls and recreates the services with the new tag.
Running multiple deployments on same host
Each deployment needs unique values for:
CM_DEPLOY_NAME– avoids container/network name conflictsCM_WEB_HOST_PORT– avoids port conflicts
Common issues
- Pull denied: PAT missing
read:packageor wrong username/PAT in the registry entry. - Port already allocated: check
CM_WEB_HOST_PORTis unique across deployments. - No port bindings applied: ensure network driver stays
bridge(nothostormacvlan).
Description
Languages
Python
49.5%
TypeScript
45.6%
Shell
4%
Dockerfile
0.8%