yiekheng 6bb85222d1 perf(web): server-side pagination + infinite-scroll for accounts/users
For 3k+ row deployments, returning the full table in one shot is the
bottleneck — the JSON payload alone is hundreds of KB and the client
mounts thousands of EditableCell instances on every visit. Pagination
with auto-fetch on scroll shrinks both the wire payload and the initial
render to a single page (200 rows).

Server (app/cm_api.py):
- /acc/ and /user/ accept ?limit, ?offset, ?prefix, ?dir, plus ?sort on
  /user/ (f_username | last_update_time). Defaults: limit=200 (capped at
  1000), offset=0, dir=desc.
- ORDER BY done in SQL with prefix-priority: rows whose username starts
  with the configured CM_PREFIX_PATTERN come first, then asc/desc by the
  sort column. The 'dir' value is whitelisted to ASC|DESC before string
  interpolation; everything else goes through parameterised binding.
- Schema verification (verify_tables_once) deferred to first request via
  a Flask before_request hook — keeps create_app() free of MySQL touches
  so unit tests + gunicorn preload still work without a live DB.

Web client:
- web/lib/api.ts: getAccountsPage / getUsersPage return { rows, hasMore }.
  hasMore = (rows.length === PAGE_SIZE), so the client knows when to
  stop fetching. Each page is its own Next.js cache entry (the URL is
  the cache key) — caching from the previous commit still applies.
- web/app/actions.ts: loadMoreAccounts / loadMoreUsers Server Actions
  for next-page requests; refreshAccounts / refreshUsers force-evict the
  cache via revalidateTag before refetching page 1.
- web/app/page.tsx + users/page.tsx: only fetch the first page now.
- web/components/{accounts,users}-table.tsx: rewrote state model. Rows
  accumulate as the user scrolls. An IntersectionObserver on a sentinel
  div near the bottom triggers loadMore when it enters the viewport
  (300px rootMargin so the next page starts loading before the user
  reaches the end). useOptimistic wraps the accumulated rows for in-
  flight edits; on success the row is committed locally so the change
  survives even though we no longer router.refresh.
- Sort toggle now refetches from page 1 with the new dir/sort param.
  Local sort over a partial set would be inconsistent.
- Mutations: delete filters from local state; create + refresh both
  reset to page 1 so the row appears in its sorted position.
- Header count shows '<loaded>+' when more pages exist so the operator
  knows what they're seeing isn't the full table.

Removed AutoRefresh:
- web/app/layout.tsx no longer mounts AutoRefresh.
- web/components/auto-refresh.tsx deleted.
- Reason: router.refresh every 30s would yank the user back to page 1
  every time, losing scroll position and accumulated rows. Manual
  Refresh button replaces it (now wired to refreshAccounts/refreshUsers
  which evict cache + refetch).

Tests: deferred verify_tables_once() means tests.test_bot_cli's
CreateAppFactoryTests pass without DB env vars again. All 38 existing
tests pass.
2026-05-03 11:29:34 +08:00

CM Bot v2 Portainer Setup (Gitea Registry)

Brief, copy/paste-ready steps to run the published images from gitea.04080616.xyz using Portainer.

What gets deployed

  • cm-api (port 3000, internal-only), cm-web (Next.js dashboard, container port 3000 → host CM_WEB_HOST_PORT), cm-telegram, cm-transfer
  • Container names prefixed with CM_DEPLOY_NAME (e.g. rex-cm-telegram-bot)
  • Docker network: ${CM_DEPLOY_NAME}-network (bridge)
  • Named volume: ${CM_DEPLOY_NAME}-web-auth-data for /data/auth (passkey JSON store)

Environment configs

Per-deployment templates live in envs/<name>/.env.example (committed). Each operator copies the example to a sibling .env (gitignored — never committed) and fills in the real secrets:

envs/
├── dev/.env.example      # Local development tier (port 8010)
├── rex/.env.example      # Rex deployment (port 8011)
└── siong/.env.example    # Siong deployment (port 8012)

For Portainer-hosted deployments (rex/siong):

cp envs/rex/.env.example envs/rex/.env
# Fill in DB_PASSWORD, CM_AGENT_*, CM_SECURITY_PIN, TELEGRAM_BOT_TOKEN, etc.
# Then load the variables into the Portainer stack environment.

For local development, see the dev tier flow:

cp envs/dev/.env.example .env
bash scripts/dev.sh up

Key variables

Variable Description
CM_DEPLOY_NAME Unique prefix for containers/network (e.g. rex-cm, siong-cm)
CM_WEB_HOST_PORT Host port for the Next.js dashboard (unique per deployment; e.g. 8010/8011/8012)
CM_AUTH_SECRET 64-hex session signing secret (bash scripts/gen_auth_secret.sh --write)
TELEGRAM_BOT_TOKEN Your Telegram bot token
DB_HOST / DB_USER / DB_PASSWORD / DB_NAME Database connection
CM_PREFIX_PATTERN Username prefix pattern
CM_AGENT_ID / CM_AGENT_PASSWORD / CM_SECURITY_PIN Agent credentials (also used as the dashboard sign-in identity)
CM_BOT_BASE_URL Bot API base URL

One-time: add the registry in Portainer

  1. Portainer → RegistriesAdd registryCustom.
  2. Name: gitea-prod (any)
  3. Registry URL: gitea.04080616.xyz
  4. Username: your Gitea username; Password: the PAT. Save.

Deploy the stack (fast path)

  1. Portainer → StacksAdd stackWeb editor.
  2. Paste the contents of docker-compose.yml from this repo (not the override).
  3. Load all variables from the appropriate envs/<name>/.env into the stack environment variables. Make sure CM_AUTH_SECRET is present (generate with bash scripts/gen_auth_secret.sh).
  4. Click Deploy the stack. Portainer will pull cm-<service>:<tag> from gitea.04080616.xyz/yiekheng and start all four containers.

Migrating an existing pre-B4 stack

The Flask web (port 8000-range) was retired and replaced by the Next.js dashboard. To upgrade:

  1. In your stack .env, drop CM_WEB_NEXT_HOST_PORT. Set CM_WEB_HOST_PORT to what CM_WEB_NEXT_HOST_PORT was (e.g. 8011/8012). Add CM_AUTH_SECRET=$(openssl rand -hex 32).
  2. Update aaPanel proxy_pass if it pointed to the old Flask port (8001/8005) — switch it to the new one (8011/8012).
  3. Redeploy the stack. The old ${CM_DEPLOY_NAME}-web-view and ${CM_DEPLOY_NAME}-web-next containers go away; a single ${CM_DEPLOY_NAME}-web takes over.

Updating to a new image tag

  1. Edit the stack → change DOCKER_IMAGE_TAGUpdate the stack.
  2. Portainer re-pulls and recreates the services with the new tag.

Running multiple deployments on same host

Each deployment needs unique values for:

  • CM_DEPLOY_NAME avoids container/network name conflicts
  • CM_WEB_HOST_PORT avoids port conflicts

Common issues

  • Pull denied: PAT missing read:package or wrong username/PAT in the registry entry.
  • Port already allocated: check CM_WEB_HOST_PORT is unique across deployments.
  • No port bindings applied: ensure network driver stays bridge (not host or macvlan).
Description
No description provided
Readme 665 KiB
Languages
Python 49.5%
TypeScript 45.6%
Shell 4%
Dockerfile 0.8%