cm_bot_v2/docs/superpowers/specs/2026-05-02-local-as-dev-design.md

Local-as-Dev Tier (Sub-Project A) Design

Date: 2026-05-02 Status: Approved (design) Sequel to: 2026-05-02-debug-mode-hotfix-design.md Out of scope (separate): rex/siong env file rotation (R2), cm_bot.py scraper resilience (R3), security hardening sub-project (C), Next.js web view (B).

Problem

cm_bot_v2 runs two production deployments — rex (port 8001) and siong (port 8005) — both deployed via Portainer from images on gitea.04080616.xyz/yiekheng. There is no formal local development tier. Local iteration today either touches a real production database or has no DB at all (web-view is the only service that boots without one). This blocks safe testing of the security hardening (sub-project C) and the planned Next.js rewrite (B), both of which need to be exercised end-to-end without poking prod data.

Goal

Add a local-only "dev tier" that spins up mysql + api-server + web-view self-contained on a developer machine, plus a Python CLI that gives the same trigger surface the Telegram bot exposes. The goal is end-to-end iteration on the bot's business logic without launching a real Telegram bot, without touching rex/siong databases, and without making automated calls to cm99.net unless the developer explicitly invokes one.

Non-Goals

• Importing a sanitized snapshot of rex/siong data into the dev DB. Seed data only; snapshot import can come later.
• Containerizing the new bot CLI. It runs from the local Python virtualenv against a port-published dev mysql.
• Changing the prod compose file (docker-compose.yml). Portainer stacks for rex/siong remain untouched.
• Migrating or rotating the existing committed envs/rex/.env and envs/siong/.env files. Those are an R2 problem.
• Building or running telegram-bot or transfer-bot automatically in dev. They are gated behind a compose bots profile and stay quiet.
• Adding curses/textual/prompt_toolkit. The interactive TUI uses stdlib input() only.
• Adding a remove/delete CLI subcommand. The current Telegram bot has no analog and the schema has no soft-delete state.

Architecture Overview

Three additions, no removals:

1. A mysql service in docker-compose.override.yml — MySQL 8.0 image, named volume for persistence, init scripts mounted at /docker-entrypoint-initdb.d/. Published to 127.0.0.1:3306:3306 so the local CLI (running outside Docker) can reach it without exposing the port to anything else on the network.
2. A new Python CLI module: app/bot_cli.py — argparse-driven, mirrors the four Telegram bot handlers plus three operational ops. With no arguments, drops into a stdlib interactive menu (TUI-style; not full curses).
3. Two shell scripts in scripts/ — dev.sh (lifecycle: up/down/reset-db/logs/status) and bot_cli.sh (env-loading wrapper for python -m app.bot_cli).

Compose service start matrix:

Command	mysql	api-server	web-view	telegram-bot	transfer-bot
dev.sh up	✓	✓	✓	(gated by bots profile)	(gated by bots profile)
bot_cli.sh ...	(already up — E2 check)	(already up)	(already up)	— (CLI is local Python)	—
Prod (Portainer using docker-compose.yml)	—	✓	✓	✓	✓

Network: api-server reaches mysql:3306 over the compose network; the local CLI reaches 127.0.0.1:3306 via the published port. DB_HOST is overridden by bot_cli.sh so a single .env works for both.

Components

1. docker-compose.override.yml changes

Add a mysql service to the existing override file (which currently only has build: directives for local image builds):

services:
  mysql:
    image: mysql:8.0
    container_name: ${CM_DEPLOY_NAME:-cm}-mysql
    restart: unless-stopped
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-devroot}
      MYSQL_DATABASE: ${DB_NAME}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASSWORD}
    ports:
      - "127.0.0.1:3306:3306"
    volumes:
      - mysql-data:/var/lib/mysql
      - ./docker/mysql/init.d:/docker-entrypoint-initdb.d:ro
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-u", "root", "-p${MYSQL_ROOT_PASSWORD:-devroot}"]
      interval: 5s
      timeout: 3s
      retries: 12
    networks:
      - bot-network

  api-server:
    depends_on:
      mysql:
        condition: service_healthy

  telegram-bot:
    profiles: ["bots"]

  transfer-bot:
    profiles: ["bots"]

volumes:
  mysql-data:
    name: ${CM_DEPLOY_NAME:-cm}-mysql-data

Notes:

• The 127.0.0.1: host-binding prefix means port 3306 is reachable from your shell only — not from other machines on the LAN. Important: the rex/siong scanner at 192.168.0.210 cannot reach this even if it tried.
• depends_on: { mysql: { condition: service_healthy } } means api-server waits for the healthcheck to pass before starting. This eliminates the cold-start race where api-server tried to connect before mysql was accepting traffic.
• Adding profiles: ["bots"] to telegram-bot and transfer-bot keeps them out of docker compose up in dev, but docker compose run --rm telegram-bot ... still works for parity tests.

2. docker/mysql/init.d/

Two SQL scripts run once on first volume creation (mysql:8 invokes everything in /docker-entrypoint-initdb.d/ in alphabetical order):

• docker/mysql/init.d/01-schema.sql — exactly the DDL from AGENTS.md (acc + user tables, utf8mb4). The CREATE DATABASE from AGENTS.md is dropped because mysql:8 image creates ${DB_NAME} via env. The USE rex_cm line is replaced with USE \${DB_NAME}``.
• docker/mysql/init.d/02-seed.sql — minimal seed: one row in acc matching CM_PREFIX_PATTERN=13c so get_next_username works, plus three filler rows.

Seed contents:

INSERT INTO acc (username, password, status, link) VALUES
  ('13c1000', 'seedpass', '', ''),
  ('13c1001', 'seedpass', '', ''),
  ('13c1002', 'seedpass', '', ''),
  ('13c1003', 'seedpass', '', '');

These are dev-only seed values — never real credentials. The scripts only run on a fresh volume (mysql refuses to re-init an initialized volume). To re-seed, run dev.sh reset-db.

3. app/bot_cli.py

import argparse
import os
import sys

from .cm_bot_hal import CM_BOT_HAL


def _print_user(user: dict) -> None:
    print(f"Username: {user['username']}")
    print(f"Password: {user['password']}")
    print(f"Link: {user['link']}")


def cmd_register(_args):
    bot = CM_BOT_HAL()
    _print_user(bot.get_user_api())


def cmd_set_pin(args):
    bot = CM_BOT_HAL()
    if not bot.is_whatsapp_url(args.link):
        print(f"ERROR: not a WhatsApp URL: {args.link}", file=sys.stderr)
        sys.exit(2)
    # Resolve names locally so we have something useful to print regardless of
    # what set_security_pin_api returns. The HAL currently returns a bool from
    # the trailing insert_user_to_table_user; the Telegram handler has the
    # same shape and a latent bug accessing result['f_username']. We avoid
    # depending on the return shape here.
    t_username, f_username = bot.get_whatsapp_link_username(args.link)
    success = bot.set_security_pin_api(args.link)
    if not success:
        print("ERROR: set_security_pin_api returned a falsy result", file=sys.stderr)
        sys.exit(1)
    print(f"OK: f_username={f_username} t_username={t_username}")


def cmd_insert_user(args):
    bot = CM_BOT_HAL()
    f_password = bot.get_user_pass_from_acc(args.f_username)
    if not f_password:
        print(f"ERROR: no password for {args.f_username}", file=sys.stderr)
        sys.exit(2)
    success = bot.insert_user_to_table_user({
        'f_username': args.f_username,
        'f_password': f_password,
        't_username': args.t_username,
        't_password': bot.security_pin,
    })
    if not success:
        print("ERROR: insert failed", file=sys.stderr)
        sys.exit(1)
    print(f"OK: inserted {args.f_username} → {args.t_username}")


def cmd_credit(args):
    bot = CM_BOT_HAL()
    print(f"Credit: {bot.get_user_credit(args.username, args.password)}")


def cmd_transfer(args):
    bot = CM_BOT_HAL()
    print(bot.transfer_credit_api(args.f_username, args.f_password, args.t_username, args.t_password))


def cmd_monitor_once(args):
    bot = CM_BOT_HAL()
    available = bot.get_all_available_acc()
    print(f"Available accounts: {len(available)} (target: {args.target})")
    if len(available) >= args.target:
        print("Already at target; nothing to do.")
        return
    for _ in range(len(available), args.target):
        try:
            user = bot.create_new_acc()
            print(f"Created: {user['username']}")
        except Exception as exc:
            print(f"ERROR creating account: {exc}", file=sys.stderr)
            sys.exit(1)


def cmd_interactive(_args):
    """Telegram-style menu in a TTY loop. stdlib only."""
    print("CM Bot CLI — interactive (type 'q' to quit, '?' for menu)")
    while True:
        print()
        print("  1                                    Register / get next account")
        print("  2 <whatsapp_link>                    Set security PIN")
        print("  3 <f_username> <t_username>          Insert into user table")
        print("  credit <username> <password>         Read account credit")
        print("  transfer <fu> <fp> <tu> <tp>         One-shot credit transfer")
        print("  monitor [N]                          Run monitor once (default 20)")
        print("  q                                    Quit")
        try:
            line = input("> ").strip()
        except (EOFError, KeyboardInterrupt):
            print()
            return
        if not line:
            continue
        if line in ("q", "quit", "exit"):
            return
        if line in ("?", "help", "menu"):
            continue
        argv = line.split()
        # Map TUI shortcuts to argparse subcommand names so we reuse the same
        # dispatch table for both modes.
        TUI_ALIASES = {"1": "register", "2": "set-pin", "3": "insert-user"}
        argv[0] = TUI_ALIASES.get(argv[0], argv[0])
        try:
            args = build_parser().parse_args(argv)
            args.func(args)
        except SystemExit:
            # argparse calls sys.exit() on parse error; swallow it to keep the
            # REPL alive instead of bailing out of the loop.
            continue
        except Exception as exc:
            print(f"ERROR: {exc}", file=sys.stderr)


def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(prog="bot_cli", description="CM Bot dev CLI (mirrors Telegram triggers).")
    sub = p.add_subparsers(dest="command")

    sp = sub.add_parser("register", aliases=["get-acc"], help="Get next available account (Telegram /1).")
    sp.set_defaults(func=cmd_register)

    sp = sub.add_parser("set-pin", help="Set security PIN from a WhatsApp link (Telegram /2).")
    sp.add_argument("link")
    sp.set_defaults(func=cmd_set_pin)

    sp = sub.add_parser("insert-user", help="Insert into user table (Telegram /3).")
    sp.add_argument("f_username")
    sp.add_argument("t_username")
    sp.set_defaults(func=cmd_insert_user)

    sp = sub.add_parser("credit", help="Read account credit balance.")
    sp.add_argument("username")
    sp.add_argument("password")
    sp.set_defaults(func=cmd_credit)

    sp = sub.add_parser("transfer", help="One-shot credit transfer.")
    sp.add_argument("f_username")
    sp.add_argument("f_password")
    sp.add_argument("t_username")
    sp.add_argument("t_password")
    sp.set_defaults(func=cmd_transfer)

    sp = sub.add_parser("monitor-once", aliases=["monitor"], help="One iteration of the auto-create monitor.")
    sp.add_argument("--target", type=int, default=20)
    sp.set_defaults(func=cmd_monitor_once)

    sp = sub.add_parser("interactive", help="Drop into the TUI menu.")
    sp.set_defaults(func=cmd_interactive)

    return p


def main(argv=None) -> int:
    parser = build_parser()
    args = parser.parse_args(argv)
    if args.command is None:
        # Default mode: interactive TUI.
        return cmd_interactive(args) or 0
    return args.func(args) or 0


if __name__ == "__main__":
    sys.exit(main())

Design notes:

• The TUI loop reuses argparse parsing so subcommand and interactive paths share validation. SystemExit is caught so a typo (e.g., 2 with no link) prints the argparse error and returns to the prompt instead of killing the REPL.
• The default python -m app.bot_cli (no args) drops into interactive mode. This matches the user's "TUI" expectation while keeping the same module usable as a one-shot script.
• Subcommand names are spelled-out verbs (register, set-pin) for shell scripting; the TUI accepts 1/2/3 shortcuts so muscle memory from Telegram works.
• The CLI imports CM_BOT_HAL directly. No new business logic. If you change the bot's behavior in cm_bot_hal.py, the CLI tracks automatically.

4. scripts/dev.sh

#!/usr/bin/env bash
set -euo pipefail

usage() {
  cat <<'EOF'
Lifecycle commands for the local dev stack (mysql + api-server + web-view).
Bots (telegram-bot, transfer-bot) are gated behind a compose 'bots' profile
and do not start with 'up'.

Usage:
  scripts/dev.sh up         Start the dev stack in the background.
  scripts/dev.sh down       Stop the stack. Volume kept (DB persists).
  scripts/dev.sh reset-db   Stop the stack AND drop the mysql volume.
  scripts/dev.sh logs       Tail logs from the running stack.
  scripts/dev.sh status     Print 'OK' if mysql container is running, else exit 1.
EOF
}

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"

SUDO="sudo"
[[ "${NO_SUDO:-0}" == "1" ]] && SUDO=""
COMPOSE=(${SUDO} docker compose -f docker-compose.yml -f docker-compose.override.yml)

[[ -f .env ]] || { echo "ERROR: .env not found. cp envs/dev/.env.example .env (then edit)." >&2; exit 2; }

case "${1:-}" in
  up)
    "${COMPOSE[@]}" up -d --build mysql api-server web-view
    "${COMPOSE[@]}" ps
    ;;
  down)
    "${COMPOSE[@]}" down
    ;;
  reset-db)
    "${COMPOSE[@]}" down --volumes
    "${COMPOSE[@]}" up -d --build mysql api-server web-view
    ;;
  logs)
    "${COMPOSE[@]}" logs -f mysql api-server web-view
    ;;
  status)
    if "${COMPOSE[@]}" ps --status running --services | grep -q '^mysql$'; then
      echo OK
    else
      echo "ERROR: dev stack not running. Run 'scripts/dev.sh up' first." >&2
      exit 1
    fi
    ;;
  -h|--help|help|"")
    usage
    [[ "${1:-}" == "" ]] && exit 1 || exit 0
    ;;
  *)
    echo "unknown command: $1" >&2
    usage >&2
    exit 1
    ;;
esac

5. scripts/bot_cli.sh

#!/usr/bin/env bash
# Run the bot CLI in the local venv. With no args, drops into the TUI menu.
# Requires: dev stack up (run scripts/dev.sh up first), .venv with deps.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"

# E2: bail if the dev stack is not running.
if ! NO_SUDO="${NO_SUDO:-0}" bash scripts/dev.sh status >/dev/null 2>&1; then
  echo "ERROR: dev stack not running. Run 'scripts/dev.sh up' first." >&2
  exit 2
fi

[[ -f .env ]] || { echo "ERROR: .env not found. cp envs/dev/.env.example .env (then edit)." >&2; exit 2; }

# Load .env into the environment (export everything between 'set -a' and 'set +a').
set -a
# shellcheck disable=SC1091
source .env
set +a

# Override DB host/port for the local CLI: docker mysql is published on
# 127.0.0.1:3306 even though api-server in-network reaches it as mysql:3306.
export DB_HOST=127.0.0.1
export DB_PORT=3306

PYTHON_BIN="${PYTHON_BIN:-${ROOT_DIR}/.venv/bin/python}"
[[ -x "${PYTHON_BIN}" ]] || { echo "ERROR: ${PYTHON_BIN} not found. Create venv: python3 -m venv .venv && .venv/bin/pip install -r requirements.txt" >&2; exit 2; }

exec "${PYTHON_BIN}" -m app.bot_cli "$@"

6. envs/dev/.env.example (committed)

# === Runtime ===
CM_DEBUG=true

# === Deployment Identity ===
CM_DEPLOY_NAME=dev-cm
CM_WEB_HOST_PORT=8000

# === Docker Registry / Build ===
CM_IMAGE_PREFIX=local
DOCKER_IMAGE_TAG=dev

# === Telegram (unused in A2 — telegram-bot is gated by 'bots' profile) ===
TELEGRAM_BOT_TOKEN=fill-only-if-running-bots-profile
TELEGRAM_ALERT_CHAT_ID=
TELEGRAM_ALERT_BOT_TOKEN=

# === Database (dev mysql in docker; bot_cli.sh overrides DB_HOST=127.0.0.1) ===
DB_HOST=mysql
DB_USER=cm
DB_PASSWORD=devpassword
DB_NAME=cm
DB_PORT=3306
DB_CONNECTION_TIMEOUT=8
DB_CONNECT_RETRIES=5
DB_CONNECT_RETRY_DELAY=2
MYSQL_ROOT_PASSWORD=devroot

# === Bot Config ===
# CM_PREFIX_PATTERN=13c MUST match the seed in docker/mysql/init.d/02-seed.sql.
CM_PREFIX_PATTERN=13c
CM_AGENT_ID=fill-with-real-agent-id-to-test-cm99-calls
CM_AGENT_PASSWORD=fill-with-real-agent-password-to-test-cm99-calls
CM_SECURITY_PIN=000000
CM_BOT_BASE_URL=https://cm99.net

Operator workflow: cp envs/dev/.env.example .env, fill in real CM_AGENT_ID / CM_AGENT_PASSWORD only if they want to invoke register / monitor-once / credit / transfer against real cm99.net. The set-pin and insert-user ops also need them.

envs/dev/.env (the operator's filled-in copy) is added to .gitignore.

7. .gitignore change

__pycache__
.DS_Store
*.html
logs
envs/dev/.env

Just envs/dev/.env — rex/siong stay tracked (their handling is R2's problem).

8. AGENTS.md updates

• Replace the section recommending raw schema setup with a pointer to bash scripts/dev.sh up.
• Add a new "Dev Tier" subsection: "Local development uses envs/dev/.env.example → .env → bash scripts/dev.sh up. The bot CLI: bash scripts/bot_cli.sh (TUI) or bash scripts/bot_cli.sh <subcommand>."
• Note: the auto-create monitor does NOT run in dev (it lives in telegram-bot which is gated by the bots profile). Use bot_cli.sh monitor-once to exercise that code path manually.

Files Created / Modified

File	Operation
docker-compose.override.yml	Modify — add mysql service, profile-gate bots, depends_on, volume
docker/mysql/init.d/01-schema.sql	Create
docker/mysql/init.d/02-seed.sql	Create
app/bot_cli.py	Create
scripts/dev.sh	Create
scripts/bot_cli.sh	Create
envs/dev/.env.example	Create
.gitignore	Modify — add envs/dev/.env
AGENTS.md	Modify — dev tier docs

No new Python or system dependencies. The bot CLI uses argparse + stdlib only; mysql:8.0 is a stable upstream image; everything else is shell + YAML.

Verification

1. Cold start. cp envs/dev/.env.example .env → bash scripts/dev.sh up. After the healthcheck settles (~10–20s), docker compose ps shows mysql, api-server, web-view all running. bash scripts/dev.sh status prints OK.
2. Schema + seed. mysql -h 127.0.0.1 -u cm -pdevpassword cm -e "SELECT username FROM acc" returns the four seed rows.
3. API smoke. curl http://localhost:3000/acc/ returns the four rows as JSON. curl http://localhost:8000/api/acc/ proxies the same through the web-view.
4. CLI no-args = TUI. bash scripts/bot_cli.sh drops into the menu. Pressing q exits cleanly.
5. CLI subcommand parity. bash scripts/bot_cli.sh register (with real agent creds) returns a username/password/link triple matching what Telegram /1 would return. bash scripts/bot_cli.sh monitor-once --target 5 reports current pool size and creates accounts up to the target.
6. Strict mode (E2). bash scripts/dev.sh down, then bash scripts/bot_cli.sh register exits 2 with ERROR: dev stack not running.
7. Reset. bash scripts/dev.sh reset-db wipes the volume; on next up, the seed is re-applied (verified by step 2).
8. Prod compose untouched. docker compose -f docker-compose.yml config renders no mysql service, no bots profile gate, no extra ports — identical output to before this change. (rex/siong Portainer stacks unaffected.)

Risk

Low. Three areas worth naming:

• Port collision. 127.0.0.1:3306 collides with a host-side mysql server if one is already running locally. Operators with such a setup would change the host port (MYSQL_HOST_PORT env override is not added in this design — YAGNI; the .env can map it manually if needed).
• bot_cli accidentally hitting prod cm99.net. register, set-pin, monitor-once, credit, transfer all call cm99.net with real agent credentials. There is no "dry-run" flag in this design. The mitigation is documentation in envs/dev/.env.example (placeholder agent creds; operator opts in by replacing them) plus the AGENTS.md note. If a sandbox cm99 environment is ever available, swap CM_BOT_BASE_URL.
• mysql startup race after reset-db. The healthcheck handles this — api-server waits — so it's covered, but worth listing.
• Latent bug in set_security_pin_api's return contract. The HAL returns a bool but cm_telegram.py:87 does result['f_username'] on it, which would TypeError in the success branch (currently swallowed by the surrounding except). The CLI's cmd_set_pin works around it by re-resolving names locally. Fixing the contract belongs in a separate change (HAL refactor), out of scope here so we don't expand A's blast radius.

Out-of-Scope Follow-Ups

• Sanitized prod snapshot import — dev.sh import-snapshot rex that mysqldumps prod and scrubs sensitive columns before loading. Useful but separate scope.
• --dry-run flag for bot_cli — record-only mode for register/transfer. Useful for testing the CLI plumbing without touching cm99.net.
• Local web-view hot-reload — currently the dev stack rebuilds the web image when files change; a bind mount + flask --reload (gated on CM_DEBUG=true) would make UI iteration instant. R3-adjacent but mostly an A++ improvement.
• Migrate rex/siong off committed env files — the R2 effort. Independent of A.