cm_bot_v2/docs/superpowers/specs/2026-05-02-local-as-dev-design.md

# Local-as-Dev Tier (Sub-Project A) Design

**Date:** 2026-05-02
**Status:** Approved (design)
**Sequel to:** [2026-05-02-debug-mode-hotfix-design.md](2026-05-02-debug-mode-hotfix-design.md)
**Out of scope (separate):** rex/siong env file rotation (R2), cm_bot.py scraper resilience (R3), security hardening sub-project (C), Next.js web view (B).

## Problem

`cm_bot_v2` runs two production deployments — `rex` (port 8001) and `siong` (port 8005) — both deployed via Portainer from images on `gitea.04080616.xyz/yiekheng`. There is no formal local development tier. Local iteration today either touches a real production database or has no DB at all (`web-view` is the only service that boots without one). This blocks safe testing of the security hardening (sub-project C) and the planned Next.js rewrite (B), both of which need to be exercised end-to-end without poking prod data.

## Goal

Add a local-only "dev tier" that spins up `mysql` + `api-server` + `web-view` self-contained on a developer machine, plus a Python CLI that gives the same trigger surface the Telegram bot exposes. The goal is end-to-end iteration on the bot's business logic without launching a real Telegram bot, without touching rex/siong databases, and without making automated calls to cm99.net unless the developer explicitly invokes one.

## Non-Goals

- Importing a sanitized snapshot of rex/siong data into the dev DB. Seed data only; snapshot import can come later.
- Containerizing the new bot CLI. It runs from the local Python virtualenv against a port-published dev mysql.
- Changing the prod compose file (`docker-compose.yml`). Portainer stacks for rex/siong remain untouched.
- Migrating or rotating the existing committed `envs/rex/.env` and `envs/siong/.env` files. Those are an R2 problem.
- Building or running `telegram-bot` or `transfer-bot` automatically in dev. They are gated behind a compose `bots` profile and stay quiet.
- Adding curses/textual/prompt_toolkit. The interactive TUI uses stdlib `input()` only.
- Adding a `remove`/`delete` CLI subcommand. The current Telegram bot has no analog and the schema has no soft-delete state.

## Architecture Overview

Three additions, no removals:

1. **A `mysql` service in `docker-compose.override.yml`** — MySQL 8.0 image, named volume for persistence, init scripts mounted at `/docker-entrypoint-initdb.d/`. Published to `127.0.0.1:3306:3306` so the local CLI (running outside Docker) can reach it without exposing the port to anything else on the network.
2. **A new Python CLI module: `app/bot_cli.py`** — argparse-driven, mirrors the four Telegram bot handlers plus three operational ops. With no arguments, drops into a stdlib interactive menu (TUI-style; not full curses).
3. **Two shell scripts in `scripts/`** — `dev.sh` (lifecycle: `up`/`down`/`reset-db`/`logs`/`status`) and `bot_cli.sh` (env-loading wrapper for `python -m app.bot_cli`).

Compose service start matrix:

| Command | mysql | api-server | web-view | telegram-bot | transfer-bot |
|---|---|---|---|---|---|
| `dev.sh up` | ✓ | ✓ | ✓ | (gated by `bots` profile) | (gated by `bots` profile) |
| `bot_cli.sh ...` | (already up — E2 check) | (already up) | (already up) | — (CLI is local Python) | — |
| Prod (Portainer using `docker-compose.yml`) | — | ✓ | ✓ | ✓ | ✓ |

Network: api-server reaches `mysql:3306` over the compose network; the local CLI reaches `127.0.0.1:3306` via the published port. `DB_HOST` is overridden by `bot_cli.sh` so a single `.env` works for both.

## Components

### 1. `docker-compose.override.yml` changes

Add a `mysql` service to the existing override file (which currently only has `build:` directives for local image builds):

```yaml
services:
  mysql:
    image: mysql:8.0
    container_name: ${CM_DEPLOY_NAME:-cm}-mysql
    restart: unless-stopped
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-devroot}
      MYSQL_DATABASE: ${DB_NAME}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASSWORD}
    ports:
      - "127.0.0.1:3306:3306"
    volumes:
      - mysql-data:/var/lib/mysql
      - ./docker/mysql/init.d:/docker-entrypoint-initdb.d:ro
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-u", "root", "-p${MYSQL_ROOT_PASSWORD:-devroot}"]
      interval: 5s
      timeout: 3s
      retries: 12
    networks:
      - bot-network

  api-server:
    depends_on:
      mysql:
        condition: service_healthy

  telegram-bot:
    profiles: ["bots"]

  transfer-bot:
    profiles: ["bots"]

volumes:
  mysql-data:
    name: ${CM_DEPLOY_NAME:-cm}-mysql-data
```

Notes:
- The `127.0.0.1:` host-binding prefix means port 3306 is reachable from your shell only — not from other machines on the LAN. Important: the rex/siong scanner at `192.168.0.210` cannot reach this even if it tried.
- `depends_on: { mysql: { condition: service_healthy } }` means `api-server` waits for the healthcheck to pass before starting. This eliminates the cold-start race where api-server tried to connect before mysql was accepting traffic.
- Adding `profiles: ["bots"]` to `telegram-bot` and `transfer-bot` keeps them out of `docker compose up` in dev, but `docker compose run --rm telegram-bot ...` still works for parity tests.

### 2. `docker/mysql/init.d/`

Two SQL scripts run once on first volume creation (mysql:8 invokes everything in `/docker-entrypoint-initdb.d/` in alphabetical order):

- `docker/mysql/init.d/01-schema.sql` — exactly the DDL from `AGENTS.md` (acc + user tables, utf8mb4). The `CREATE DATABASE` from AGENTS.md is dropped because mysql:8 image creates `${DB_NAME}` via env. The `USE rex_cm` line is replaced with `USE \`${DB_NAME}\``.
- `docker/mysql/init.d/02-seed.sql` — minimal seed: one row in `acc` matching `CM_PREFIX_PATTERN=13c` so `get_next_username` works, plus three filler rows.

Seed contents:

```sql
INSERT INTO acc (username, password, status, link) VALUES
  ('13c1000', 'seedpass', '', ''),
  ('13c1001', 'seedpass', '', ''),
  ('13c1002', 'seedpass', '', ''),
  ('13c1003', 'seedpass', '', '');
```

These are dev-only seed values — never real credentials. The scripts only run on a fresh volume (mysql refuses to re-init an initialized volume). To re-seed, run `dev.sh reset-db`.

### 3. `app/bot_cli.py`

```python
import argparse
import os
import sys

from .cm_bot_hal import CM_BOT_HAL


def _print_user(user: dict) -> None:
    print(f"Username: {user['username']}")
    print(f"Password: {user['password']}")
    print(f"Link: {user['link']}")


def cmd_register(_args):
    bot = CM_BOT_HAL()
    _print_user(bot.get_user_api())


def cmd_set_pin(args):
    bot = CM_BOT_HAL()
    if not bot.is_whatsapp_url(args.link):
        print(f"ERROR: not a WhatsApp URL: {args.link}", file=sys.stderr)
        sys.exit(2)
    # Resolve names locally so we have something useful to print regardless of
    # what set_security_pin_api returns. The HAL currently returns a bool from
    # the trailing insert_user_to_table_user; the Telegram handler has the
    # same shape and a latent bug accessing result['f_username']. We avoid
    # depending on the return shape here.
    t_username, f_username = bot.get_whatsapp_link_username(args.link)
    success = bot.set_security_pin_api(args.link)
    if not success:
        print("ERROR: set_security_pin_api returned a falsy result", file=sys.stderr)
        sys.exit(1)
    print(f"OK: f_username={f_username} t_username={t_username}")


def cmd_insert_user(args):
    bot = CM_BOT_HAL()
    f_password = bot.get_user_pass_from_acc(args.f_username)
    if not f_password:
        print(f"ERROR: no password for {args.f_username}", file=sys.stderr)
        sys.exit(2)
    success = bot.insert_user_to_table_user({
        'f_username': args.f_username,
        'f_password': f_password,
        't_username': args.t_username,
        't_password': bot.security_pin,
    })
    if not success:
        print("ERROR: insert failed", file=sys.stderr)
        sys.exit(1)
    print(f"OK: inserted {args.f_username} → {args.t_username}")


def cmd_credit(args):
    bot = CM_BOT_HAL()
    print(f"Credit: {bot.get_user_credit(args.username, args.password)}")


def cmd_transfer(args):
    bot = CM_BOT_HAL()
    print(bot.transfer_credit_api(args.f_username, args.f_password, args.t_username, args.t_password))


def cmd_monitor_once(args):
    bot = CM_BOT_HAL()
    available = bot.get_all_available_acc()
    print(f"Available accounts: {len(available)} (target: {args.target})")
    if len(available) >= args.target:
        print("Already at target; nothing to do.")
        return
    for _ in range(len(available), args.target):
        try:
            user = bot.create_new_acc()
            print(f"Created: {user['username']}")
        except Exception as exc:
            print(f"ERROR creating account: {exc}", file=sys.stderr)
            sys.exit(1)


def cmd_interactive(_args):
    """Telegram-style menu in a TTY loop. stdlib only."""
    print("CM Bot CLI — interactive (type 'q' to quit, '?' for menu)")
    while True:
        print()
        print("  1                                    Register / get next account")
        print("  2 <whatsapp_link>                    Set security PIN")
        print("  3 <f_username> <t_username>          Insert into user table")
        print("  credit <username> <password>         Read account credit")
        print("  transfer <fu> <fp> <tu> <tp>         One-shot credit transfer")
        print("  monitor [N]                          Run monitor once (default 20)")
        print("  q                                    Quit")
        try:
            line = input("> ").strip()
        except (EOFError, KeyboardInterrupt):
            print()
            return
        if not line:
            continue
        if line in ("q", "quit", "exit"):
            return
        if line in ("?", "help", "menu"):
            continue
        argv = line.split()
        # Map TUI shortcuts to argparse subcommand names so we reuse the same
        # dispatch table for both modes.
        TUI_ALIASES = {"1": "register", "2": "set-pin", "3": "insert-user"}
        argv[0] = TUI_ALIASES.get(argv[0], argv[0])
        try:
            args = build_parser().parse_args(argv)
            args.func(args)
        except SystemExit:
            # argparse calls sys.exit() on parse error; swallow it to keep the
            # REPL alive instead of bailing out of the loop.
            continue
        except Exception as exc:
            print(f"ERROR: {exc}", file=sys.stderr)


def build_parser() -> argparse.ArgumentParser:
    p = argparse.ArgumentParser(prog="bot_cli", description="CM Bot dev CLI (mirrors Telegram triggers).")
    sub = p.add_subparsers(dest="command")

    sp = sub.add_parser("register", aliases=["get-acc"], help="Get next available account (Telegram /1).")
    sp.set_defaults(func=cmd_register)

    sp = sub.add_parser("set-pin", help="Set security PIN from a WhatsApp link (Telegram /2).")
    sp.add_argument("link")
    sp.set_defaults(func=cmd_set_pin)

    sp = sub.add_parser("insert-user", help="Insert into user table (Telegram /3).")
    sp.add_argument("f_username")
    sp.add_argument("t_username")
    sp.set_defaults(func=cmd_insert_user)

    sp = sub.add_parser("credit", help="Read account credit balance.")
    sp.add_argument("username")
    sp.add_argument("password")
    sp.set_defaults(func=cmd_credit)

    sp = sub.add_parser("transfer", help="One-shot credit transfer.")
    sp.add_argument("f_username")
    sp.add_argument("f_password")
    sp.add_argument("t_username")
    sp.add_argument("t_password")
    sp.set_defaults(func=cmd_transfer)

    sp = sub.add_parser("monitor-once", aliases=["monitor"], help="One iteration of the auto-create monitor.")
    sp.add_argument("--target", type=int, default=20)
    sp.set_defaults(func=cmd_monitor_once)

    sp = sub.add_parser("interactive", help="Drop into the TUI menu.")
    sp.set_defaults(func=cmd_interactive)

    return p


def main(argv=None) -> int:
    parser = build_parser()
    args = parser.parse_args(argv)
    if args.command is None:
        # Default mode: interactive TUI.
        return cmd_interactive(args) or 0
    return args.func(args) or 0


if __name__ == "__main__":
    sys.exit(main())
```

Design notes:

- The TUI loop reuses `argparse` parsing so subcommand and interactive paths share validation. SystemExit is caught so a typo (e.g., `2` with no link) prints the argparse error and returns to the prompt instead of killing the REPL.
- The default `python -m app.bot_cli` (no args) drops into interactive mode. This matches the user's "TUI" expectation while keeping the same module usable as a one-shot script.
- Subcommand names are spelled-out verbs (`register`, `set-pin`) for shell scripting; the TUI accepts `1`/`2`/`3` shortcuts so muscle memory from Telegram works.
- The CLI imports `CM_BOT_HAL` directly. No new business logic. If you change the bot's behavior in `cm_bot_hal.py`, the CLI tracks automatically.

### 4. `scripts/dev.sh`

```bash
#!/usr/bin/env bash
set -euo pipefail

usage() {
  cat <<'EOF'
Lifecycle commands for the local dev stack (mysql + api-server + web-view).
Bots (telegram-bot, transfer-bot) are gated behind a compose 'bots' profile
and do not start with 'up'.

Usage:
  scripts/dev.sh up         Start the dev stack in the background.
  scripts/dev.sh down       Stop the stack. Volume kept (DB persists).
  scripts/dev.sh reset-db   Stop the stack AND drop the mysql volume.
  scripts/dev.sh logs       Tail logs from the running stack.
  scripts/dev.sh status     Print 'OK' if mysql container is running, else exit 1.
EOF
}

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"

SUDO="sudo"
[[ "${NO_SUDO:-0}" == "1" ]] && SUDO=""
COMPOSE=(${SUDO} docker compose -f docker-compose.yml -f docker-compose.override.yml)

[[ -f .env ]] || { echo "ERROR: .env not found. cp envs/dev/.env.example .env (then edit)." >&2; exit 2; }

case "${1:-}" in
  up)
    "${COMPOSE[@]}" up -d --build mysql api-server web-view
    "${COMPOSE[@]}" ps
    ;;
  down)
    "${COMPOSE[@]}" down
    ;;
  reset-db)
    "${COMPOSE[@]}" down --volumes
    "${COMPOSE[@]}" up -d --build mysql api-server web-view
    ;;
  logs)
    "${COMPOSE[@]}" logs -f mysql api-server web-view
    ;;
  status)
    if "${COMPOSE[@]}" ps --status running --services | grep -q '^mysql$'; then
      echo OK
    else
      echo "ERROR: dev stack not running. Run 'scripts/dev.sh up' first." >&2
      exit 1
    fi
    ;;
  -h|--help|help|"")
    usage
    [[ "${1:-}" == "" ]] && exit 1 || exit 0
    ;;
  *)
    echo "unknown command: $1" >&2
    usage >&2
    exit 1
    ;;
esac
```

### 5. `scripts/bot_cli.sh`

```bash
#!/usr/bin/env bash
# Run the bot CLI in the local venv. With no args, drops into the TUI menu.
# Requires: dev stack up (run scripts/dev.sh up first), .venv with deps.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT_DIR}"

# E2: bail if the dev stack is not running.
if ! NO_SUDO="${NO_SUDO:-0}" bash scripts/dev.sh status >/dev/null 2>&1; then
  echo "ERROR: dev stack not running. Run 'scripts/dev.sh up' first." >&2
  exit 2
fi

[[ -f .env ]] || { echo "ERROR: .env not found. cp envs/dev/.env.example .env (then edit)." >&2; exit 2; }

# Load .env into the environment (export everything between 'set -a' and 'set +a').
set -a
# shellcheck disable=SC1091
source .env
set +a

# Override DB host/port for the local CLI: docker mysql is published on
# 127.0.0.1:3306 even though api-server in-network reaches it as mysql:3306.
export DB_HOST=127.0.0.1
export DB_PORT=3306

PYTHON_BIN="${PYTHON_BIN:-${ROOT_DIR}/.venv/bin/python}"
[[ -x "${PYTHON_BIN}" ]] || { echo "ERROR: ${PYTHON_BIN} not found. Create venv: python3 -m venv .venv && .venv/bin/pip install -r requirements.txt" >&2; exit 2; }

exec "${PYTHON_BIN}" -m app.bot_cli "$@"
```

### 6. `envs/dev/.env.example` (committed)

```
# === Runtime ===
CM_DEBUG=true

# === Deployment Identity ===
CM_DEPLOY_NAME=dev-cm
CM_WEB_HOST_PORT=8000

# === Docker Registry / Build ===
CM_IMAGE_PREFIX=local
DOCKER_IMAGE_TAG=dev

# === Telegram (unused in A2 — telegram-bot is gated by 'bots' profile) ===
TELEGRAM_BOT_TOKEN=fill-only-if-running-bots-profile
TELEGRAM_ALERT_CHAT_ID=
TELEGRAM_ALERT_BOT_TOKEN=

# === Database (dev mysql in docker; bot_cli.sh overrides DB_HOST=127.0.0.1) ===
DB_HOST=mysql
DB_USER=cm
DB_PASSWORD=devpassword
DB_NAME=cm
DB_PORT=3306
DB_CONNECTION_TIMEOUT=8
DB_CONNECT_RETRIES=5
DB_CONNECT_RETRY_DELAY=2
MYSQL_ROOT_PASSWORD=devroot

# === Bot Config ===
# CM_PREFIX_PATTERN=13c MUST match the seed in docker/mysql/init.d/02-seed.sql.
CM_PREFIX_PATTERN=13c
CM_AGENT_ID=fill-with-real-agent-id-to-test-cm99-calls
CM_AGENT_PASSWORD=fill-with-real-agent-password-to-test-cm99-calls
CM_SECURITY_PIN=000000
CM_BOT_BASE_URL=https://cm99.net
```

Operator workflow: `cp envs/dev/.env.example .env`, fill in real `CM_AGENT_ID` / `CM_AGENT_PASSWORD` only if they want to invoke `register` / `monitor-once` / `credit` / `transfer` against real cm99.net. The `set-pin` and `insert-user` ops also need them.

`envs/dev/.env` (the operator's filled-in copy) is added to `.gitignore`.

### 7. `.gitignore` change

```
__pycache__
.DS_Store
*.html
logs
envs/dev/.env
```

Just `envs/dev/.env` — rex/siong stay tracked (their handling is R2's problem).

### 8. `AGENTS.md` updates

- Replace the section recommending raw schema setup with a pointer to `bash scripts/dev.sh up`.
- Add a new "Dev Tier" subsection: "Local development uses `envs/dev/.env.example` → `.env` → `bash scripts/dev.sh up`. The bot CLI: `bash scripts/bot_cli.sh` (TUI) or `bash scripts/bot_cli.sh <subcommand>`."
- Note: the auto-create monitor does NOT run in dev (it lives in `telegram-bot` which is gated by the `bots` profile). Use `bot_cli.sh monitor-once` to exercise that code path manually.

## Files Created / Modified

| File | Operation |
|---|---|
| `docker-compose.override.yml` | Modify — add mysql service, profile-gate bots, depends_on, volume |
| `docker/mysql/init.d/01-schema.sql` | Create |
| `docker/mysql/init.d/02-seed.sql` | Create |
| `app/bot_cli.py` | Create |
| `scripts/dev.sh` | Create |
| `scripts/bot_cli.sh` | Create |
| `envs/dev/.env.example` | Create |
| `.gitignore` | Modify — add `envs/dev/.env` |
| `AGENTS.md` | Modify — dev tier docs |

No new Python or system dependencies. The bot CLI uses argparse + stdlib only; mysql:8.0 is a stable upstream image; everything else is shell + YAML.

## Verification

1. **Cold start.** `cp envs/dev/.env.example .env` → `bash scripts/dev.sh up`. After the healthcheck settles (~10–20s), `docker compose ps` shows mysql, api-server, web-view all `running`. `bash scripts/dev.sh status` prints `OK`.
2. **Schema + seed.** `mysql -h 127.0.0.1 -u cm -pdevpassword cm -e "SELECT username FROM acc"` returns the four seed rows.
3. **API smoke.** `curl http://localhost:3000/acc/` returns the four rows as JSON. `curl http://localhost:8000/api/acc/` proxies the same through the web-view.
4. **CLI no-args = TUI.** `bash scripts/bot_cli.sh` drops into the menu. Pressing `q` exits cleanly.
5. **CLI subcommand parity.** `bash scripts/bot_cli.sh register` (with real agent creds) returns a username/password/link triple matching what Telegram `/1` would return. `bash scripts/bot_cli.sh monitor-once --target 5` reports current pool size and creates accounts up to the target.
6. **Strict mode (E2).** `bash scripts/dev.sh down`, then `bash scripts/bot_cli.sh register` exits 2 with `ERROR: dev stack not running.`
7. **Reset.** `bash scripts/dev.sh reset-db` wipes the volume; on next `up`, the seed is re-applied (verified by step 2).
8. **Prod compose untouched.** `docker compose -f docker-compose.yml config` renders no mysql service, no bots profile gate, no extra ports — identical output to before this change. (rex/siong Portainer stacks unaffected.)

## Risk

Low. Three areas worth naming:

- **Port collision.** `127.0.0.1:3306` collides with a host-side mysql server if one is already running locally. Operators with such a setup would change the host port (`MYSQL_HOST_PORT` env override is *not* added in this design — YAGNI; the .env can map it manually if needed).
- **bot_cli accidentally hitting prod cm99.net.** `register`, `set-pin`, `monitor-once`, `credit`, `transfer` all call cm99.net with real agent credentials. There is no "dry-run" flag in this design. The mitigation is documentation in `envs/dev/.env.example` (placeholder agent creds; operator opts in by replacing them) plus the AGENTS.md note. If a sandbox cm99 environment is ever available, swap `CM_BOT_BASE_URL`.
- **mysql startup race after `reset-db`.** The healthcheck handles this — api-server waits — so it's covered, but worth listing.
- **Latent bug in `set_security_pin_api`'s return contract.** The HAL returns a bool but `cm_telegram.py:87` does `result['f_username']` on it, which would TypeError in the success branch (currently swallowed by the surrounding except). The CLI's `cmd_set_pin` works around it by re-resolving names locally. Fixing the contract belongs in a separate change (HAL refactor), out of scope here so we don't expand A's blast radius.

## Out-of-Scope Follow-Ups

- **Sanitized prod snapshot import** — `dev.sh import-snapshot rex` that mysqldumps prod and scrubs sensitive columns before loading. Useful but separate scope.
- **`--dry-run` flag for bot_cli** — record-only mode for `register`/`transfer`. Useful for testing the CLI plumbing without touching cm99.net.
- **Local `web-view` hot-reload** — currently the dev stack rebuilds the web image when files change; a bind mount + flask `--reload` (gated on `CM_DEBUG=true`) would make UI iteration instant. R3-adjacent but mostly an A++ improvement.
- **Migrate rex/siong off committed env files** — the R2 effort. Independent of A.