For 3k+ row deployments, returning the full table in one shot is the
bottleneck — the JSON payload alone is hundreds of KB and the client
mounts thousands of EditableCell instances on every visit. Pagination
with auto-fetch on scroll shrinks both the wire payload and the initial
render to a single page (200 rows).
Server (app/cm_api.py):
- /acc/ and /user/ accept ?limit, ?offset, ?prefix, ?dir, plus ?sort on
/user/ (f_username | last_update_time). Defaults: limit=200 (capped at
1000), offset=0, dir=desc.
- ORDER BY done in SQL with prefix-priority: rows whose username starts
with the configured CM_PREFIX_PATTERN come first, then asc/desc by the
sort column. The 'dir' value is whitelisted to ASC|DESC before string
interpolation; everything else goes through parameterised binding.
- Schema verification (verify_tables_once) deferred to first request via
a Flask before_request hook — keeps create_app() free of MySQL touches
so unit tests + gunicorn preload still work without a live DB.
Web client:
- web/lib/api.ts: getAccountsPage / getUsersPage return { rows, hasMore }.
hasMore = (rows.length === PAGE_SIZE), so the client knows when to
stop fetching. Each page is its own Next.js cache entry (the URL is
the cache key) — caching from the previous commit still applies.
- web/app/actions.ts: loadMoreAccounts / loadMoreUsers Server Actions
for next-page requests; refreshAccounts / refreshUsers force-evict the
cache via revalidateTag before refetching page 1.
- web/app/page.tsx + users/page.tsx: only fetch the first page now.
- web/components/{accounts,users}-table.tsx: rewrote state model. Rows
accumulate as the user scrolls. An IntersectionObserver on a sentinel
div near the bottom triggers loadMore when it enters the viewport
(300px rootMargin so the next page starts loading before the user
reaches the end). useOptimistic wraps the accumulated rows for in-
flight edits; on success the row is committed locally so the change
survives even though we no longer router.refresh.
- Sort toggle now refetches from page 1 with the new dir/sort param.
Local sort over a partial set would be inconsistent.
- Mutations: delete filters from local state; create + refresh both
reset to page 1 so the row appears in its sorted position.
- Header count shows '<loaded>+' when more pages exist so the operator
knows what they're seeing isn't the full table.
Removed AutoRefresh:
- web/app/layout.tsx no longer mounts AutoRefresh.
- web/components/auto-refresh.tsx deleted.
- Reason: router.refresh every 30s would yank the user back to page 1
every time, losing scroll position and accumulated rows. Manual
Refresh button replaces it (now wired to refreshAccounts/refreshUsers
which evict cache + refetch).
Tests: deferred verify_tables_once() means tests.test_bot_cli's
CreateAppFactoryTests pass without DB env vars again. All 38 existing
tests pass.
Eliminates the per-request DB hit when:
- A user opens a tab they've recently visited (within 30s)
- The 30s AutoRefresh fires while no mutations have happened
- Multiple browser tabs are open and switch between Accounts/Users
- A passing-by request races another request for the same data
How it works:
- web/lib/api.ts: fetchApi() now forwards the next.revalidate/tags
options to Next.js's data cache. getAccounts/getUsers tag their
responses ('accounts'/'users') with a 30-second freshness window.
- web/app/actions.ts: every mutation (update/create/delete X 2 tables)
calls revalidateTag() so the next GET for that table bypasses the
cache and re-reads from MySQL. Stale data never lingers after a write.
The cache lives in the cm-web Node process (per worker). For our
2-worker setup that's at most 2 cached copies; the next AutoRefresh
tick after the 30s window expires triggers exactly one DB read per
worker. If the operator manually clicks Refresh, that's a router.refresh
which also re-fetches.
Tradeoffs:
- External DB writes (e.g., the cm99.net monitor inserting a row) won't
appear in the dashboard until the 30s window elapses or a mutation
happens. The previous behavior had a 30s ceiling too (auto-refresh
interval), so the perceived freshness is unchanged.
- Memory: each cached payload is a few KB to a few hundred KB. Trivial.
If you want stricter freshness later, drop CACHE_REVALIDATE_SECONDS in
web/lib/api.ts. If you want pagination on top of this, the cache key
becomes per-URL automatically, so /acc/?offset=200 caches separately
from /acc/?offset=0 — no further work needed.
api-server gets /create-acc-data and /create-user-data POST routes
that INSERT into the respective tables with required-field validation.
Frontend adds an 'Add' button next to Refresh in each table head;
opens a native <dialog> form with all fields. Inputs use 16px font on
phone (sm:text-[13px] desktop) so iOS doesn't auto-zoom.
A small form-dialog-shell helper centralizes the modal chrome,
field label, and input class so create-account-dialog and
create-user-dialog stay focused on their fields and validation.
Adds × delete button per row in both tables (desktop column +
mobile card header). Click → native <dialog> confirm modal with
Esc/backdrop-cancel, destructive red button, error inline.
Wires deleteAccount/deleteUser Server Actions calling the new
api-server routes; revalidatePath refreshes the list on success.
EditableCell input switches to text-base (16px) on phone (sm:text-[13px]
above 640px), preventing iOS Safari auto-zoom-on-focus that was
shifting the layout when the soft keyboard appeared.
