From 50cd3ecc6c544960763b9747623fd1a845a33ed2 Mon Sep 17 00:00:00 2001 From: Mia Wu Date: Tue, 2 Jun 2026 11:59:31 -0400 Subject: [PATCH] minor ui change --- CLAUDE.md | 144 ----------------------- DESIGNPLAN.md | 24 +++- IMPLEMENTATION_PLAN.md | 142 ++++++++++------------ dashboard/src/components/ConfigModal.jsx | 2 +- 4 files changed, 81 insertions(+), 231 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index a5bdc4b..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,144 +0,0 @@ -# CLAUDE.md — Project Context for AI Assistants - -## What this project is - -A real-time web dashboard that tracks WiFi test execution progress. It compares a **target tests directory** (`.ini` files defining every test that must run) against a **results directory** (folders created when each test completes). The dashboard shows completion rates, elapsed time, estimated time remaining, and a filterable table of every test. - ---- - -## How to run - -### Backend (Python) -```powershell -cd server -.\.venv\Scripts\Activate.ps1 # activate venv -python app.py # starts Flask on port 3001 -``` - -### Frontend (dev) -```powershell -cd dashboard -npm run dev # Vite dev server; proxies /api → localhost:3001 -``` - -### Frontend (production build) -```powershell -cd dashboard -npm run build # outputs to dashboard/dist/ -# Flask serves dist/ automatically when app.py is running -``` - ---- - -## Tech stack - -| Layer | Technology | -|---|---| -| Frontend | React 18 + Vite, TailwindCSS v4 (`@tailwindcss/vite`), TanStack React Query | -| Backend | Python 3, Flask, flask-cors | -| Database | SQLite via Python stdlib `sqlite3` (WAL mode, single persistent connection + RLock) | -| File watching | `watchdog` Python library | -| Real-time | Server-Sent Events (SSE) — one-way push from backend to browser | - ---- - -## Key files - -### Backend (`server/`) - -| File | Purpose | -|---|---| -| `app.py` | Flask app, all route handlers, `bootstrap()`, `__main__` | -| `db_py.py` | SQLite schema init, all query helpers (`upsert_test`, `mark_completed`, `reset_by_file_id_and_device`, `get_config`, `set_config`, etc.) | -| `scanner.py` | `full_scan()`, `scan_targets()`, `scan_results()`, `process_result_dir()`, `parse_deleted_result_dir_name()` | -| `parser.py` | `parse_target_filename()`, `parse_result_filename()`, `parse_timestamp()`, `parse_elapsed_time()`, `parse_tput_rssi()` | -| `watcher.py` | Two watchdog `Observer` instances — `_TargetHandler` and `_ResultsHandler`. `start_watching()` / `stop_watching()` | -| `sse_py.py` | Per-client `Queue` registry, `stream_events()` generator, `broadcast(data)` | - -### Frontend (`dashboard/src/`) - -| File | Purpose | -|---|---| -| `lib/api.js` | Thin `fetch` wrapper; all API calls go through `apiFetch('/path')` | -| `hooks/useStats.js` | Fetches `/api/stats`; **owns the SSE `EventSource`**; invalidates `['stats']` and `['tests']` on `update` events | -| `hooks/useTests.js` | Fetches `/api/tests` | -| `hooks/useConfig.js` | Fetches/saves `/api/config` | -| `components/ConfigModal.jsx` | Settings modal: directory paths (always editable) + avg time overrides | -| `components/DirectoryBrowser.jsx` | Server-side folder picker using `/api/browse` | - ---- - -## Data model - -### How tests are identified - -- **Target files** live in subdirectories of the target directory. Format: `TC_WIFI_.ini`. Files named `GLOBAL.ini` are skipped. -- **`test_id`**: segment matching `R\d+[A-Z0-9]+` (e.g., `R2COERXAX014`) -- **Result directories**: top-level subdirectories of the results directory. A test is **completed** when a result directory name contains the same `test_id`. -- **Result folder name example**: `COE_CGW453_R2COERXAX014_TPT3E_RSSI70_STA56_2GHZ_CH1_BW20_TCP_MIMOFD_SONFD_MESHFD_LPI_UL` - -### Tags parsed from target filename - -`interference`, `device`, `test_point`, `rssi`, `station`, `band`, `channel`, `bandwidth`, `direction` - -`rotation` is parsed from the **parent folder name** (segment matching `ROT\d+`). - -### Database tables - -**`tests`** — one row per target `.ini` file: -- `id` (TEXT PK) — `parent_dir/base_filename` -- `test_id`, `parent_dir`, `filename`, `interference`, `device`, `rotation`, `test_point`, `station`, `band`, `channel`, `bandwidth`, `rssi`, `direction` -- `completed` (0/1), `completed_at` (ISO timestamp), `duration_seconds` (REAL) -- `tput_results` (JSON array of `{station, tput, dlRssi, ulRssi}`) - -**`config`** — key/value store: -- `target_dir`, `results_dir` -- `avg_time_coe`, `avg_time_p2p`, `avg_time_p3p` (seconds as string; NULL = use calculated average) - ---- - -## API routes - -| Method | Path | Description | -|---|---|---| -| GET | `/api/tests` | All tests; supports query filters (`completed`, `interference`, `device`, `rotation`, `testPoint`, `station`, `band`, `channel`, `bandwidth`, `rssi`, `direction`) | -| GET | `/api/stats` | Aggregated stats: overall, per-device, timing with estimates | -| GET | `/api/events` | SSE stream; sends `data: {"type":"update"}\n\n` on any directory change | -| GET | `/api/config` | Current config values | -| POST | `/api/config` | Update config; triggers full rescan + rewatcher if dirs changed | -| POST | `/api/config/rescan` | Force full rescan without changing config | -| GET | `/api/browse?path=...` | List subdirectories at a server path (omit `path` for drive roots) | - ---- - -## Watcher architecture - -Two `watchdog.Observer` instances run in daemon threads: - -1. **`_TargetHandler`** — watches `target_dir` recursively. Any add/delete/move of a `TC_WIFI_*.ini` file schedules a **debounced full scan** (1 second timer, cancels and restarts on rapid changes). - -2. **`_ResultsHandler`** — watches `results_dir` recursively. Handles events surgically: - - Directory created → `process_result_dir()` + broadcast - - Directory deleted → `reset_by_file_id_and_device()` + broadcast - - Directory moved in/out/renamed → appropriate reset/process + broadcast - - File created/deleted inside a result dir → `process_result_dir()` + broadcast - -### Critical Windows quirk -When a directory is deleted (including Recycle Bin), `watchdog` calls `os.path.isdir()` at event-processing time. The directory is already gone, so it returns `False`, making `event.is_directory = False` even for directory events. **All handlers check `_is_direct_child_dir(path)` directly** — never gate on `event.is_directory` for delete/move cases. - ---- - -## SSE implementation notes - -- `sse_py.py`: each connected client gets a `queue.Queue`. `broadcast()` calls `put_nowait(payload)` on all queues. `stream_events()` is a generator yielding `f"data: {payload}\n\n"` (real newlines — `\n` not `\\n`). -- `useStats.js`: creates `new EventSource('/api/events')` once on mount. On `{"type":"update"}` message, invalidates React Query keys `['stats']` and `['tests']`. -- Vite dev proxy forwards `/api/events` to Flask, keeping the SSE connection alive. - ---- - -## Known issues / gotchas - -- `config.json` in `server/` is a legacy file used for a one-time migration to SQLite on first run. It is no longer needed once `dashboard.db` exists. -- The backend entrypoint is `app.py`, not `main.py`. -- Flask dev server (`app.run`) is used directly — no gunicorn/waitress configured yet. -- Port is `3001` (configurable via `PORT` env var). diff --git a/DESIGNPLAN.md b/DESIGNPLAN.md index 63c5bd2..996b4c9 100644 --- a/DESIGNPLAN.md +++ b/DESIGNPLAN.md @@ -1,28 +1,38 @@ ## Design Plan -**Inputs**: target tests directory, result tests directory +**Inputs**: target tests directory, reference result tests directory, DUT result tests directory **Output**: a dashboard that displays completion rate, elapsed time, estimated time remaining, and a list of all tests that can be filtered by tags **Requirements**: - A test is marked completed when there is a filename match in result tests directory -- When there is a change in the input directories, data should update automatically or when refreshed - Dashboard can be accessed by different devices, but input directories are stored locally on one machine - - Add SSO later +- Login required in order to view dashboard + - Viewer + - Only able to view + - Admin + - Able to access and modify settings +- Can enter exclusions for tests that are not ran in settings Front end UI requirements: -- Completion rate +- Completion rate and number of tests completed - Overall completion rate - CGW 452 completion rate + - number of tests completed for each type - CGW453 completion rate + - number of tests completed for each type - Time elapsed - Estimated Time remaining +- Estimated date to complete + - 16 hrs on weekdays + - 24 hrs on weekends - List of all tests - Can filter by Tags: - Completed: Boolean + - Type - Device: String - - Rotation: String + - Rotation: String - Test Point: String - Station: String - Band: String @@ -30,3 +40,7 @@ Front end UI requirements: - Bandwidth: String - RSSI: String - Direction: String + - A P2P test with COE pairings should be displayed together + - P3P tests should be paired together by throttled/unthrottled + - Completed tests + - Display the avg throughput for each station diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index d8e7c94..f13f07e 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -1,6 +1,6 @@ # Implementation Plan — Test Dashboard -> **Status as of May 2026**: fully implemented and running. Backend migrated from the original Node.js plan to Python (Flask). See current tech stack and structure below. +> **Status as of June 2026**: fully implemented and running. Backend is Python (Flask) with JWT authentication and role-based access control. ## 1. Overview @@ -14,10 +14,9 @@ A web dashboard that monitors test execution progress by comparing a **target te |---|---|---| | Frontend | React + Vite | Dev server proxies `/api` → `localhost:3001` | | Backend | Python 3 + Flask | Replaced original Node.js/Express plan | -| Real-time | Server-Sent Events (SSE) | One-way push from backend to browser | -| Directory watching | watchdog | Python filesystem watcher (Windows-compatible) | +| Auth | JWT (PyJWT) | Bearer token auth with `admin` and `viewer` roles | | Database | SQLite (via `sqlite3` stdlib) | WAL mode; single persistent connection with RLock | -| Frontend state | TanStack React Query | Cache + SSE-driven invalidation | +| Frontend state | TanStack React Query | Query cache + manual invalidation after mutations | | UI | TailwindCSS v4 | Via `@tailwindcss/vite` plugin | --- @@ -35,7 +34,7 @@ test_house_dashboard/ │ ├── index.html │ ├── package.json │ └── src/ -│ ├── App.jsx – root layout, SSE wiring via useStats +│ ├── App.jsx – root layout, auth gate, modal settings entry │ ├── main.jsx │ ├── components/ │ │ ├── StatCard.jsx – metric card (label / value / sub) @@ -44,22 +43,20 @@ test_house_dashboard/ │ │ ├── FilterPanel.jsx – dropdown tag filters │ │ ├── TimeDisplay.jsx – elapsed / estimated time display │ │ ├── StatusBadge.jsx – completed / pending indicator -│ │ ├── ConfigModal.jsx – settings modal (dirs + avg time overrides) -│ │ └── DirectoryBrowser.jsx – server-side folder picker (uses /api/browse) +│ │ └── ConfigModal.jsx – settings modal (dirs + avg time overrides) │ ├── hooks/ -│ │ ├── useStats.js – fetches stats; owns the SSE EventSource +│ │ ├── useAuth.js – login session state + role checks +│ │ ├── useStats.js – fetches stats │ │ ├── useTests.js – fetches test list │ │ └── useConfig.js – fetches/saves config │ └── lib/ -│ └── api.js – thin fetch wrapper; BASE = '/api' +│ └── api.js – fetch wrapper with JWT header injection │ └── server/ ← Python backend - ├── app.py – Flask app, all routes, bootstrap(), __main__ + ├── app.py – Flask app, JWT auth, role guards, bootstrap(), __main__ ├── db_py.py – SQLite schema, query helpers, config CRUD ├── scanner.py – full_scan(), scan_targets(), scan_results(), process_result_dir() ├── parser.py – parse_target_filename(), parse_result_filename(), etc. - ├── watcher.py – watchdog observers for target + results dirs - ├── sse_py.py – SSE queue registry + broadcast() ├── requirements.txt ├── dashboard.db – SQLite database (auto-created) └── .venv/ – Python virtual environment @@ -185,9 +182,20 @@ CREATE TABLE config ( key TEXT PRIMARY KEY, value TEXT NOT NULL ); + +-- Stores dashboard users +CREATE TABLE users ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + username TEXT NOT NULL UNIQUE, + password_hash TEXT NOT NULL, + role TEXT NOT NULL, -- admin | viewer + is_active INTEGER NOT NULL DEFAULT 1, + created_at TEXT NOT NULL DEFAULT (datetime('now')) +); -- Config keys: -- target_dir – path to target tests directory -- results_dir – path to results directory +-- results_dir_ref – optional reference results directory -- avg_time_coe – manual avg seconds per COE test (NULL = use calculated) -- avg_time_p2p – manual avg seconds per P2P test (NULL = use calculated) -- avg_time_p3p – manual avg seconds per P3P test (NULL = use calculated) @@ -201,12 +209,23 @@ CREATE TABLE config ( | Method | Path | Description | |---|---|---| +| `POST` | `/api/auth/login` | Login with username/password; returns JWT + user role | +| `GET` | `/api/auth/me` | Resolve current authenticated user from JWT | +| `GET` | `/api/users` | Admin-only list of users | +| `POST` | `/api/users` | Admin-only create user (`admin` / `viewer`) | | `GET` | `/api/tests` | All tests; supports query filters (see below) | | `GET` | `/api/stats` | Aggregated stats object | -| `GET` | `/api/events` | SSE stream — pushes `update` events on directory change | +| `GET` | `/api/scan-status` | Returns active scan state | | `GET` | `/api/config` | Current directory paths and avg time overrides | -| `POST` | `/api/config` | Set `targetDir`, `resultsDir`, avg time overrides; triggers re-scan | -| `GET` | `/api/browse` | List subdirectories at a given server path (directory picker) | +| `POST` | `/api/config` | Save config and optionally trigger full re-scan | +| `POST` | `/api/config/rescan` | Force full re-scan using saved config | +| `POST` | `/api/config/rescan-results` | Re-process only results directories | + +Auth/role rules: + +- All dashboard data routes require a valid JWT. +- `/api/config*` and `/api/users*` are admin-only. +- Frontend stores JWT in localStorage key `dashboard_jwt` and sends `Authorization: Bearer `. ### `GET /api/tests` Query Parameters @@ -277,44 +296,13 @@ CREATE TABLE config ( - `estimatedRemainingSeconds` = `(avgCOE × COE_remaining) + (avgP2P × P2P_remaining) + (avgP3P × P3P_remaining)` - If any type has remaining tests but no avg time (calculated or manual), that type contributes `null` to the sum and the UI flags it as **input required** -### `GET /api/browse` — Directory Browser - -Allows the frontend to navigate the server's local filesystem so users can pick directories without typing paths manually. - -**Query parameters**: -- `path` (optional) — absolute path to list. If omitted or empty, returns the filesystem roots (e.g., `C:\`, `D:\` on Windows). - -**Response**: -```json -{ - "path": "C:\\Tests", - "parent": "C:\\", - "dirs": [ - { "name": "Target", "path": "C:\\Tests\\Target" }, - { "name": "Results", "path": "C:\\Tests\\Results" } - ] -} -``` - -- Only **directories** are returned (no files). -- Hidden directories (names starting with `.`) are excluded. -- If `path` does not exist or is not a directory, returns `400`. -- `parent` is `null` when already at a filesystem root. - ---- - ## 7. Real-time Updates -- Backend registers an SSE endpoint at `GET /api/events` (`sse_py.py`) -- `watchdog` watches both target and results directories via two separate `Observer` instances (`watcher.py`) -- On any change, the backend updates SQLite and calls `broadcast({"type": "update"})`, which pushes `data: {"type":"update"}\n\n` to all connected SSE clients -- `useStats.js` owns the `EventSource('/api/events')` connection; on `update` it calls `queryClient.invalidateQueries` for both `['stats']` and `['tests']` +Current implementation does not use SSE/watchdog. -**Windows watchdog quirk — `is_directory` unreliable on delete**: when a directory is deleted, watchdog calls `os.path.isdir()` to set `event.is_directory`, but the directory is already gone by then, so it returns `False`. All delete/move handlers therefore check `_is_direct_child_dir(path)` directly instead of relying on `event.is_directory`. - -**Result directory deleted**: `reset_by_file_id_and_device(test_id, device)` resets `completed = 0`, `completed_at = NULL`, `duration_seconds = NULL`. broadcast fires unconditionally regardless of whether the folder name was parseable. - -**Recycle Bin delete on Windows**: fires a `MovedEvent` (`src` = result dir, `dest` = `$RECYCLE.BIN\...`). Handled by `on_moved` via `_is_direct_child_dir` on the source path. +- Stats/tests/config are fetched via React Query. +- Mutations (`saveConfig`, `Save Times`, `Save Results`) manually invalidate affected query keys. +- Full scans/rescans run only when triggered by settings actions. --- @@ -324,7 +312,7 @@ Allows the frontend to navigate the server's local filesystem so users can pick ``` ┌──────────────────────────────────────────────────────────────┐ -│ Test Dashboard [⟳ Refresh] [⚙ Settings] │ +│ Test Dashboard [⚙ Settings] │ ├──────────┬──────────┬──────────┬────────────┬───────────────┤ │ Overall │ CGW452 │ CGW453 │ Time │ Est. Remaining │ │ 60% │ 70% │ 50% │ Elapsed │ │ @@ -341,7 +329,7 @@ Allows the frontend to navigate the server's local filesystem so users can pick └──────────────────────────────────────────────────────────────┘ ``` -**Inline avg-time input**: When a type has remaining tests but zero completed, a warning banner appears with inline input fields for the missing avg time(s). Submitting saves to `POST /api/config` and immediately recalculates the estimate. +**Auth gate**: App shows a sign-in screen first. Dashboard renders only when authenticated. Settings button appears only for admin users. ### 8.2 Component Breakdown @@ -350,29 +338,24 @@ Allows the frontend to navigate the server's local filesystem so users can pick | `StatCard` | Displays a metric (label, value, sub-value); used for completion rate + counts | | `CompletionBar` | Progress bar with percentage label | | `TimeDisplay` | Formats seconds into `Xh Ym`; shows elapsed and estimated remaining | -| `AvgTimeWarning` | Banner shown when a type has remaining tests but no avg time; contains inline inputs | | `FilterPanel` | Dropdown filters for each tag including Interference; maintains filter state | -| `TestTable` | Virtualized (react-window) table of tests; columns sortable; File ID as primary ID column | -| `ConfigModal` | Form for directory paths (with picker button) + manual avg time overrides per type | -| `DirectoryBrowser` | Inline folder picker inside `ConfigModal`; navigates the server filesystem via `/api/browse` | +| `TestTable` | Filtered table of tests with status and parsed tags | +| `ConfigModal` | Admin settings modal for directories, avg overrides, and SMB credentials | | `StatusBadge` | Green checkmark or grey circle for completed/pending | ### 8.3 Settings / Config A gear icon opens `ConfigModal` with two sections: -1. **Directories** — target and results directory paths. Each path field has a **Browse** button that opens the `DirectoryBrowser`: - - Starts at the filesystem root (lists available drives on Windows) - - Displays current path as a clickable breadcrumb - - Lists subdirectories; clicking one navigates into it - - **Select This Folder** button confirms the selection and populates the path field - - **Cancel** closes the browser without changing the field - - On save, triggers a full re-scan and re-watch +1. **Directories** — target/results paths plus optional reference results path and exclusion rules. + - On Save All, backend updates config and triggers full re-scan when directory/exclusion fields changed. 2. **Avg Time Overrides** — three number inputs (COE, P2P, P3P) in minutes. Each shows: - - The calculated average from completed tests (if any), labelled `Auto: 7m 30s` - - A manual override field; when filled, it takes precedence over the calculated value - - Clear button to remove the override and revert to calculated - - If no completed tests exist for a type, the field is highlighted with a required indicator + - Manual override values converted to seconds and saved in config. +3. **SMB Credentials** — domain, username, password for network share access. +4. **Action buttons**: + - Save All + - Save Times + - Save Results (`/api/config/rescan-results`) All config values are POSTed to `POST /api/config` as key/value pairs and persisted in SQLite. @@ -398,26 +381,23 @@ In production, Flask serves the built React `dist/` as static files via `send_fr ## 10. Implementation Status ### Completed -- [x] Python Flask backend (`app.py`) with all API routes +- [x] Python Flask backend (`app.py`) with JWT auth and role guards - [x] SQLite schema, WAL mode, thread-safe helpers (`db_py.py`) -- [x] Tag parsing from target filenames and parent folder names (`parser.py`) -- [x] Full directory scan on startup / config change (`scanner.py`) -- [x] Watchdog filesystem watchers for both target and results dirs (`watcher.py`) - - [x] Windows `is_directory` timing bug fixed (check path directly) - - [x] Recycle Bin delete handled via `on_moved` -- [x] SSE broadcast with correct `\n\n` terminators (`sse_py.py`) -- [x] All frontend components and hooks -- [x] Config modal: directories always editable (lock removed) +- [x] Users table + user management APIs (admin-only) +- [x] Default user seeding from environment (`DEFAULT_ADMIN_*`, `DEFAULT_VIEWER_*`) +- [x] Tag parsing and scan logic (`parser.py`, `scanner.py`) +- [x] Full scan / rescan / results-only rescan flows +- [x] Frontend auth flow (login screen, JWT persistence, role-based UI) +- [x] Config modal (directories, avg overrides, SMB credentials) - [x] Vite dev proxy + Flask static serving for production - [x] Config migrated from `config.json` → SQLite on first run ### Remaining / Future -- [ ] SSO / authentication (noted in design plan) +- [ ] Admin UI for user create/list (currently API-only) +- [ ] Invalid test results +- [ ] Watchdog automatic updates - [ ] Test detail view (click a row to see tput/RSSI breakdown) -- [ ] Production deployment docs (systemd / Task Scheduler service) -2. Install `@tanstack/react-query`, `axios`, `tailwindcss`, `lucide-react` -3. Build `api.js` — base fetch helpers + SSE subscription hook -4. Build `useTests` and `useStats` hooks +- [ ] Production deployment docs (service startup, backup, secrets handling) 5. Build `StatCard`, `CompletionBar`, `TimeDisplay`, `StatusBadge` 6. Build `App.jsx` layout with stats row diff --git a/dashboard/src/components/ConfigModal.jsx b/dashboard/src/components/ConfigModal.jsx index 53ffb50..3c9419f 100644 --- a/dashboard/src/components/ConfigModal.jsx +++ b/dashboard/src/components/ConfigModal.jsx @@ -160,7 +160,7 @@ export default function ConfigModal({ onClose }) { {[ { key: 'target_dir', label: 'Target Tests Directory' }, { key: 'results_dir', label: 'Results Directory (DUT)' }, - { key: 'results_dir_ref', label: 'Results Directory (Reference)' }, + { key: 'results_dir_ref', label: 'Results Directory (REF)' }, ].map(({ key, label }) => (