Mia.Wu/test_dashboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

minor ui change

Browse Source
This commit is contained in:
Mia Wu
2026-06-02 11:59:31 -04:00
parent a1b120076b
commit 50cd3ecc6c
4 changed files with 81 additions and 231 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
-144
View File
@@ -1,144 +0,0 @@
# CLAUDE.md — Project Context for AI Assistants
## What this project is
A real-time web dashboard that tracks WiFi test execution progress. It compares a **target tests directory** (`.ini` files defining every test that must run) against a **results directory** (folders created when each test completes). The dashboard shows completion rates, elapsed time, estimated time remaining, and a filterable table of every test.
---
## How to run
### Backend (Python)
```powershell
cd server
.\.venv\Scripts\Activate.ps1 # activate venv
python app.py # starts Flask on port 3001
```
### Frontend (dev)
```powershell
cd dashboard
npm run dev # Vite dev server; proxies /api → localhost:3001
```
### Frontend (production build)
```powershell
cd dashboard
npm run build # outputs to dashboard/dist/
# Flask serves dist/ automatically when app.py is running
```
---
## Tech stack
| Layer | Technology |
|---|---|
| Frontend | React 18 + Vite, TailwindCSS v4 (`@tailwindcss/vite`), TanStack React Query |
| Backend | Python 3, Flask, flask-cors |
| Database | SQLite via Python stdlib `sqlite3` (WAL mode, single persistent connection + RLock) |
| File watching | `watchdog` Python library |
| Real-time | Server-Sent Events (SSE) — one-way push from backend to browser |
---
## Key files
### Backend (`server/`)
| File | Purpose |
|---|---|
| `app.py` | Flask app, all route handlers, `bootstrap()`, `__main__` |
| `db_py.py` | SQLite schema init, all query helpers (`upsert_test`, `mark_completed`, `reset_by_file_id_and_device`, `get_config`, `set_config`, etc.) |
| `scanner.py` | `full_scan()`, `scan_targets()`, `scan_results()`, `process_result_dir()`, `parse_deleted_result_dir_name()` |
| `parser.py` | `parse_target_filename()`, `parse_result_filename()`, `parse_timestamp()`, `parse_elapsed_time()`, `parse_tput_rssi()` |
| `watcher.py` | Two watchdog `Observer` instances — `_TargetHandler` and `_ResultsHandler`. `start_watching()` / `stop_watching()` |
| `sse_py.py` | Per-client `Queue` registry, `stream_events()` generator, `broadcast(data)` |
### Frontend (`dashboard/src/`)
| File | Purpose |
|---|---|
| `lib/api.js` | Thin `fetch` wrapper; all API calls go through `apiFetch('/path')` |
| `hooks/useStats.js` | Fetches `/api/stats`; **owns the SSE `EventSource`**; invalidates `['stats']` and `['tests']` on `update` events |
| `hooks/useTests.js` | Fetches `/api/tests` |
| `hooks/useConfig.js` | Fetches/saves `/api/config` |
| `components/ConfigModal.jsx` | Settings modal: directory paths (always editable) + avg time overrides |
| `components/DirectoryBrowser.jsx` | Server-side folder picker using `/api/browse` |
---
## Data model
### How tests are identified
- **Target files** live in subdirectories of the target directory. Format: `TC_WIFI_<tags>.ini`. Files named `GLOBAL.ini` are skipped.
- **`test_id`**: segment matching `R\d+[A-Z0-9]+` (e.g., `R2COERXAX014`)
- **Result directories**: top-level subdirectories of the results directory. A test is **completed** when a result directory name contains the same `test_id`.
- **Result folder name example**: `COE_CGW453_R2COERXAX014_TPT3E_RSSI70_STA56_2GHZ_CH1_BW20_TCP_MIMOFD_SONFD_MESHFD_LPI_UL`
### Tags parsed from target filename
`interference`, `device`, `test_point`, `rssi`, `station`, `band`, `channel`, `bandwidth`, `direction`
`rotation` is parsed from the **parent folder name** (segment matching `ROT\d+`).
### Database tables
**`tests`** — one row per target `.ini` file:
- `id` (TEXT PK) — `parent_dir/base_filename`
- `test_id`, `parent_dir`, `filename`, `interference`, `device`, `rotation`, `test_point`, `station`, `band`, `channel`, `bandwidth`, `rssi`, `direction`
- `completed` (0/1), `completed_at` (ISO timestamp), `duration_seconds` (REAL)
- `tput_results` (JSON array of `{station, tput, dlRssi, ulRssi}`)
**`config`** — key/value store:
- `target_dir`, `results_dir`
- `avg_time_coe`, `avg_time_p2p`, `avg_time_p3p` (seconds as string; NULL = use calculated average)
---
## API routes
| Method | Path | Description |
|---|---|---|
| GET | `/api/tests` | All tests; supports query filters (`completed`, `interference`, `device`, `rotation`, `testPoint`, `station`, `band`, `channel`, `bandwidth`, `rssi`, `direction`) |
| GET | `/api/stats` | Aggregated stats: overall, per-device, timing with estimates |
| GET | `/api/events` | SSE stream; sends `data: {"type":"update"}\n\n` on any directory change |
| GET | `/api/config` | Current config values |
| POST | `/api/config` | Update config; triggers full rescan + rewatcher if dirs changed |
| POST | `/api/config/rescan` | Force full rescan without changing config |
| GET | `/api/browse?path=...` | List subdirectories at a server path (omit `path` for drive roots) |
---
## Watcher architecture
Two `watchdog.Observer` instances run in daemon threads:
1. **`_TargetHandler`** — watches `target_dir` recursively. Any add/delete/move of a `TC_WIFI_*.ini` file schedules a **debounced full scan** (1 second timer, cancels and restarts on rapid changes).
2. **`_ResultsHandler`** — watches `results_dir` recursively. Handles events surgically:
- Directory created → `process_result_dir()` + broadcast
- Directory deleted → `reset_by_file_id_and_device()` + broadcast
- Directory moved in/out/renamed → appropriate reset/process + broadcast
- File created/deleted inside a result dir → `process_result_dir()` + broadcast
### Critical Windows quirk
When a directory is deleted (including Recycle Bin), `watchdog` calls `os.path.isdir()` at event-processing time. The directory is already gone, so it returns `False`, making `event.is_directory = False` even for directory events. **All handlers check `_is_direct_child_dir(path)` directly** — never gate on `event.is_directory` for delete/move cases.
---
## SSE implementation notes
- `sse_py.py`: each connected client gets a `queue.Queue`. `broadcast()` calls `put_nowait(payload)` on all queues. `stream_events()` is a generator yielding `f"data: {payload}\n\n"` (real newlines — `\n` not `\\n`).
- `useStats.js`: creates `new EventSource('/api/events')` once on mount. On `{"type":"update"}` message, invalidates React Query keys `['stats']` and `['tests']`.
- Vite dev proxy forwards `/api/events` to Flask, keeping the SSE connection alive.
---
## Known issues / gotchas
- `config.json` in `server/` is a legacy file used for a one-time migration to SQLite on first run. It is no longer needed once `dashboard.db` exists.
- The backend entrypoint is `app.py`, not `main.py`.
- Flask dev server (`app.run`) is used directly — no gunicorn/waitress configured yet.
- Port is `3001` (configurable via `PORT` env var).
DESIGNPLAN.md
+19 -5
View File
@@ -1,28 +1,38 @@
## Design Plan
**Inputs**: target tests directory, result tests directory
**Inputs**: target tests directory, reference result tests directory, DUT result tests directory
**Output**: a dashboard that displays completion rate, elapsed time, estimated time remaining, and a list of all tests that can be filtered by tags
**Requirements**:
- A test is marked completed when there is a filename match in result tests directory
- When there is a change in the input directories, data should update automatically or when refreshed
- Dashboard can be accessed by different devices, but input directories are stored locally on one machine
- Add SSO later
- Login required in order to view dashboard
- Viewer
- Only able to view
- Admin
- Able to access and modify settings
- Can enter exclusions for tests that are not ran in settings
Front end UI requirements:
- Completion rate
- Completion rate and number of tests completed
- Overall completion rate
- CGW 452 completion rate
- number of tests completed for each type
- CGW453 completion rate
- number of tests completed for each type
- Time elapsed
- Estimated Time remaining
- Estimated date to complete
- 16 hrs on weekdays
- 24 hrs on weekends
- List of all tests
- Can filter by Tags:
- Completed: Boolean
- Type
- Device: String
- Rotation: String
- Rotation: String
- Test Point: String
- Station: String
- Band: String
@@ -30,3 +40,7 @@ Front end UI requirements:
- Bandwidth: String
- RSSI: String
- Direction: String
- A P2P test with COE pairings should be displayed together
- P3P tests should be paired together by throttled/unthrottled
- Completed tests
- Display the avg throughput for each station
IMPLEMENTATION_PLAN.md
+61 -81
View File
@@ -1,6 +1,6 @@
# Implementation Plan — Test Dashboard
> **Status as of May 2026**: fully implemented and running. Backend migrated from the original Node.js plan to Python (Flask). See current tech stack and structure below.
> **Status as of June 2026**: fully implemented and running. Backend is Python (Flask) with JWT authentication and role-based access control.
## 1. Overview
@@ -14,10 +14,9 @@ A web dashboard that monitors test execution progress by comparing a **target te
|---|---|---|
| Frontend | React + Vite | Dev server proxies `/api` → `localhost:3001` |
| Backend | Python 3 + Flask | Replaced original Node.js/Express plan |
| Real-time | Server-Sent Events (SSE) | One-way push from backend to browser |
| Directory watching | watchdog | Python filesystem watcher (Windows-compatible) |
| Auth | JWT (PyJWT) | Bearer token auth with `admin` and `viewer` roles |
| Database | SQLite (via `sqlite3` stdlib) | WAL mode; single persistent connection with RLock |
| Frontend state | TanStack React Query | Cache + SSE-driven invalidation |
| Frontend state | TanStack React Query | Query cache + manual invalidation after mutations |
| UI | TailwindCSS v4 | Via `@tailwindcss/vite` plugin |
---
@@ -35,7 +34,7 @@ test_house_dashboard/
│ ├── index.html
│ ├── package.json
│ └── src/
│ ├── App.jsx – root layout, SSE wiring via useStats
│ ├── App.jsx – root layout, auth gate, modal settings entry
│ ├── main.jsx
│ ├── components/
│ │ ├── StatCard.jsx – metric card (label / value / sub)
@@ -44,22 +43,20 @@ test_house_dashboard/
│ │ ├── FilterPanel.jsx – dropdown tag filters
│ │ ├── TimeDisplay.jsx – elapsed / estimated time display
│ │ ├── StatusBadge.jsx – completed / pending indicator
│ │ ├── ConfigModal.jsx – settings modal (dirs + avg time overrides)
│ │ └── DirectoryBrowser.jsx – server-side folder picker (uses /api/browse)
│ │ └── ConfigModal.jsx – settings modal (dirs + avg time overrides)
│ ├── hooks/
│ │ ├── useStats.js – fetches stats; owns the SSE EventSource
│ │ ├── useAuth.js – login session state + role checks
│ │ ├── useStats.js – fetches stats
│ │ ├── useTests.js – fetches test list
│ │ └── useConfig.js – fetches/saves config
│ └── lib/
│ └── api.js – thin fetch wrapper; BASE = '/api'
│ └── api.js – fetch wrapper with JWT header injection
│
└── server/ ← Python backend
├── app.py – Flask app, all routes, bootstrap(), __main__
├── app.py – Flask app, JWT auth, role guards, bootstrap(), __main__
├── db_py.py – SQLite schema, query helpers, config CRUD
├── scanner.py – full_scan(), scan_targets(), scan_results(), process_result_dir()
├── parser.py – parse_target_filename(), parse_result_filename(), etc.
├── watcher.py – watchdog observers for target + results dirs
├── sse_py.py – SSE queue registry + broadcast()
├── requirements.txt
├── dashboard.db – SQLite database (auto-created)
└── .venv/ – Python virtual environment
@@ -185,9 +182,20 @@ CREATE TABLE config (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
-- Stores dashboard users
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
role TEXT NOT NULL, -- admin | viewer
is_active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
-- Config keys:
-- target_dir – path to target tests directory
-- results_dir – path to results directory
-- results_dir_ref – optional reference results directory
-- avg_time_coe – manual avg seconds per COE test (NULL = use calculated)
-- avg_time_p2p – manual avg seconds per P2P test (NULL = use calculated)
-- avg_time_p3p – manual avg seconds per P3P test (NULL = use calculated)
@@ -201,12 +209,23 @@ CREATE TABLE config (
| Method | Path | Description |
|---|---|---|
| `POST` | `/api/auth/login` | Login with username/password; returns JWT + user role |
| `GET` | `/api/auth/me` | Resolve current authenticated user from JWT |
| `GET` | `/api/users` | Admin-only list of users |
| `POST` | `/api/users` | Admin-only create user (`admin` / `viewer`) |
| `GET` | `/api/tests` | All tests; supports query filters (see below) |
| `GET` | `/api/stats` | Aggregated stats object |
| `GET` | `/api/events` | SSE stream — pushes `update` events on directory change |
| `GET` | `/api/scan-status` | Returns active scan state |
| `GET` | `/api/config` | Current directory paths and avg time overrides |
| `POST` | `/api/config` | Set `targetDir`, `resultsDir`, avg time overrides; triggers re-scan |
| `GET` | `/api/browse` | List subdirectories at a given server path (directory picker) |
| `POST` | `/api/config` | Save config and optionally trigger full re-scan |
| `POST` | `/api/config/rescan` | Force full re-scan using saved config |
| `POST` | `/api/config/rescan-results` | Re-process only results directories |
Auth/role rules:
- All dashboard data routes require a valid JWT.
- `/api/config*` and `/api/users*` are admin-only.
- Frontend stores JWT in localStorage key `dashboard_jwt` and sends `Authorization: Bearer <token>`.
### `GET /api/tests` Query Parameters
@@ -277,44 +296,13 @@ CREATE TABLE config (
- `estimatedRemainingSeconds` = `(avgCOE × COE_remaining) + (avgP2P × P2P_remaining) + (avgP3P × P3P_remaining)`
- If any type has remaining tests but no avg time (calculated or manual), that type contributes `null` to the sum and the UI flags it as **input required**
### `GET /api/browse` — Directory Browser
Allows the frontend to navigate the server's local filesystem so users can pick directories without typing paths manually.
**Query parameters**:
- `path` (optional) — absolute path to list. If omitted or empty, returns the filesystem roots (e.g., `C:\`, `D:\` on Windows).
**Response**:
```json
{
"path": "C:\\Tests",
"parent": "C:\\",
"dirs": [
{ "name": "Target", "path": "C:\\Tests\\Target" },
{ "name": "Results", "path": "C:\\Tests\\Results" }
]
}
```
- Only **directories** are returned (no files).
- Hidden directories (names starting with `.`) are excluded.
- If `path` does not exist or is not a directory, returns `400`.
- `parent` is `null` when already at a filesystem root.
---
## 7. Real-time Updates
- Backend registers an SSE endpoint at `GET /api/events` (`sse_py.py`)
- `watchdog` watches both target and results directories via two separate `Observer` instances (`watcher.py`)
- On any change, the backend updates SQLite and calls `broadcast({"type": "update"})`, which pushes `data: {"type":"update"}\n\n` to all connected SSE clients
- `useStats.js` owns the `EventSource('/api/events')` connection; on `update` it calls `queryClient.invalidateQueries` for both `['stats']` and `['tests']`
Current implementation does not use SSE/watchdog.
**Windows watchdog quirk — `is_directory` unreliable on delete**: when a directory is deleted, watchdog calls `os.path.isdir()` to set `event.is_directory`, but the directory is already gone by then, so it returns `False`. All delete/move handlers therefore check `_is_direct_child_dir(path)` directly instead of relying on `event.is_directory`.
**Result directory deleted**: `reset_by_file_id_and_device(test_id, device)` resets `completed = 0`, `completed_at = NULL`, `duration_seconds = NULL`. broadcast fires unconditionally regardless of whether the folder name was parseable.
**Recycle Bin delete on Windows**: fires a `MovedEvent` (`src` = result dir, `dest` = `$RECYCLE.BIN\...`). Handled by `on_moved` via `_is_direct_child_dir` on the source path.
- Stats/tests/config are fetched via React Query.
- Mutations (`saveConfig`, `Save Times`, `Save Results`) manually invalidate affected query keys.
- Full scans/rescans run only when triggered by settings actions.
---
@@ -324,7 +312,7 @@ Allows the frontend to navigate the server's local filesystem so users can pick
```
┌──────────────────────────────────────────────────────────────┐
│ Test Dashboard [⟳ Refresh] [⚙ Settings] │
│ Test Dashboard [⚙ Settings] │
├──────────┬──────────┬──────────┬────────────┬───────────────┤
│ Overall │ CGW452 │ CGW453 │ Time │ Est. Remaining │
│ 60% │ 70% │ 50% │ Elapsed │ │
@@ -341,7 +329,7 @@ Allows the frontend to navigate the server's local filesystem so users can pick
└──────────────────────────────────────────────────────────────┘
```
**Inline avg-time input**: When a type has remaining tests but zero completed, a warning banner appears with inline input fields for the missing avg time(s). Submitting saves to `POST /api/config` and immediately recalculates the estimate.
**Auth gate**: App shows a sign-in screen first. Dashboard renders only when authenticated. Settings button appears only for admin users.
### 8.2 Component Breakdown
@@ -350,29 +338,24 @@ Allows the frontend to navigate the server's local filesystem so users can pick
| `StatCard` | Displays a metric (label, value, sub-value); used for completion rate + counts |
| `CompletionBar` | Progress bar with percentage label |
| `TimeDisplay` | Formats seconds into `Xh Ym`; shows elapsed and estimated remaining |
| `AvgTimeWarning` | Banner shown when a type has remaining tests but no avg time; contains inline inputs |
| `FilterPanel` | Dropdown filters for each tag including Interference; maintains filter state |
| `TestTable` | Virtualized (react-window) table of tests; columns sortable; File ID as primary ID column |
| `ConfigModal` | Form for directory paths (with picker button) + manual avg time overrides per type |
| `DirectoryBrowser` | Inline folder picker inside `ConfigModal`; navigates the server filesystem via `/api/browse` |
| `TestTable` | Filtered table of tests with status and parsed tags |
| `ConfigModal` | Admin settings modal for directories, avg overrides, and SMB credentials |
| `StatusBadge` | Green checkmark or grey circle for completed/pending |
### 8.3 Settings / Config
A gear icon opens `ConfigModal` with two sections:
1. **Directories** — target and results directory paths. Each path field has a **Browse** button that opens the `DirectoryBrowser`:
- Starts at the filesystem root (lists available drives on Windows)
- Displays current path as a clickable breadcrumb
- Lists subdirectories; clicking one navigates into it
- **Select This Folder** button confirms the selection and populates the path field
- **Cancel** closes the browser without changing the field
- On save, triggers a full re-scan and re-watch
1. **Directories** — target/results paths plus optional reference results path and exclusion rules.
- On Save All, backend updates config and triggers full re-scan when directory/exclusion fields changed.
2. **Avg Time Overrides** — three number inputs (COE, P2P, P3P) in minutes. Each shows:
- The calculated average from completed tests (if any), labelled `Auto: 7m 30s`
- A manual override field; when filled, it takes precedence over the calculated value
- Clear button to remove the override and revert to calculated
- If no completed tests exist for a type, the field is highlighted with a required indicator
- Manual override values converted to seconds and saved in config.
3. **SMB Credentials** — domain, username, password for network share access.
4. **Action buttons**:
- Save All
- Save Times
- Save Results (`/api/config/rescan-results`)
All config values are POSTed to `POST /api/config` as key/value pairs and persisted in SQLite.
@@ -398,26 +381,23 @@ In production, Flask serves the built React `dist/` as static files via `send_fr
## 10. Implementation Status
### Completed
- [x] Python Flask backend (`app.py`) with all API routes
- [x] Python Flask backend (`app.py`) with JWT auth and role guards
- [x] SQLite schema, WAL mode, thread-safe helpers (`db_py.py`)
- [x] Tag parsing from target filenames and parent folder names (`parser.py`)
- [x] Full directory scan on startup / config change (`scanner.py`)
- [x] Watchdog filesystem watchers for both target and results dirs (`watcher.py`)
- [x] Windows `is_directory` timing bug fixed (check path directly)
- [x] Recycle Bin delete handled via `on_moved`
- [x] SSE broadcast with correct `\n\n` terminators (`sse_py.py`)
- [x] All frontend components and hooks
- [x] Config modal: directories always editable (lock removed)
- [x] Users table + user management APIs (admin-only)
- [x] Default user seeding from environment (`DEFAULT_ADMIN_*`, `DEFAULT_VIEWER_*`)
- [x] Tag parsing and scan logic (`parser.py`, `scanner.py`)
- [x] Full scan / rescan / results-only rescan flows
- [x] Frontend auth flow (login screen, JWT persistence, role-based UI)
- [x] Config modal (directories, avg overrides, SMB credentials)
- [x] Vite dev proxy + Flask static serving for production
- [x] Config migrated from `config.json` → SQLite on first run
### Remaining / Future
- [ ] SSO / authentication (noted in design plan)
- [ ] Admin UI for user create/list (currently API-only)
- [ ] Invalid test results
- [ ] Watchdog automatic updates
- [ ] Test detail view (click a row to see tput/RSSI breakdown)
- [ ] Production deployment docs (systemd / Task Scheduler service)
2. Install `@tanstack/react-query`, `axios`, `tailwindcss`, `lucide-react`
3. Build `api.js` — base fetch helpers + SSE subscription hook
4. Build `useTests` and `useStats` hooks
- [ ] Production deployment docs (service startup, backup, secrets handling)
5. Build `StatCard`, `CompletionBar`, `TimeDisplay`, `StatusBadge`
6. Build `App.jsx` layout with stats row
dashboard/src/components/ConfigModal.jsx
+1 -1
View File
@@ -160,7 +160,7 @@ export default function ConfigModal({ onClose }) {
{[
{ key: 'target_dir', label: 'Target Tests Directory' },
{ key: 'results_dir', label: 'Results Directory (DUT)' },
{ key: 'results_dir_ref', label: 'Results Directory (Reference)' },
{ key: 'results_dir_ref', label: 'Results Directory (REF)' },
].map(({ key, label }) => (
<div key={key}>
<label className="text-slate-400 text-xs block mb-1">{label}</label>