483 lines
21 KiB
Markdown
483 lines
21 KiB
Markdown
# Implementation Plan — Test Dashboard
|
||
|
||
> **Status as of May 2026**: fully implemented and running. Backend migrated from the original Node.js plan to Python (Flask). See current tech stack and structure below.
|
||
|
||
## 1. Overview
|
||
|
||
A web dashboard that monitors test execution progress by comparing a **target tests directory** (what should run) against a **results directory** (what has run). The backend runs on one machine with access to both directories; the frontend is accessible from any device on the network.
|
||
|
||
---
|
||
|
||
## 2. Tech Stack
|
||
|
||
| Layer | Technology | Notes |
|
||
|---|---|---|
|
||
| Frontend | React + Vite | Dev server proxies `/api` → `localhost:3001` |
|
||
| Backend | Python 3 + Flask | Replaced original Node.js/Express plan |
|
||
| Real-time | Server-Sent Events (SSE) | One-way push from backend to browser |
|
||
| Directory watching | watchdog | Python filesystem watcher (Windows-compatible) |
|
||
| Database | SQLite (via `sqlite3` stdlib) | WAL mode; single persistent connection with RLock |
|
||
| Frontend state | TanStack React Query | Cache + SSE-driven invalidation |
|
||
| UI | TailwindCSS v4 | Via `@tailwindcss/vite` plugin |
|
||
|
||
---
|
||
|
||
## 3. Project Structure
|
||
|
||
```
|
||
test_house_dashboard/
|
||
├── DESIGNPLAN.md
|
||
├── IMPLEMENTATION_PLAN.md
|
||
├── CLAUDE.md ← project context for AI assistants
|
||
│
|
||
├── dashboard/ ← React frontend
|
||
│ ├── vite.config.js – proxy /api → localhost:3001
|
||
│ ├── index.html
|
||
│ ├── package.json
|
||
│ └── src/
|
||
│ ├── App.jsx – root layout, SSE wiring via useStats
|
||
│ ├── main.jsx
|
||
│ ├── components/
|
||
│ │ ├── StatCard.jsx – metric card (label / value / sub)
|
||
│ │ ├── CompletionBar.jsx – progress bar
|
||
│ │ ├── TestTable.jsx – filterable test list table
|
||
│ │ ├── FilterPanel.jsx – dropdown tag filters
|
||
│ │ ├── TimeDisplay.jsx – elapsed / estimated time display
|
||
│ │ ├── StatusBadge.jsx – completed / pending indicator
|
||
│ │ ├── ConfigModal.jsx – settings modal (dirs + avg time overrides)
|
||
│ │ └── DirectoryBrowser.jsx – server-side folder picker (uses /api/browse)
|
||
│ ├── hooks/
|
||
│ │ ├── useStats.js – fetches stats; owns the SSE EventSource
|
||
│ │ ├── useTests.js – fetches test list
|
||
│ │ └── useConfig.js – fetches/saves config
|
||
│ └── lib/
|
||
│ └── api.js – thin fetch wrapper; BASE = '/api'
|
||
│
|
||
└── server/ ← Python backend
|
||
├── app.py – Flask app, all routes, bootstrap(), __main__
|
||
├── db_py.py – SQLite schema, query helpers, config CRUD
|
||
├── scanner.py – full_scan(), scan_targets(), scan_results(), process_result_dir()
|
||
├── parser.py – parse_target_filename(), parse_result_filename(), etc.
|
||
├── watcher.py – watchdog observers for target + results dirs
|
||
├── sse_py.py – SSE queue registry + broadcast()
|
||
├── requirements.txt
|
||
├── dashboard.db – SQLite database (auto-created)
|
||
└── .venv/ – Python virtual environment
|
||
```
|
||
|
||
---
|
||
|
||
## 4. File ID & Tag Parsing
|
||
|
||
### 4.1 File ID & Completion Matching
|
||
|
||
A **test is completed** when its **file ID** appears in any result directory name.
|
||
|
||
**File ID** = the test case code segment extracted from the filename — the segment matching the pattern `R\d+[A-Z0-9]+` (e.g., `R2COERXAC003`, `R5P2PRXAX012`).
|
||
|
||
**Target files** live inside subdirectories of the target directory. All target files use the prefix `TC_WIFI_` and extension `.ini`. Each parent directory also contains a `GLOBAL.ini` file that must be **skipped** by the scanner:
|
||
|
||
```
|
||
TP_WIFI_ATT_CDR_GRP1_ROT1_CGW452_WNC_REGRESSION/
|
||
GLOBAL.ini ← skip
|
||
TC_WIFI_COE_CGW452_R2COERXAC003_TPT3E_RSSI70_STA4_2GHZ_CH1_BW20_TCP_MIMOFD_SONFD_MESHFD_LPI_UL.ini
|
||
↑
|
||
test_id = R2COERXAC003
|
||
```
|
||
|
||
Scanner filter: include only files where `filename.startsWith('TC_WIFI_') && filename.endsWith('.ini')`.
|
||
|
||
**Result directories** live as top-level subdirectories of the results directory. The directory name contains the same file ID:
|
||
|
||
```
|
||
COE_CGW452_R2COERXAC003_TPT3E_RSSI70_STA4_2GHZ_CH1_BW20_TCP_MIMOFD_SONFD_MESHFD_LPI_UL/
|
||
↑
|
||
test_id = R2COERXAC003
|
||
COE_CGW452_R2COERXAC003_..._2026-05-16-07-30-14 ← target log file (contains test_id in name)
|
||
other_file.txt ← ignored
|
||
...other files
|
||
```
|
||
|
||
A test is **completed** when a result directory name contains the target's `test_id`.
|
||
|
||
**Duration per test**: the result directory may contain multiple `.txt` files. Only the `.txt` file(s) whose name includes the `test_id` are scanned. All other `.txt` files are ignored.
|
||
|
||
Filter: `filename.endsWith('.txt') && filename.includes(test_id)`
|
||
|
||
Scan the matched file for the line:
|
||
|
||
```
|
||
[2026-05-16 09:21:02,105 INFO] Elapsed time : 1:51:00.660866
|
||
```
|
||
|
||
Regex: `/\[.*?INFO\]\s+Elapsed time\s*:\s*([\d]+:[\d]{2}:[\d]{2}\.[\d]+)/`
|
||
|
||
The captured group (`1:51:00.660866`) is parsed as `H:MM:SS.microseconds` and converted to total seconds stored in `duration_seconds`. If no matching line is found, `duration_seconds` is left `NULL` and excluded from the elapsed sum and avg calculations.
|
||
|
||
**Re-runs**: if multiple `.txt` files match the `test_id` filter (re-run logs), use the one with the **latest** timestamp in its filename for both `completed_at` and `duration_seconds`.
|
||
|
||
### 4.2 Tag Parsing
|
||
|
||
Tags are parsed from two sources:
|
||
|
||
**From the parent folder name** (e.g., `TP_WIFI_ATT_CDR_GRP1_ROT1_CGW452_WNC_REGRESSION`):
|
||
| Tag | Pattern | Example |
|
||
|---|---|---|
|
||
| Rotation | `ROT\d+` | `ROT1` |
|
||
| Device | `CGW\d+` | `CGW452` |
|
||
|
||
**From the target filename** (after stripping `TC_WIFI_` prefix and `.ini` extension):
|
||
```
|
||
COE_CGW452_R2COERXAC003_TPT3E_RSSI70_STA4_2GHZ_CH1_BW20_..._UL
|
||
↑
|
||
first segment = interference type
|
||
```
|
||
|
||
| Tag | Extraction | Example |
|
||
|---|---|---|
|
||
| **Interference** | First segment — fixed enum: `COE`, `P2P`, `P3P` | `COE` |
|
||
| Device | Regex `CGW\d+` | `CGW452` |
|
||
| File ID | Regex `R\d+[A-Z0-9]+` | `R2COERXAC003` |
|
||
| Test Point | Regex `TPT\w+` | `TPT3E` |
|
||
| RSSI | Regex `RSSI\d+` | `RSSI70` |
|
||
| Station | Regex `STA\d+` | `STA4` |
|
||
| Band | Regex `\dGHZ` | `2GHZ` |
|
||
| Channel | Regex `CH\d+` | `CH1` |
|
||
| Bandwidth | Regex `BW\d+` | `BW20` |
|
||
| Direction | Last segment — fixed enum: `UL`, `DL`, `BI` | `UL` |
|
||
|
||
**Completion Timestamp**: extracted from the timestamped result file name inside the matching result directory (e.g., `..._2026-05-16-07-30-14` → `2026-05-16 07:30:14`).
|
||
|
||
**Interference type** is also validated against the result directory name's first segment (`COE_...`, `P2P_...`, `P3P_...`) as a cross-check.
|
||
|
||
---
|
||
|
||
## 5. Database Schema
|
||
|
||
```sql
|
||
-- Stores one row per target test file
|
||
CREATE TABLE tests (
|
||
id TEXT PRIMARY KEY, -- full derived ID (filename without prefix/ext)
|
||
test_id TEXT NOT NULL, -- short test case code, e.g. R2COERXAC003
|
||
parent_dir TEXT NOT NULL, -- parent folder name (TP_WIFI_...)
|
||
filename TEXT NOT NULL, -- original .ini filename
|
||
completed INTEGER NOT NULL DEFAULT 0, -- 0 or 1
|
||
completed_at TEXT, -- ISO timestamp from result file name
|
||
duration_seconds REAL, -- per-test duration in seconds (from result file or fs timestamps)
|
||
|
||
-- parsed tags
|
||
interference TEXT, -- COE | P2P | P3P
|
||
device TEXT, -- CGW452 | CGW453
|
||
rotation TEXT,
|
||
test_point TEXT,
|
||
station TEXT,
|
||
band TEXT,
|
||
channel TEXT,
|
||
bandwidth TEXT,
|
||
rssi TEXT,
|
||
direction TEXT,
|
||
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||
);
|
||
|
||
-- Stores configuration: directory paths AND manual avg time overrides
|
||
CREATE TABLE config (
|
||
key TEXT PRIMARY KEY,
|
||
value TEXT NOT NULL
|
||
);
|
||
-- Config keys:
|
||
-- target_dir – path to target tests directory
|
||
-- results_dir – path to results directory
|
||
-- avg_time_coe – manual avg seconds per COE test (NULL = use calculated)
|
||
-- avg_time_p2p – manual avg seconds per P2P test (NULL = use calculated)
|
||
-- avg_time_p3p – manual avg seconds per P3P test (NULL = use calculated)
|
||
```
|
||
|
||
---
|
||
|
||
## 6. Backend API
|
||
|
||
### Endpoints
|
||
|
||
| Method | Path | Description |
|
||
|---|---|---|
|
||
| `GET` | `/api/tests` | All tests; supports query filters (see below) |
|
||
| `GET` | `/api/stats` | Aggregated stats object |
|
||
| `GET` | `/api/events` | SSE stream — pushes `update` events on directory change |
|
||
| `GET` | `/api/config` | Current directory paths and avg time overrides |
|
||
| `POST` | `/api/config` | Set `targetDir`, `resultsDir`, avg time overrides; triggers re-scan |
|
||
| `GET` | `/api/browse` | List subdirectories at a given server path (directory picker) |
|
||
|
||
### `GET /api/tests` Query Parameters
|
||
|
||
```
|
||
?completed=true|false
|
||
&interference=COE|P2P|P3P
|
||
&device=CGW452
|
||
&rotation=ROT1
|
||
&testPoint=TPT3E
|
||
&station=STA4
|
||
&band=2GHZ
|
||
&channel=CH1
|
||
&bandwidth=BW20
|
||
&rssi=RSSI70
|
||
&direction=UL
|
||
```
|
||
|
||
### `GET /api/stats` Response Shape
|
||
|
||
```json
|
||
{
|
||
"overall": {
|
||
"total": 200,
|
||
"completed": 120,
|
||
"completionRate": 0.60
|
||
},
|
||
"cgw452": {
|
||
"total": 100,
|
||
"completed": 70,
|
||
"completionRate": 0.70
|
||
},
|
||
"cgw453": {
|
||
"total": 100,
|
||
"completed": 50,
|
||
"completionRate": 0.50
|
||
},
|
||
"timing": {
|
||
"elapsedSeconds": 172800,
|
||
"estimatedRemainingSeconds": 115200,
|
||
"byType": {
|
||
"COE": {
|
||
"avgSeconds": 450,
|
||
"avgSource": "calculated",
|
||
"remaining": 80
|
||
},
|
||
"P2P": {
|
||
"avgSeconds": 300,
|
||
"avgSource": "manual",
|
||
"remaining": 50
|
||
},
|
||
"P3P": {
|
||
"avgSeconds": 600,
|
||
"avgSource": "calculated",
|
||
"remaining": 30
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**Timing logic**:
|
||
|
||
- `elapsedSeconds` = `SUM(duration_seconds)` for all completed tests (sum of individual test durations, not wall-clock)
|
||
- For each interference type (COE, P2P, P3P):
|
||
- `avgSeconds` = `AVG(duration_seconds) WHERE interference = type AND completed = 1`
|
||
- If no completed tests exist for that type, `avgSeconds` = value from `config` table (`avg_time_coe` / `avg_time_p2p` / `avg_time_p3p`); `avgSource` = `"manual"`
|
||
- If a manual override is set in config even when calculated data exists, the manual value takes precedence; `avgSource` = `"manual_override"`
|
||
- `estimatedRemainingSeconds` = `(avgCOE × COE_remaining) + (avgP2P × P2P_remaining) + (avgP3P × P3P_remaining)`
|
||
- If any type has remaining tests but no avg time (calculated or manual), that type contributes `null` to the sum and the UI flags it as **input required**
|
||
|
||
### `GET /api/browse` — Directory Browser
|
||
|
||
Allows the frontend to navigate the server's local filesystem so users can pick directories without typing paths manually.
|
||
|
||
**Query parameters**:
|
||
- `path` (optional) — absolute path to list. If omitted or empty, returns the filesystem roots (e.g., `C:\`, `D:\` on Windows).
|
||
|
||
**Response**:
|
||
```json
|
||
{
|
||
"path": "C:\\Tests",
|
||
"parent": "C:\\",
|
||
"dirs": [
|
||
{ "name": "Target", "path": "C:\\Tests\\Target" },
|
||
{ "name": "Results", "path": "C:\\Tests\\Results" }
|
||
]
|
||
}
|
||
```
|
||
|
||
- Only **directories** are returned (no files).
|
||
- Hidden directories (names starting with `.`) are excluded.
|
||
- If `path` does not exist or is not a directory, returns `400`.
|
||
- `parent` is `null` when already at a filesystem root.
|
||
|
||
---
|
||
|
||
## 7. Real-time Updates
|
||
|
||
- Backend registers an SSE endpoint at `GET /api/events` (`sse_py.py`)
|
||
- `watchdog` watches both target and results directories via two separate `Observer` instances (`watcher.py`)
|
||
- On any change, the backend updates SQLite and calls `broadcast({"type": "update"})`, which pushes `data: {"type":"update"}\n\n` to all connected SSE clients
|
||
- `useStats.js` owns the `EventSource('/api/events')` connection; on `update` it calls `queryClient.invalidateQueries` for both `['stats']` and `['tests']`
|
||
|
||
**Windows watchdog quirk — `is_directory` unreliable on delete**: when a directory is deleted, watchdog calls `os.path.isdir()` to set `event.is_directory`, but the directory is already gone by then, so it returns `False`. All delete/move handlers therefore check `_is_direct_child_dir(path)` directly instead of relying on `event.is_directory`.
|
||
|
||
**Result directory deleted**: `reset_by_file_id_and_device(test_id, device)` resets `completed = 0`, `completed_at = NULL`, `duration_seconds = NULL`. broadcast fires unconditionally regardless of whether the folder name was parseable.
|
||
|
||
**Recycle Bin delete on Windows**: fires a `MovedEvent` (`src` = result dir, `dest` = `$RECYCLE.BIN\...`). Handled by `on_moved` via `_is_direct_child_dir` on the source path.
|
||
|
||
---
|
||
|
||
## 8. Frontend Components
|
||
|
||
### 8.1 Layout
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────┐
|
||
│ Test Dashboard [⟳ Refresh] [⚙ Settings] │
|
||
├──────────┬──────────┬──────────┬────────────┬───────────────┤
|
||
│ Overall │ CGW452 │ CGW453 │ Time │ Est. Remaining │
|
||
│ 60% │ 70% │ 50% │ Elapsed │ │
|
||
│ 120/200 │ 70/100 │ 50/100 │ 48h 0m │ 32h 0m │
|
||
├──────────┴──────────┴──────────┴────────────┴───────────────┤
|
||
│ ⚠ No P2P results yet — avg time required for estimate │
|
||
│ COE avg: 7m 30s (calc) P2P avg: [___] min P3P avg: 10m │
|
||
├──────────────────────────────────────────────────────────────┤
|
||
│ [Completed ▾] [Interference ▾] [Device ▾] [Band ▾] ... │
|
||
├──────────────────────────────────────────────────────────────┤
|
||
│ File ID │ Type │ Device │ Rotation │ TP │ ... │ Status │
|
||
│ R2COERXAC003│ COE │ CGW452 │ ROT1 │ TPT3E│ ... │ ✓ │
|
||
│ R5P2PRXAX012│ P2P │ CGW452 │ ROT1 │ TPT1C│ ... │ ○ │
|
||
└──────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Inline avg-time input**: When a type has remaining tests but zero completed, a warning banner appears with inline input fields for the missing avg time(s). Submitting saves to `POST /api/config` and immediately recalculates the estimate.
|
||
|
||
### 8.2 Component Breakdown
|
||
|
||
| Component | Responsibility |
|
||
|---|---|
|
||
| `StatCard` | Displays a metric (label, value, sub-value); used for completion rate + counts |
|
||
| `CompletionBar` | Progress bar with percentage label |
|
||
| `TimeDisplay` | Formats seconds into `Xh Ym`; shows elapsed and estimated remaining |
|
||
| `AvgTimeWarning` | Banner shown when a type has remaining tests but no avg time; contains inline inputs |
|
||
| `FilterPanel` | Dropdown filters for each tag including Interference; maintains filter state |
|
||
| `TestTable` | Virtualized (react-window) table of tests; columns sortable; File ID as primary ID column |
|
||
| `ConfigModal` | Form for directory paths (with picker button) + manual avg time overrides per type |
|
||
| `DirectoryBrowser` | Inline folder picker inside `ConfigModal`; navigates the server filesystem via `/api/browse` |
|
||
| `StatusBadge` | Green checkmark or grey circle for completed/pending |
|
||
|
||
### 8.3 Settings / Config
|
||
|
||
A gear icon opens `ConfigModal` with two sections:
|
||
|
||
1. **Directories** — target and results directory paths. Each path field has a **Browse** button that opens the `DirectoryBrowser`:
|
||
- Starts at the filesystem root (lists available drives on Windows)
|
||
- Displays current path as a clickable breadcrumb
|
||
- Lists subdirectories; clicking one navigates into it
|
||
- **Select This Folder** button confirms the selection and populates the path field
|
||
- **Cancel** closes the browser without changing the field
|
||
- On save, triggers a full re-scan and re-watch
|
||
2. **Avg Time Overrides** — three number inputs (COE, P2P, P3P) in minutes. Each shows:
|
||
- The calculated average from completed tests (if any), labelled `Auto: 7m 30s`
|
||
- A manual override field; when filled, it takes precedence over the calculated value
|
||
- Clear button to remove the override and revert to calculated
|
||
- If no completed tests exist for a type, the field is highlighted with a required indicator
|
||
|
||
All config values are POSTed to `POST /api/config` as key/value pairs and persisted in SQLite.
|
||
|
||
---
|
||
|
||
## 9. Vite Proxy Configuration
|
||
|
||
`vite.config.js` proxies all `/api` requests to the Python backend:
|
||
|
||
```js
|
||
server: {
|
||
host: true, // expose on LAN so other devices can reach the dev server
|
||
proxy: {
|
||
'/api': 'http://localhost:3001'
|
||
}
|
||
}
|
||
```
|
||
|
||
In production, Flask serves the built React `dist/` as static files via `send_from_directory`. Build with `npm run build` inside `dashboard/`.
|
||
|
||
---
|
||
|
||
## 10. Implementation Status
|
||
|
||
### Completed
|
||
- [x] Python Flask backend (`app.py`) with all API routes
|
||
- [x] SQLite schema, WAL mode, thread-safe helpers (`db_py.py`)
|
||
- [x] Tag parsing from target filenames and parent folder names (`parser.py`)
|
||
- [x] Full directory scan on startup / config change (`scanner.py`)
|
||
- [x] Watchdog filesystem watchers for both target and results dirs (`watcher.py`)
|
||
- [x] Windows `is_directory` timing bug fixed (check path directly)
|
||
- [x] Recycle Bin delete handled via `on_moved`
|
||
- [x] SSE broadcast with correct `\n\n` terminators (`sse_py.py`)
|
||
- [x] All frontend components and hooks
|
||
- [x] Config modal: directories always editable (lock removed)
|
||
- [x] Vite dev proxy + Flask static serving for production
|
||
- [x] Config migrated from `config.json` → SQLite on first run
|
||
|
||
### Remaining / Future
|
||
- [ ] SSO / authentication (noted in design plan)
|
||
- [ ] Test detail view (click a row to see tput/RSSI breakdown)
|
||
- [ ] Production deployment docs (systemd / Task Scheduler service)
|
||
2. Install `@tanstack/react-query`, `axios`, `tailwindcss`, `lucide-react`
|
||
3. Build `api.js` — base fetch helpers + SSE subscription hook
|
||
4. Build `useTests` and `useStats` hooks
|
||
5. Build `StatCard`, `CompletionBar`, `TimeDisplay`, `StatusBadge`
|
||
6. Build `App.jsx` layout with stats row
|
||
|
||
### Phase 3 — Test List & Filters
|
||
1. Build `FilterPanel` with controlled filter state
|
||
2. Build `TestTable` with all columns + sort
|
||
3. Wire filters to `GET /api/tests` query params
|
||
4. Add `ConfigModal` for directory path configuration
|
||
|
||
### Phase 4 — Real-time & Polish
|
||
1. Connect SSE stream in `useTests` / `useStats` to auto-invalidate queries
|
||
2. Add manual refresh button
|
||
3. Add loading skeletons and error states
|
||
4. Mobile-responsive layout adjustments
|
||
5. Test on secondary device over local network
|
||
|
||
### Phase 5 — SSO (Future)
|
||
- Add authentication middleware to the Express server (e.g., `passport.js` with an OAuth/OIDC provider)
|
||
- Protect all `/api` routes and the static frontend behind the auth middleware
|
||
- Add session management (`express-session` + a session store)
|
||
|
||
---
|
||
|
||
## 11. Key Dependencies
|
||
|
||
### Server (`server/package.json`)
|
||
```json
|
||
{
|
||
"dependencies": {
|
||
"better-sqlite3": "^9.x",
|
||
"chokidar": "^4.x",
|
||
"cors": "^2.x",
|
||
"dotenv": "^16.x",
|
||
"express": "^4.x"
|
||
}
|
||
}
|
||
```
|
||
|
||
### Client (`dashboard/package.json` additions)
|
||
```json
|
||
{
|
||
"dependencies": {
|
||
"@tanstack/react-query": "^5.x",
|
||
"axios": "^1.x",
|
||
"lucide-react": "^0.x",
|
||
"tailwindcss": "^4.x"
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 12. Resolved Design Decisions
|
||
|
||
| # | Decision |
|
||
|---|---|
|
||
| 1 | Target prefix is always `TC_WIFI_`, extension always `.ini`. `GLOBAL.ini` present in each parent directory is skipped by the scanner. |
|
||
| 2 | Interference types are a fixed enum: `COE`, `P2P`, `P3P`. |
|
||
| 3 | Duration parsed from result `.txt` log line `[... INFO] Elapsed time : H:MM:SS.ffffff`. Tests with no parseable line excluded from avg/sum. |
|
||
| 4 | If multiple result log files exist in a result directory (re-run), use the file with the **latest** timestamp in its name. |
|
||
| 5 | Backend runs on port **3001**. |
|
||
| 6 | When a result directory is deleted, the matching test is reset to pending (`completed=0`, `completed_at=NULL`, `duration_seconds=NULL`). |
|