scheduler/DESIGNPLAN.md

Scheduler

A test scheduler for the NJTH which compiles a schedule according to the rules and priority of the tests.

Inputs

• Target test directory, test results directories
• Estimated runtime for each type of test for calculations
• Top priority tests
  • Allow user to manually override and enter tests that needs to be run tonight
• Day time testing (Y/N)
  • if yes, enter how many hours

TestID	Type	ROT	COE_Pairs	RX_TX	Priority	Scheduled_on	Completed
P2PRXAX001	P2P	ROT1		RX	3		0
P2PRXAX004	P2P	ROT1	[COERXAX001, COERXAX002]	AX	2		0
P2PTXAX001	P2P	ROT1		TX	3		0
COERXAX001	COE	ROT1		RX	3		0
COERXAX002	COE	ROT1		RX	3		0
P2PRXAX014	P2P	ROT2		RX	3		0
P2PTXAX014	P2P	ROT2		TX	3		0

Output

Backend

• Compile a schedule that would require the lowest amount of time in total according to the following rules and priority:
  • Rules
    • Two tests' overlapping stations are located at the same test points, they can be run in the same night
      • i.e. when two tests have the same rotation
      • P2PRXAX001, P2PRXAX004, P2PTXAX001, COERXAX001, COERXAX002 can be run on the same night;
      • P2PRXAX014, P2PTXAX014 can be run on the same night
    • Up link down link p2p should run b2b (RX and TX)
      • Ex. P2PRXAX001 and P2PTXAX001 should run b2b
    • COE P2P bundle should run b2b
      • Ex. P2PRXAX001 and COERXAX001 and COERXAX002 should run b2b
  • Priority
    • 1: highest priority (run tonight)
    • 5: lowest priority
    • General priority: COE P2P pair -> P2P only -> P3P
      • Set priority of each test according to the type when first read in the tests P2P with COE pairing -> 2, P2P only -> 3, and P3P -> 4
    • Change priority to 1 for manually entered top priority tests, meaning it will be run tonight
  • Calculation
    • Use the estimate run time for each type of test to fit as many tests as possible in a shift of testing. One shift at night on weekdays is 16hrs, on weekends 24 hours with 1hr margin
    • Estimate run time
      • P2P: 85 mins
      • COE: 115 mins
      • P3P: 105 mins
    • If day time testing for today is yes,
  • Failed test management
    • After there's a modification to the result directories (results are in), and there are tests that are scheduled for last night but were not added to the result directories, meaning the test failed/invalid
    • Estimate how much time for rerun:
      • If user choose to rerun for next day, move the whole schedule down by a day

Frontend

• Calendar

  	Sun	Mon	Tue	Wed	Thu	Fri	Sat
  S1	TestId	TestId	TestId	TestId	TestId	TestId	TestId
  S2	TestId	TestId	TestId	TestId	TestId	TestId	TestId
  S3	TestId	TestId	TestId	TestId	TestId	TestId	TestId

  • List the order that the tests should be conducted after compiling
  • Split one day into 3 eight-hour shift
    • Shift 1 is daytime, on weekdays don't include unless day time testing for today is on. On weekends it is included
    • Shift 2 and 3 are night time
  • Hover over each tests to see test details: type, rotation
  • Display one week at a time, arrows to go to prev/next week
  • Today will be hightlighted or boxed
  • Colors:
    • Pending tests: gray
    • Completed tests: green
    • Failed tests: red
    • Invalid tests: yellow

• Options (on the right of the calendar)

  • Day time testing for today: Yes or No
    • If yes, include shift 1 in todays testing
  • Top priority tests
    • Enter a list of comma separated TestIDs to make those test priority 1

• Test Config for tonight

  • After the schedule is compiled, show where each station should be placed at what test point

• Estimated Completion Date

• Settings

  • Estimated run time for each type of test
  • Test exclusion
    • User can enter rules to exclude tests that will not be run
  • Manual priority override
  • Holidays
    • User can enter holidays and that day will be counted as a weekend (24hrs)

• Failed Tests

  • Display a banner with:
    • A message saying what tests failed last night and requires a rerun
    • Estimate rerun time
    • Yes/No button for rerun during the day
      • If yes, display today's shift 1 with the rerun tests
      • If no, the rerun tests will be ran tonight, the whole schedule is shift down by one day, update the calendar

Layout

┌──────────────────────────────────────────────────────────────┐
| CGW453 Scheduler
|   ┌────────────────────────────────────┐  COMPLETION DATE
|   |                                    |
|   |              CALENDAR              |
|   |                                    |
|   |                                    |
|
|
|
|
|
└──────────────────────────────────────────────────────────────┘

Implementation

Backend

• Python, FastAPI, SQLite for storage
• graph.py
  • graph
    • Node: test
    • Edge: compatible
    • Adjacency list
    • Graph will not be modified
  • active_tests (set)
    • Keep track of tests that are not completed
    • Remove test from active_tests when test completed or is excluded
  • priorities (dict)
    • Keep track of test's priority
• Methods

  • build_graph()

  • compatible(test1, test2) -> bool

    • all overlapping stations are at the same test point

  • Degree(t): len(graph[t] & active_tests)

  • Base node selection: priority -> degree

    base = min( active_tests, key=lambda t: (priority[t], -len(graph[t] & active_tests)))

Frontend

• React