Documentation
Sentinel Vision for Windows, version 0.1.0. Four programs that run detection and pose models on a webcam or a video and turn what they find into statuses.
Download and run
- Download SentinelVision-windows-x64.zip.
- Unzip it anywhere. Nothing is installed and nothing is written outside the folder.
- Double-click
SentinelOffice.exe,SentinelShop.exe,SentinelHome.exeorSentinelExam.exe.
Each program starts on its sample clip. The clip has nothing drawn in it. The outlines and statuses are computed as it plays.
The programs are not code-signed, so Windows may ask before the first run. Choose "More info", then "Run anyway".
Keep the folder together. The programs read models, samples, zones and fonts at start and share the _internal folder. To use them on another PC, copy the whole folder.
It needs Windows 10 or 11, 64-bit. It runs on the CPU and uses a DirectX 12 graphics card when one is available.
The four programs
| Program | Reports | Models it loads |
|---|---|---|
| SentinelOffice | Each seat as Working, Absent or Phone usage. Seats are learned from the first second. | Objects |
| SentinelShop | Zones you draw: someone entering, someone staying past a time, more people than a limit. | Objects |
| SentinelHome | Zones by what is in them: a person, a vehicle or a pet. Pets are recognised and raise no alert. | Objects |
| SentinelExam | Each candidate's head direction, the desk it points at, phones, and a risk score from 0 to 100. | Pose and objects |
Keys
| Key | Does |
|---|---|
| W | Webcam. Press again for the next camera. |
| O | Open a video file. |
| S | Back to the sample clip. |
| M | Switch between the small model and the nano model. |
| [ ] | Smaller or larger model input: 640, 960 or 1280 pixels. |
| R | Start again. Seats are mapped afresh and timers go to zero. |
| Z | Draw a zone (Shop and Home). Click each corner, press Enter, then pick the rule with a number key. |
| C | Clear all zones (Shop and Home). |
| P | Pause. |
| H | Show or hide the key hints. |
| F | Full screen. |
| X | Save a screenshot to logs. |
| Q | Quit. |
The readout and the log
The box in the bottom corner of the window shows what is running: the model, its input size, whether it is on the GPU or the CPU, the inference time for the last frame, the frame rate and the number of detections.
Every status change is written to a CSV file in logs, one file per session, with the clock time, the time into the stream, the source, the seat or zone, and what changed.
A program that cannot start writes the reason to sentinel.log in its folder.
Rules and thresholds
A status is one rule applied to what the models found. The rules are the same ones the demo clips on the site were made with.
Office
| Status | Rule |
|---|---|
| Working | A person is detected in the seat and no phone is detected in their hands. It means the person is at the desk. |
| Absent | No person in the seat for 0.75 seconds. Clears after 0.4 seconds back. |
| Phone usage | A phone is detected inside the person's outline, away from where phones were lying during the first second, for 0.35 seconds. |
Zones (Shop and Home)
| Rule | Fires when |
|---|---|
| Entry | A person, vehicle or pet, whichever the zone is set to, is inside for about a third of a second. The status stays up briefly after the zone empties. |
| Dwell | The same tracked person has been inside for 5 seconds. |
| Occupancy | More people than the limit are inside for 0.6 seconds. |
Zones you draw use the centre of each detected box. The zone files for the sample clips are in zones, and yours are saved there as custom_shop.json and custom_home.json.
Exam
- During the first second, each seated person becomes a seat with a desk zone around where their wrists rest and a baseline head direction.
- Head direction is the line from the midpoint of the ears to the nose. Yaw is how far that line has turned from the candidate's own baseline, in degrees.
- Past 38 degrees, the look has left the candidate's own paper. The target is the best-aligned desk within 30 degrees of the head direction and within seven shoulder-widths.
- A phone counts when it is inside the candidate's outline or desk zone for 0.35 seconds.
| Risk score | Value | Meaning |
|---|---|---|
| Look hold | 0.35 s | A look must last this long before it scores. |
| Look rate | 42 a second | Added while the head points at another desk. |
| Decay | 7 a second | Removed the rest of the time. |
| Phone | 85 | A phone at the desk lifts the score to at least this. |
| Review | 25 | Normal becomes Review. |
| Flagged | 60 | Review becomes Flagged. |
Looking up, or away at nothing, scores nothing. A flag is a prompt for an invigilator to check the footage at that time. The program makes no finding about what the candidate was doing.
Model files
The models folder holds four ONNX files. The same files are in the repository.
| File | Size in bytes | SHA-256 |
|---|---|---|
| yolo26n.onnx | 10,419,737 | a772bebdb74762b8567986ac206a5ecafb27e481c572cfa41435e3300acc4642 |
| yolo26s.onnx | 38,769,190 | 2a0b6ac6d7d441fbc71c574af44f8694f62fda33e940078c16037c105268e01a |
| yolo26n-pose.onnx | 12,534,179 | b690600113b28104f1041c7467842f43bf122b4df1037d2a598129dacc3e8287 |
| yolo26s-pose.onnx | 42,267,783 | 209b1d2b54efc281e2980d40e9eedd638999517802cba1fcf5bc59e35d7968e6 |
To check a file on Windows:
certutil -hashfile models\yolo26s.onnx SHA256
Each model takes one RGB image scaled to 0 to 1, at any width and height that are multiples of 32. The object models return 300 rows of box, confidence and class. The pose models return 300 rows of box, confidence and 17 keypoints. They are opset 17 exports of Ultralytics YOLO26 with its end-to-end head, so the rows are final detections.
Speed
Inference time per frame on a 2-core 2.1 GHz Xeon with no graphics card:
| Model and input | Office, Shop, Home | Exam |
|---|---|---|
| Nano, 640 px | 40 to 57 ms | 94 ms |
| Nano, 1280 px | 165 to 173 ms | 380 ms |
| Small, 640 px | 119 to 158 ms | 256 ms |
| Small, 1280 px | 463 to 553 ms | 1,037 ms |
Exam is slower because it runs the pose model and the object model on every frame.
The program chooses the small model at 1280 pixels for a video file on a graphics card, and 640 pixels on a CPU or for a webcam, where people are close to the camera. M, [ and ] change that while it runs.
When the PC cannot keep up with a file, frames are dropped so the clip stays at real speed. The rules run on clip time, so events fire at the same moments either way.
Source and build
The source is at github.com/forgehk/sentinel-vision: the programs, the rules, the tests and the workflow that builds each release.
That workflow runs on GitHub's Windows machines. It runs the tests, builds the folder, starts each of the four programs on its sample clip and stops unless each one reports the expected events. Only then is the zip published, with its SHA-256. The build log is public.
To build it yourself, install Python 3.10 or newer, download the source and double-click app\build_exe.bat. To run it without building:
cd app
run.bat exam
Limits
- One person on a webcam cannot be flagged for looking at another desk, because there is no other desk. The yaw figure still moves and a phone still flags.
- Head direction is measured in the picture from one camera. It reads a head turned sideways well, and a head tilted toward or away from the camera poorly. With the nose hidden, no direction is reported.
- The gaze target is the best-aligned nearby desk. It is not a measurement of what the eyes are on.
- Phones are small. The small model at 1280 pixels finds a phone held at a desk a few metres away. The nano model at 640 pixels usually does not.
- Seats are learned from the first second. Press R to map them again after people have sat down.
- The release check runs on a machine with no camera and no graphics card, so webcam input and GPU inference are not part of it.
Licence
Sentinel Vision is released under the GNU Affero General Public License v3.0. The YOLO26 models are from Ultralytics and carry the same licence. Closed-source commercial use of the models needs an Ultralytics Enterprise licence.