Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine

Grow cameras: a saved camera profile per Pi camera, re-applied at every start, and a Tune panel with the box drawn once and colour correction under grow light #591

Merged opened this in answer to

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

Merged

vesper/arm:vesper/584-grow-cameras-see-the-plants-a-saved-came → main 4 commits fork

  • Merged. vesper/584-grow-cameras-see-the-plants-a-saved-came was merged into main as 17c46ef.

Review

Sign in to review this change.
13 files changed +1390 −15
Software/README.md +84
@@ -900,6 +900,90 @@ The vision kit in `Software/vision` works against this server and stream:
900900 `SO101_ARM=http://<pi>:8101 SO101_CAMERA=http://<pi>:8080/stream`, then the
901901 `just vision-*` recipes. See `Software/vision/QUEST.md`.
902902
903 ### Camera profile: seeing the plants under a grow light
904
905 A webcam meters on the brightest thing it sees. In the grow box that is the
906 light, so on auto exposure the box comes out as a white blob on blue. The fix
907 is a fixed exposure and white balance. A webcam forgets both on every re-plug
908 and reboot, so the Pi applies a saved **profile** each time the camera starts.
909
910 **Until a profile is saved, each camera gets the owner's own choice** from
911 2026-10-26 (`OWNER_DEFAULTS` in `src/camera-profile.ts`). Camera 1, the side
912 XWF-1080P on :8080, gets auto exposure, auto white balance and backlight
913 compensation 1. Camera 2, the overhead eMeet C960 on :8081, gets brightness
914 115, contrast 120 and saturation 150. The Tune panel starts from these, and
915 `so101 camera-profile <n>` applies them on the Pi when nothing has been saved,
916 so they survive a re-plug from the first deploy on.
917
918 **Tune it on /admin → Cameras → Tune.** The panel shows the camera once a
919 second and has a slider for exposure, white balance, gain, focus, backlight
920 compensation, brightness, contrast and saturation. Each control can be *leave* (the camera's own value), *auto* or
921 *fixed*. A slider goes on the camera at once (`POST /admin/cameras/<id>/try`),
922 and **Save** keeps it (`PUT /admin/cameras/<id>`). *Grow-box settings* loads
923 what worked on 2026-10-26: exposure 180 (×100 µs), white balance 6500 K,
924 backlight compensation 0.
925
926 **Mark the box.** Drag on the picture to mark the box. The region is kept in
927 fractions of the frame, so a region drawn on the 640×360 stream crops a
928 1920×1080 still the same way (`cropRect` in `src/ui/still-frame.js`). The
929 check under the pictures only looks at that region. When the region is mostly
930 blown out or black, it says: *Camera can't see the plants: move it inside or
931 beside the box, below the light.* That means the camera is in the wrong place.
932 The overhead camera, which looks down at the top of the light panel, is the
933 case it was written for.
934
935 **Colour.** A blue/violet grow light leaves little red and green, so colour is
936 never true under it. *Measure colour* takes gray-world gains over the region,
937 ignoring blown-out and black pixels, and caps each channel at ×4. The right
938 picture shows the box with the gains applied, labelled "corrected". The gains
939 are saved with the camera. `whiteCardGains` takes them from a shot of a white
940 card under the light instead.
941
942 **On the Pi**, the arm server keeps camera n's profile in
943 `camera-profiles/<n>.json` beside its poses, and `so101 camera-profile <n>`
944 puts it on the camera. The camera's device is read from `/etc/so101/camera<n>`:
945 the `/dev/v4l/by-id` path alone, or as `DEVICE=<path>`. Each control is one
946 `v4l2-ctl --set-ctrl` call made with an argument list, never a shell. A
947 control the camera lacks (the XWF-1080P has no focus) is reported and the rest
948 still go on. With no profile saved, it applies the owner's default for that camera, if
949 there is one, and otherwise leaves the camera alone. It never stops a stream
950 from starting.
951
952 ```sh
953 sudo apt-get install v4l-utils
954 v4l2-ctl -d /dev/v4l/by-id/<camera>-video-index0 --list-ctrls # the names and ranges this camera has
955 ls /dev/v4l/by-id/ | sudo tee /etc/so101/camera2 # then keep only camera 2's -video-index0 line
956 sudo systemctl edit so101-camera@.service
957 ```
958
959 ```ini
960 [Service]
961 ExecStartPre=-/usr/bin/node <so101 tree>/bin/so101 camera-profile %i --file <the poses.json so101 serve uses>
962 ```
963
964 Use the arm server's tree (the one `so101.service` runs) and the same
965 `--file` as its `so101 serve`, given as an absolute path, so both read the
966 same `camera-profiles/`. The leading `-` keeps the stream starting even if the
967 command fails. To re-apply on a re-plug, restart the unit when its camera
968 appears:
969
970 ```sh
971 echo 'ACTION=="add", SUBSYSTEM=="video4linux", ATTR{index}=="0", RUN+="/usr/bin/systemctl restart --no-block [email protected] [email protected]"' \
972 | sudo tee /etc/udev/rules.d/90-so101-camera.rules
973 sudo udevadm control --reload
974 ```
975
976 **Tuning is page-only on purpose.** An exposure is judged by looking at the
977 picture, which neither the app's terminal (`tty`) nor the CLI can show. So
978 `so101 camera-profile` only re-applies what the panel saved. On a Pi with no
979 browser at hand, `v4l2-ctl --set-ctrl` tries a value by hand, and the panel
980 keeps it.
981
982 The relay never passes `/camera-profile/` through to the arm server. Only the
983 owner's two routes above reach it. The simulation has no camera to tune: Save
984 keeps the look in the relay's camera list and says the Pi did not take it.
985 The arm server must be running on the Pi for a profile to reach the camera.
986
903987 ### H.264 renditions on 8082-8084
904988
905989 Beside the MJPEG stream, the Pi 4's hardware encoder (`/dev/video11`, the
Software/so101/src/camera-control.ts added +90
@@ -0,0 +1,90 @@
1 /**
2 * The Pi's side of a camera profile (src/camera-profile.ts): where camera n is, where its saved profile is kept, and
3 * running `v4l2-ctl` to put a profile on it. The arm server calls it when the owner tries or saves a profile on /admin;
4 * `so101 camera-profile <n>` calls it from the camera service's ExecStartPre, so the profile survives a re-plug or a reboot.
5 *
6 * Runs under Node as well as Bun (the CLI is Node): nothing here reaches bun:sqlite. v4l2-ctl is run with execFile, an
7 * argument list built from checked numbers, never a shell.
8 */
9
10 import { execFile } from "node:child_process";
11 import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
12 import { dirname, join } from "node:path";
13 import { controls, parseDevice, parseProfile, ProfileError, v4l2Calls, type CameraProfile } from "./camera-profile.ts";
14
15 /** The camera services are so101-camera@1 … @8. */
16 export const MAX_UNIT = 8;
17
18 export function parseUnit(text: unknown): number {
19 const n = typeof text === "number" ? text : typeof text === "string" && /^\d{1,2}$/.test(text) ? Number(text) : NaN;
20 if (!Number.isInteger(n) || n < 1 || n > MAX_UNIT) throw new ProfileError(`a camera on the Pi is numbered 1 to ${MAX_UNIT}, like so101-camera@1`);
21 return n;
22 }
23
24 /**
25 * Camera n's device, from /etc/so101/camera<n>: the file holds the /dev/v4l/by-id path, alone or as DEVICE=<path>
26 * (so the same file can be the service's EnvironmentFile).
27 */
28 export function cameraDevice(unit: number, etc = "/etc/so101"): string {
29 const file = join(etc, `camera${parseUnit(unit)}`);
30 if (!existsSync(file)) throw new ProfileError(`${file} is missing: write camera ${unit}'s /dev/v4l/by-id path there`);
31 const lines = readFileSync(file, "utf8").split("\n").map((l) => l.trim()).filter((l) => l && !l.startsWith("#"));
32 const line = lines.find((l) => l.startsWith("DEVICE=")) ?? lines.find((l) => l.startsWith("/dev/"));
33 if (!line) throw new ProfileError(`${file} names no device: write camera ${unit}'s /dev/v4l/by-id path there`);
34 return parseDevice(line.replace(/^DEVICE=/, "").replace(/^["']|["']$/g, ""));
35 }
36
37 /** Where camera n's saved profile is kept, beside the arm server's poses. */
38 export function profileFile(dataDir: string, unit: number): string {
39 return join(dataDir, "camera-profiles", `${parseUnit(unit)}.json`);
40 }
41
42 /** The saved profile, or null when none was saved (or the file is unreadable, which is said on stderr). */
43 export function loadProfile(file: string): CameraProfile | null {
44 if (!existsSync(file)) return null;
45 try {
46 return parseProfile(JSON.parse(readFileSync(file, "utf8")));
47 } catch (err) {
48 console.error(`camera profile: ${file} unreadable (${(err as Error).message})`);
49 return null;
50 }
51 }
52
53 export function saveProfile(file: string, profile: CameraProfile): void {
54 mkdirSync(dirname(file), { recursive: true });
55 const tmp = `${file}.tmp`;
56 writeFileSync(tmp, JSON.stringify(parseProfile(profile)));
57 renameSync(tmp, file);
58 }
59
60 /** Runs one `v4l2-ctl` call; resolves with its error text, or "" when it worked. */
61 export type RunV4l2 = (args: string[]) => Promise<string>;
62
63 export const runV4l2: RunV4l2 = (args) =>
64 new Promise((resolve) => {
65 execFile("v4l2-ctl", args, { timeout: 3000 }, (err, _stdout, stderr) => {
66 if (!err) return resolve("");
67 const code = (err as NodeJS.ErrnoException).code;
68 resolve(code === "ENOENT" ? "v4l2-ctl is not installed (sudo apt-get install v4l-utils)" : (stderr || err.message).trim().split("\n")[0]!.slice(0, 200));
69 });
70 });
71
72 export interface Applied {
73 /** The controls the camera took, as name=value. */
74 set: string[];
75 /** The ones it refused, and why: a control this camera lacks is the usual one. */
76 failed: { control: string; error: string }[];
77 }
78
79 /** Put a profile on the camera at `device`, one control at a time; a refused control is reported, never thrown. */
80 export async function applyProfile(device: string, profile: CameraProfile, run: RunV4l2 = runV4l2): Promise<Applied> {
81 const calls = v4l2Calls(device, profile);
82 const names = controls(profile).map((c) => `${c.name}=${c.value}`);
83 const out: Applied = { set: [], failed: [] };
84 for (const [i, args] of calls.entries()) {
85 const error = await run(args);
86 if (error) out.failed.push({ control: names[i]!, error });
87 else out.set.push(names[i]!);
88 }
89 return out;
90 }
Software/so101/src/camera-list.html +210 −4
@@ -9,14 +9,27 @@
99 body { font: 15px/1.5 system-ui, sans-serif; max-width: 42rem; margin: 2rem auto; padding: 0 1rem; }
1010 li { display: flex; gap: .75rem; align-items: baseline; padding: .4rem 0; border-bottom: 1px solid var(--line); list-style: none; }
1111 ul { padding: 0; } li span.src { color: var(--dim); flex: 1; overflow-wrap: anywhere; }
12 form { display: flex; gap: .5rem; flex-wrap: wrap; } input, button { font: inherit; padding: .3rem .5rem; } li button { padding: .5rem .9rem; } label { align-self: center; } #msg { color: #c33; min-height: 1.5em; }
12 form { display: flex; gap: .5rem; flex-wrap: wrap; } input, button, select { font: inherit; padding: .3rem .5rem; } li button { padding: .5rem .9rem; } label { align-self: center; } #msg { color: #c33; min-height: 1.5em; }
13 #tune { border-top: 2px solid var(--line); margin-top: 1.5rem; padding-top: .5rem; }
14 .views { display: grid; grid-template-columns: 1fr 1fr; gap: .5rem; } @media (max-width: 34rem) { .views { grid-template-columns: 1fr; } }
15 figure { margin: 0; } figcaption { color: var(--dim); font-size: 13px; }
16 canvas { display: block; width: 100%; aspect-ratio: 16 / 9; object-fit: contain; background: #0002; touch-action: none; }
17 #view { cursor: crosshair; }
18 .ctl { display: grid; grid-template-columns: 7.5rem 6.5rem 1fr 4rem; gap: .5rem; align-items: center; margin: .3rem 0; }
19 .ctl output { font-variant-numeric: tabular-nums; text-align: right; } .ctl input[type=range]:disabled { opacity: .35; }
20 /* a phone: the label and its mode on one line, the slider and its value under them, the slider as wide as the screen */
21 @media (max-width: 34rem) { .ctl { grid-template-columns: 1fr auto; row-gap: .15rem; margin: .6rem 0; } .ctl input[type=range] { min-height: 44px; } }
22 #box { border: 1px solid var(--line); border-radius: 4px; margin: .5rem 0; padding: .3rem .6rem .5rem; display: flex; gap: .5rem 1rem; flex-wrap: wrap; }
23 #box label { display: flex; gap: .3rem; align-items: center; } #box input { width: 4.5rem; }
24 #seen { min-height: 1.5em; } #seen.bad { color: #c33; font-weight: 600; }
25 .row { display: flex; gap: .5rem; flex-wrap: wrap; margin: .5rem 0; } #note { min-height: 1.5em; } #note.bad { color: #c33; }
1326 </style>
1427 </head>
1528 <body>
1629 <p><a href="/">← the arm</a> · <a href="/admin">Admin</a></p>
1730 <h1>Cameras</h1>
1831 <p>Each camera is streamed once from its source and shared by everyone watching. Viewers see a change within a few seconds.</p>
19 <noscript><p>This page needs JavaScript to list and change cameras.</p></noscript>
32 <noscript><p>This page needs JavaScript to list and change cameras, and to tune them.</p></noscript>
2033 <ul id="list"></ul>
2134 <form id="add">
2235 <label for="name">Name</label>
@@ -26,7 +39,34 @@
2639 <button>Add camera</button>
2740 </form>
2841 <p id="msg" role="status"></p>
29 <script>
42 <section id="tune" hidden aria-labelledby="tuneTitle">
43 <h2 id="tuneTitle">Tune</h2>
44 <p>Drag on the picture, or type the box's edges below in percent of the frame, to mark the box: stills are cropped to it, and the colour correction and the check below look only there. Sliders go on the camera at once; <b>Save</b> keeps them, and the Pi puts them back each time the camera starts.</p>
45 <div class="views">
46 <figure><canvas id="view" width="640" height="360" role="img" aria-label="The camera's picture, one snapshot a second, with the box marked in yellow" aria-describedby="boxSays"></canvas><figcaption id="viewCap">as the camera sees it</figcaption></figure>
47 <figure><canvas id="fixed" width="640" height="360" role="img" aria-label="Only the box, with the colour correction applied when there is one" aria-describedby="fixedCap"></canvas><figcaption id="fixedCap">the box</figcaption></figure>
48 </div>
49 <fieldset id="box">
50 <legend id="boxSays">The box: the whole frame</legend>
51 <label>Left <input id="bx" type="number" min="0" max="98" step="1" inputmode="numeric">%</label>
52 <label>Top <input id="by" type="number" min="0" max="98" step="1" inputmode="numeric">%</label>
53 <label>Width <input id="bw" type="number" min="2" max="100" step="1" inputmode="numeric">%</label>
54 <label>Height <input id="bh" type="number" min="2" max="100" step="1" inputmode="numeric">%</label>
55 </fieldset>
56 <p id="seen" role="status"></p>
57 <div id="ctls"></div>
58 <div class="row">
59 <button type="button" id="grow">Grow-box settings</button>
60 <button type="button" id="measure">Measure colour</button>
61 <button type="button" id="uncorrect">No correction</button>
62 <button type="button" id="whole">Whole frame</button>
63 <label for="unit">Pi camera</label><select id="unit"><option value="">not the Pi's</option></select>
64 </div>
65 <div class="row"><button type="button" id="save">Save</button><button type="button" id="closeTune">Close</button></div>
66 <p id="note" role="status" aria-live="polite"></p>
67 </section>
68 <script type="module">
69 import { CANT_SEE, applyGains, cropRect, grayWorldGains, regionFromDrag, regionFromPercent, regionToPercent, visibility } from "/ui/__VERSION__/still-frame.js";
3070 const $ = (id) => document.getElementById(id);
3171 const say = (t) => { $("msg").textContent = t; };
3272 async function call(method, path, body) {
@@ -42,9 +82,11 @@
4282 const li = document.createElement("li");
4383 const name = document.createElement("b"); name.textContent = c.name;
4484 const src = document.createElement("span"); src.className = "src"; src.textContent = c.source;
85 const tune = document.createElement("button"); tune.textContent = "Tune"; tune.setAttribute("aria-label", `Tune ${c.name}`);
86 tune.onclick = () => open(c);
4587 const rm = document.createElement("button"); rm.textContent = "Remove"; rm.setAttribute("aria-label", `Remove ${c.name}`);
4688 rm.onclick = () => call("DELETE", `/admin/cameras/${c.id}`).then(load, (e) => say(e.message));
47 li.append(name, src, rm);
89 li.append(name, src, tune, rm);
4890 return li;
4991 }));
5092 say("");
@@ -55,6 +97,170 @@
5597 call("POST", "/admin/cameras", { name: $("name").value, source: $("source").value }).then(() => { $("add").reset(); load(); }, (err) => say(err.message));
5698 };
5799 load();
100
101 // ---------------------------------------------------------------- Tune
102 /** Each control: its profile field, its label, its range, and whether the camera can run it itself. */
103 const CONTROLS = [
104 { key: "exposure", label: "Exposure", min: 1, max: 2000, step: 1, auto: true, unit: "×100 µs" },
105 { key: "whiteBalance", label: "White balance", min: 2800, max: 6500, step: 100, auto: true, unit: "K" },
106 { key: "gain", label: "Gain", min: 0, max: 255, step: 1, auto: false, unit: "" },
107 { key: "focus", label: "Focus", min: 0, max: 1023, step: 5, auto: true, unit: "" },
108 { key: "backlight", label: "Backlight comp.", min: 0, max: 8, step: 1, auto: false, unit: "" },
109 { key: "brightness", label: "Brightness", min: -64, max: 255, step: 1, auto: false, unit: "" },
110 { key: "contrast", label: "Contrast", min: 0, max: 255, step: 1, auto: false, unit: "" },
111 { key: "saturation", label: "Saturation", min: 0, max: 255, step: 1, auto: false, unit: "" },
112 ];
113 const GROW = { exposure: 180, whiteBalance: 6500, backlight: 0 };
114 for (let n = 1; n <= 8; n++) $("unit").append(new Option(`so101-camera@${n}`, String(n)));
115 /** @type {{ id: number, name: string } | null} */ let cam = null;
116 let region = null, gains = null, timer = 0, trying = 0, frame = null, drag = null;
117 const view = $("view"), fixed = $("fixed");
118
119 for (const c of CONTROLS) {
120 const row = document.createElement("div"); row.className = "ctl";
121 const label = document.createElement("label"); label.textContent = c.label; label.htmlFor = `r-${c.key}`;
122 const mode = document.createElement("select"); mode.id = `m-${c.key}`; mode.setAttribute("aria-label", `${c.label}: leave, auto or fixed`);
123 mode.append(new Option("leave", "leave"), ...(c.auto ? [new Option("auto", "auto")] : []), new Option("fixed", "fixed"));
124 const range = Object.assign(document.createElement("input"), { type: "range", id: `r-${c.key}`, min: c.min, max: c.max, step: c.step });
125 const out = document.createElement("output"); out.id = `o-${c.key}`;
126 const shown = () => { range.disabled = mode.value !== "fixed"; out.textContent = mode.value === "fixed" ? `${range.value}${c.unit ? " " + c.unit : ""}` : mode.value; };
127 mode.onchange = () => { shown(); tryLater(); };
128 range.oninput = () => { mode.value = "fixed"; shown(); tryLater(); };
129 row.append(label, mode, range, out);
130 $("ctls").append(row);
131 c.show = shown;
132 }
133
134 function profile() {
135 const p = {};
136 for (const c of CONTROLS) {
137 const mode = $(`m-${c.key}`).value;
138 if (mode === "auto") p[c.key] = "auto";
139 else if (mode === "fixed") p[c.key] = Number($(`r-${c.key}`).value);
140 }
141 return p;
142 }
143 function showProfile(p) {
144 for (const c of CONTROLS) {
145 const v = p[c.key];
146 $(`m-${c.key}`).value = v === undefined ? "leave" : v === "auto" ? "auto" : "fixed";
147 if (typeof v === "number") $(`r-${c.key}`).value = String(v);
148 c.show();
149 }
150 }
151 /** The panel's one status line: what the Pi did, and every error, in the same live region. */
152 const tell = (text, bad = false) => { $("note").textContent = text; $("note").className = bad ? "bad" : ""; };
153 const sayPi = (pi) => {
154 if (!pi) return tell("");
155 if (pi.error) return tell(`The Pi did not take it: ${pi.error}`, true);
156 tell(`Camera ${pi.saved ? "kept" : "set"}: ${pi.set.join(", ") || "nothing"}${pi.failed.length ? `. It has no ${pi.failed.map((f) => f.control.split("=")[0]).join(", ")}` : ""}.`);
157 };
158 function tryLater() {
159 clearTimeout(timer);
160 timer = setTimeout(async () => {
161 if (!cam) return;
162 const mine = ++trying;
163 try { const r = await call("POST", `/admin/cameras/${cam.id}/try`, { profile: profile() }); if (mine === trying) sayPi(r.pi); }
164 catch (e) { tell(e.message, true); }
165 }, 300);
166 }
167
168 async function open(c) {
169 const all = await call("GET", "/admin/cameras").catch((e) => (say(e.message), []));
170 const full = all.find((x) => x.id === c.id);
171 if (!full) return;
172 cam = { id: full.id, name: full.name };
173 region = full.look.region; gains = full.look.gains;
174 $("tuneTitle").textContent = `Tune: ${full.name}`;
175 $("unit").value = full.look.unit === null ? "" : String(full.look.unit);
176 showProfile(full.look.profile);
177 sayPi(null);
178 showBox();
179 $("tune").hidden = false;
180 $("tune").scrollIntoView({ behavior: "smooth", block: "start" });
181 grab();
182 }
183 $("closeTune").onclick = () => { cam = null; $("tune").hidden = true; };
184
185 /** The camera now, one snapshot a second while the panel is open. */
186 async function grab() {
187 const id = cam?.id;
188 if (id === undefined) return;
189 try {
190 const r = await fetch(`/camera/${id}/snapshot?t=${Date.now()}`, { cache: "no-store" });
191 if (!r.ok) throw new Error(`the camera did not answer (${r.status})`);
192 const bitmap = await createImageBitmap(await r.blob());
193 const c = document.createElement("canvas"); c.width = bitmap.width; c.height = bitmap.height;
194 const g = c.getContext("2d", { willReadFrequently: true }); g.drawImage(bitmap, 0, 0);
195 frame = { canvas: c, pixels: g.getImageData(0, 0, c.width, c.height) };
196 $("viewCap").textContent = `as the camera sees it, ${c.width}×${c.height}`;
197 paint();
198 } catch (e) { $("viewCap").textContent = e.message; }
199 if (cam?.id === id) setTimeout(grab, 1000);
200 }
201
202 /** The number inputs and the legend follow the region, however it was set. */
203 function showBox() {
204 const p = regionToPercent(region);
205 for (const k of ["x", "y", "w", "h"]) { const el = $(`b${k}`); if (document.activeElement !== el) el.value = String(p[k]); }
206 $("boxSays").textContent = region ? `The box: ${p.w}% wide and ${p.h}% tall, ${p.x}% from the left and ${p.y}% from the top` : "The box: the whole frame";
207 }
208 for (const k of ["x", "y", "w", "h"]) {
209 $(`b${k}`).onchange = () => {
210 try {
211 region = regionFromPercent({ x: $("bx").value, y: $("by").value, w: $("bw").value, h: $("bh").value });
212 tell(""); showBox(); paint();
213 } catch (e) { tell(e.message, true); }
214 };
215 }
216
217 function paint() {
218 showBox();
219 if (!frame) return;
220 const { canvas, pixels } = frame;
221 view.width = canvas.width; view.height = canvas.height;
222 const v = view.getContext("2d");
223 v.drawImage(canvas, 0, 0);
224 const rect = cropRect(region, canvas.width, canvas.height);
225 if (region) {
226 v.fillStyle = "#0008";
227 v.fillRect(0, 0, view.width, rect.y); v.fillRect(0, rect.y + rect.h, view.width, view.height - rect.y - rect.h);
228 v.fillRect(0, rect.y, rect.x, rect.h); v.fillRect(rect.x + rect.w, rect.y, view.width - rect.x - rect.w, rect.h);
229 v.strokeStyle = "#ffd400"; v.lineWidth = Math.max(2, view.width / 320); v.strokeRect(rect.x, rect.y, rect.w, rect.h);
230 }
231 const box = new ImageData(rect.w, rect.h);
232 for (let row = 0; row < rect.h; row++) box.data.set(pixels.data.subarray(((rect.y + row) * pixels.width + rect.x) * 4, ((rect.y + row) * pixels.width + rect.x + rect.w) * 4), row * rect.w * 4);
233 const seen = visibility(pixels.data, pixels.width, rect);
234 if (gains) applyGains(box.data, gains);
235 fixed.width = rect.w; fixed.height = rect.h;
236 fixed.getContext("2d").putImageData(box, 0, 0);
237 $("fixedCap").textContent = `${region ? "the box" : "the whole frame"}, ${gains ? `corrected (×${gains.r} ×${gains.g} ×${gains.b})` : "not corrected"}`;
238 const pct = (f) => `${Math.round(f * 100)}%`;
239 $("seen").className = seen.verdict === "ok" ? "" : "bad";
240 $("seen").textContent = seen.verdict === "ok" ? `The camera can see the box: ${pct(seen.blown)} blown out, ${pct(seen.dark)} black.` : `${CANT_SEE} (${pct(seen.blown)} blown out, ${pct(seen.dark)} black)`;
241 }
242
243 const at = (e) => { const b = view.getBoundingClientRect(); return { x: (e.clientX - b.left) / b.width, y: (e.clientY - b.top) / b.height }; };
244 view.onpointerdown = (e) => { drag = at(e); view.setPointerCapture(e.pointerId); };
245 view.onpointermove = (e) => { if (!drag) return; const r = regionFromDrag(drag, at(e)); if (r.w > 0.02 && r.h > 0.02) { region = r; paint(); } };
246 view.onpointerup = () => { drag = null; };
247 $("whole").onclick = () => { region = null; paint(); };
248 $("grow").onclick = () => { showProfile(GROW); tryLater(); };
249 $("uncorrect").onclick = () => { gains = null; paint(); };
250 $("measure").onclick = () => {
251 if (!frame) return;
252 const found = grayWorldGains(frame.pixels.data, frame.pixels.width, cropRect(region, frame.canvas.width, frame.canvas.height));
253 if (!found) return tell("Too little of the box is neither blown out nor black to judge its colour.", true);
254 gains = found; paint();
255 };
256 $("save").onclick = async () => {
257 if (!cam) return;
258 const round = (r) => r && { x: +r.x.toFixed(4), y: +r.y.toFixed(4), w: +r.w.toFixed(4), h: +r.h.toFixed(4) };
259 try {
260 const r = await call("PUT", `/admin/cameras/${cam.id}`, { unit: $("unit").value ? Number($("unit").value) : null, region: round(region), gains, profile: profile() });
261 if (r.pi) sayPi(r.pi); else tell("Saved.");
262 } catch (e) { tell(e.message, true); }
263 };
58264 </script>
59265 </body>
60266 </html>
Software/so101/src/camera-look.ts added +66
@@ -0,0 +1,66 @@
1 /**
2 * How a listed camera should look, kept with it in the camera list (src/cameras.ts): its profile on the Pi
3 * (src/camera-profile.ts), which so101-camera@<unit> it is there, the region of the frame that is the grow box, and the
4 * colour correction its stills get. Plain data, checked here at the boundary.
5 */
6
7 import { defaultProfile, parseProfile, ProfileError, type CameraProfile } from "./camera-profile.ts";
8 import { parseUnit } from "./camera-control.ts";
9 import { parseGains, parseRegion, type Gains, type Rect } from "./ui/still-frame.js";
10
11 export interface CameraLook {
12 profile: CameraProfile;
13 /** so101-camera@<unit> on the Pi; null when the camera is not one of the Pi's. */
14 unit: number | null;
15 /** The part of the frame that is the box, in fractions; null for the whole frame. */
16 region: Rect | null;
17 /** The colour correction for stills; null for none. */
18 gains: Gains | null;
19 }
20
21 const KEYS = new Set(["profile", "unit", "region", "gains"]);
22
23 /**
24 * The Pi serves so101-camera@n on port 8079 + n (8080 for the first): the unit a camera's source is, when it is one of those.
25 * Only a guess for a new camera; the owner can set the unit on the Tune panel.
26 */
27 export function unitOf(source: string): number | null {
28 try {
29 const port = Number(new URL(source).port);
30 return port >= 8080 && port <= 8087 ? port - 8079 : null;
31 } catch {
32 return null;
33 }
34 }
35
36 /** The look a camera has before anyone tunes it: the owner's chosen profile for its unit (src/camera-profile.ts). */
37 export function defaultLook(source: string): CameraLook {
38 const unit = unitOf(source);
39 return { profile: defaultProfile(unit), unit, region: null, gains: null };
40 }
41
42 /** A change to a look → the look after it; fields left out keep their value. Throws a ProfileError saying what is wrong. */
43 export function changeLook(now: CameraLook, input: unknown): CameraLook {
44 if (input === null || typeof input !== "object" || Array.isArray(input)) throw new ProfileError("send {profile?, unit?, region?, gains?}");
45 const o = input as Record<string, unknown>;
46 for (const key of Object.keys(o)) if (!KEYS.has(key)) throw new ProfileError(`a camera's look has no "${key.slice(0, 40)}"`);
47 const next = { ...now };
48 if (Object.hasOwn(o, "profile")) next.profile = parseProfile(o.profile);
49 if (Object.hasOwn(o, "unit")) next.unit = o.unit === null ? null : parseUnit(o.unit);
50 try {
51 if (Object.hasOwn(o, "region")) next.region = parseRegion(o.region);
52 if (Object.hasOwn(o, "gains")) next.gains = parseGains(o.gains);
53 } catch (err) {
54 throw new ProfileError((err as Error).message);
55 }
56 return next;
57 }
58
59 /** A look read back from the camera list's file: whatever does not check out is the default. */
60 export function readLook(saved: unknown, source: string): CameraLook {
61 try {
62 return changeLook(defaultLook(source), saved ?? {});
63 } catch {
64 return defaultLook(source);
65 }
66 }
Software/so101/src/camera-profile.test.ts added +149
@@ -0,0 +1,149 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { defaultProfile, GROW_BOX, parseDevice, parseProfile, ProfileError, v4l2Calls } from "./camera-profile.ts";
6 import { defaultLook } from "./camera-look.ts";
7 import { applyProfile, cameraDevice, loadProfile, parseUnit, profileFile, saveProfile } from "./camera-control.ts";
8 import { main } from "./cli.ts";
9
10 const XWF = "/dev/v4l/by-id/usb-XWF-1080P-video-index0";
11 const dirs: string[] = [];
12 const temp = () => {
13 const dir = mkdtempSync(join(tmpdir(), "camera-profile-"));
14 dirs.push(dir);
15 return dir;
16 };
17 afterEach(() => {
18 for (const d of dirs.splice(0)) rmSync(d, { recursive: true, force: true });
19 });
20
21 describe("a camera profile", () => {
22 test("the grow box's hand-tuned settings become the v4l2-ctl calls the owner typed, modes first", () => {
23 expect(v4l2Calls(XWF, GROW_BOX)).toEqual([
24 ["-d", XWF, "--set-ctrl=auto_exposure=1"],
25 ["-d", XWF, "--set-ctrl=white_balance_automatic=0"],
26 ["-d", XWF, "--set-ctrl=exposure_time_absolute=180"],
27 ["-d", XWF, "--set-ctrl=white_balance_temperature=6500"],
28 ["-d", XWF, "--set-ctrl=backlight_compensation=0"],
29 ]);
30 });
31
32 test("auto hands a control back to the camera; a field left out is left alone", () => {
33 expect(v4l2Calls("/dev/video0", { exposure: "auto", focus: "auto", gain: 40 }).map((a) => a[2])).toEqual([
34 "--set-ctrl=auto_exposure=3",
35 "--set-ctrl=focus_automatic_continuous=1",
36 "--set-ctrl=gain=40",
37 ]);
38 expect(v4l2Calls("/dev/video0", {})).toEqual([]);
39 expect(v4l2Calls("/dev/video0", { focus: 300 }).map((a) => a[2])).toEqual(["--set-ctrl=focus_automatic_continuous=0", "--set-ctrl=focus_absolute=300"]);
40 });
41
42 test("anything but a checked number or auto is refused at the boundary", () => {
43 for (const bad of [null, [], "x", { exposure: "180" }, { exposure: 1.5 }, { exposure: 0 }, { gain: "auto" }, { gain: 256 }, { whiteBalance: 100 }, { colour: 1 }, { exposure: "180; rm -rf /" }]) {
44 expect(() => parseProfile(bad)).toThrow(ProfileError);
45 }
46 expect(parseProfile({ exposure: null, gain: 0 })).toEqual({ gain: 0 });
47 });
48
49 test("a device is a path under /dev, nothing that climbs out or carries a flag", () => {
50 expect(parseDevice(` ${XWF} `)).toBe(XWF);
51 for (const bad of ["/dev/../etc/passwd", "video0", "-d /dev/video0", "/dev/video0 --all", "", 3]) expect(() => parseDevice(bad)).toThrow(ProfileError);
52 });
53 });
54
55 describe("the owner's chosen settings (#584, 2026-10-26)", () => {
56 test("each camera starts from what the owner set by hand, as the v4l2-ctl calls they typed", () => {
57 expect(v4l2Calls("/dev/video2", defaultProfile(2)).map((a) => a[2])).toEqual(["--set-ctrl=brightness=115", "--set-ctrl=contrast=120", "--set-ctrl=saturation=150"]);
58 expect(v4l2Calls(XWF, defaultProfile(1)).map((a) => a[2])).toEqual(["--set-ctrl=auto_exposure=3", "--set-ctrl=white_balance_automatic=1", "--set-ctrl=backlight_compensation=1"]);
59 expect(defaultProfile(3)).toEqual({});
60 expect(defaultProfile(null)).toEqual({});
61 });
62
63 test("a new camera on the Pi's ports starts from its unit's default; one elsewhere from nothing", () => {
64 expect(defaultLook("http://100.89.95.68:8081/stream").profile).toEqual({ brightness: 115, contrast: 120, saturation: 150 });
65 expect(defaultLook("http://100.89.95.68:8080/stream").profile).toEqual({ exposure: "auto", whiteBalance: "auto", backlight: 1 });
66 expect(defaultLook("http://cam.local:9000/stream").profile).toEqual({});
67 defaultLook("http://100.89.95.68:8081/stream").profile.saturation = 1; // a copy: the defaults themselves never change
68 expect(defaultProfile(2).saturation).toBe(150);
69 });
70
71 test("brightness, contrast and saturation are checked like the rest", () => {
72 expect(parseProfile({ brightness: -10, contrast: 0, saturation: 255 })).toEqual({ brightness: -10, contrast: 0, saturation: 255 });
73 for (const bad of [{ brightness: 300 }, { contrast: -1 }, { saturation: "auto" }, { saturation: 1.5 }]) expect(() => parseProfile(bad)).toThrow(ProfileError);
74 });
75 });
76
77 describe("the Pi's side", () => {
78 test("a saved profile round-trips through the file the camera service reads, into the calls it runs", async () => {
79 const data = temp();
80 const file = profileFile(data, 1);
81 expect(loadProfile(file)).toBeNull();
82 saveProfile(file, GROW_BOX);
83 const etc = temp();
84 writeFileSync(join(etc, "camera1"), `# the side camera\nDEVICE=${XWF}\n`);
85 const ran: string[][] = [];
86 const applied = await applyProfile(cameraDevice(1, etc), loadProfile(file)!, async (args) => (ran.push(args), args[2]!.includes("backlight") ? "unknown control 'backlight_compensation'" : ""));
87 expect(ran).toEqual(v4l2Calls(XWF, GROW_BOX));
88 expect(applied.set).toEqual(["auto_exposure=1", "white_balance_automatic=0", "exposure_time_absolute=180", "white_balance_temperature=6500"]);
89 expect(applied.failed).toEqual([{ control: "backlight_compensation=0", error: "unknown control 'backlight_compensation'" }]);
90 });
91
92 test("camera n's device comes from /etc/so101/camera<n>, a bare path or DEVICE=; a missing one says what to write", () => {
93 const etc = temp();
94 writeFileSync(join(etc, "camera2"), "/dev/video2\n");
95 expect(cameraDevice(2, etc)).toBe("/dev/video2");
96 expect(() => cameraDevice(3, etc)).toThrow(/camera3 is missing/);
97 writeFileSync(join(etc, "camera4"), "DEVICE=/dev/../etc/shadow\n");
98 expect(() => cameraDevice(4, etc)).toThrow(ProfileError);
99 });
100
101 test("units are 1 to 8; an unreadable profile file is no profile", () => {
102 expect(parseUnit("2")).toBe(2);
103 for (const bad of ["0", "9", "../1", 1.5, "1/../../x"]) expect(() => parseUnit(bad)).toThrow(ProfileError);
104 const data = temp();
105 mkdirSync(join(data, "camera-profiles"));
106 writeFileSync(profileFile(data, 1), "{nope");
107 expect(loadProfile(profileFile(data, 1))).toBeNull();
108 });
109 });
110
111 describe("so101 camera-profile <n>, the camera service's ExecStartPre", () => {
112 const quiet = async (argv: string[]) => {
113 const [log, err] = [console.log, process.stderr.write.bind(process.stderr)];
114 const said: string[] = [];
115 console.log = (...a: unknown[]) => void said.push(a.join(" "));
116 process.stderr.write = ((t: string) => (said.push(String(t)), true)) as typeof process.stderr.write;
117 try {
118 return { code: await main(argv), said: said.join("\n") };
119 } finally {
120 console.log = log;
121 process.stderr.write = err;
122 }
123 };
124
125 test("with no profile saved and no owner's default it leaves the camera alone and lets the stream start", async () => {
126 const data = temp();
127 const { code, said } = await quiet(["camera-profile", "3", "--file", join(data, "poses.json")]);
128 expect(code).toBe(0);
129 expect(said).toContain("no profile saved for camera 3");
130 expect(said).toContain("leaving the camera as it is");
131 });
132
133 test("with no profile saved, the overhead camera gets the owner's 2026-10-26 settings at its start", async () => {
134 const data = temp();
135 const { code, said } = await quiet(["camera-profile", "2", "--file", join(data, "poses.json"), "--device", "/dev/null-camera"]);
136 expect(code).toBe(0); // a control the camera refuses (here: no such device) is reported, never a failed start
137 expect(said).toContain("using the owner's default");
138 for (const c of ["brightness=115", "contrast=120", "saturation=150"]) expect(said).toContain(c);
139 });
140
141 test("a bad unit or device is refused before anything runs", async () => {
142 const data = temp();
143 saveProfile(profileFile(data, 1), GROW_BOX);
144 expect((await quiet(["camera-profile", "9"])).code).toBe(1);
145 const bad = await quiet(["camera-profile", "1", "--file", join(data, "poses.json"), "--device", "/dev/video0;reboot"]);
146 expect(bad.code).toBe(1);
147 expect(bad.said).toContain("a camera device is a path under /dev");
148 });
149 });
Software/so101/src/camera-profile.ts added +121
@@ -0,0 +1,121 @@
1 /**
2 * A camera's profile: the few UVC controls that decide whether it sees anything under a grow light (exposure, white
3 * balance, gain, focus, backlight compensation, brightness, contrast, saturation), as plain data, and the `v4l2-ctl` invocations that put it on the camera.
4 *
5 * Every field is optional: one left out is left as the camera has it. A webcam forgets all of this on a re-plug or a
6 * reboot, so the Pi applies the saved profile each time its camera service starts (README, "Camera profile").
7 *
8 * The invocations are argument lists for execFile, never a shell line: each value is a number checked here, and each
9 * control goes in its own call, so a control one camera lacks (the XWF-1080P has no focus) does not stop the others.
10 * Modes come before values: a UVC camera refuses an exposure time while auto exposure is on.
11 */
12
13 export interface CameraProfile {
14 /** "auto", or a fixed exposure time in 100 µs units (exposure_time_absolute). */
15 exposure?: "auto" | number;
16 /** "auto", or a fixed colour temperature in kelvin. */
17 whiteBalance?: "auto" | number;
18 gain?: number;
19 /** "auto" (continuous autofocus), or a fixed focus position. */
20 focus?: "auto" | number;
21 /** backlight_compensation: 0 keeps a bright box from darkening everything else. */
22 backlight?: number;
23 brightness?: number;
24 contrast?: number;
25 saturation?: number;
26 }
27
28 export class ProfileError extends Error {}
29
30 /** The range each number may take: wide enough for any UVC webcam, narrow enough that a typo is caught. */
31 export const RANGES = {
32 exposure: [1, 10_000],
33 whiteBalance: [2000, 10_000],
34 gain: [0, 255],
35 focus: [0, 1023],
36 backlight: [0, 8],
37 // UVC cameras disagree on these: -64..64 on some, 0..255 on others (the eMeet C960 is 0..255)
38 brightness: [-255, 255],
39 contrast: [0, 255],
40 saturation: [0, 255],
41 } as const satisfies Record<keyof CameraProfile, readonly [number, number]>;
42
43 /** What the owner tuned by hand on the grow box's side camera on 2026-10-26 (#584): the inside of the dome and the tray show. */
44 export const GROW_BOX: CameraProfile = { exposure: 180, whiteBalance: 6500, backlight: 0 };
45
46 /**
47 * What the owner chose by hand on the workshop's Pi on 2026-10-26 (#584), per so101-camera unit: each camera's profile
48 * until one is saved. Camera 1 is the side XWF-1080P on :8080 ("decent" on auto, with backlight compensation); camera 2
49 * the overhead eMeet C960 on :8081 ("saturation up a little … looked good").
50 */
51 export const OWNER_DEFAULTS: Readonly<Record<number, CameraProfile>> = {
52 1: { exposure: "auto", whiteBalance: "auto", backlight: 1 },
53 2: { brightness: 115, contrast: 120, saturation: 150 },
54 };
55
56 /** Camera `unit`'s profile before anyone saves one: the owner's choice where there is one, else nothing set. */
57 export function defaultProfile(unit: number | null): CameraProfile {
58 return unit !== null && Object.hasOwn(OWNER_DEFAULTS, unit) ? { ...OWNER_DEFAULTS[unit] } : {};
59 }
60
61 const AUTO_ALLOWED = new Set<keyof CameraProfile>(["exposure", "whiteBalance", "focus"]);
62
63 /** Any JSON → a profile, or a ProfileError saying which field is wrong. Unknown fields are refused, not dropped. */
64 export function parseProfile(input: unknown): CameraProfile {
65 if (input === null || typeof input !== "object" || Array.isArray(input)) throw new ProfileError("a camera profile is an object");
66 const out: CameraProfile = {};
67 for (const [key, value] of Object.entries(input)) {
68 if (!Object.hasOwn(RANGES, key)) throw new ProfileError(`a camera profile has no "${key.slice(0, 40)}"`);
69 const field = key as keyof CameraProfile;
70 if (value === undefined || value === null) continue;
71 if (value === "auto" && AUTO_ALLOWED.has(field)) {
72 (out as Record<string, unknown>)[field] = "auto";
73 continue;
74 }
75 const [lo, hi] = RANGES[field];
76 if (typeof value !== "number" || !Number.isInteger(value) || value < lo || value > hi) {
77 throw new ProfileError(`${field} is ${AUTO_ALLOWED.has(field) ? '"auto" or ' : ""}a whole number from ${lo} to ${hi}`);
78 }
79 (out as Record<string, unknown>)[field] = value;
80 }
81 return out;
82 }
83
84 /** One control to set, by its V4L2 name, in the order they must be set. */
85 export interface Control {
86 name: string;
87 value: number;
88 }
89
90 /** The controls a profile sets, modes first. */
91 export function controls(profile: CameraProfile): Control[] {
92 const modes: Control[] = [];
93 const values: Control[] = [];
94 const { exposure, whiteBalance, focus, gain, backlight, brightness, contrast, saturation } = profile;
95 // auto_exposure: 1 is manual, 3 is aperture priority (the webcam's "auto")
96 if (exposure === "auto") modes.push({ name: "auto_exposure", value: 3 });
97 else if (exposure !== undefined) modes.push({ name: "auto_exposure", value: 1 }), values.push({ name: "exposure_time_absolute", value: exposure });
98 if (whiteBalance === "auto") modes.push({ name: "white_balance_automatic", value: 1 });
99 else if (whiteBalance !== undefined) modes.push({ name: "white_balance_automatic", value: 0 }), values.push({ name: "white_balance_temperature", value: whiteBalance });
100 if (focus === "auto") modes.push({ name: "focus_automatic_continuous", value: 1 });
101 else if (focus !== undefined) modes.push({ name: "focus_automatic_continuous", value: 0 }), values.push({ name: "focus_absolute", value: focus });
102 if (backlight !== undefined) values.push({ name: "backlight_compensation", value: backlight });
103 if (gain !== undefined) values.push({ name: "gain", value: gain });
104 if (brightness !== undefined) values.push({ name: "brightness", value: brightness });
105 if (contrast !== undefined) values.push({ name: "contrast", value: contrast });
106 if (saturation !== undefined) values.push({ name: "saturation", value: saturation });
107 return [...modes, ...values];
108 }
109
110 /** A device path: /dev/video0 or /dev/v4l/by-id/usb-…-video-index0, nothing that climbs out of /dev. */
111 export function parseDevice(text: unknown): string {
112 const path = typeof text === "string" ? text.trim() : "";
113 if (!/^\/dev\/[A-Za-z0-9._:+\/-]{1,200}$/.test(path) || path.split("/").includes("..")) throw new ProfileError("a camera device is a path under /dev, like /dev/v4l/by-id/<camera>-video-index0");
114 return path;
115 }
116
117 /** The argument list of each `v4l2-ctl` call that puts the profile on the camera at `device`, one control per call. */
118 export function v4l2Calls(device: string, profile: CameraProfile): string[][] {
119 const dev = parseDevice(device);
120 return controls(parseProfile(profile)).map((c) => ["-d", dev, `--set-ctrl=${c.name}=${c.value}`]);
121 }
Software/so101/src/camera-tune.test.ts added +192
@@ -0,0 +1,192 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { mkdtempSync, readFileSync } from "node:fs";
3 import { tmpdir } from "node:os";
4 import { join } from "node:path";
5 import { FeetechBus } from "./bus.ts";
6 import { FakeBus } from "./fake-bus.ts";
7 import { CameraHub } from "./cameras.ts";
8 import { GROW_BOX, v4l2Calls, type CameraProfile } from "./camera-profile.ts";
9 import { loadProfile, profileFile } from "./camera-control.ts";
10 import { sign } from "./identity.ts";
11 import { Lobby } from "./lobby.ts";
12 import { startRelay, type RunningRelay } from "./relay.ts";
13 import { startServer, type RunningServer } from "./server.ts";
14
15 // The Tune panel's path, end to end: /admin/cameras/<id> on the relay → PUT /camera-profile/<unit> on the arm server →
16 // the profile on the camera and in the file `so101 camera-profile <n>` reads at the camera service's next start.
17
18 const owner = { "content-type": "application/json", "sec-fetch-site": "same-origin" };
19 const XWF = "/dev/v4l/by-id/usb-XWF-1080P-video-index0";
20 let arm: RunningServer | null = null;
21 let relay: RunningRelay | null = null;
22 let hub: CameraHub | null = null;
23 afterEach(async () => {
24 await relay?.close();
25 await arm?.close();
26 hub?.close();
27 relay = arm = hub = null;
28 });
29
30 /** An arm server whose camera control records what it was asked, and a relay with one camera on the Pi's port 8080. */
31 async function setup(control = true) {
32 const dataDir = mkdtempSync(join(tmpdir(), "so101-tune-"));
33 const asked: { unit: number; profile: CameraProfile }[] = [];
34 arm = await startServer(new FeetechBus(new FakeBus([1, 2, 6])), {
35 host: "127.0.0.1",
36 port: 0,
37 ids: [1, 2, 6],
38 posesFile: join(dataDir, "poses.json"),
39 cameraControl: control
40 ? async (unit, profile) => {
41 asked.push({ unit, profile });
42 return { set: v4l2Calls(XWF, profile).map((a) => a[2]!.slice("--set-ctrl=".length)), failed: [] };
43 }
44 : undefined,
45 });
46 const camerasFile = join(dataDir, "cameras.json");
47 hub = new CameraHub({ file: camerasFile, seed: [{ name: "Side", source: "http://127.0.0.1:8080/stream" }] });
48 relay = await startRelay({ port: 0, simulation: `http://127.0.0.1:${arm.port}`, cameras: hub });
49 const base = `http://127.0.0.1:${relay.port}`;
50 return { base, dataDir, camerasFile, asked };
51 }
52
53 describe("tuning a camera", () => {
54 test("Save keeps the look with the camera and the profile on the Pi, where the camera service's next start reads it", async () => {
55 const { base, dataDir, camerasFile, asked } = await setup();
56 const region = { x: 0.3, y: 0.25, w: 0.4, h: 0.5 };
57 const r = await fetch(`${base}/admin/cameras/1`, { method: "PUT", headers: owner, body: JSON.stringify({ profile: GROW_BOX, region, gains: { r: 2, g: 1.5, b: 0.6 } }) });
58 expect(r.status).toBe(200);
59 const body = (await r.json()) as { look: unknown; pi: { saved: boolean; set: string[] } };
60 expect(body.look).toEqual({ profile: GROW_BOX, unit: 1, region, gains: { r: 2, g: 1.5, b: 0.6 } });
61 expect(body.pi.saved).toBe(true);
62 expect(body.pi.set).toContain("exposure_time_absolute=180");
63 expect(asked).toEqual([{ unit: 1, profile: GROW_BOX }]);
64
65 // the relay's list keeps the look across a restart
66 expect(JSON.parse(readFileSync(camerasFile, "utf8")).cameras[0].look).toEqual(body.look);
67 expect(new CameraHub({ file: camerasFile }).look(1)).toEqual(body.look as never);
68 // the Pi keeps the profile where `so101 camera-profile 1` finds it, and it turns into the calls the owner typed by hand
69 expect(v4l2Calls(XWF, loadProfile(profileFile(dataDir, 1))!).map((a) => a[2])).toEqual([
70 "--set-ctrl=auto_exposure=1",
71 "--set-ctrl=white_balance_automatic=0",
72 "--set-ctrl=exposure_time_absolute=180",
73 "--set-ctrl=white_balance_temperature=6500",
74 "--set-ctrl=backlight_compensation=0",
75 ]);
76 });
77
78 test("a slider's try goes on the camera but is not kept; a region alone is not sent to the Pi", async () => {
79 const { base, dataDir, asked } = await setup();
80 const tried = await fetch(`${base}/admin/cameras/1/try`, { method: "POST", headers: owner, body: JSON.stringify({ profile: { exposure: 300 } }) });
81 expect(tried.status).toBe(200);
82 expect(((await tried.json()) as { pi: { saved: boolean } }).pi.saved).toBe(false);
83 expect(asked).toEqual([{ unit: 1, profile: { exposure: 300 } }]);
84 expect(loadProfile(profileFile(dataDir, 1))).toBeNull();
85 const region = await fetch(`${base}/admin/cameras/1`, { method: "PUT", headers: owner, body: JSON.stringify({ region: null }) });
86 expect(((await region.json()) as { pi: unknown }).pi).toBeNull();
87 expect(asked.length).toBe(1);
88 });
89
90 test("bad input is refused at the boundary and the camera is left as it was", async () => {
91 const { base, asked } = await setup();
92 const put = (body: string, path = "/admin/cameras/1") => fetch(`${base}${path}`, { method: "PUT", headers: owner, body });
93 expect((await put("{nope")).status).toBe(400);
94 expect((await put(JSON.stringify({ profile: { exposure: "180; reboot" } }))).status).toBe(400);
95 expect((await put(JSON.stringify({ region: { x: 0.9, y: 0, w: 0.5, h: 0.5 } }))).status).toBe(400);
96 expect((await put(JSON.stringify({ gains: { r: 9, g: 1, b: 1 } }))).status).toBe(400);
97 expect((await put(JSON.stringify({ unit: 12 }))).status).toBe(400);
98 expect((await put(JSON.stringify({ colour: 1 }))).status).toBe(400);
99 expect((await put("{}", "/admin/cameras/9")).status).toBe(404);
100 expect((await fetch(`${base}/admin/cameras/1/try`, { method: "DELETE", headers: owner })).status).toBe(405);
101 expect(hub!.has(1)).toBe(true);
102 expect((await fetch(`${base}/admin/cameras/1`, { method: "PUT", headers: { ...owner, "sec-fetch-site": "cross-site" }, body: "{}" })).status).toBe(403);
103 expect(asked).toEqual([]);
104 expect(hub!.look(1)).toEqual({ profile: { exposure: "auto", whiteBalance: "auto", backlight: 1 }, unit: 1, region: null, gains: null });
105 });
106
107 test("nobody reaches the arm server's camera profiles through the relay, not even with a good body", async () => {
108 const { base, asked } = await setup();
109 expect((await fetch(`${base}/camera-profile/1`, { method: "PUT", headers: owner, body: JSON.stringify(GROW_BOX) })).status).toBe(404);
110 expect((await fetch(`${base}/camera-profile/1`)).status).toBe(404);
111 expect(asked).toEqual([]);
112 });
113
114 test("the simulation has no camera to tune: the look is kept and the panel is told why the Pi did not take it", async () => {
115 const { base } = await setup(false);
116 const r = await fetch(`${base}/admin/cameras/1`, { method: "PUT", headers: owner, body: JSON.stringify({ profile: GROW_BOX }) });
117 expect(r.status).toBe(200);
118 expect(((await r.json()) as { pi: { error: string } }).pi.error).toContain("simulation");
119 expect(hub!.look(1)!.profile).toEqual(GROW_BOX);
120 });
121
122 test("only the owner tunes a camera", async () => {
123 hub = new CameraHub({ seed: [{ name: "Side", source: "http://127.0.0.1:8080/stream" }] });
124 const key = "k".repeat(32);
125 const lobby = new Lobby({ identityKey: key });
126 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", cameras: hub, lobby });
127 const later = Date.now() + 60_000;
128 const as = (identity: string | null) => {
129 const headers: Record<string, string> = { ...owner };
130 if (identity) Object.assign(headers, { "x-forge-identity": identity, "x-forge-signature": sign(identity, key) });
131 return fetch(`http://127.0.0.1:${relay!.port}/admin/cameras/1`, { method: "PUT", headers, body: JSON.stringify({ region: { x: 0, y: 0, w: 0.5, h: 0.5 } }) });
132 };
133 expect((await as(null)).status).toBe(401);
134 expect((await as(`u1|alice|user|${later}`)).status).toBe(403);
135 expect(hub.look(1)!.region).toBeNull();
136 expect((await as(`u2|trevato|admin|${later}`)).status).toBe(200);
137 expect(hub.look(1)!.region).toEqual({ x: 0, y: 0, w: 0.5, h: 0.5 });
138 lobby.close();
139 });
140
141 test("the cameras page loads the still-frame module from a path the arm server serves", async () => {
142 const { base } = await setup();
143 const page = await (await fetch(`${base}/admin/camera-list`)).text();
144 const path = /"(\/ui\/[0-9a-f]{12}\/still-frame\.js)"/.exec(page)?.[1];
145 expect(path).toBeDefined();
146 const module = await fetch(`${base}${path}`);
147 expect(module.status).toBe(200);
148 expect(await module.text()).toContain("export function grayWorldGains");
149 });
150 });
151
152 describe("the Tune panel, read rather than rendered (there is no browser here)", () => {
153 const page = readFileSync(new URL("./camera-list.html", import.meta.url), "utf8");
154
155 test("on a 360 px phone each control stacks: label and mode on one line, a full-width slider and its value under them", () => {
156 const wide = /\.ctl \{[^}]*grid-template-columns: ([^;]+);/.exec(page)?.[1];
157 expect(wide).toBe("7.5rem 6.5rem 1fr 4rem");
158 const narrow = /@media \(max-width: (\d+)rem\) \{ \.ctl \{[^}]*grid-template-columns: ([^;]+);/.exec(page);
159 expect(Number(narrow?.[1]) * 16).toBeGreaterThanOrEqual(360);
160 // two columns, filled in source order (label, mode, range, output): the slider gets the whole 1fr of its own row
161 expect(narrow?.[2]).toBe("1fr auto");
162 expect(page.indexOf('row.append(label, mode, range, out)')).toBeGreaterThan(0);
163 });
164
165 test("the box can be set without a pointer, and the pictures say what they are", () => {
166 for (const [id, name] of [["bx", "Left"], ["by", "Top"], ["bw", "Width"], ["bh", "Height"]]) {
167 expect(page).toMatch(new RegExp(`<label>${name} <input id="${id}" type="number"`));
168 }
169 expect(page).toContain("regionFromPercent({ x: $(\"bx\").value");
170 for (const id of ["view", "fixed"]) expect(page).toMatch(new RegExp(`<canvas id="${id}"[^>]*role="img" aria-label="[^"]{20,}"`));
171 });
172
173 test("the panel has one live status line for results and errors alike, and says it needs JavaScript", () => {
174 const panel = page.slice(page.indexOf('<section id="tune"'), page.indexOf("</section>"));
175 expect(panel.match(/role="status"/g)?.length).toBe(2); // #seen (can it see the box) and #note (everything else)
176 expect(page).not.toContain('$("pi")');
177 expect(page).toMatch(/<noscript>[^<]*<p>[^<]*tune them/);
178 });
179 });
180
181 describe("the arm server's camera profiles", () => {
182 test("a unit outside 1-8 or a bad profile is refused before anything runs", async () => {
183 const { asked } = await setup();
184 const at = `http://127.0.0.1:${arm!.port}`;
185 expect((await fetch(`${at}/camera-profile/9`, { method: "PUT", body: "{}" })).status).toBe(400);
186 expect((await fetch(`${at}/camera-profile/..%2F1`, { method: "PUT", body: "{}" })).status).toBe(400);
187 expect((await fetch(`${at}/camera-profile/1`, { method: "PUT", body: JSON.stringify({ gain: -1 }) })).status).toBe(400);
188 expect(asked).toEqual([]);
189 expect(await (await fetch(`${at}/camera-profile/1`)).json()).toEqual({ unit: 1, saved: false, profile: { exposure: "auto", whiteBalance: "auto", backlight: 1 } });
190 expect(await (await fetch(`${at}/camera-profile/3`)).json()).toEqual({ unit: 3, saved: false, profile: {} });
191 });
192 });
Software/so101/src/cameras.ts +23 −3
@@ -9,12 +9,15 @@
99 import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
1010 import { dirname } from "node:path";
1111 import { CameraFeed, type FeedOptions } from "./camera-feed.ts";
12 import { changeLook, defaultLook, readLook, type CameraLook } from "./camera-look.ts";
1213
1314 export interface CameraSpec {
1415 id: number;
1516 name: string;
1617 /** The stream's URL, normalised: scheme, host, port, path. */
1718 source: string;
19 /** Its profile on the Pi, its region and its colour correction (src/camera-look.ts). */
20 look: CameraLook;
1821 }
1922
2023 export interface CameraInfo extends CameraSpec {
@@ -69,7 +72,9 @@ export class CameraHub {
6972 if (this.file && existsSync(this.file)) {
7073 try {
7174 const saved = JSON.parse(readFileSync(this.file, "utf8")) as { next?: number; cameras?: CameraSpec[] };
72 this.specs = (saved.cameras ?? []).filter((c) => Number.isSafeInteger(c.id) && typeof c.name === "string" && typeof c.source === "string");
75 this.specs = (saved.cameras ?? [])
76 .filter((c) => Number.isSafeInteger(c.id) && typeof c.name === "string" && typeof c.source === "string")
77 .map((c) => ({ id: c.id, name: c.name, source: c.source, look: readLook(c.look, c.source) }));
7378 this.next = Math.max(saved.next ?? 1, ...this.specs.map((c) => c.id + 1));
7479 } catch (err) {
7580 console.error(`cameras: ${this.file} unreadable (${(err as Error).message}); starting from the default`);
@@ -81,7 +86,7 @@ export class CameraHub {
8186 private seed(seed: HubOptions["seed"]): void {
8287 this.specs = [];
8388 this.next = 1;
84 for (const s of seed ?? []) this.specs.push({ id: this.next++, name: s.name, source: s.source });
89 for (const s of seed ?? []) this.specs.push({ id: this.next++, name: s.name, source: s.source, look: defaultLook(s.source) });
8590 this.save();
8691 }
8792
@@ -98,12 +103,27 @@ export class CameraHub {
98103
99104 add(name: unknown, source: unknown): CameraSpec {
100105 if (this.specs.length >= MAX_CAMERAS) throw new CameraError(`at most ${MAX_CAMERAS} cameras`);
101 const spec = { id: this.next++, name: parseName(name), source: parseSource(source) };
106 const at = parseSource(source);
107 const spec = { id: this.next++, name: parseName(name), source: at, look: defaultLook(at) };
102108 this.specs.push(spec);
103109 this.save();
104110 return spec;
105111 }
106112
113 /** Change a camera's look and keep it; null for an unknown camera. Throws a ProfileError for a bad change. */
114 setLook(id: number, change: unknown): CameraLook | null {
115 const spec = this.specs.find((c) => c.id === id);
116 if (!spec) return null;
117 spec.look = changeLook(spec.look, change);
118 this.save();
119 return spec.look;
120 }
121
122 /** A listed camera's look, or null when unknown. */
123 look(id: number): CameraLook | null {
124 return this.specs.find((c) => c.id === id)?.look ?? null;
125 }
126
107127 remove(id: number): boolean {
108128 const at = this.specs.findIndex((c) => c.id === id);
109129 if (at < 0) return false;
Software/so101/src/cli.ts +25 −1
@@ -1,4 +1,7 @@
11 import { closeSync, openSync, readFileSync, renameSync, writeFileSync } from "node:fs";
2 import { dirname } from "node:path";
3 import { applyProfile, cameraDevice, loadProfile, parseUnit, profileFile } from "./camera-control.ts";
4 import { defaultProfile, parseDevice } from "./camera-profile.ts";
25 import { parseArgs } from "node:util";
36 import { SerialPort } from "serialport";
47 import { FeetechBus, range } from "./bus.ts";
@@ -47,6 +50,9 @@ usage: so101 <command> [options]
4750 pose list | pose delete <name>
4851 serve [--listen 0.0.0.0:8101] [--file poses.json] [--model <dir with so101_new_calib.urdf>]
4952 HTTP API + WebSocket target stream for this arm (run where the follower is)
53 camera-profile <n> [--file poses.json] [--device /dev/v4l/by-id/…]
54 put the profile saved on /admin on so101-camera@n's camera (its ExecStartPre);
55 the device is read from /etc/so101/camera<n> unless given
5056 leader --to ws://host:8101 [--hz 60]
5157 read this leader arm and drive a remote follower through its serve
5258 setup [--dry-run] guided: find both arms, reuse calibration, then drive
@@ -85,6 +91,7 @@ const OPTIONS = {
8591 to: { type: "string" },
8692 model: { type: "string" },
8793 ssh: { type: "string" },
94 device: { type: "string" },
8895 } as const;
8996
9097 interface Values {
@@ -111,6 +118,7 @@ interface Values {
111118 model?: string;
112119 to?: string;
113120 ssh?: string;
121 device?: string;
114122 }
115123
116124 type Command = (values: Values, positionals: string[]) => Promise<void>;
@@ -522,7 +530,8 @@ const COMMANDS: Record<string, Command> = {
522530 const port = bounded(integer(portText ?? "8101", "--listen port"), 1, 65535, "--listen port");
523531 return withBus(values, async (bus) => {
524532 const { startServer } = await import("./server.ts"); // lazily: it reaches bun:sqlite, which Node lacks
525 const server = await startServer(bus, { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000 });
533 const cameraControl = (unit: number, profile: Parameters<typeof applyProfile>[1]) => applyProfile(cameraDevice(unit), profile);
534 const server = await startServer(bus, { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000, cameraControl });
526535 console.log(`serving ${bus.path} on http://${host}:${server.port} — open it in a browser; the API is listed in src/server.ts`);
527536 console.log("ctrl-c to stop");
528537 await new Promise<void>((resolve) => process.once("SIGINT", () => resolve()));
@@ -532,6 +541,21 @@ const COMMANDS: Record<string, Command> = {
532541 });
533542 },
534543
544 "camera-profile": async (values, [unitText]) => {
545 if (unitText === undefined) throw new Error("usage: so101 camera-profile <n> [--file poses.json] [--device /dev/v4l/by-id/…]");
546 const unit = parseUnit(unitText);
547 const file = profileFile(dirname(values.file ?? "poses.json"), unit);
548 const saved = loadProfile(file);
549 // Until the owner saves one, the profile they chose by hand on 2026-10-26 (#584); a camera with neither streams as it likes.
550 const profile = saved ?? defaultProfile(unit);
551 if (!Object.keys(profile).length) return console.log(`no profile saved for camera ${unit} (${file}); leaving the camera as it is`);
552 if (!saved) console.log(`no profile saved for camera ${unit} (${file}); using the owner's default`);
553 const device = values.device === undefined ? cameraDevice(unit) : parseDevice(values.device);
554 const { set, failed } = await applyProfile(device, profile);
555 console.log(`camera ${unit} (${device}): set ${set.join(" ") || "nothing"}`);
556 for (const f of failed) console.log(` not set: ${f.control} (${f.error})`);
557 },
558
535559 leader: (values) => {
536560 if (!values.to) throw new Error("usage: so101 leader --to ws://host:8101 [--hz 60]");
537561 const hz = bounded(integer(values.hz ?? "60", "--hz"), 1, 120, "--hz");
Software/so101/src/relay.ts +68 −7
@@ -53,6 +53,10 @@
5353 * GET /story, /story/arm.webp the story of the arm for anyone, docs/story.md with the camera and the week's scorecard; its preview photo (src/story.ts)
5454 * GET /admin/camera-list the owner's page for the camera list
5555 * POST /admin/cameras {"name", "source"}, DELETE /admin/cameras/<id> the owner changes the list; viewers see it within seconds
56 * PUT /admin/cameras/<id> {"profile"?, "unit"?, "region"?, "gains"?} the owner's Tune panel saves a camera's look (src/camera-look.ts);
57 * a profile goes on to the Pi's camera and is kept there (the arm server's PUT /camera-profile/<unit>)
58 * POST /admin/cameras/<id>/try {"profile"} put a profile on the Pi's camera without keeping it: the panel's sliders, live
59 * (the arm server's own /camera-profile/ is never passed through: only these two reach it, for the owner)
5660 * Resting (the owner closed it, or a real arm unheard for two minutes; src/opening.ts): / is the resting card
5761 * (src/resting-page.ts), /status answers {"open": false, "reason", "note", "reopenAt", "lastOpenAt"} without asking the
5862 * arm, every turn has ended, the cameras and /follow are refused (and those open are closed), and nothing goes to the arm
@@ -64,6 +68,7 @@
6468 * turn or no turn: GET answers it or null, PUT {"joints": {...}} stores it
6569 */
6670
71 import { createHash } from "node:crypto";
6772 import { readFileSync } from "node:fs";
6873 import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
6974 import type { Duplex } from "node:stream";
@@ -97,6 +102,7 @@ import { readTuning } from "./tuning-panel.ts";
97102 import { readServos } from "./servo-panel.ts";
98103 import { parseFps, type FeedOptions } from "./camera-feed.ts";
99104 import { CameraError, CameraHub } from "./cameras.ts";
105 import { parseProfile, ProfileError, type CameraProfile } from "./camera-profile.ts";
100106 import { VideoFeed, type VideoSources } from "./video-feed.ts";
101107 import type { VideoCard } from "./video-panel.ts";
102108 import { LADDER } from "./video-fanout.ts";
@@ -189,8 +195,15 @@ const MAX_BODY = 1_000_000;
189195 const VIDEO = /^\/camera\/(\d{1,9})\/video$/;
190196 const CAMERA_WS = /^\/camera\/(\d{1,9})\/ws$/;
191197 const CAMERA = /^\/camera\/(\d{1,9})\/(stream|state|snapshot)$/;
192 const ADMIN_CAMERA = /^\/admin\/cameras(?:\/(\d{1,9}))?$/;
193 const CAMERA_PAGE = readFileSync(new URL("./camera-list.html", import.meta.url), "utf8");
198 const ADMIN_CAMERA = /^\/admin\/cameras(?:\/(\d{1,9})(\/try)?)?$/;
199 /** The cameras page, with the still-frame module under a name that changes with its content (the arm server's /ui/ takes any). */
200 const STILL_FRAME = new URL("./ui/still-frame.js", import.meta.url);
201 const CAMERA_PAGE = readFileSync(new URL("./camera-list.html", import.meta.url), "utf8").replaceAll(
202 "/ui/__VERSION__/",
203 `/ui/${createHash("sha256").update(readFileSync(STILL_FRAME)).digest("hex").slice(0, 12)}/`,
204 );
205 /** The arm server's camera profiles: the relay asks for them itself, for the owner, and passes nobody's request on. */
206 const CAMERA_PROFILE = /^\/camera-profile(?:\/|$)/;
194207 /** Served from this process whichever arm is live: they describe the app, not the arm. */
195208 const LOCAL = /^\/(?:index\.html|leader-map\.js)?$|^\/(?:model|vendor|printables|ui|app)(?:\/|$)/;
196209 /** The page itself, which the relay writes the first view's facts into. */
@@ -695,11 +708,57 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
695708 res.end(out);
696709 };
697710
698 /** The owner's changes to the camera list. */
699 const administer = async (req: IncomingMessage, res: ServerResponse, id: number | null) => {
711 /**
712 * Put a profile on so101-camera@<unit>'s camera through the arm server, kept there for the camera's next start unless only
713 * tried: what it set and what the camera refused, or why it could not.
714 */
715 const toPi = async (unit: number | null, profile: CameraProfile, keep: boolean): Promise<Record<string, unknown>> => {
716 if (unit === null) return { error: "this camera is not one of the Pi's (no so101-camera unit): set its unit on the Tune panel" };
717 try {
718 const answer = await fetch(`${remote && onArm() ? remote.http : simulation}/camera-profile/${unit}${keep ? "" : "?try=1"}`, {
719 method: "PUT",
720 headers: { "content-type": "application/json" },
721 body: JSON.stringify(profile),
722 signal: AbortSignal.timeout(Math.max(readTimeoutMs, 5000)),
723 cache: "no-store",
724 });
725 const body = (await answer.json().catch(() => ({}))) as Record<string, unknown>;
726 if (answer.status === 404) return { error: "the arm server on the Pi is older than camera profiles: deploy it" };
727 if (!answer.ok) return { error: typeof body.error === "string" ? body.error.slice(0, 200) : `the arm server answered ${answer.status}` };
728 const list = (v: unknown) => (Array.isArray(v) ? v.slice(0, 16) : []);
729 return { saved: body.saved === true, set: list(body.set).filter((c) => typeof c === "string"), failed: list(body.failed) };
730 } catch (err) {
731 return { error: `the arm server did not answer (${reason(err)})` };
732 }
733 };
734
735 /** The owner's changes to the camera list, and to each camera's look. */
736 const administer = async (req: IncomingMessage, res: ServerResponse, id: number | null, trying: boolean) => {
700737 const who = lobby ? lobby.who(req) : null;
701738 if (lobby && !who?.owner) return json(res, who ? 403 : 401, { error: "only the owner may change the cameras" });
702739 if (req.method === "GET" && id === null) return json(res, 200, cameras.list());
740 if (id !== null && (trying ? req.method === "POST" : req.method === "PUT")) {
741 const look = cameras.look(id);
742 if (!look) return json(res, 404, { error: "no such camera" });
743 let input: Record<string, unknown>;
744 try {
745 input = JSON.parse((await readBody(req)).toString("utf8") || "{}");
746 } catch {
747 return json(res, 400, { error: "send JSON" });
748 }
749 try {
750 if (trying) {
751 if (input === null || typeof input !== "object") throw new ProfileError("send {\"profile\"}");
752 return json(res, 200, { pi: await toPi(look.unit, parseProfile(input.profile), false) });
753 }
754 const saved = cameras.setLook(id, input)!;
755 const pi = input !== null && typeof input === "object" && Object.hasOwn(input, "profile") ? await toPi(saved.unit, saved.profile, true) : null;
756 return json(res, 200, { look: saved, pi });
757 } catch (err) {
758 if (err instanceof ProfileError) return json(res, 400, { error: err.message });
759 throw err;
760 }
761 }
703762 if (req.method === "POST" && id === null) {
704763 let input: { name?: unknown; source?: unknown };
705764 try {
@@ -714,8 +773,9 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
714773 throw err;
715774 }
716775 }
717 if (req.method === "DELETE" && id !== null) return cameras.remove(id) ? json(res, 200, { ok: true }) : json(res, 404, { error: "no such camera" });
718 return json(res, 405, { error: "GET or POST /admin/cameras, DELETE /admin/cameras/<id>" });
776 if (req.method === "DELETE" && id !== null && !trying) return cameras.remove(id) ? json(res, 200, { ok: true }) : json(res, 404, { error: "no such camera" });
777 if (trying) return json(res, 405, { error: "POST /admin/cameras/<id>/try" });
778 return json(res, 405, { error: "GET or POST /admin/cameras, PUT or DELETE /admin/cameras/<id>" });
719779 };
720780
721781 /** A person's own calibration: signed in is enough, it never touches the arm. */
@@ -805,7 +865,8 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
805865 return res.end(CAMERA_PAGE);
806866 }
807867 const cameraAdmin = ADMIN_CAMERA.exec(url.pathname);
808 if (cameraAdmin) return await administer(req, res, cameraAdmin[1] === undefined ? null : Number(cameraAdmin[1]));
868 if (cameraAdmin) return await administer(req, res, cameraAdmin[1] === undefined ? null : Number(cameraAdmin[1]), cameraAdmin[2] !== undefined);
869 if (CAMERA_PROFILE.test(url.pathname)) return json(res, 404, { error: "no such route" });
809870 if (lobby && url.pathname === "/admin/lift") return await handleLift(lobby, req, res, readBody);
810871 if (lobby && (url.pathname === "/admin" || url.pathname === "/admin/health")) return await handleAdmin(lobby, req, res, url, readBody, askPrecision, askServos, videoCards, options.piSync ? () => options.piSync!.versions() : undefined, tuneP);
811872 if (serveStory(req, res, url.pathname, { cameras: () => (restingNow() ? [] : cameras.list()), numbers: storyNumbers })) return;
Software/so101/src/server.ts +40
@@ -29,6 +29,10 @@
2929 * POST /play {"name": "wave", "rate"?: 1, "torque"?: 70, "release"?: false}
3030 * GET /calibration {"camera": {...}|null, "handeye": {...}|null, ...} what the vision kit published
3131 * PUT /calibration/<kind> store that JSON next to the poses (kind: camera, camera2, handeye, ...)
32 * GET /camera-profile/<n> {"unit", "saved", "profile"}: camera n's saved profile (src/camera-profile.ts), else the owner's default for it
33 * PUT /camera-profile/<n>[?try=1] {"exposure"?, "whiteBalance"?, "gain"?, "focus"?, "backlight"?, "brightness"?, "contrast"?, "saturation"?} put it on so101-camera@n's
34 * camera now, one control at a time, and keep it for the camera service's next start (?try=1: only put it on);
35 * {"saved", "set", "failed"}. Without camera control (the simulation) nothing runs: 501, and nothing is kept
3236 * GET /settings the safe zone: {"zone": "enforce"|"warn"|"off", "floorMarginMm", "baseMarginMm", "jointOffsetsDeg"},
3337 * and the safety limits of src/safety.ts (speed, acceleration, rate, dead-man, park, gripper force, joint limits)
3438 * PUT /settings change any of those; "servoGains" takes partial per-joint patches ({"2": {"p": 80}}), joints 1-5, 0-254
@@ -78,6 +82,8 @@ import { GainSync } from "./servo-gain-sync.ts";
7882 import { TUNABLE, ParamsError, factoryValues, readServoParams, recordFactory, validateWrite, writeServoParams } from "./servo-params.ts";
7983 import { WebSocketServer, type WebSocket } from "ws";
8084 import { browserModule } from "./browser-modules.ts";
85 import { defaultProfile, parseProfile, ProfileError, type CameraProfile } from "./camera-profile.ts";
86 import { loadProfile, parseUnit, profileFile, saveProfile, type Applied } from "./camera-control.ts";
8187 import { Clock, CommandGate, HELD_CLOSE, isRelease, sentAtOf } from "./command-gate.ts";
8288 import { FLOOD_CLOSE, FrameRate, frameBytes, MAX_FRAME_BYTES, TOO_BIG_CLOSE } from "./frame-limit.ts";
8389 import type { FeetechBus } from "./bus.ts";
@@ -136,6 +142,8 @@ export interface ServerOptions {
136142 now?: () => number;
137143 /** Longest /watch goes without sending an unchanged status, ms (200): keep it inside the relay link's staleMs. */
138144 watchQuietMs?: number;
145 /** Puts a profile on so101-camera@<unit>'s camera (src/camera-control.ts); only `so101 serve` on the Pi gives one, so the simulation never runs v4l2-ctl. */
146 cameraControl?: (unit: number, profile: CameraProfile) => Promise<Applied>;
139147 }
140148
141149 export interface RunningServer {
@@ -827,6 +835,38 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
827835 res.writeHead(200, { "content-type": "text/javascript; charset=utf-8", "cache-control": "no-cache", etag: served.etag });
828836 return res.end(served.js);
829837 }
838 const cameraProfile = /^\/camera-profile\/([^/]*)$/.exec(url.pathname);
839 if (cameraProfile) {
840 let unit: number;
841 try {
842 unit = parseUnit(cameraProfile[1]);
843 } catch (err) {
844 throw new HttpError(400, (err as Error).message);
845 }
846 if (req.method === "GET") {
847 const saved = loadProfile(profileFile(dataDir, unit));
848 return json(res, 200, { unit, saved: saved !== null, profile: saved ?? defaultProfile(unit) });
849 }
850 if (req.method !== "PUT") throw new HttpError(405, "GET or PUT /camera-profile/<n>");
851 let profile: CameraProfile;
852 try {
853 profile = parseProfile(await readBody(req));
854 } catch (err) {
855 if (err instanceof ProfileError) throw new HttpError(400, err.message);
856 throw err;
857 }
858 if (!options.cameraControl) return json(res, 501, { error: "no camera control here: this is the simulation, not the Pi", saved: false, set: [], failed: [] });
859 let applied: Applied;
860 try {
861 applied = await options.cameraControl(unit, profile);
862 } catch (err) {
863 if (err instanceof ProfileError) return json(res, 409, { error: err.message, saved: false, set: [], failed: [] });
864 throw err;
865 }
866 const keep = url.searchParams.get("try") !== "1";
867 if (keep) saveProfile(profileFile(dataDir, unit), profile);
868 return json(res, 200, { saved: keep, ...applied });
869 }
830870 if (req.method === "PUT" && url.pathname.startsWith("/calibration/")) {
831871 const kind = url.pathname.slice("/calibration/".length);
832872 if (!/^[a-z][a-z0-9_-]{0,31}$/.test(kind)) throw new HttpError(400, "calibration kind: lowercase letters, digits, - and _");
Software/so101/src/ui/still-frame.js added +201
@@ -0,0 +1,201 @@
1 /**
2 * What a still of the grow box needs before anyone looks at it: cropped to the region the owner drew (the box, not the
3 * room), colour-corrected for a blue/violet grow light, and checked for whether the plants can be seen at all.
4 * Plain ESM on RGBA pixels (a canvas's ImageData), served at /ui/ for the cameras page and imported by the tests and the relay.
5 *
6 * The region is kept in fractions of the frame, so one drawn on the 640×360 stream crops a 1920×1080 still the same way.
7 */
8
9 /** @typedef {{ x: number, y: number, w: number, h: number }} Rect */
10 /** @typedef {{ r: number, g: number, b: number }} Gains */
11
12 /** The hint the grow card and the Tune panel show when the region is mostly blown out or black. */
13 export const CANT_SEE = "Camera can't see the plants: move it inside or beside the box, below the light.";
14
15 /** A corrected channel is never pushed more than this: past it a blue-only light gives noise, not colour. */
16 export const MAX_GAIN = 4;
17 export const MIN_GAIN = 0.25;
18
19 /** Luma at or above this is blown out, at or below DARK is black. */
20 export const BLOWN = 245;
21 export const DARK = 16;
22
23 /**
24 * Any JSON → a region in fractions of the frame, or null for "the whole frame". Throws an Error naming what is wrong.
25 * @param {unknown} input
26 * @returns {Rect | null}
27 */
28 export function parseRegion(input) {
29 if (input === null || input === undefined) return null;
30 if (typeof input !== "object" || Array.isArray(input)) throw new Error("a region is {x, y, w, h}, fractions of the frame");
31 const o = /** @type {Record<string, unknown>} */ (input);
32 const keys = Object.keys(o).sort().join(",");
33 if (keys !== "h,w,x,y") throw new Error("a region is {x, y, w, h}, fractions of the frame");
34 const num = (/** @type {unknown} */ v) => (typeof v === "number" && Number.isFinite(v) ? v : NaN);
35 const [x, y, w, h] = [num(o.x), num(o.y), num(o.w), num(o.h)];
36 if (![x, y, w, h].every((v) => v >= 0 && v <= 1)) throw new Error("a region's x, y, w and h are each from 0 to 1");
37 if (w < 0.02 || h < 0.02 || x + w > 1.0001 || y + h > 1.0001) throw new Error("a region lies inside the frame and is at least 2% wide and tall");
38 const r4 = (/** @type {number} */ v) => Math.round(v * 10_000) / 10_000;
39 return { x: r4(x), y: r4(y), w: r4(Math.min(w, 1 - x)), h: r4(Math.min(h, 1 - y)) };
40 }
41
42 /**
43 * Two corners dragged on the picture, in fractions, → a region (either drag direction).
44 * @param {{ x: number, y: number }} a
45 * @param {{ x: number, y: number }} b
46 * @returns {Rect}
47 */
48 export function regionFromDrag(a, b) {
49 const clamp = (/** @type {number} */ v) => Math.min(1, Math.max(0, v));
50 const [x0, x1] = [Math.min(clamp(a.x), clamp(b.x)), Math.max(clamp(a.x), clamp(b.x))];
51 const [y0, y1] = [Math.min(clamp(a.y), clamp(b.y)), Math.max(clamp(a.y), clamp(b.y))];
52 return { x: x0, y: y0, w: x1 - x0, h: y1 - y0 };
53 }
54
55 /**
56 * A region → the four whole percentages the Tune panel's number inputs show (the whole frame for null).
57 * @param {Rect | null} region
58 * @returns {{ x: number, y: number, w: number, h: number }}
59 */
60 export function regionToPercent(region) {
61 const r = region ?? { x: 0, y: 0, w: 1, h: 1 };
62 const pct = (/** @type {number} */ v) => Math.round(v * 100);
63 return { x: pct(r.x), y: pct(r.y), w: pct(r.w), h: pct(r.h) };
64 }
65
66 /**
67 * The number inputs' percentages → a region, kept inside the frame (a box pushed past an edge is cut at it), or null
68 * when they are the whole frame. Throws an Error naming what is wrong, like parseRegion.
69 * @param {{ x: unknown, y: unknown, w: unknown, h: unknown }} pct
70 * @returns {Rect | null}
71 */
72 export function regionFromPercent(pct) {
73 const [x, y, w, h] = [pct.x, pct.y, pct.w, pct.h].map((v) => (typeof v === "number" ? v : typeof v === "string" && v.trim() ? Number(v) : NaN));
74 if (![x, y, w, h].every((v) => Number.isFinite(v))) throw new Error("the box's left, top, width and height are each a number of percent");
75 const fx = Math.min(1, Math.max(0, /** @type {number} */ (x) / 100));
76 const fy = Math.min(1, Math.max(0, /** @type {number} */ (y) / 100));
77 const fw = Math.min(1 - fx, /** @type {number} */ (w) / 100);
78 const fh = Math.min(1 - fy, /** @type {number} */ (h) / 100);
79 if (fx === 0 && fy === 0 && fw >= 1 && fh >= 1) return null;
80 return parseRegion({ x: fx, y: fy, w: fw, h: fh });
81 }
82
83 /**
84 * A region (fractions, or null for all of it) → whole pixels of a width×height frame, inside it and at least 1×1.
85 * @param {Rect | null} region
86 * @param {number} width
87 * @param {number} height
88 * @returns {Rect}
89 */
90 export function cropRect(region, width, height) {
91 if (!region) return { x: 0, y: 0, w: width, h: height };
92 const x = Math.min(width - 1, Math.max(0, Math.round(region.x * width)));
93 const y = Math.min(height - 1, Math.max(0, Math.round(region.y * height)));
94 const right = Math.min(width, Math.max(x + 1, Math.round((region.x + region.w) * width)));
95 const bottom = Math.min(height, Math.max(y + 1, Math.round((region.y + region.h) * height)));
96 return { x, y, w: right - x, h: bottom - y };
97 }
98
99 /**
100 * Visit each pixel of `rect` in RGBA `data` of a frame `width` pixels wide.
101 * @param {ArrayLike<number>} data
102 * @param {number} width
103 * @param {Rect} rect
104 * @param {(r: number, g: number, b: number) => void} visit
105 */
106 function each(data, width, rect, visit) {
107 for (let row = rect.y; row < rect.y + rect.h; row++) {
108 for (let col = rect.x, i = (row * width + col) * 4; col < rect.x + rect.w; col++, i += 4) visit(data[i] ?? 0, data[i + 1] ?? 0, data[i + 2] ?? 0);
109 }
110 }
111
112 const luma = (/** @type {number} */ r, /** @type {number} */ g, /** @type {number} */ b) => 0.2126 * r + 0.7152 * g + 0.0722 * b;
113
114 /**
115 * Gray-world: the scene averages to gray, so the gains that make each channel's mean the same are the light's colour
116 * undone. Blown-out and black pixels say nothing about the light and are left out. Null when too little is left to judge.
117 * @param {ArrayLike<number>} data RGBA
118 * @param {number} width
119 * @param {Rect} rect
120 * @returns {Gains | null}
121 */
122 export function grayWorldGains(data, width, rect) {
123 let [r, g, b, n] = [0, 0, 0, 0];
124 each(data, width, rect, (pr, pg, pb) => {
125 const top = Math.max(pr, pg, pb);
126 if (top >= 250 || top <= DARK) return;
127 r += pr;
128 g += pg;
129 b += pb;
130 n++;
131 });
132 if (n < Math.max(16, rect.w * rect.h * 0.05) || !r || !g || !b) return null;
133 const gray = (r + g + b) / 3;
134 const gain = (/** @type {number} */ sum) => Math.round(Math.min(MAX_GAIN, Math.max(MIN_GAIN, gray / sum)) * 1000) / 1000;
135 return { r: gain(r), g: gain(g), b: gain(b) };
136 }
137
138 /**
139 * Gains from a white card shot under the grow light: the card's mean, made white.
140 * @param {ArrayLike<number>} data RGBA
141 * @param {number} width
142 * @param {Rect} card the card's pixels
143 * @returns {Gains | null}
144 */
145 export function whiteCardGains(data, width, card) {
146 let [r, g, b, n] = [0, 0, 0, 0];
147 each(data, width, card, (pr, pg, pb) => ((r += pr), (g += pg), (b += pb), n++));
148 if (!n || !r || !g || !b) return null;
149 const top = Math.max(r, g, b);
150 const gain = (/** @type {number} */ sum) => Math.round(Math.min(MAX_GAIN, Math.max(MIN_GAIN, top / sum)) * 1000) / 1000;
151 return { r: gain(r), g: gain(g), b: gain(b) };
152 }
153
154 /**
155 * Any JSON → gains, or null for "uncorrected". Throws an Error naming what is wrong.
156 * @param {unknown} input
157 * @returns {Gains | null}
158 */
159 export function parseGains(input) {
160 if (input === null || input === undefined) return null;
161 const o = /** @type {Record<string, unknown>} */ (input);
162 if (typeof input !== "object" || Array.isArray(input) || Object.keys(o).sort().join(",") !== "b,g,r") throw new Error("a colour correction is {r, g, b}");
163 const ok = (/** @type {unknown} */ v) => typeof v === "number" && v >= MIN_GAIN && v <= MAX_GAIN;
164 if (!ok(o.r) || !ok(o.g) || !ok(o.b)) throw new Error(`each colour gain is from ${MIN_GAIN} to ${MAX_GAIN}`);
165 return { r: /** @type {number} */ (o.r), g: /** @type {number} */ (o.g), b: /** @type {number} */ (o.b) };
166 }
167
168 /**
169 * Multiply each pixel's channels by the gains, in place; alpha is left alone.
170 * @param {Uint8ClampedArray | number[]} data RGBA
171 * @param {Gains} gains
172 */
173 export function applyGains(data, gains) {
174 const clamp = (/** @type {number} */ v) => (v > 255 ? 255 : v < 0 ? 0 : Math.round(v));
175 for (let i = 0; i + 3 < data.length; i += 4) {
176 data[i] = clamp((data[i] ?? 0) * gains.r);
177 data[i + 1] = clamp((data[i + 1] ?? 0) * gains.g);
178 data[i + 2] = clamp((data[i + 2] ?? 0) * gains.b);
179 }
180 }
181
182 /**
183 * How much of the region is blown out or black, and whether the plants could be seen there.
184 * @param {ArrayLike<number>} data RGBA
185 * @param {number} width
186 * @param {Rect} rect
187 * @returns {{ blown: number, dark: number, verdict: "ok" | "blown" | "dark" }}
188 */
189 export function visibility(data, width, rect) {
190 let [blown, dark, n] = [0, 0, 0];
191 each(data, width, rect, (r, g, b) => {
192 const y = luma(r, g, b);
193 if (y >= BLOWN) blown++;
194 else if (y <= DARK) dark++;
195 n++;
196 });
197 const [fb, fd] = n ? [blown / n, dark / n] : [0, 1];
198 const round = (/** @type {number} */ v) => Math.round(v * 1000) / 1000;
199 const verdict = fb >= 0.5 || (fb + fd >= 0.7 && fb >= fd) ? "blown" : fd >= 0.6 || fb + fd >= 0.7 ? "dark" : "ok";
200 return { blown: round(fb), dark: round(fd), verdict };
201 }
Software/so101/src/ui/still-frame.test.ts added +121
@@ -0,0 +1,121 @@
1 import { describe, expect, test } from "bun:test";
2 import { applyGains, CANT_SEE, cropRect, grayWorldGains, parseGains, parseRegion, regionFromDrag, regionFromPercent, regionToPercent, visibility, whiteCardGains } from "./still-frame.js";
3
4 /** A width×height RGBA frame painted by `at(x, y)`. */
5 function frame(width: number, height: number, at: (x: number, y: number) => [number, number, number]): Uint8ClampedArray {
6 const data = new Uint8ClampedArray(width * height * 4);
7 for (let y = 0; y < height; y++) for (let x = 0; x < width; x++) data.set([...at(x, y), 255], (y * width + x) * 4);
8 return data;
9 }
10
11 /**
12 * The grow box as the side camera saw it on 2026-10-26: a blown-out light panel along the top, the box in the middle
13 * (gray tray, green leaves) lit blue/violet, and the dark room around it. 64×36, a tenth of the 640×360 stream.
14 */
15 const W = 64;
16 const H = 36;
17 const BOX = { x: 24, y: 12, w: 20, h: 16 };
18 const growBox = frame(W, H, (x, y) => {
19 if (y < 6) return [255, 255, 255]; // the panel
20 const inBox = x >= BOX.x && x < BOX.x + BOX.w && y >= BOX.y && y < BOX.y + BOX.h;
21 if (!inBox) return [8, 8, 12]; // the room
22 const leaf = (x + y) % 3 === 0;
23 // what a blue/violet light makes of a gray tray and green leaves: little red, little green, a lot of blue
24 return leaf ? [30, 70, 160] : [50, 40, 190];
25 });
26
27 describe("the crop", () => {
28 test("a region in fractions crops the stream and a full-resolution still to the same part of the scene", () => {
29 const region = { x: 0.375, y: 1 / 3, w: 0.3125, h: 4 / 9 };
30 expect(cropRect(region, 64, 36)).toEqual(BOX);
31 expect(cropRect(region, 640, 360)).toEqual({ x: 240, y: 120, w: 200, h: 160 });
32 expect(cropRect(region, 1920, 1080)).toEqual({ x: 720, y: 360, w: 600, h: 480 });
33 expect(cropRect(null, 1920, 1080)).toEqual({ x: 0, y: 0, w: 1920, h: 1080 });
34 });
35
36 test("a crop never leaves the frame and is never empty", () => {
37 expect(cropRect({ x: 0.99, y: 0.99, w: 0.01, h: 0.01 }, 10, 10)).toEqual({ x: 9, y: 9, w: 1, h: 1 });
38 expect(cropRect({ x: 0.5, y: 0, w: 0.6, h: 1 }, 100, 50)).toEqual({ x: 50, y: 0, w: 50, h: 50 });
39 });
40
41 test("a drag in either direction is the same region", () => {
42 expect(regionFromDrag({ x: 0.7, y: 0.8 }, { x: 0.2, y: 0.1 })).toEqual(regionFromDrag({ x: 0.2, y: 0.1 }, { x: 0.7, y: 0.8 }));
43 expect(regionFromDrag({ x: -1, y: 0.5 }, { x: 2, y: 0.5 })).toEqual({ x: 0, y: 0.5, w: 1, h: 0 });
44 });
45
46 test("the keyboard's four percentages and a drawn region are the same box", () => {
47 expect(regionFromPercent({ x: "30", y: "25", w: "40", h: "50" })).toEqual({ x: 0.3, y: 0.25, w: 0.4, h: 0.5 });
48 expect(regionToPercent({ x: 0.3, y: 0.25, w: 0.4, h: 0.5 })).toEqual({ x: 30, y: 25, w: 40, h: 50 });
49 expect(regionToPercent(null)).toEqual({ x: 0, y: 0, w: 100, h: 100 });
50 expect(regionFromPercent({ x: 0, y: 0, w: 100, h: 100 })).toBeNull();
51 expect(regionFromPercent({ x: 80, y: 0, w: 50, h: 100 })).toEqual({ x: 0.8, y: 0, w: 0.2, h: 1 }); // cut at the edge
52 for (const bad of [{ x: "", y: 0, w: 10, h: 10 }, { x: "a", y: 0, w: 10, h: 10 }, { x: 0, y: 0, w: 1, h: 10 }, { x: 100, y: 0, w: 10, h: 10 }]) expect(() => regionFromPercent(bad)).toThrow();
53 });
54
55 test("a region from outside is checked: inside the frame, at least 2% each way, nothing else in it", () => {
56 expect(parseRegion({ x: 0.25, y: 0.25, w: 0.5, h: 0.5 })).toEqual({ x: 0.25, y: 0.25, w: 0.5, h: 0.5 });
57 expect(parseRegion(null)).toBeNull();
58 for (const bad of [[], "x", { x: 0, y: 0, w: 1 }, { x: 0, y: 0, w: 1, h: 1, z: 1 }, { x: -0.1, y: 0, w: 0.5, h: 0.5 }, { x: 0.6, y: 0, w: 0.6, h: 0.5 }, { x: 0, y: 0, w: 0.01, h: 0.5 }, { x: "0", y: 0, w: 1, h: 1 }]) {
59 expect(() => parseRegion(bad)).toThrow();
60 }
61 });
62 });
63
64 describe("colour under the grow light", () => {
65 test("gray-world on the box takes the blue cast out: the corrected tray is near gray and the leaves turn greener than the tray", () => {
66 const gains = grayWorldGains(growBox, W, BOX)!;
67 expect(gains.b).toBeLessThan(1);
68 expect(gains.r).toBeGreaterThan(1);
69 const tray = [50, 40, 190];
70 const leaf = [30, 70, 160];
71 const corrected = [...tray, 255, ...leaf, 255];
72 applyGains(corrected, gains);
73 const [tr, tg, tb] = corrected;
74 expect(Math.max(tr!, tg!, tb!) - Math.min(tr!, tg!, tb!)).toBeLessThan(Math.max(...tray) - Math.min(...tray)); // closer to gray
75 expect(corrected[5]! / corrected[6]!).toBeGreaterThan(tg! / tb!); // the leaf's green stands out from the tray
76 expect(corrected[3]).toBe(255); // alpha untouched
77 });
78
79 test("the panel and the room do not steer the correction; with nothing left to judge there is none", () => {
80 expect(grayWorldGains(growBox, W, { x: 0, y: 0, w: W, h: H })).toEqual(grayWorldGains(growBox, W, BOX));
81 expect(grayWorldGains(growBox, W, { x: 0, y: 0, w: W, h: 6 })).toBeNull();
82 });
83
84 test("no gain passes the cap, however little red a violet light leaves", () => {
85 const violet = frame(8, 8, () => [1, 20, 240]);
86 const gains = grayWorldGains(violet, 8, { x: 0, y: 0, w: 8, h: 8 })!;
87 expect(gains.r).toBe(4);
88 expect(gains.b).toBeGreaterThanOrEqual(0.25);
89 });
90
91 test("a white card under the light becomes white", () => {
92 const card = frame(4, 4, () => [80, 100, 200]);
93 const gains = whiteCardGains(card, 4, { x: 0, y: 0, w: 4, h: 4 })!;
94 expect(gains).toEqual({ r: 2.5, g: 2, b: 1 });
95 applyGains(card, gains);
96 expect([...card.slice(0, 3)]).toEqual([200, 200, 200]);
97 });
98
99 test("gains from outside are checked", () => {
100 expect(parseGains({ r: 1.2, g: 1, b: 0.5 })).toEqual({ r: 1.2, g: 1, b: 0.5 });
101 expect(parseGains(undefined)).toBeNull();
102 for (const bad of [{ r: 5, g: 1, b: 1 }, { r: 1, g: 1 }, { r: 1, g: 1, b: 1, a: 1 }, [1, 1, 1], { r: "1", g: 1, b: 1 }]) expect(() => parseGains(bad)).toThrow();
103 });
104 });
105
106 describe("can the camera see the plants", () => {
107 test("the overhead camera's view, the top of the light panel, is flagged as blown out", () => {
108 const overhead = frame(W, H, (x, y) => (Math.hypot(x - 32, y - 18) < 30 ? [255, 250, 255] : [120, 90, 200]));
109 expect(visibility(overhead, W, { x: 0, y: 0, w: W, h: H }).verdict).toBe("blown");
110 });
111
112 test("the side camera's region on the box is fine; the whole frame of the dark room is not", () => {
113 expect(visibility(growBox, W, BOX)).toEqual({ blown: 0, dark: 0, verdict: "ok" });
114 const night = frame(W, H, () => [6, 6, 10]);
115 expect(visibility(night, W, cropRect(null, W, H)).verdict).toBe("dark");
116 });
117
118 test("the hint says what to do", () => {
119 expect(CANT_SEE).toBe("Camera can't see the plants: move it inside or beside the box, below the light.");
120 });
121 });

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs