trevato/arm

Public: anyone can read this; the crew can change it.

Migrated from github.com/Trevato/SO-ARM100

workbenchthis machine
More
Mapg aCommitsg mBranchesg bRunsg rEventsFlows

Grow stills at 1920×1080, cropped to the camera's region and colour-corrected (split from #584) #603

Merged vespercrew opened this

Check · Review · Land done · Live — Landed; nothing deploys from this branch, so this is the end of its path.

ConversationFiles
Merged

vesper/arm:vesper/589-grow-stills-at-1920-1080-cropped-to-the → main 1 commit fork

  • Merged. vesper/589-grow-stills-at-1920-1080-cropped-to-the was merged into main as d2bd69c.

Review

Sign in to review this change.
13 files changed +585 −33
Software/README.md +3 −1
@@ -1407,4 +1407,6 @@ The methods behind the vision kit, so you can read the originals:
14071407 model follows; [three.js](https://threejs.org) and
14081408 [urdf-loader](https://github.com/gkjohnson/urdf-loaders) for the 3D view;
14091409 the [Web Serial API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API)
1410 for the leader arm in the browser; [µStreamer](https://github.com/pikvm/ustreamer) for the cameras.
1410 for the leader arm in the browser; [µStreamer](https://github.com/pikvm/ustreamer) for the cameras;
1411 [jpeg-js](https://github.com/jpeg-js/jpeg-js) (pure JavaScript, no dependencies of its own) to crop and
1412 colour-correct the grow box's stills on the relay, which has no other way to decode a JPEG.
Software/so101/README.md +10 −1
@@ -93,7 +93,16 @@ The app is a workshop: one Pi, several **stations**, each with its cameras and c
9393
9494 The hosted app takes a still from the grow station's first camera every 10 minutes (`GROW_STILL_MINUTES`, 1 to 1440) and keeps it under `FORGE_APP_DATA/grow-stills`: 30 days, and no more than `GROW_STILLS_CAP_MB` (500) in all, the oldest going first. A still taken while the light is off is flagged dark and left out of the timelapse. The card plays today, the last 7 days or all of it, scrubbable, built in the browser from the stills (nothing is encoded). Under it: day N since the planted date, the owner's notes, light hours per day from the light's transitions, and "canopy (green area)" per day, the share of green pixels inside the box drawn in Tune. That is a measure of green area, not of health; the owner's browser works it out for stills that have none and sends it back.
9595
96 `GET /grow/stills?from=&to=&after=&limit=&dark=1` pages stills (`{stills, next, count, bytes, capBytes}`), `GET /grow/stills/<ms>.jpg` one still, `GET /grow/log` the log and chart numbers; `POST /grow/log {planted?, note?}` and `POST /grow/canopy {readings}` are the owner's, and same-origin only. The chart is empty until the owner has opened the card once, since only that browser works out the readings. Without JavaScript the card shows the latest still and a link to `/grow/log`. From the terminal: `/grow log` reads the log, `/grow planted <YYYY-MM-DD>|none` and `/grow note <text>` (owner) write it through the same routes; the card shows a note on its next load.
96 **Full size, cropped, corrected.** When the grow camera is one of the Pi's (Tune shows its unit, `so101-camera@<n>`) and the arm is online, each still is grabbed at 1920×1080 (or the camera's nearest size), not taken from the stream. ustreamer holds the camera, so the arm server pauses that camera's stream for the grab: `POST /camera-still/<n>` stops `so101-camera@<n>`, takes one MJPEG frame with `v4l2-ctl` after 15 frames to settle, and starts the stream again whatever happened (src/camera-still.ts). The stream comes back at its own resolution, 640×360 on the grow camera, after a gap of about two seconds every ten minutes. The relay then crops the still to the box drawn on Tune, with `cropRect(look.region, width, height)` on the still's own size, and keeps that crop. When the camera has colour gains (*Measure colour* on Tune), a copy with the gains applied is kept beside it, and the card shows that copy with "corrected" beside its time. The **Corrected** button switches back to the raw colours. The raw still is never corrected; with no region it is the camera's JPEG byte for byte. Decoding a 1080p JPEG takes about half a second, so the relay does it in a worker (src/grow-frame.ts) and pages are not held up. The codec is jpeg-js, the one dependency this added: pure JavaScript with none of its own, since neither Bun nor Node can decode a JPEG. If the Pi cannot grab a still (no unit, arm offline, no sudo rule), the stream's own snapshot is used and cropped the same way. The pause needs the arm server's user to be allowed to stop and start the camera units, and nothing else:
97
98 ```sh
99 echo 'so101 ALL=(root) NOPASSWD: /usr/bin/systemctl stop [email protected], /usr/bin/systemctl start [email protected], /usr/bin/systemctl stop [email protected], /usr/bin/systemctl start [email protected]' \
100 | sudo tee /etc/sudoers.d/so101-camera-still && sudo chmod 440 /etc/sudoers.d/so101-camera-still && sudo visudo -c
101 ```
102
103 (Replace `so101` with the user `so101.service` runs as.) The white photo light around each still (#589's optional part) is not done: the grow light's switch has no photo light yet.
104
105 `GET /grow/stills?from=&to=&after=&limit=&dark=1` pages stills (`{stills, next, count, bytes, capBytes}`; each still says `cropped` and `corrected`), `GET /grow/stills/<ms>.jpg` one still, `GET /grow/stills/<ms>-corrected.jpg` its corrected copy, `GET /grow/log` the log and chart numbers; `POST /grow/log {planted?, note?}` and `POST /grow/canopy {readings}` are the owner's, and same-origin only. The chart is empty until the owner has opened the card once, since only that browser works out the readings. Without JavaScript the card shows the latest still and a link to `/grow/log`. From the terminal: `/grow log` reads the log, `/grow planted <YYYY-MM-DD>|none` and `/grow note <text>` (owner) write it through the same routes; the card shows a note on its next load.
97106
98107 ## The story page: /story
99108
Software/so101/bun.lock +3
@@ -5,6 +5,7 @@
55 "": {
66 "name": "so101",
77 "dependencies": {
8 "jpeg-js": "^0.4.4",
89 "serialport": "^13",
910 "three": "^0.186.1",
1011 "urdf-loader": "^0.13.1",
@@ -96,6 +97,8 @@
9697
9798 "debug": ["[email protected]", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-6WTZ/IxCY/T6BALoZHaE4ctp9xm+Z5kY/pzYaCHRFeyVhojxlrm+46y68HA6hr0TcwEssoxNiDEUJQjfPZ/RYA=="],
9899
100 "jpeg-js": ["[email protected]", "", {}, "sha512-WZzeDOEtTOBK4Mdsar0IqEU5sMr3vSV2RqkAIzUEV2BHnUfKGyswWFPFwK5EeDo93K3FohSHbLAjj0s1Wzd+dg=="],
101
99102 "ms": ["[email protected]", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
100103
101104 "node-addon-api": ["[email protected]", "", {}, "sha512-8VOpLHFrOQlAH+qA0ZzuGRlALRA6/LVh8QJldbrC4DY0hXoMP0l4Acq8TzFC018HztWiRqyCEj2aTWY2UvnJUg=="],
Software/so101/package.json +1
@@ -13,6 +13,7 @@
1313 "test": "bun test"
1414 },
1515 "dependencies": {
16 "jpeg-js": "^0.4.4",
1617 "serialport": "^13",
1718 "three": "^0.186.1",
1819 "urdf-loader": "^0.13.1",
Software/so101/src/camera-still.ts added +81
@@ -0,0 +1,81 @@
1 /**
2 * The Pi's side of a grow still at full size. ustreamer holds the camera, so a 1920×1080 still pauses the stream:
3 * stop so101-camera@n, grab one MJPEG frame at 1920×1080 with `v4l2-ctl`, start the stream again. The stream comes
4 * back at its own --resolution (640×360 on the grow camera), and it is started again however the grab went.
5 *
6 * A camera without 1920×1080 gets its nearest size from the driver; the relay reads the size from the JPEG, so the
7 * crop still lands on the region. Every command is an argument list built from a checked unit number and the device
8 * path from /etc/so101/camera<n>, never a shell. Stopping and starting the unit needs sudo for exactly those two
9 * commands (the README has the sudoers lines); `sudo -n` fails at once rather than waiting for a password.
10 */
11 import { execFile } from "node:child_process";
12 import { mkdtempSync, readFileSync, rmSync } from "node:fs";
13 import { tmpdir } from "node:os";
14 import { join } from "node:path";
15 import { parseUnit } from "./camera-control.ts";
16
17 export const STILL_WIDTH = 1920;
18 export const STILL_HEIGHT = 1080;
19 /** Frames thrown away while the camera settles after it opens: the first ones of a webcam are dark or soft. */
20 export const SETTLE_FRAMES = 15;
21
22 /** Runs one command; resolves with its error text, or "" when it worked. */
23 export type Run = (command: string, args: string[], timeoutMs: number) => Promise<string>;
24
25 export const run: Run = (command, args, timeoutMs) =>
26 new Promise((resolve) => {
27 execFile(command, args, { timeout: timeoutMs }, (err, _stdout, stderr) => {
28 if (!err) return resolve("");
29 const code = (err as NodeJS.ErrnoException).code;
30 resolve(code === "ENOENT" ? `${command} is not installed` : (stderr || err.message).trim().split("\n")[0]!.slice(0, 200));
31 });
32 });
33
34 export class StillError extends Error {
35 /** Another still was being taken: try again later, nothing is wrong with the camera. */
36 readonly busy: boolean;
37 constructor(message: string, busy = false) {
38 super(message);
39 this.busy = busy;
40 }
41 }
42
43 /** The commands, in order: pause the stream, grab into `file`, start the stream. */
44 export function stillCommands(unit: number, device: string, file: string): { stop: [string, string[]]; grab: [string, string[]]; start: [string, string[]] } {
45 const service = `so101-camera@${parseUnit(unit)}.service`;
46 return {
47 stop: ["sudo", ["-n", "systemctl", "stop", service]],
48 grab: ["v4l2-ctl", ["-d", device, `--set-fmt-video=width=${STILL_WIDTH},height=${STILL_HEIGHT},pixelformat=MJPG`, "--stream-mmap=3", `--stream-skip=${SETTLE_FRAMES}`, "--stream-count=1", `--stream-to=${file}`]],
49 start: ["sudo", ["-n", "systemctl", "start", service]],
50 };
51 }
52
53 let busy = false;
54
55 /**
56 * One 1920×1080 JPEG from camera `unit` at `device`, the stream paused around it. Throws a StillError saying what
57 * went wrong; the stream is started again either way. One at a time: a second call while one runs is refused.
58 */
59 export async function grabStill(unit: number, device: string, exec: Run = run): Promise<Uint8Array> {
60 if (busy) throw new StillError("a still is already being taken", true);
61 busy = true;
62 const dir = mkdtempSync(join(tmpdir(), "so101-still-"));
63 const c = stillCommands(unit, device, join(dir, "still.jpg"));
64 try {
65 const stopped = await exec(...c.stop, 10_000);
66 if (stopped) throw new StillError(`could not pause the stream: ${stopped}`);
67 try {
68 const grabbed = await exec(...c.grab, 10_000);
69 if (grabbed) throw new StillError(`could not grab a still: ${grabbed}`);
70 const bytes = new Uint8Array(readFileSync(join(dir, "still.jpg")));
71 if (bytes.length < 4 || bytes[0] !== 0xff || bytes[1] !== 0xd8) throw new StillError("the camera gave no JPEG (is it set to MJPEG?)");
72 return bytes;
73 } finally {
74 const started = await exec(...c.start, 10_000);
75 if (started) console.error(`camera still: could not start so101-camera@${unit} again: ${started}`);
76 }
77 } finally {
78 busy = false;
79 rmSync(dir, { recursive: true, force: true });
80 }
81 }
Software/so101/src/cli.ts +3 −1
@@ -1,6 +1,7 @@
11 import { closeSync, openSync, readFileSync, renameSync, writeFileSync } from "node:fs";
22 import { dirname } from "node:path";
33 import { applyProfile, cameraDevice, loadProfile, parseUnit, profileFile } from "./camera-control.ts";
4 import { grabStill } from "./camera-still.ts";
45 import { defaultProfile, parseDevice } from "./camera-profile.ts";
56 import { parseArgs } from "node:util";
67 import { SerialPort } from "serialport";
@@ -534,7 +535,8 @@ const COMMANDS: Record<string, Command> = {
534535 return withBus(values, async (bus) => {
535536 const { startServer } = await import("./server.ts"); // lazily: it reaches bun:sqlite, which Node lacks
536537 const cameraControl = (unit: number, profile: Parameters<typeof applyProfile>[1]) => applyProfile(cameraDevice(unit), profile);
537 const server = await startServer(bus, { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000, cameraControl, lightMs: 15_000 });
538 const cameraStill = (unit: number) => grabStill(unit, cameraDevice(unit));
539 const server = await startServer(bus, { host, port, posesFile: values.file ?? "poses.json", modelDir: values.model, gainSyncMs: 2000, cameraControl, cameraStill, lightMs: 15_000 });
538540 console.log(`serving ${bus.path} on http://${host}:${server.port} — open it in a browser; the API is listed in src/server.ts`);
539541 console.log("ctrl-c to stop");
540542 await new Promise<void>((resolve) => process.once("SIGINT", () => resolve()));
Software/so101/src/grow-frame-worker.ts added +11
@@ -0,0 +1,11 @@
1 /** The worker src/grow-frame.ts runs `finishStill` in, so a 1080p decode never holds up the relay's pages. */
2 import { parentPort } from "node:worker_threads";
3 import { finishStill, type StillLook } from "./grow-frame.ts";
4
5 parentPort?.once("message", ({ bytes, look }: { bytes: Uint8Array; look: StillLook }) => {
6 try {
7 parentPort!.postMessage({ ok: true, out: finishStill(bytes, look) });
8 } catch (err) {
9 parentPort!.postMessage({ ok: false, error: (err as Error).message });
10 }
11 });
Software/so101/src/grow-frame.test.ts added +235
@@ -0,0 +1,235 @@
1 import { afterEach, describe, expect, test } from "bun:test";
2 import { mkdtempSync, readdirSync, rmSync, writeFileSync } from "node:fs";
3 import { createServer, type Server } from "node:http";
4 import { tmpdir } from "node:os";
5 import { join } from "node:path";
6 import jpeg from "jpeg-js";
7 import { FeetechBus } from "./bus.ts";
8 import { CameraHub } from "./cameras.ts";
9 import { grabStill, STILL_HEIGHT, STILL_WIDTH, StillError, stillCommands, type Run } from "./camera-still.ts";
10 import { FakeBus } from "./fake-bus.ts";
11 import { finishStill, finishStillApart, jpegSize } from "./grow-frame.ts";
12 import { GrowLog } from "./grow-log.ts";
13 import { GrowStills } from "./grow-stills.ts";
14 import { startRelay, type RunningRelay } from "./relay.ts";
15 import { startServer, type RunningServer } from "./server.ts";
16 import { StationHub } from "./stations.ts";
17 import { canopyRegion, stillView } from "./ui/grow-timelapse.js";
18
19 const dirs: string[] = [];
20 const tmp = () => {
21 const d = mkdtempSync(join(tmpdir(), "grow-frame-"));
22 dirs.push(d);
23 return d;
24 };
25 let relay: RunningRelay | undefined;
26 let server: RunningServer | undefined;
27 let camera: Server | undefined;
28 afterEach(async () => {
29 await relay?.close();
30 await server?.close();
31 camera?.close();
32 relay = server = camera = undefined;
33 dirs.splice(0).forEach((d) => rmSync(d, { recursive: true, force: true }));
34 });
35 const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
36
37 /**
38 * The fixture still: 192×108 (a tenth of 1920×1080). The left half is grey, the right half is the box: plants
39 * (green) on soil, under a grow light that leaves the red channel weak.
40 */
41 const W = 192;
42 const H = 108;
43 function fixture(): Uint8Array<ArrayBuffer> {
44 const data = new Uint8Array(W * H * 4);
45 for (let y = 0; y < H; y++) {
46 for (let x = 0; x < W; x++) {
47 const i = (y * W + x) * 4;
48 const box = x >= W / 2;
49 data.set(box ? (y < H / 2 ? [30, 120, 90, 255] : [40, 60, 100, 255]) : [128, 128, 128, 255], i);
50 }
51 }
52 return Uint8Array.from(jpeg.encode({ data, width: W, height: H }, 95).data);
53 }
54 /** The mean of each channel of a JPEG. */
55 function means(bytes: Uint8Array): { r: number; g: number; b: number; width: number; height: number } {
56 const { data, width, height } = jpeg.decode(bytes, { useTArray: true, formatAsRGBA: true });
57 let [r, g, b] = [0, 0, 0];
58 for (let i = 0; i < data.length; i += 4) (r += data[i]!), (g += data[i + 1]!), (b += data[i + 2]!);
59 const n = data.length / 4;
60 return { r: r / n, g: g / n, b: b / n, width, height };
61 }
62 const RIGHT_HALF = { x: 0.5, y: 0, w: 0.5, h: 1 };
63 const GAINS = { r: 2, g: 1, b: 0.8 };
64
65 describe("a still is cropped and corrected", () => {
66 test("cropped to the region by cropRect on its own size; the corrected copy has the gains, the raw one does not", () => {
67 const still = fixture();
68 const before = still.slice();
69 const out = finishStill(still, { region: RIGHT_HALF, gains: GAINS });
70 expect(out).toMatchObject({ cropped: true, width: W, height: H });
71 const raw = means(out.raw);
72 const fixed = means(out.corrected!);
73 expect([raw.width, raw.height]).toEqual([W / 2, H]);
74 expect([fixed.width, fixed.height]).toEqual([W / 2, H]);
75 // The box alone: none of the grey half is in the crop (its red would lift the mean past 50).
76 expect(raw.r).toBeGreaterThan(30);
77 expect(raw.r).toBeLessThan(40);
78 expect(fixed.r / raw.r).toBeCloseTo(2, 1);
79 expect(fixed.g / raw.g).toBeCloseTo(1, 1);
80 expect(fixed.b / raw.b).toBeCloseTo(0.8, 1);
81 expect(still).toEqual(before);
82 });
83
84 test("with no region the raw still is the camera's bytes, untouched; gains alone still give a corrected copy", () => {
85 const still = fixture();
86 const plain = finishStill(still, { region: null, gains: null });
87 expect(plain.raw).toBe(still);
88 expect(plain).toMatchObject({ corrected: null, cropped: false, width: W, height: H });
89 const gained = finishStill(still, { region: null, gains: GAINS });
90 expect(gained.raw).toBe(still);
91 expect(gained.cropped).toBe(false);
92 expect(means(gained.corrected!).r / means(still).r).toBeCloseTo(2, 1);
93 });
94
95 test("the size comes from the JPEG's header, and a still that is not one is refused", () => {
96 expect(jpegSize(fixture())).toEqual({ width: W, height: H });
97 expect(jpegSize(new Uint8Array([0xff, 0xd8, 0xff, 0xd9]))).toBeNull();
98 expect(finishStill(new Uint8Array([0xff, 0xd8, 1, 2, 3]), { region: null, gains: null })).toMatchObject({ width: 0, height: 0, cropped: false });
99 expect(() => finishStill(new Uint8Array([0xff, 0xd8, 1, 2, 3]), { region: RIGHT_HALF, gains: null })).toThrow();
100 });
101
102 test("in a worker it gives the same still, and a bad one is an error, not a hang", async () => {
103 const still = fixture();
104 const apart = await finishStillApart(still, { region: RIGHT_HALF, gains: GAINS });
105 const here = finishStill(still, { region: RIGHT_HALF, gains: GAINS });
106 expect(new Uint8Array(apart.raw)).toEqual(Uint8Array.from(here.raw));
107 expect(new Uint8Array(apart.corrected!)).toEqual(Uint8Array.from(here.corrected!));
108 await expect(finishStillApart(new Uint8Array([0xff, 0xd8, 1, 2]), { region: RIGHT_HALF, gains: null })).rejects.toThrow();
109 });
110 });
111
112 describe("the store keeps the corrected copy beside the raw still", () => {
113 test("both are read back as given, counted together, flagged, and go together", () => {
114 const dir = tmp();
115 const s = new GrowStills(dir, { keepMs: 60_000 });
116 const out = finishStill(fixture(), { region: RIGHT_HALF, gains: GAINS });
117 const T0 = 1_760_000_000_000;
118 s.add(out.raw, T0, false, { cropped: true, corrected: out.corrected });
119 expect(s.read(T0)).toEqual(out.raw);
120 expect(s.read(T0, true)).toEqual(out.corrected!);
121 expect(s.page({}).stills).toEqual([{ at: T0, dark: false, cropped: true, corrected: true, bytes: out.raw.length + out.corrected!.length }]);
122 s.add(fixture(), T0 + 120_000, false);
123 expect(s.read(T0 + 120_000, true)).toBeNull();
124 expect(s.page({}).stills.map((x) => [x.at, x.cropped, x.corrected])).toEqual([[T0 + 120_000, false, false]]);
125 expect(readdirSync(dir).filter((f) => f.endsWith(".jpg"))).toEqual([`${T0 + 120_000}.jpg`]);
126 });
127
128 test("a corrected copy that is not a JPEG is refused, and nothing is written", () => {
129 const dir = tmp();
130 expect(() => new GrowStills(dir).add(fixture(), 1_760_000_000_000, false, { corrected: new Uint8Array([1, 2, 3, 4]) })).toThrow();
131 expect(readdirSync(dir)).toEqual([]);
132 });
133 });
134
135 describe("the card", () => {
136 test("shows the corrected copy labelled so, unless turned off; a cropped still's canopy is not cropped again", () => {
137 expect(stillView({ at: 5, corrected: true }, true)).toEqual({ src: "/grow/stills/5-corrected.jpg", label: "corrected" });
138 expect(stillView({ at: 5, corrected: true }, false)).toEqual({ src: "/grow/stills/5.jpg", label: "" });
139 expect(stillView({ at: 5 }, true)).toEqual({ src: "/grow/stills/5.jpg", label: "" });
140 expect(canopyRegion({ cropped: true }, RIGHT_HALF)).toBeNull();
141 expect(canopyRegion({ cropped: false }, RIGHT_HALF)).toBe(RIGHT_HALF);
142 });
143 });
144
145 describe("a 1920×1080 still on the Pi, the stream paused around it", () => {
146 const DEV = "/dev/v4l/by-id/usb-XWF-1080P-video-index0";
147 /** A fake runner: records each command, and writes `still` where v4l2-ctl was told to. */
148 const runner = (still: Uint8Array | null, fail: Partial<Record<"stop" | "grab" | "start", string>> = {}) => {
149 const ran: string[] = [];
150 const exec: Run = async (command, args) => {
151 const step = command === "v4l2-ctl" ? "grab" : (args[2] as "stop" | "start");
152 ran.push(step);
153 if (step === "grab" && still && !fail.grab) writeFileSync(args.find((a) => a.startsWith("--stream-to="))!.slice("--stream-to=".length), still);
154 return fail[step] ?? "";
155 };
156 return { ran, exec };
157 };
158
159 test("stops the camera's service, grabs one MJPEG frame at 1920×1080 after it settles, and starts it again", async () => {
160 const c = stillCommands(2, DEV, "/x/still.jpg");
161 expect(c.stop).toEqual(["sudo", ["-n", "systemctl", "stop", "[email protected]"]]);
162 expect(c.start).toEqual(["sudo", ["-n", "systemctl", "start", "[email protected]"]]);
163 expect(c.grab[1]).toContain(`--set-fmt-video=width=${STILL_WIDTH},height=${STILL_HEIGHT},pixelformat=MJPG`);
164 expect(c.grab[1]).toContain("--stream-count=1");
165 expect(() => stillCommands(9, DEV, "/x")).toThrow();
166 const still = fixture();
167 const { ran, exec } = runner(still);
168 expect(await grabStill(2, DEV, exec)).toEqual(still);
169 expect(ran).toEqual(["stop", "grab", "start"]);
170 });
171
172 test("the stream is started again when the grab fails or gives no JPEG; one still at a time", async () => {
173 const failed = runner(null, { grab: "Device or resource busy" });
174 await expect(grabStill(1, DEV, failed.exec)).rejects.toThrow(/could not grab a still: Device or resource busy/);
175 expect(failed.ran).toEqual(["stop", "grab", "start"]);
176 const junk = runner(new Uint8Array([1, 2, 3, 4, 5]));
177 await expect(grabStill(1, DEV, junk.exec)).rejects.toThrow(/no JPEG/);
178 expect(junk.ran).toEqual(["stop", "grab", "start"]);
179 const noSudo = runner(fixture(), { stop: "sudo: a password is required" });
180 await expect(grabStill(1, DEV, noSudo.exec)).rejects.toThrow(/could not pause the stream/);
181 expect(noSudo.ran).toEqual(["stop"]);
182 let release = () => {};
183 const slow: Run = (_c, args) => (args[2] === "stop" ? new Promise((r) => (release = () => r(""))) : Promise.resolve(""));
184 const first = grabStill(1, DEV, slow).catch((e: Error) => e);
185 const second = await grabStill(1, DEV, slow).catch((e: StillError) => e);
186 expect(second).toBeInstanceOf(StillError);
187 expect((second as StillError).busy).toBe(true);
188 release();
189 await first;
190 });
191
192 test("the arm server answers POST /camera-still/<n> with the JPEG; the simulation says it has no camera", async () => {
193 const still = fixture();
194 const units: number[] = [];
195 const data = tmp();
196 server = await startServer(new FeetechBus(new FakeBus([1])), { host: "127.0.0.1", port: 0, ids: [1], posesFile: join(data, "poses.json"), cameraStill: async (unit) => (units.push(unit), still) });
197 const base = `http://127.0.0.1:${server.port}`;
198 const got = await fetch(`${base}/camera-still/2`, { method: "POST" });
199 expect(got.headers.get("content-type")).toBe("image/jpeg");
200 expect(new Uint8Array(await got.arrayBuffer())).toEqual(still);
201 expect(units).toEqual([2]);
202 expect((await fetch(`${base}/camera-still/2`)).status).toBe(405);
203 expect((await fetch(`${base}/camera-still/9`, { method: "POST" })).status).toBe(400);
204 await server.close();
205 server = await startServer(new FeetechBus(new FakeBus([1])), { host: "127.0.0.1", port: 0, ids: [1], posesFile: join(data, "poses.json") });
206 expect((await fetch(`http://127.0.0.1:${server.port}/camera-still/1`, { method: "POST" })).status).toBe(501);
207 });
208 });
209
210 test("the relay keeps each still cropped to the camera's region, with a corrected copy at -corrected.jpg", async () => {
211 const dir = tmp();
212 const still = fixture();
213 camera = createServer((_req, res) => {
214 res.writeHead(200, { "content-type": "image/jpeg" });
215 res.end(still);
216 });
217 await new Promise<void>((r) => camera!.listen(0, "127.0.0.1", r));
218 const port = (camera.address() as { port: number }).port;
219 const stations = new StationHub();
220 stations.replace([{ id: "grow", name: "Grow box", kind: "grow", cameras: [1], controls: [] }]);
221 const cameras = new CameraHub({ seed: [{ name: "Grow cam", source: `http://127.0.0.1:${port}/stream` }] });
222 cameras.setLook(1, { region: RIGHT_HALF, gains: GAINS });
223 const stills = new GrowStills(join(dir, "stills"));
224 relay = await startRelay({ port: 0, simulation: "http://127.0.0.1:1", cameras, stations, grow: { stills, log: new GrowLog(join(dir, "log.json")), everyMs: 60 }, env: {} });
225 for (const end = Date.now() + 5000; stills.usage().count < 1 && Date.now() < end; ) await sleep(20);
226 const base = `http://127.0.0.1:${relay.port}`;
227 const page = (await (await fetch(`${base}/grow/stills`)).json()) as { stills: { at: number; cropped: boolean; corrected: boolean }[] };
228 expect(page.stills[0]).toMatchObject({ cropped: true, corrected: true });
229 const at = page.stills[0]!.at;
230 const raw = means(new Uint8Array(await (await fetch(`${base}/grow/stills/${at}.jpg`)).arrayBuffer()));
231 const corrected = means(new Uint8Array(await (await fetch(`${base}/grow/stills/${at}-corrected.jpg`)).arrayBuffer()));
232 expect([raw.width, raw.height, corrected.width, corrected.height]).toEqual([W / 2, H, W / 2, H]);
233 expect(corrected.r / raw.r).toBeCloseTo(2, 1);
234 expect((await fetch(`${base}/grow/stills/1234567890-corrected.jpg`)).status).toBe(404);
235 });
Software/so101/src/grow-frame.ts added +104
@@ -0,0 +1,104 @@
1 /**
2 * A grow still made ready to keep: cropped to the camera's region (`cropRect` on the still's own size, 1920×1080 when
3 * the Pi could grab one) and, when the camera has gains, a colour-corrected copy beside it (`applyGains`). The raw
4 * still is never corrected; with no region it is the camera's bytes untouched. The maths is src/ui/still-frame.js;
5 * the JPEG codec is jpeg-js (pure JS, no dependencies of its own).
6 *
7 * Decoding and encoding a 1080p frame takes about half a second, so the relay runs `finishStill` in a worker
8 * (src/grow-frame-worker.ts) through `finishStillApart`, and the pages it serves meanwhile are not held up.
9 */
10 import { Worker } from "node:worker_threads";
11 import jpeg from "jpeg-js";
12 import { applyGains, cropRect, type Gains, type Rect } from "./ui/still-frame.js";
13
14 /** What the still is cropped and corrected with: the camera's look (src/camera-look.ts). */
15 export interface StillLook {
16 region: Rect | null;
17 gains: Gains | null;
18 }
19
20 export interface Finished {
21 /** The still to keep: cropped when the camera has a region, its colours as the camera gave them. */
22 raw: Uint8Array;
23 /** The same pixels with the camera's gains applied, labelled "corrected"; null when the camera has none. */
24 corrected: Uint8Array | null;
25 /** True when `raw` is the region alone, so nothing downstream crops it again. */
26 cropped: boolean;
27 /** The size the camera gave, before the crop (0×0 when it was kept as it came and its header names none). */
28 width: number;
29 height: number;
30 }
31
32 /** Quality of a re-encoded still: high, since a timelapse is a still looked at many times. */
33 export const QUALITY = 90;
34 /** Bigger than any webcam still: a decode is refused past this rather than allowed to eat the relay's memory. */
35 const MAX_MP = 16;
36
37 /** The width and height in a JPEG's frame header, without decoding it; null when there is none. */
38 export function jpegSize(bytes: Uint8Array): { width: number; height: number } | null {
39 let i = 2;
40 while (i + 9 < bytes.length) {
41 if (bytes[i] !== 0xff) return null;
42 const marker = bytes[i + 1]!;
43 if (marker === 0xff) {
44 i++;
45 continue;
46 }
47 const length = (bytes[i + 2]! << 8) | bytes[i + 3]!;
48 // SOF0 to SOF15, leaving out DHT (C4), JPG (C8) and DAC (CC), which share the range.
49 if (marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc) {
50 return { height: (bytes[i + 5]! << 8) | bytes[i + 6]!, width: (bytes[i + 7]! << 8) | bytes[i + 8]! };
51 }
52 i += 2 + length;
53 }
54 return null;
55 }
56
57 /** One JPEG still and the camera's look → the still to keep and, with gains, its corrected copy. Throws on a JPEG it cannot read. */
58 export function finishStill(bytes: Uint8Array, look: StillLook): Finished {
59 if (!look.region && !look.gains) return { raw: bytes, corrected: null, cropped: false, ...(jpegSize(bytes) ?? { width: 0, height: 0 }) };
60 const frame = jpeg.decode(bytes, { useTArray: true, formatAsRGBA: true, maxResolutionInMP: MAX_MP, maxMemoryUsageInMB: 256 });
61 const { width, height } = frame;
62 const rect = cropRect(look.region, width, height);
63 const whole = rect.x === 0 && rect.y === 0 && rect.w === width && rect.h === height;
64 const pixels = whole ? frame.data : crop(frame.data, width, rect);
65 const raw = whole ? bytes : new Uint8Array(jpeg.encode({ data: pixels, width: rect.w, height: rect.h }, QUALITY).data);
66 let corrected: Uint8Array | null = null;
67 if (look.gains) {
68 const copy = new Uint8ClampedArray(pixels);
69 applyGains(copy, look.gains);
70 corrected = new Uint8Array(jpeg.encode({ data: copy, width: rect.w, height: rect.h }, QUALITY).data);
71 }
72 return { raw, corrected, cropped: !whole, width, height };
73 }
74
75 /** The rows of `rect` out of an RGBA frame `width` pixels wide. */
76 function crop(data: Uint8Array, width: number, rect: Rect): Uint8Array {
77 const out = new Uint8Array(rect.w * rect.h * 4);
78 for (let row = 0; row < rect.h; row++) {
79 const from = ((rect.y + row) * width + rect.x) * 4;
80 out.set(data.subarray(from, from + rect.w * 4), row * rect.w * 4);
81 }
82 return out;
83 }
84
85 /** `finishStill` in a worker of its own, ended when it answers or after `timeoutMs`. */
86 export function finishStillApart(bytes: Uint8Array, look: StillLook, timeoutMs = 20_000): Promise<Finished> {
87 if (!look.region && !look.gains) return Promise.resolve(finishStill(bytes, look));
88 return new Promise((resolve, reject) => {
89 const worker = new Worker(new URL("./grow-frame-worker.ts", import.meta.url));
90 const timer = setTimeout(() => done(new Error("finishing the still took too long")), timeoutMs);
91 let settled = false;
92 const done = (err: Error | null, out?: Finished) => {
93 if (settled) return;
94 settled = true;
95 clearTimeout(timer);
96 void worker.terminate();
97 if (err) reject(err);
98 else resolve(out!);
99 };
100 worker.once("message", (m: { ok: true; out: Finished } | { ok: false; error: string }) => (m.ok ? done(null, m.out) : done(new Error(m.error))));
101 worker.once("error", (err: unknown) => done(err instanceof Error ? err : new Error(String(err))));
102 worker.postMessage({ bytes, look });
103 });
104 }
Software/so101/src/grow-stills.ts +51 −19
@@ -2,6 +2,10 @@
22 * The grow box's stills: one JPEG per capture, kept for 30 days under a byte cap (the oldest goes first), each flagged
33 * when the light was off. A still's name is its time, so the store is its own index and survives any restart.
44 * The canopy reading of each still (src/ui/grow-canopy.js, worked out in the browser) is kept beside them.
5 *
6 * Names: `<ms>.jpg`, with `d` before the dot when it is dark and `k` when it is cropped to the camera's region
7 * (src/grow-frame.ts), so the browser does not crop it again. A colour-corrected copy is the same name with `c`
8 * before the dot; it goes with its still. The raw still is never overwritten by the corrected one.
59 */
610 import { existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
711 import { join } from "node:path";
@@ -19,7 +23,12 @@ export interface Still {
1923 at: number;
2024 /** The light was off: the picture is of a dark box. */
2125 dark: boolean;
26 /** Both files' bytes, the corrected copy's too. */
2227 bytes: number;
28 /** Cropped to the camera's region when it was taken; older stills are the whole frame. */
29 cropped: boolean;
30 /** A colour-corrected copy is kept beside it, at /grow/stills/<at>-corrected.jpg. */
31 corrected: boolean;
2332 /** Green-pixel fraction of the owner's region, 0 to 1, once the browser has worked it out. */
2433 green?: number;
2534 }
@@ -30,7 +39,9 @@ export interface Page {
3039 next: number | null;
3140 }
3241
33 const NAME = /^(\d{10,15})(d?)\.jpg$/;
42 const NAME = /^(\d{10,15})(d?)(k?)(c?)\.jpg$/;
43
44 type Kept = { at: number; dark: boolean; cropped: boolean };
3445
3546 /** "TEN_MIN" env text → ms between stills, within bounds; the default for anything else. */
3647 export function parseEvery(text: string | undefined): number {
@@ -64,30 +75,46 @@ export class GrowStills {
6475 }
6576
6677 private files(): Still[] {
67 const out: Still[] = [];
78 const raw = new Map<number, Still>();
79 const copies = new Map<number, number>();
6880 for (const name of readdirSync(this.dir)) {
6981 const m = NAME.exec(name);
7082 if (!m) continue;
7183 try {
7284 const at = Number(m[1]);
73 const green = this.green.get(at);
74 out.push({ at, dark: m[2] === "d", bytes: statSync(join(this.dir, name)).size, ...(green === undefined ? {} : { green }) });
85 const bytes = statSync(join(this.dir, name)).size;
86 if (m[4] === "c") copies.set(at, bytes);
87 else {
88 const green = this.green.get(at);
89 raw.set(at, { at, dark: m[2] === "d", bytes, cropped: m[3] === "k", corrected: false, ...(green === undefined ? {} : { green }) });
90 }
7591 } catch {
7692 /* removed while we looked */
7793 }
7894 }
79 return out.sort((a, b) => a.at - b.at);
95 for (const [at, bytes] of copies) {
96 const s = raw.get(at);
97 if (s) (s.bytes += bytes), (s.corrected = true);
98 }
99 return [...raw.values()].sort((a, b) => a.at - b.at);
80100 }
81101
82 private path(s: { at: number; dark: boolean }): string {
83 return join(this.dir, `${s.at}${s.dark ? "d" : ""}.jpg`);
102 private path(s: Kept, corrected = false): string {
103 return join(this.dir, `${s.at}${s.dark ? "d" : ""}${s.cropped ? "k" : ""}${corrected ? "c" : ""}.jpg`);
84104 }
85105
86 /** Keep a still, then drop what is past its days and, oldest first, what is past the cap. */
87 add(jpeg: Uint8Array, at: number, dark: boolean): void {
88 if (!Number.isSafeInteger(at) || at < 1e9 || jpeg.length < 4 || jpeg[0] !== 0xff || jpeg[1] !== 0xd8) throw new Error("not a JPEG still");
89 writeFileSync(`${this.path({ at, dark })}.tmp`, jpeg);
90 renameSync(`${this.path({ at, dark })}.tmp`, this.path({ at, dark }));
106 /**
107 * Keep a still (and, when given, its colour-corrected copy), then drop what is past its days and, oldest first,
108 * what is past the cap. `cropped` says the still is the camera's region alone.
109 */
110 add(jpeg: Uint8Array, at: number, dark: boolean, o: { cropped?: boolean; corrected?: Uint8Array | null } = {}): void {
111 const isJpeg = (b: Uint8Array) => b.length >= 4 && b[0] === 0xff && b[1] === 0xd8;
112 if (!Number.isSafeInteger(at) || at < 1e9 || !isJpeg(jpeg) || (o.corrected && !isJpeg(o.corrected))) throw new Error("not a JPEG still");
113 const kept = { at, dark, cropped: o.cropped === true };
114 const write = (file: string, bytes: Uint8Array) => (writeFileSync(`${file}.tmp`, bytes), renameSync(`${file}.tmp`, file));
115 // The copy first: a still never shows as corrected before its copy is there.
116 if (o.corrected) write(this.path(kept, true), o.corrected);
117 write(this.path(kept), jpeg);
91118 this.prune(at);
92119 }
93120
@@ -99,10 +126,12 @@ export class GrowStills {
99126 const gone: Still[] = [];
100127 for (const s of all) {
101128 if (s.at >= cutoff && total <= this.capBytes) break;
102 try {
103 unlinkSync(this.path(s));
104 } catch {
105 /* already gone */
129 for (const file of s.corrected ? [this.path(s, true), this.path(s)] : [this.path(s)]) {
130 try {
131 unlinkSync(file);
132 } catch {
133 /* already gone */
134 }
106135 }
107136 total -= s.bytes;
108137 gone.push(s);
@@ -137,10 +166,13 @@ export class GrowStills {
137166 return { count: all.length, bytes: all.reduce((n, s) => n + s.bytes, 0), capBytes: this.capBytes };
138167 }
139168
140 read(at: number): Uint8Array | null {
169 /** A still's bytes, or its colour-corrected copy's; null when there is none. */
170 read(at: number, corrected = false): Uint8Array | null {
141171 for (const dark of [false, true]) {
142 const file = this.path({ at, dark });
143 if (existsSync(file)) return readFileSync(file);
172 for (const cropped of [true, false]) {
173 const file = this.path({ at, dark, cropped }, corrected);
174 if (existsSync(file)) return readFileSync(file);
175 }
144176 }
145177 return null;
146178 }
Software/so101/src/relay.ts +29 −7
@@ -108,6 +108,7 @@ import { CameraError, CameraHub } from "./cameras.ts";
108108 import { ARM_ONLY, StationError, StationHub } from "./stations.ts";
109109 import { renderWorkshop, type StationView } from "./workshop-page.ts";
110110 import { DAY_MS, GrowStills, KEEP_DAYS } from "./grow-stills.ts";
111 import { finishStillApart } from "./grow-frame.ts";
111112 import { canopyByDay, dayNumber, GrowLog, GrowLogError, lightHours } from "./grow-log.ts";
112113 import { parseProfile, ProfileError, type CameraProfile } from "./camera-profile.ts";
113114 import { VideoFeed, type VideoSources } from "./video-feed.ts";
@@ -960,9 +961,9 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
960961 const v = url.searchParams.get(name);
961962 return v === null || !/^\d{1,15}$/.test(v) ? undefined : Number(v);
962963 };
963 const still = /^\/grow\/stills\/(\d{10,15})\.jpg$/.exec(url.pathname);
964 const still = /^\/grow\/stills\/(\d{10,15})(-corrected)?\.jpg$/.exec(url.pathname);
964965 if (req.method === "GET" && still) {
965 const bytes = grow.stills.read(Number(still[1]));
966 const bytes = grow.stills.read(Number(still[1]), still[2] !== undefined);
966967 if (!bytes) return json(res, 404, { error: "no such still" }), true;
967968 res.writeHead(200, { "content-type": "image/jpeg", "cache-control": "public, max-age=31536000, immutable", "content-length": String(bytes.length) });
968969 res.end(bytes);
@@ -1005,18 +1006,39 @@ export async function startRelay(options: RelayOptions): Promise<RunningRelay> {
10051006 return false;
10061007 };
10071008
1008 /** Take one still of the grow box's camera, flagged dark when the light is off. Quiet on failure: a missed still is a gap, not an error. */
1009 /**
1010 * Take one still of the grow box's camera, flagged dark when the light is off. A camera that is one of the Pi's is
1011 * grabbed at 1920×1080 by the arm server (its stream paused for it, src/camera-still.ts); otherwise, or when that
1012 * fails, the stream's own snapshot is taken. The still is cropped to the camera's region and, with gains, a corrected
1013 * copy kept beside it (src/grow-frame.ts, in a worker). Quiet on failure: a missed still is a gap, not an error.
1014 */
10091015 const takeStill = async () => {
10101016 const grow = options.grow;
10111017 const growStation = stations.list().find((st) => st.kind === "grow");
10121018 const id = growStation?.cameras[0];
10131019 const source = id === undefined ? null : cameras.source(id);
1014 if (!grow || source === null || restingNow()) return;
1020 if (!grow || id === undefined || source === null || restingNow()) return;
1021 const at = Date.now();
1022 const look = cameras.look(id);
1023 const read = async (answer: Response) => {
1024 const bytes = new Uint8Array(await answer.arrayBuffer());
1025 return answer.ok && bytes.length < 8_000_000 ? bytes : null;
1026 };
10151027 try {
10161028 const light = (await lightHooks.read())?.light;
1017 const answer = await fetch(source.replace(/\/[^/]*$/, "/snapshot"), { signal: AbortSignal.timeout(Math.min(readTimeoutMs, 5000)) });
1018 const bytes = new Uint8Array(await answer.arrayBuffer());
1019 if (answer.ok && bytes.length < 8_000_000) grow.stills.add(bytes, Date.now(), light?.configured === true && light.on === false);
1029 const dark = light?.configured === true && light.on === false;
1030 let bytes: Uint8Array | null = null;
1031 if (look?.unit && remote && onArm()) {
1032 bytes = await fetch(`${remote.http}/camera-still/${look.unit}`, { method: "POST", signal: AbortSignal.timeout(20_000) })
1033 .then(read)
1034 .catch(() => null);
1035 }
1036 bytes ??= await read(await fetch(source.replace(/\/[^/]*$/, "/snapshot"), { signal: AbortSignal.timeout(Math.min(readTimeoutMs, 5000)) }));
1037 if (!bytes) return;
1038 const done = await finishStillApart(bytes, { region: look?.region ?? null, gains: look?.gains ?? null }).catch(() => null);
1039 // A still that will not decode is kept whole and uncorrected: a picture is still better than a gap.
1040 if (done) grow.stills.add(done.raw, at, dark, { cropped: done.cropped, corrected: done.corrected });
1041 else grow.stills.add(bytes, at, dark);
10201042 } catch {
10211043 /* the camera or the Pi did not answer: no still this time */
10221044 }
Software/so101/src/server.ts +24
@@ -33,6 +33,8 @@
3333 * PUT /camera-profile/<n>[?try=1] {"exposure"?, "whiteBalance"?, "gain"?, "focus"?, "backlight"?, "brightness"?, "contrast"?, "saturation"?} put it on so101-camera@n's
3434 * camera now, one control at a time, and keep it for the camera service's next start (?try=1: only put it on);
3535 * {"saved", "set", "failed"}. Without camera control (the simulation) nothing runs: 501, and nothing is kept
36 * POST /camera-still/<n> one 1920×1080 JPEG from so101-camera@n's camera, its stream paused for the grab (src/camera-still.ts);
37 * 409 while another is taken, 502 when the grab failed, 501 in the simulation
3638 * GET /settings the safe zone: {"zone": "enforce"|"warn"|"off", "floorMarginMm", "baseMarginMm", "jointOffsetsDeg"},
3739 * and the safety limits of src/safety.ts (speed, acceleration, rate, dead-man, park, gripper force, joint limits)
3840 * PUT /settings change any of those; "servoGains" takes partial per-joint patches ({"2": {"p": 80}}), joints 1-5, 0-254
@@ -87,6 +89,7 @@ import { WebSocketServer, type WebSocket } from "ws";
8789 import { browserModule } from "./browser-modules.ts";
8890 import { defaultProfile, parseProfile, ProfileError, type CameraProfile } from "./camera-profile.ts";
8991 import { loadProfile, parseUnit, profileFile, saveProfile, type Applied } from "./camera-control.ts";
92 import { StillError } from "./camera-still.ts";
9093 import { Clock, CommandGate, HELD_CLOSE, isRelease, sentAtOf } from "./command-gate.ts";
9194 import { FLOOD_CLOSE, FrameRate, frameBytes, MAX_FRAME_BYTES, TOO_BIG_CLOSE } from "./frame-limit.ts";
9295 import type { FeetechBus } from "./bus.ts";
@@ -152,6 +155,8 @@ export interface ServerOptions {
152155 watchQuietMs?: number;
153156 /** Puts a profile on so101-camera@<unit>'s camera (src/camera-control.ts); only `so101 serve` on the Pi gives one, so the simulation never runs v4l2-ctl. */
154157 cameraControl?: (unit: number, profile: CameraProfile) => Promise<Applied>;
158 /** Grabs one full-size still from so101-camera@<unit>, pausing its stream (src/camera-still.ts); only on the Pi. */
159 cameraStill?: (unit: number) => Promise<Uint8Array>;
155160 }
156161
157162 export interface RunningServer {
@@ -883,6 +888,25 @@ export async function startServer(bus: FeetechBus, options: ServerOptions = {}):
883888 if (keep) saveProfile(profileFile(dataDir, unit), profile);
884889 return json(res, 200, { saved: keep, ...applied });
885890 }
891 const cameraStill = /^\/camera-still\/([^/]*)$/.exec(url.pathname);
892 if (cameraStill) {
893 if (req.method !== "POST") throw new HttpError(405, "POST /camera-still/<n>");
894 let unit: number;
895 try {
896 unit = parseUnit(cameraStill[1]);
897 } catch (err) {
898 throw new HttpError(400, (err as Error).message);
899 }
900 if (!options.cameraStill) return json(res, 501, { error: "no camera here: this is the simulation, not the Pi" });
901 let bytes: Uint8Array;
902 try {
903 bytes = await options.cameraStill(unit);
904 } catch (err) {
905 return json(res, err instanceof StillError && err.busy ? 409 : 502, { error: (err as Error).message });
906 }
907 res.writeHead(200, { "content-type": "image/jpeg", "cache-control": "no-store", "content-length": String(bytes.length) });
908 return res.end(bytes);
909 }
886910 if (req.method === "PUT" && url.pathname.startsWith("/calibration/")) {
887911 const kind = url.pathname.slice("/calibration/".length);
888912 if (!/^[a-z][a-z0-9_-]{0,31}$/.test(kind)) throw new HttpError(400, "calibration kind: lowercase letters, digits, - and _");
Software/so101/src/ui/grow-timelapse.js +30 −4
@@ -2,6 +2,8 @@
22 /**
33 * The grow card's timelapse, log and canopy chart, built in the browser from the stills the relay keeps
44 * (GET /grow/stills, /grow/stills/<at>.jpg, /grow/log): no video is encoded anywhere. Plain ESM served at /ui/.
5 * A still with a colour-corrected copy (/grow/stills/<at>-corrected.jpg) is shown corrected, labelled so, unless
6 * the "Corrected" button is turned off.
57 * The page has `[data-grow]` holding the camera's box as `data-region` JSON and `data-owner="1"` for the owner.
68 */
79 import { greenFraction } from "./grow-canopy.js";
@@ -35,6 +37,23 @@ export function chartPoints(days, width, height, pad = 6) {
3537 .join(" ");
3638 }
3739
40 /**
41 * Where a still is drawn from, and the label beside its time: its corrected copy when it has one and that is wanted.
42 * @param {{ at: number, corrected?: boolean }} still
43 * @param {boolean} wantCorrected
44 */
45 export function stillView(still, wantCorrected) {
46 const corrected = wantCorrected && still.corrected === true;
47 return { src: `/grow/stills/${still.at}${corrected ? "-corrected" : ""}.jpg`, label: corrected ? "corrected" : "" };
48 }
49
50 /**
51 * The region a still's canopy is counted in: none for a still already cropped to it.
52 * @param {{ cropped?: boolean }} still
53 * @param {unknown} region
54 */
55 export const canopyRegion = (still, region) => (still.cropped ? null : region);
56
3857 /** @param {string} url @returns {Promise<any>} */
3958 const get = async (url) => {
4059 const res = await fetch(url, { cache: "no-store" });
@@ -73,7 +92,7 @@ async function measure(stills, region) {
7392 const g = canvas.getContext("2d");
7493 g.drawImage(img, 0, 0, canvas.width, canvas.height);
7594 const { data } = g.getImageData(0, 0, canvas.width, canvas.height);
76 found.push({ at: s.at, green: greenFraction(data, canvas.width, canvas.height, region) });
95 found.push({ at: s.at, green: greenFraction(data, canvas.width, canvas.height, canopyRegion(s, region)) });
7796 } catch {
7897 // A still that will not decode is skipped: it has no reading, which is the honest answer.
7998 }
@@ -107,9 +126,10 @@ export async function growCard(root) {
107126 const slider = el("input", undefined, { type: "range", min: "0", max: "0", value: "0", "aria-label": "Scrub the timelapse" });
108127 const when = el("output", "");
109128 const play = el("button", "Play", { type: "button" });
129 const fix = el("button", "Corrected", { type: "button", "aria-pressed": "true", hidden: "", title: "Show the stills with the camera's colour correction" });
110130 const buttons = Object.entries(RANGES).map(([key, label]) => el("button", label, { type: "button", "data-range": key }));
111131 const bar = el("div", undefined, { class: "controls" });
112 bar.append(...buttons, play);
132 bar.append(...buttons, play, fix);
113133 const lapse = el("div", undefined, { class: "lapse" });
114134 lapse.append(bar, view, slider, when);
115135 const log = el("div", undefined, { class: "growlog" });
@@ -121,8 +141,9 @@ export async function growCard(root) {
121141 const show = (i) => {
122142 const s = stills[i];
123143 if (!s) return;
124 view.src = `/grow/stills/${s.at}.jpg`;
125 when.textContent = new Date(s.at).toLocaleString([], { dateStyle: "medium", timeStyle: "short" });
144 const v = stillView(s, fix.getAttribute("aria-pressed") === "true");
145 view.src = v.src;
146 when.textContent = new Date(s.at).toLocaleString([], { dateStyle: "medium", timeStyle: "short" }) + (v.label ? ` · ${v.label}` : "");
126147 slider.value = String(i);
127148 };
128149 const stop = () => {
@@ -137,11 +158,16 @@ export async function growCard(root) {
137158 timer = setInterval(() => (Number(slider.value) + 1 >= stills.length ? stop() : show(Number(slider.value) + 1)), 150);
138159 });
139160 slider.addEventListener("input", () => (stop(), show(Number(slider.value))));
161 fix.addEventListener("click", () => {
162 fix.setAttribute("aria-pressed", String(fix.getAttribute("aria-pressed") !== "true"));
163 show(Number(slider.value));
164 });
140165 const pick = async (range) => {
141166 stop();
142167 for (const b of buttons) b.setAttribute("aria-pressed", String(b.dataset.range === range));
143168 stills = await load(rangeStart(range));
144169 slider.max = String(Math.max(0, stills.length - 1));
170 fix.hidden = !stills.some((s) => s.corrected);
145171 if (!stills.length) {
146172 view.removeAttribute("src");
147173 when.textContent = "No stills in this range yet.";

↑↓ navigate ↵ open ⌘↵ its app esc close

Keyboard shortcuts

Docs