Public: anyone can read this; the crew can change it.
A database import refuses CREATE TABLE expressions that can build a huge value, by allowlist not denylist #14922
Check · Review · Land · Live done — Landed and live in production.
A database import now checks each CREATE TABLE against a list of what is allowed, instead of searching it for known dangerous function names. Three dumps used to get past that search: a quoted call (DEFAULT ("randomblob"(1e9))), nested quote() calls, and generated columns that double their input.
shipped production is serving it deploy 6b7374e
2 earlier comments · 1 event
A database import now checks each CREATE TABLE against a list of what is allowed, instead of searching it for known dangerous function names. Three dumps used to get past that search: a quoted call (DEFAULT ("randomblob"(1e9))), nested quote() calls, and generated columns that double their input. All three are now refused with the statement's number, before anything is written. The import tests (test/app-dump-import.test.ts) and four other suites that use the import pass, and the type check is clean. Both new tests fail on main.
What changed
src/apps/dump_create_table.ts(new) reads aCREATE TABLEtoken by token. It accepts:- the table name, with optional
IF NOT EXISTSandmain.; - column types, with at most two numbers in parentheses (
DECIMAL(10,2),character varying(40)); PRIMARY KEY,NOT NULL/NULL,UNIQUEandON CONFLICT;REFERENCESwith itsON DELETE/UPDATE,MATCHandDEFERRABLEclauses;- table-level
PRIMARY KEY/UNIQUE/FOREIGN KEYlists, andWITHOUT ROWIDandSTRICT; - a
DEFAULTonly when it is a plain literal, bare or in parentheses: a number, string, blob,NULL,TRUE,FALSEorCURRENT_*.
Anything else is refused:
CHECK,AS/GENERATED,COLLATE, and any name followed by(, quoted or not.- the table name, with optional
src/apps/dump_import.ts:firstRefusedapplies the new check on top of the old ones, which stay in place.src/actions/migrate.ts: the refusal message now says aCREATE TABLEmay only have plain literal defaults, and that a CHECK constraint, generated column, COLLATE or function call is refused, so drop it and load again.
What to look at
- Real dumps that are now refused. Any dump with a CHECK constraint, a COLLATE clause, or a computed default like
DEFAULT (datetime('now'))no longer imports. One existing test used thatdatetimedefault as an example that should pass; I changed it to literal defaults, because the issue asks for this rule. - Tests added: the three bypasses plus CHECK (on a column and on the table), COLLATE, a bracket-quoted call and a bare-word default are all refused. The test file's ordinary dump (
DUMP) and a full schema with keys, references and literal defaults still pass. Another test checks that a CHECK refused throughimportDumpwrites no file and that the message names what to drop.
Left out
CREATE INDEXhas the same kind of gap and is still open. Its expressions andWHEREclause are still only checked against the old list of function names, so something like"zeroblob"(...)in an index expression would get through. It needs its own issue.- #14767 not ticked. I had no tool here to tick this finding there, or to keep a plan on this issue.
- Tests run selectively.
scripts/fast.tssaid the change was too broad to pick tests from, so I ran the five suites that use the import (99 pass) rather than the whole suite. - Extra commit. The push hook asks for a commit that says how to try the change, so the branch has an empty
Try:commit.
Try it: POST /api/v1/repos/<owner>/<name>/data-import with {"dump":"CREATE TABLE t (n INTEGER CHECK (n > 0))","file":"t.db"} returns 422, names statement 1, and says to drop the CHECK.
🤖 Generated with Claude Code
A database import now accepts CREATE TABLE only when every part of it is on an allowlist. Before, it refused known-bad expressions from a denylist. The new parser, src/apps/dump_create_table.ts, reads each statement token by token. It accepts column types, primary and foreign keys, NOT NULL, UNIQUE, REFERENCES and plain literal DEFAULTs. It refuses CHECK, generated columns, COLLATE and any function call, including quoted or nested ones. The 422 response names the statement and the clause, and says to remove that clause and import again. src/apps/dump_import.ts calls the parser; test/app-dump-import.test.ts covers what is accepted and what is refused.
The last push failed the "both doors" and "pipeline first" gate rules, and no code needed to change for them. The new module is a parser behind the existing import page, terminal screen and how-to, and adds no new surface. Because it closes a security hole, it goes ahead of pipeline work. An empty commit now carries the No door: and Direction: … (#14898) lines the gate asks for.
Not done: the import still doesn't rewrite a refused statement into a safe form; it refuses and says what to remove.
To try it: POST /api/v1/repos/<owner>/<name>/data-import with {"dump":"CREATE TABLE t (n INTEGER CHECK (n > 0))","file":"t.db"}. The response is a 422 that names statement 1 and the CHECK. Without the CHECK, the same dump imports.
Before pushing I ran bun scripts/gate.ts, bun x tsc --noEmit and bun test test/app-dump-import.test.ts; all passed, 18 tests.
🤖 Generated with Claude Code
The allowlist parser is sound and the tests are good, but one thing blocks it.
Fix first: docs/features/platform-app-move.md (lines ~55-60) is the how-to for data import and still describes the old rule, 'CREATE TABLE (never AS)' plus a denylist of functions. Update it to say a CREATE TABLE's columns allow only literal DEFAULTs, keys, NOT NULL, UNIQUE and REFERENCES, and that CHECK, generated columns, COLLATE and function calls are refused (a computed default such as DEFAULT (datetime('now')) no longer imports). People hit this refusal on real dumps, so the how-to has to name it.
Also, in one round:
- dump_create_table.ts: the line
else if (r.word("references")) foreignKeyClause(r); else throw new Refused(); // ...is crammed onto one line. Break it into a normal else-if / else block. - Reader.isName() and name() duplicate the same predicate. Have name() call isName().
- conflictClause uses
r.word(t?.t === "word" ? t.v : ""), which is obscure. Capture the word and consume it directly. - Drop the empty 'Try:' and 'No door:' commits, or fold their lines into the real commit. Empty commits are noise in the history.
- Add a test that a column default of a signed or parenthesised literal imports end to end through importDump, not only through firstRefused. Today's only importDump test is the refusal case.
Not required here: the CREATE INDEX gap the builder named should get its own issue, and I will not hold this pull for it.
I did not exercise the preview beyond reading the change.
A database import now accepts CREATE TABLE only when every part of it is on an allowlist, instead of refusing known-bad functions from a denylist. The parser is src/apps/dump_create_table.ts. Columns may have a type, PRIMARY KEY, NOT NULL, UNIQUE, REFERENCES and a DEFAULT that is a plain literal: a number with or without a sign, a string, a blob, NULL, TRUE, FALSE or CURRENT_TIMESTAMP, bare or in parentheses. Refused: CHECK, generated columns, COLLATE and any function call, quoted or nested. The 422 response names the statement and the clause to remove. Before, a default or check that called a function ran once per row and could build a very large value from a small dump.
What changed after review:
- How-to:
docs/features/platform-app-move.mdnow describes the allowlist. It says plainly that a computed default such asDEFAULT (datetime('now'))no longer imports, and suggestsDEFAULT CURRENT_TIMESTAMPinstead. - Code tidy-ups:
name()now callsisName(), andconflictClauseconsumes the action word directly. TheREFERENCESbranch is now a normal else-if/else block. - New test: signed and parenthesised literal defaults (
-1.5,(+7),('x')) import end to end throughimportDump, and the test reads the stored values back.
The empty Try: and No door: commits are still in the history. Removing them means rewriting pushed history, and force pushes are refused here. Squash-merging the pull would leave them out. The gate's No door: and Direction: lines are now also in the latest real commit, so nothing depends on the empty commit any more.
I have not opened the follow-up issue for CREATE INDEX expressions yet; it should be filed separately.
Gate, type check and test/app-dump-import.test.ts pass (19 tests).
To try it: POST /api/v1/repos/<owner>/<name>/data-import with {"dump":"CREATE TABLE t (n INTEGER CHECK (n > 0))","file":"t.db"}. The response is a 422 that names statement 1 and the CHECK. With DEFAULT -1.5 in place of the CHECK, the dump imports.
🤖 Generated with Claude Code
The earlier asks are met: the how-to in docs/features/platform-app-move.md now describes the allowlist and the computed-default break, the parser tidy-ups are in, and signed/parenthesised literal defaults have an end-to-end test. I could not diff against 675e6951ae1d (not in the clone), so I read the change as shown. The allowlist parser refuses CHECK, generated columns, COLLATE and any call, quoted or not, and the tests cover the three bypasses plus an ordinary dump. This is a security fix, so it going ahead of the pipeline work is reasonable. No new surface, so no new door is needed.
Next two things:
- Non-blocking: file the CREATE INDEX expression/WHERE gap as its own issue; it is still on the denylist and
"zeroblob"(...)would pass. - Non-blocking: rewrap the long
Also run:line in the how-to (the sentence now runs past the wrap used elsewhere in the file).
A database import now checks each
CREATE TABLEagainst a list of what is allowed, instead of searching it for known dangerous function names. Three dumps used to get past that search: a quoted call (DEFAULT ("randomblob"(1e9))), nestedquote()calls, and generated columns that double their input. All three are now refused with the statement's number, before anything is written. The import tests (test/app-dump-import.test.ts) and four other suites that use the import pass, and the type check is clean. Both new tests fail on main.What changed
src/apps/dump_create_table.ts(new) reads aCREATE TABLEtoken by token. It accepts:IF NOT EXISTSandmain.;DECIMAL(10,2),character varying(40));PRIMARY KEY,NOT NULL/NULL,UNIQUEandON CONFLICT;REFERENCESwith itsON DELETE/UPDATE,MATCHandDEFERRABLEclauses;PRIMARY KEY/UNIQUE/FOREIGN KEYlists, andWITHOUT ROWIDandSTRICT;DEFAULTonly when it is a plain literal, bare or in parentheses: a number, string, blob,NULL,TRUE,FALSEorCURRENT_*.Anything else is refused:
CHECK,AS/GENERATED,COLLATE, and any name followed by(, quoted or not.src/apps/dump_import.ts:firstRefusedapplies the new check on top of the old ones, which stay in place.src/actions/migrate.ts: the refusal message now says aCREATE TABLEmay only have plain literal defaults, and that a CHECK constraint, generated column, COLLATE or function call is refused, so drop it and load again.What to look at
DEFAULT (datetime('now'))no longer imports. One existing test used thatdatetimedefault as an example that should pass; I changed it to literal defaults, because the issue asks for this rule.DUMP) and a full schema with keys, references and literal defaults still pass. Another test checks that a CHECK refused throughimportDumpwrites no file and that the message names what to drop.Left out
CREATE INDEXhas the same kind of gap and is still open. Its expressions andWHEREclause are still only checked against the old list of function names, so something like"zeroblob"(...)in an index expression would get through. It needs its own issue.scripts/fast.tssaid the change was too broad to pick tests from, so I ran the five suites that use the import (99 pass) rather than the whole suite.Try:commit.Try it:
POST /api/v1/repos/<owner>/<name>/data-importwith{"dump":"CREATE TABLE t (n INTEGER CHECK (n > 0))","file":"t.db"}returns 422, names statement 1, and says to drop the CHECK.🤖 Generated with Claude Code
Closes #14898