Design: FerretDB automatic security & speed remediation, logging and reports

Status: Design for approval · Owner: xet7 · Related: WeKan.md (the WeKan side and the shared on-disk layout), Snap-Core.

This is the FerretDB (v1, SQLite backend — the wekan/FerretDB fork on main-v1) counterpart of WeKan.md. It specifies, for the database process:

1. Automatic remediation of security and performance problems it can handle itself.
2. Reporting each such event to WeKan, which records it.

Crucially, FerretDB does NOT write to any database or file itself. It surfaces a problem to WeKan (as an error/warning on the operation, or via its log stream), and WeKan records it into the eventlog collection with a normal Meteor JavaScript query (WeKan.md §3). This keeps storage DB-agnostic — the feature works identically whether WeKan runs on FerretDB or MongoDB — and there is no separate FerretDB UI or logger DB; events appear in WeKan's Admin Panel → Reports → Security / Speed.

1. Threat model (what FerretDB can remediate)

FerretDB sits behind WeKan and speaks the MongoDB wire protocol over a local socket; it is not internet-facing, so classic web vulns (SSRF/XSS/CSRF) do not apply here. The FerretDB-relevant categories are:

Performance categories (the bigger part): see §5.

2. How FerretDB reports problems to WeKan (no DB writes by FerretDB)

FerretDB must not create files or open its own log database. Instead it returns the problem to WeKan, and WeKan saves it:

• On an operation error (e.g. SQLITE_BUSY exhausted, an orphaned-table adopt, a rejected name): FerretDB returns a normal MongoDB-wire error/warning to WeKan. WeKan's DB-call sites catch it and call securityLog.record(...) / speedLog.record(...), which insert into eventlog (source ferretdb/sqlite.*).
• On a slow statement: the slow-query WARN already added for #6480 (internal/util/fsql, FERRETDB_SLOW_QUERY_THRESHOLD) is emitted to FerretDB's log stream; a thin WeKan-side ingestor (or the operation's own timing) turns it into a speedLog.record(...) with source:'sqlite.query' and a short statement-shape detail.

So every FerretDB event ends up as a document in the same eventlog collection WeKan writes (rows labelled by source = sqlite…/ferretdb… vs WeKan's localizeAvatar/setAvatarUrl), shown on WeKan's one Reports screen. No statfs file guard and no separate summary are needed — storage and counting are the WeKan DB's job (WeKan.md §3, §5).

3. The reporting surface in FerretDB (no logger package)

There is no seclog Go package and no SQLite log file. FerretDB only needs to make each problem observable to WeKan; the recording is WeKan's job. Concretely:

• Keep returning precise, classifiable errors/warnings on the wire for the remediation points in §4 (orphan-table adopt, non-finite sanitize, name reject, pool contention, busy-timeout wait), so WeKan can attribute and record them.
• Keep the slow-query WARN (internal/util/fsql) and, where cheap, add a short structured field (statement shape, elapsed ms) to make WeKan-side ingestion trivial. Never log document contents or credentials — only a classification and timing.

WeKan maps these to eventlog documents via the category catalog (models/lib/securityCategories.js) using source:'sqlite.*'.

3b. Canary tokens — operations the client never issues

The database is reached over a local socket by one application, whose driver is a Meteor 3 one. That makes a class of operations interesting by their mere presence: server-side JavaScript evaluation, an aggregation writing its result into a collection, dropping a database, a server-administration command. The driver does not send them. A request that does is either a bug or somebody who has reached the socket and is looking around — and both are worth telling the operator about.

internal/util/canary (paired with tests in the same package) is the FerretDB half of WeKan.md §12, and it keeps the same three properties:

• SILENT. A tripped canary returns the ordinary "operation not supported by this build" refusal — the same answer an unimplemented command gets. The id is appended as canary:<id>, which the client parses and the operator reads; nothing in it says that anything was detected or recorded, so a probe cannot tell a watched operation from an unimplemented one and route around the watched ones. A Go test asserts the message contains none of detect, record, log, alert.
• BOUNDED. The package writes nothing — no file, no table, no counter. It is a map lookup and an error value, so a caller hammering it in a loop costs this process one string comparison per request and this package no memory at all. This is what §2's "FerretDB does not write to any database or file" means for canaries, and a Go test asserts the package is stateless.
• ATTRIBUTED. The marker names which canary, so the operator's report says "tried to run server-side JavaScript" rather than "an error".

The canaries:

Check(op) returns nil for everything else, and a Go test pins the ordinary vocabulary — find, insert, update, aggregate, $match, $group, $lookup, … — as not tripping. A canary that fires on normal traffic is worse than no canary: it buries the real ones.

How it reaches the operator

Exactly as §2 describes for every other FerretDB event — the database reports, the client records:

FerretDB                                     WeKan
  Check(op) → canary:<id> in the error   ──►  recordDatabaseProblem()
                                                │ databaseCanaryId() reads the marker
                                                ▼
                                              tripCanary('database.canary')   ← rate-limited,
                                                │                               aggregated,
                                                ▼                               attributed
                                              eventlog (stream:'security')
                                                │
                                                ▼
                                     Admin Panel → Problems → Security

Two things that are deliberate on the WeKan side (server/lib/databaseProblems.js):

1. The id is not trusted as a category. It is matched against a known list, and anything else is recorded generically. An error string is attacker-influenced, and a marker parsed out of one must never be able to choose which security category it lands in — or to inject anything, which is why the extractor accepts only [a-z][a-z0-9.-]{0,60}.
2. A canary goes to the security stream and does not fall through to the database stream. Filing it under "the database said something" would put an intrusion attempt in the list of things to triage as configuration problems.

On MongoDB there is no FerretDB to mark anything, and these operations simply never appear — the WeKan side is inert, and the feature degrades to nothing rather than misbehaving. That is the same property as §2: storage and reporting are WeKan's job, so nothing here depends on which database is underneath.

4. Security remediation points → logger

Each is a place where FerretDB already does the safe thing (mostly shipped for #6467/#6476/ #6480/#6481); this design adds the event record so the behavior is counted and reportable.

5. Performance remediation (the main FerretDB work)

Auto-remediated (already shipped for #6480 and follow-ups):

• SQLite connection pragmas as defaults (operator override wins): synchronous(normal) (fewer fsyncs under WAL), cache_size(-16384), mmap_size(134217728), temp_store(memory), busy_timeout(30000), journal_mode(wal) — see internal/backends/sqlite/metadata/pool/uri.go.
• Filter pushdown ($in/$regex/ranges) turning WeKan's whole-collection scans into indexed lookups; bounded warm connection pool; unlimited open connections to avoid cursor starvation.

Detected-but-not-auto-fixed → reported to WeKan, which records a speed event (source sqlite.*):

• A statement at or above FERRETDB_SLOW_QUERY_THRESHOLD (default 1s) — the existing slow-query WARN (internal/util/fsql) is surfaced to WeKan, which records a speed event (category:'slow-query', source:'sqlite.query').
• A query that could not be pushed down and fell back to a full scan + in-Go filter (category:'no-pushdown').
• WAL file growth beyond a threshold, or a checkpoint taking too long (category:'wal-growth' / 'slow-checkpoint').
• Connection-pool wait time above a threshold (category:'pool-wait').

Each carries a short Detail (statement shape — not its bound argument values) and is counted per category, summarized like WeKan.md §5, and shown in WeKan's Reports → Speed.

6. Tests & negative tests

• Go tests assert the remediation paths return the precise, classifiable errors WeKan keys on ( drop + counter), the summary tally (counts per category/Bleed/severity), Detail truncation/newline-stripping, and the non-blocking drop-on-full-channel path. Negative tests: malformed event, oversize detail, unwritable directory (no panic, event dropped).
• Extend the slow-query test (internal/util/fsql/slow_test.go) to assert a slow statement also produces a speed event.
• Because this sandbox has no Go runtime, the logic is additionally validated with a Python mirror (path builder, disk-guard arithmetic, tally) that is actually run, per the project rule; the Go tests are for the maintainer to run with go test ./....

7. Relationship to WeKan.md

• Same directory tree and disk-space discipline; different filename prefix (ferretdb-).
• WeKan's Reports → Security / Speed / Tests read the one eventlog collection, so one admin screen shows both processes, each row labelled by source (wekan… vs sqlite…/ferretdb…).
• Category/*Bleed naming is shared; FerretDB adds a few DB-specific generic names (OrphanTableBleed, NonFiniteBleed, SlowQueryBleed) that do not (yet) have hall-of-fame pages — the Report shows the general category alongside them.