Back to Wekan

Rewriting WeKan in Free Pascal

docs/Design/Multiverse/FreePascal.md

11.1323.3 KB
Original Source

Rewriting WeKan in Free Pascal

This page evaluates Free Pascal as a possible implementation language for WeKan. It uses the dependencies and design choices already explored in the local Omi and Wami prototypes. It is a design exploration, not a commitment to replace the current Meteor application.

Omi demonstrates a maintained, standalone Free Pascal HTTP server using fphttpapp, HTTPDefs, httproute, fpjson, jsonparser and a statically linked SQLite amalgamation. Wami demonstrates server-rendered WeKan-shaped pages for modern, no-JavaScript and retro browsers, backed by a proposed SQLite schema. Wami's scale-up design selects mORMot 2, or alternatively Brook, only when the small fcl-web server is no longer sufficient.

References:

Summary

Free Pascal could produce a small native WeKan server for more operating systems and CPUs than the current Node.js distribution. Static HTML forms can make the core board workflow usable without JavaScript, including old and text-mode browsers. Modern browsers can progressively add drag-and-drop and richer widgets without making them prerequisites for reading or changing a board.

This is not a source translation. Meteor currently supplies accounts, methods, publications, subscriptions, DDP over SockJS, Minimongo, Tracker reactivity, optimistic method simulation and the build lifecycle. The selected Pascal stack does not reproduce those facilities automatically. They must be replaced by the server-rendered Wami interaction model, retained through a compatibility layer, or implemented explicitly.

The most coherent first target is therefore not a pixel-identical Meteor clone. It is the Wami design: a server-rendered, SQLite-backed, accessible core WeKan with immediate form submissions and optional browser enhancements. Compatibility with existing data, permissions, imports and integrations remains mandatory.

Existing prototype decisions

AreaOmi selectionWami selectionConsequence for a Free Pascal WeKan
CompilerFree Pascal 3.x, native executableFree Pascal, including AmigaOS/AROS/MorphOS targetsKeep the portable language subset small and test every promised target.
HTTP serverFCL fphttpapp, HTTPDefs, httprouteThe same lightweight server initiallyThis is the default scale-down stack.
High-scale HTTP/realtimeNot required by OmimORMot 2; Brook as an alternativeAdd only after measurements show that fphttpapp is insufficient.
TLS and public edgeReverse proxyCaddy 2 reverse proxyKeep certificate automation, HTTP/2/3 and edge policy outside the minimal app binary.
DataSQLite amalgamation linked into the executableSQLite schema modeled from WeKan dataOne local database file is the primary offline/small-server design.
JSONfpjson, jsonparserJSON fields and structure detectionPrefer FCL units at small scale; mORMot JSON is an optional modern-server optimization.
HTMLGenerated server-side, no cookies or required JavaScriptHTML 4-compatible pages with forms and buttonsCore operations must work before progressive JavaScript is loaded.
Modern interactionNone requiredinteract.js for drag-and-drop and multi-touchKeep it as an optional browser asset, not a server dependency.
Authentication modelStateful sessions, signed hidden POST fields, token rotation and context bindingFile-backed prototype loginReuse the server-owned session concept, but redesign password storage and proxy-aware binding for production.
Assets and translationsFiles beside the executable; WeKan JSON locale filesExisting WeKan CSS, images and locale files copied for experimentsAdd a generated Pascal resource/embed step for a true single-file build.
AttachmentsFilesystem operations and streamingFilesystem paths with metadata in SQLiteStream large files and never load whole attachments into memory.

Omi and Wami are prototypes, not proof that every production requirement is complete. In particular, plaintext prototype password files must not become the production account store, and IP/User-Agent binding must account for trusted reverse proxies and mobile address changes.

Dependency equivalents

The entries below prefer dependencies already selected by Omi and Wami. Other libraries are mentioned only where those prototypes deliberately identify a scale-up option or where the current WeKan feature has no selected implementation. A custom entry means product behaviour must be designed and tested.

Platform, realtime and data

Current WeKan dependency or facilitySelected Free Pascal equivalentCompatibility and migration notes
Meteor application platformFCL fphttpapp + HTTPDefs + httproute, with explicit application servicesThis is Omi and Wami's small default. There is no single Meteor-equivalent Pascal package.
Node.js runtimeFree Pascal native executableRemoves the server JavaScript runtime. Optional browser tooling and assets remain separate.
Meteor methodsRouted POST forms and JSON endpoints in httproutePreserve validation, authorization, error results, idempotency and audit side effects. Wami prefers immediate form submission for core operations.
DDP over SockJSCustom compatibility service, or replace it with normal HTTP plus optional WebSocketsThe Wami server-rendered design does not require DDP. Existing Meteor clients would require a tested DDP/SockJS bridge.
Publications and subscriptionsServer-rendered partial/full responses; optional mORMot 2 WebSocket PubSubAuthorization must be rechecked for every response and realtime event. A broadcast queue alone is not a reactive database observer.
TrackerServer request/response state; optional browser enhancement statePascal server threads do not replace Tracker's browser dependency graph. The no-JS design avoids requiring that graph.
ReactiveVar, ReactiveDict and SessionSigned form state, server session records and ordinary Pascal records/classesDefine ownership, expiry and concurrency explicitly. Never trust hidden fields merely because the server generated them.
MinimongoNo client database in the minimal designRender only visible data from SQLite. A rich offline modern client would need a separate browser store and reconciliation protocol.
MongoDB Meteor driverFerretDB as a separate compatibility service, or a custom database adapterNo MongoDB driver was selected in Omi/Wami. Direct SQLite is the selected Wami path; Mongo compatibility needs its own adapter and conformance suite.
FerretDB v1Keep as a separate process when existing MongoDB documents must remain authoritativeBoth being native programs does not create an in-process API. SQLite mode should not depend on FerretDB internals.
MongoDB collectionsWami's SQLite schema plus a repository/data-mapper layerThe current schema is exploratory and stores many values as text. Normalize types, indexes, foreign keys and JSON fields based on measured queries.
aldeed:collection2 and SimpleSchemaPascal record/class types plus custom boundary validatorsCompile-time types do not validate HTTP fields, JSON, imported data or old rows.
matb33:collection-hooksExplicit service procedures and transaction hooksPreserve activities, rules, webhooks, attachment cleanup and denormalized fields in the same transaction where possible.
BSON and EJSONfpjson/jsonparser for APIs; custom BSON/EJSON codecs only for compatibilityObject IDs, dates, binary data and special numbers need round-trip fixtures before old data can be declared compatible.
check and argument auditingTyped parsers, length/range checks and centralized request validationEvery route must reject unknown, missing and malformed fields before authorization-sensitive work.
Synced CronDedicated Pascal worker thread plus persistent SQLite job and lease tablesSleep or an in-memory timer alone does not survive restarts or coordinate replicas.
High-scale servermORMot 2; Brook is the selected alternativeWami intentionally defers these larger dependencies until load testing justifies them. Do not assume benchmark claims apply to WeKan.

Templates and browser UI

Current WeKan dependency or facilitySelected Free Pascal equivalentCompatibility and migration notes
Jade templatesPascal HTML rendering helpers, as used by Omi/WamiPug is Jade's JavaScript successor, but it is not required by the selected server-rendered Pascal design. A template engine could be added later if helpers become unmanageable.
BlazeServer-rendered HTML pages and formsHelpers, events and lifecycle code become route handlers and view helpers. Preserve names and selectors needed by accessibility tools and tests where practical.
Flow Routerhttproute server routesExisting board, card, public and authentication URLs remain compatibility requirements.
Tracker reactivityPage reloads or targeted modern-browser updatesThe core path must remain functional without JavaScript. Optional realtime enhancement must not create a second authorization model.
jQuery and jQuery UINo dependency for core UI; retain selected client files only where still neededPrefer HTML controls and CSS. Every retained widget remains a browser dependency, not a Pascal library.
Touch Punch and dragscrollWami's selected interact.js enhancementUse Pointer Events through the library where supported; forms and selection controls provide the non-drag fallback.
Multi-card draginteract.js multi-touch experimentKeep keyboard and checkbox/button alternatives so touch gestures are never the only operation path.
AutosizeNormal textarea plus optional small browser scriptThe server cannot measure rendered browser controls.
HotkeysAccess keys and ordinary HTML navigation; optional browser keyboard handlerAvoid shortcuts that conflict with assistive technology or text entry.
TextcompleteFull-page or form-based selection in the minimal UI; optional modern componentMentions and emoji can work through explicit selection before a caret-aware enhancement exists.
FullCalendarServer-rendered table calendarWami specifically prefers one table for screen-reader compatibility. Rich interaction may progressively enhance that table.
Font AwesomeExisting copied CSS/fonts for modern browsers; text labels for universal UIIcons must not be the only accessible name or status indicator.
DOMPurifyAvoid injecting untrusted HTML; escape output in Pascal and use a strict server sanitizer for allowed markupOmi's HtmlEncode pattern is the baseline. A new sanitizer requires adversarial fixtures; simple string replacement is insufficient.
Markdown-it and pluginsOmi's server-side Markdown subset, expanded behind a tested renderer interfaceExact compatibility, raw HTML handling and sanitization are more important than matching every plugin immediately.
Temml/math renderingRetain an optional browser rendererBasic browsers may show the source expression. Do not make math JavaScript block the rest of a card.
i18next and sprintffpjson locale loading and an escaped Pascal t() helperOmi already consumes WeKan-style JSON locale files. Placeholder inventories and fallback rules must match English exactly.
JSZipServer-side archive implementation or retained optional browser assetNo archive unit is selected yet. Any choice needs traversal, expanded-size and entry-count limits.

Authentication, integrations and files

Current WeKan dependency or facilitySelected Free Pascal equivalentCompatibility and migration notes
Accounts PasswordOmi-style server sessions and brute-force lockout, upgraded with a maintained password KDF and secure random sourceDo not retain plaintext password files. Support migration from current hashes, resume-token revocation and constant-time verification.
Session cookiesOmi's signed hidden POST fields and rotating one-use counter; cookies may remain an optional modern modeOmi deliberately avoids cookies. Bind tokens to the action and expiry; make IP binding configurable because mobile networks and proxies change addresses.
CSRF protectionAction-bound one-use form token checked on every state changeToken rotation is useful only with replay-safe server state and correct concurrent-tab behaviour.
OIDC, OAuth, LDAP and CASCustom adapters over maintained Pascal HTTP/TLS and protocol libraries; none selected by Omi/Wami yetTreat these as prototype gates. Provider discovery, signatures, redirects, TLS and logout require integration tests before parity is claimed.
SandstormCustom header/capability adapterPreserve identity, sharing and lifecycle semantics; it is independent of HTML rendering.
Meteor EmailSMTP client selected after supported-platform testingNo mail dependency is selected yet. TLS availability on retro targets will differ from modern Linux and Windows.
ostrio:filesOmi-style filesystem storage with SQLite metadata and TFileStream responsesCanonicalize paths, authorize before opening, support ranges, and stream rather than buffering large files.
Local attachmentsFilesystem beside a writable data directoryKeep executables/assets read-only and data outside the install directory where platform conventions require it.
AWS S3, Azure Blob and Google Cloud StorageStorage interface plus provider adapters; no SDK selected by Omi/WamiDo not hand-code cloud signing casually. A helper service is acceptable when a maintained Pascal SDK is unavailable.
WebhooksPascal HTTP client with explicit TLS, timeout, redirect and SSRF policyUser-configured URLs must not reach loopback, metadata services or private networks unless explicitly allowed.
PDFKitServer-side PDF adapter or an external helperNo selected Pascal PDF library exists in the prototypes. Verify Unicode, fonts and pagination against fixtures.
ExcelJSSpreadsheet adapter or an external helperNo selected library exists. Preserve XLSX import/export behaviour before removing the current implementation.
Papa ParseFPC CSV parsing using Classes/SysUtils or a small audited parserCover quoting, embedded newlines, encodings, delimiters and spreadsheet formula injection.
Archiver and unzipperFPC archive units selected per target, or a supervised helperEnforce path containment, entry limits and decompression limits consistently.
Filesystem globbingFindFirst, FindNext, FindClose and explicit matching helpersRestrict every operation to a resolved root; do not concatenate untrusted paths.

Build, tests and distribution

Current WeKan dependency or facilitySelected Free Pascal equivalentCompatibility and migration notes
Meteor build tool and Rspackfpc build script plus an explicit asset-generation stepOmi compiles SQLite C to an object and links it into the executable. CSS/JS/images/translations need their own reproducible resource step.
npm dependency downloadRepository-owned minimal code and pinned source archivesWami's goal is offline compilation. Vendored code still needs provenance, checksums, licenses and security update procedures.
SQLite runtimePinned SQLite amalgamation statically linked, as in OmiThis produces no separate SQLite runtime library, but the database file remains external writable data.
Static web assetsFPC resources compiled into the binary, or files beside it during developmentOmi currently uses adjacent files. Embedding them is additional work required for a true one-file server.
Mocha, Chai and SinonFPCUnit plus small unit executables and HTTP fixture testsKeep existing JavaScript tests as behavioural specifications during migration.
PlaywrightKeep Playwright for modern Chromium, Firefox and WebKitAdd HTML-level tests with JavaScript disabled and selected retro-browser smoke tests. Free Pascal does not replace browser automation.
Import testsShared fixtures run against Meteor and Pascal implementationsRound-trip existing board, user, permission, attachment and activity shapes.
Database conformanceIdentical operation fixtures for SQLite mode, MongoDB and FerretDB adaptersExplicitly document intentional SQLite semantic differences rather than hiding them.
Docker and ComposeKeep for modern server deploymentsThe Pascal binary can make the image smaller, while Caddy and database topology remain deployment choices.
Snap, Flatpak and AppImageKeep where usefulNative binaries simplify packaging but do not remove metadata, sandbox permissions, migrations or updates.
Retro platformsDirect FPC builds with platform-specific feature matricesfphttpapp, SQLite, TLS, threads and mORMot do not have identical capabilities on every FPC target.
Single executableFPC executable + linked SQLite + compiled resourcesThis can contain the application and read-only assets. Writable SQLite data and attachments must remain outside it.

Advantages

Scale down first

The Omi/Wami stack can run without Node.js, npm or a client JavaScript runtime. Simple HTML, a native server and SQLite can substantially reduce startup, memory and disk needs. This also gives old, text-mode and accessibility-oriented browsers a useful core interface instead of an unsupported blank page.

Broad native platform coverage

Free Pascal supports modern Linux, Windows and macOS as well as targets that are not served by current Node.js releases. The promise must be per dependency, not per compiler: a CPU supported by FPC may still lack compatible threads, TLS, SQLite locking, WebSockets or mORMot optimizations.

Progressive enhancement

Wami separates product operations from drag-and-drop gestures. Checkboxes, buttons and form submissions create an auditable baseline; modern browsers can add interact.js, richer styling and realtime updates. This benefits keyboard navigation and provides a fallback when scripts fail or networks are unreliable.

Native SQLite deployment

Omi demonstrates linking a pinned SQLite amalgamation into the executable. A single database file is attractive for personal, offline and small-team WeKan installations, backups and transfers. Streaming attachments separately avoids inflating that file with large binary content.

Disadvantages and risks

The selected stack does not replace Meteor semantics

fphttpapp handles requests and fpjson parses JSON. Neither provides reactive queries, DDP, optimistic writes, subscription teardown or accounts. The Wami server-rendered design intentionally changes those mechanics; compatibility tests must distinguish acceptable redesign from missing behaviour.

Ecosystem gaps

Pascal has fewer current, widely reviewed packages for OIDC, cloud SDKs, office documents, browser tooling and some security protocols than JavaScript, Go or the JVM. External helper processes may be safer and cheaper than maintaining private protocol implementations, although they weaken the one-file goal.

Two scale targets can split the implementation

The small FCL server and the high-scale mORMot 2 server should share domain and storage interfaces, not become two unrelated WeKans. Adding mORMot pre-emptively would conflict with Wami's minimal/offline goal; adding it late without clean boundaries would require another rewrite.

SQLite changes database behaviour

MongoDB documents, arrays and update operators do not map mechanically to a wide SQLite schema of text columns. Transactions, ordering, nullability, indexes and concurrent writes need a deliberate relational design. Wami's schema is valuable input, not yet a production migration guarantee.

Retro compatibility constrains dependencies

Modern TLS, Unicode, threading and filesystem guarantees cannot be assumed on every old target. A small offline/LAN edition may support fewer integrations than the modern server edition. The feature matrix must say so explicitly rather than silently weakening security to make an old platform compile.

Effects on performance and tests

Native compilation and the small FCL stack should make server startup and pure unit tests fast. Server-rendered pages may reduce browser JavaScript parsing and reactive work. SQLite can be fast for a single-server workload when transactions, indexes and journal mode suit the operating system.

Performance claims in Wami's scale notes are hypotheses until measured with WeKan data and behaviour. WAL mode is not available or appropriate on every retro filesystem, and one writer remains a serialization point. Benchmark board loads, permission filtering, card moves, activities, attachments and realtime fan-out with realistic contention.

The complete test suite still needs real browsers, imports, databases and integrations. Add a JavaScript-disabled browser profile and HTML form tests, but keep Playwright's three modern engines. Faster Pascal unit tests complement those tests; they do not make them unnecessary.

Suggested prototype

Extend Wami rather than beginning a third unrelated Pascal experiment:

  1. Extract reusable HTML escaping, translation, routing, session and SQLite code from Omi into small reviewed Pascal units.
  2. Normalize the Wami SQLite schema for users, boards, swimlanes, lists, cards, memberships and activities, with typed fields and required indexes.
  3. Implement login with a production password KDF, secure randomness, lockout, revocation and Omi-style one-use action tokens.
  4. Render one board using only visible cards, with keyboard-accessible forms to create, move and archive a card without JavaScript.
  5. Add interact.js as progressive enhancement while keeping the same server authorization and form fallback.
  6. Stream an attachment from a canonicalized filesystem path with range and size tests.
  7. Run the same permission, import and browser fixtures against Meteor and Wami, including JavaScript-disabled and reconnect cases.
  8. Load-test fphttpapp first. Prototype mORMot 2 behind the same interfaces only if measured concurrency or WebSocket requirements exceed it.
  9. Build offline for modern amd64/arm64 and one retro target, documenting which TLS, SQLite, realtime and integration features each artifact supports.

Recommendation

Use Omi's maintained Free Pascal server as the implementation reference and Wami's scale-down, server-rendered design as the product reference. Start with FCL fphttpapp, httproute, fpjson, statically linked SQLite, server-rendered HTML and optional interact.js. Keep Caddy at the public TLS edge. Do not add mORMot 2 or Brook until a compatible vertical slice has been measured and the small stack is demonstrably the limit.

Treat a Free Pascal WeKan as a deliberate accessible/offline architecture, not as a line-by-line port of Blaze and Meteor. A single executable containing the server, SQLite engine and read-only assets is realistic; writable databases, attachments and external identity/storage services remain outside the binary.