one-os-dev-wayland-html5-compositor-greenfield-analysis/00-overview.md

00 — What Greenfield Is and How It's Structured

The one-paragraph summary

Greenfield is a Wayland compositor that runs partly in the browser (the actual compositing and rendering, written in TypeScript against WebGL / WebCodecs / WASM) and partly on a Linux host (a Node.js process that hosts real Linux Wayland apps via libwayland-server and bridges their wire traffic to the browser over WebSocket). The two halves together let you display real Linux applications (GTK3/4, Qt, xterm, X11 apps via XWayland) inside a browser tab, as if they were native browser apps. An optional Emscripten-based SDK lets you compile separate Wayland apps to WASM and run them entirely in the browser with no Linux host required.

Repo layout

greenfield/
├── README.md                          # "The in-browser wayland compositor"
├── package.json                       # yarn 4.5.0 workspaces: libs/*, packages/*, examples/webapps/*, examples/compositor/*
├── .yarnrc.yml                        # nodeLinker: pnpm, global cache off
├── yarn.lock                          # 9368 lines
├── publish.sh                         # yarn workspaces foreach npm publish
├── protocol/                          # Wayland XML protocol definitions (4 files)
│   ├── wayland.xml                    # Core Wayland protocol (124KB)
│   ├── xdg-shell.xml                  # Desktop shell protocol (48KB)
│   ├── wayland-drm.xml                # DRM buffer formats (8.8KB)
│   └── web-bitmapbuf-unstable-v1.xml  # *** Greenfield custom extension (2KB) ***
├── libs/                              # 9 internal @gfld/* libraries
│   ├── common/                        # @gfld/common - shared client/server runtime (wire format)
│   ├── client-protocol/               # @gfld/client-protocol - browser-side generated Wayland bindings
│   ├── compositor-protocol/           # @gfld/compositor-protocol - Node.js-side generated Wayland bindings
│   ├── client-generator/               # CLI: XML → TS browser protocol source
│   ├── compositor-generator/          # CLI: XML → TS Node protocol source
│   ├── compositor-proxy-generator/    # CLI: XML → TS proxy protocol source
│   ├── compositor-wasm/               # pixman + xkbcommon compiled to WASM
│   ├── compositor-ffmpeg-h264/        # FFmpeg n5.1.2 compiled to WASM (H.264 software fallback decoder)
│   └── xtsb/                          # X11/XCB TypeScript bindings (vendored)
├── packages/                          # 4 user-facing @gfld/* packages
│   ├── compositor/                    # @gfld/compositor - browser-side Wayland compositor library (AGPL)
│   ├── compositor-shell/              # @gfld/compositor-shell - Preact+Tailwind SPA shell (AGPL) *** OUT OF SCOPE ***
│   ├── compositor-proxy/              # @gfld/compositor-proxy - native Wayland compositor proxy (AGPL, Linux-only)
│   └── compositor-proxy-cli/          # @gfld/compositor-proxy-cli - CLI binary (AGPL, Linux-only)
├── sdk/                               # Greenfield WebAssembly SDK (for browser-resident WASM apps)
│   ├── README.md
│   ├── build.sh                       # clones emsdk 3.1.60, builds 30+ sysrootlibs
│   ├── sdk_env.sh                     # exports PKG_CONFIG_LIBDIR + PATH=toolkit/bin
│   ├── toolkit/                       # gfcc/gf++/gfar/gfnm wrappers around emcc + wayland-scanner + meson cross inis
│   └── sysrootlibs/                   # 34 build scripts for cross-compiling graphics stack (wayland, pixman, cairo, pango, gtk4, etc.)
├── examples/
│   ├── compositor/experimental-fullscreen/   # vite PWA kiosk running a single remote app
│   ├── docker-compose/                # Dockerfile + applications.json + docker-compose.yml for gtk3-demo/gtk4-demo/xterm/glxgears
│   ├── sdk/simple-shm/                # Weston simple-shm.c built with gfcc + wayland-scanner
│   ├── sdk/weston/                    # Weston clients (terminal, dnd, editor, keyboard, flower, ...) meson-built
│   ├── sdk/gtk4/                      # GTK4 demos meson-built to WASM
│   └── webapps/                       # Pure-JS browser demos: simple-shm, webgl, weston-clients (symlink), gtk4
├── docs/                              # Jekyll site (CNAME: udev.be/greenfield)
│   └── pages/{components, design, future_plans, getting_started, sdk}
└── docker/                            # Production Dockerfile for compositor-proxy-cli

The dual-side architecture

Greenfield is split into two cooperating halves, connected over WebSocket (or, in the future, WebTransport). Understanding this split is fundamental to everything else in this bundle.

┌─────────────────────────────────────────────────────────────────────────────┐
│                              BROWSER SIDE (TypeScript + WASM)                │
│                                                                             │
│  ┌─────────────────────────┐    ┌──────────────────────────────────────────┐ │
│  │ @gfld/compositor-shell │    │ @gfld/compositor                          │ │
│  │ (Preact + Tailwind SPA)│───▶│  - Session, Compositor, Surface, Seat     │ │
│  │ *** OUT OF SCOPE ***    │    │  - Keyboard, Pointer, Touch, Xkb (WASM)   │ │
│  └─────────────────────────┘    │  - Output (WebGL canvas)                  │ │
│                                 │  - render/ (WebGL shaders: YUV→RGB, etc.) │ │
│  ┌─────────────────────────┐    │  - desktop/ (Floating vs AlwaysFullScreen) │ │
│  │  JS/WASM webapps       │───▶│  - remote/ (Channel, kcp.ts, decoders)    │ │
│  │  (use @gfld/client-    │    │  - browser/ (dnd, selection, input)        │ │
│  │   protocol + common)   │    │                                            │ │
│  └─────────────────────────┘    └──────────────┬──────────────┬─────────────┘ │
│                                                 │              │               │
│                       depends on                ▼              ▼               │
│                ┌─────────────────┐  ┌──────────────┐  ┌──────────────────┐   │
│                │ @gfld/compositor│  │ @gfld/common │  │ @gfld/xtsb        │   │
│                │ -protocol       │  │ (Connection, │  │ (X11/XCB bindings,│   │
│                │ (Node-side      │  │  wire format)│  │  used by both     │   │
│                │  generated      │  │              │  │  compositor+proxy)│   │
│                │  bindings)       │  │              │  └──────────────────┘   │
│                └─────────────────┘  └──────────────┘                          │
└─────────────────────────────────────────────────────────────────────────────┘
                                          │
                                          │ WebSocket (TCP, perMessageDeflate=false)
                                          │ + KCP-over-WebSocket (reliable UDP, ready for WebTransport)
                                          │ + 9 endpoint paths per session
                                          ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                            NATIVE SIDE (Linux, Node 18+, C)                 │
│                                                                             │
│  ┌──────────────────────────┐    ┌──────────────────────────────────────────┐ │
│  │ @gfld/compositor-proxy-cli│───▶│ @gfld/compositor-proxy                   │ │
│  │  - main.ts: http + ws     │    │  - Session, SessionController            │ │
│  │    WebSocketServer        │    │  - NativeWaylandClientSession            │ │
│  │  - forks SessionProcess   │    │  - NativeWaylandCompositorSession        │ │
│  │    per browser session    │    │  - NativeAppContext (launch)             │ │
│  │  - --bind-port=8081       │    │  - SurfaceBufferEncoding                 │ │
│  │  - --applications=<json>  │    │  - Channel, FrameFeedback                │ │
│  │  - --encoder=x264|nvh264  │    │  - XWaylandSession                       │ │
│  │  - --render-device=/dev/  │    │  - kcp.ts (1100-line TypeScript impl)     │ │
│  │    dri/renderD128         │    │  - wayland-server.ts (TS wrapper)         │ │
│  └──────────────────────────┘    │  - io/, encoding/, protocol/              │ │
│                                  └──────────────┬──────────────┬─────────────┘ │
│                                                 │              │               │
│                              depends on         ▼              ▼               │
│                ┌────────────────────────┐  ┌────────────────────────────────┐ │
│                │ @gfld/compositor-proxy-│  │ native/ (C, built via CMake +   │ │
│                │  generator             │  │ Ninja)                          │ │
│                │ (generates *_interface │  │  - wayland/src/addon.c          │ │
│                │  and *_interceptor.js) │  │  - encoding/src/gst_frame_     │ │
│                └────────────────────────┘  │           encoder.c             │ │
│                                            │  - poll/src/poll.c (libuv)      │ │
│                                            │  - wayland-server/ (patched)    │ │
│                                            │  - westfield-*.c (EGL, dmabuf,   │ │
│                                            │     XWayland, etc.)             │ │
│                                            └────────────────────────────────┘ │
│                                                                             │
│  Native Wayland apps (GTK3/4, Qt, xterm, X11 via XWayland) attach to the    │
│  proxy as if it were a real wl_display. The proxy encodes each surface's    │
│  buffer to H.264 via GStreamer (x264 software or nvh264 hardware) using     │
│  zero-copy dmabuf, sends frames over WebSocket/KCP to the browser.          │
└─────────────────────────────────────────────────────────────────────────────┘

Tech-stack table

Category	Component	Tech / Version	Role
Languages	Browser TS	TypeScript ^5.5.2	All @gfld/* TS packages
	Native C	C99	packages/compositor-proxy/native/
	Shell	bash	sdk/build.sh, toolkit wrappers
	Python	python3 (xcbgen)	libs/xtsb protocol generation
Build tools	Workspace	Yarn 4.5.0 (Berry, pnpm nodeLinker)	Monorepo workspace mgmt
	Browser bundler	Vite ^5.3.1	compositor-shell, examples
	TS compiler	tsc ^5.5.2	libs/* and packages/* build
	Native build	CMake 3.13+ + Ninja	packages/compositor-proxy build:native
	Native pkg	@vercel/pkg (Brotli) + @vercel/ncc	compositor-proxy-cli single-binary
	Lint/format	ESLint ^8.57 + Prettier ^3.3.2	All TS packages
	Test	Jest ^29.7.0 + ts-jest ^29.1.5	libs/xtsb, packages/compositor-proxy
WASM toolchain	Emscripten (libs build)	emsdk 3.1.46	libs/compositor-wasm, libs/compositor-ffmpeg-h264
	Emscripten (SDK build)	emsdk 3.1.60	sdk/build.sh (sysrootlibs)
	Meson cross	meson-gf-cross.ini (host_machine: wasm32/emscripten)	sdk/toolkit
Wayland / GL stack	wayland.xml / xdg-shell.xml / wayland-drm.xml	upstream freedesktop.org	protocol/
	wayland-scanner	vendored at sdk/toolkit/bin/wayland-scanner	C header/codegen for native clients
	libwayland (server)	vendored + patched at packages/compositor-proxy/native/wayland/src/wayland-server/	native compositor server side
	wl_drm + linux-dmabuf-unstable-v1	implemented in native/wayland/src/wlr_*.c	zero-copy dmabuf to GStreamer
Custom protocol	web-bitmapbuf-unstable-v1.xml	Greenfield extension (Erik De Rijcke, 2023)	browser JS/WASM clients wrap HTML ImageBitmap as wl_buffer via fd
Native GL/media runtime	GStreamer	1.20 (built from gitlab.freedesktop.org in docker)	H.264 encoding pipeline, plugins: base/good/bad/ugly/gl + nvcodec
	GStreamer encoders	x264 (software), nvh264 (NVIDIA hardware), vaapih264 (Intel, experimental)	--encoder flag on compositor-proxy-cli
	Mesa / OpenGL / EGL	libgbm1, libdrm2, libopengl0, libglvnd0, libegl1-mesa	GPU buffer mgmt + EGL rendering
	NVIDIA	libnvidia-egl-wayland1, libnvidia-egl-gbm1	optional HW accel
Browser transport	WebSocket	ws ^8.17.1 (Node), native WebSocket (browser)	primary transport (TCP)
	KCP	TypeScript impl (1100+ lines per side)	planned reliable UDP — future switch to WebTransport
	WebTransport	mentioned in docs/design as future	not yet implemented
	XWayland	xwayland package + xauth + xxd	X11 backward compat on native side
Browser rendering	WebGL	WebGL1 native, desynchronized:true	Output.ts canvas, render/*.ts shaders
	WebCodecs API	@types/dom-webcodecs ^0.1.11	H.264 GPU decode in browser (preferred path)
	WebAssembly (FFmpeg)	libs/compositor-ffmpeg-h264 (FFmpeg n5.1.2)	software fallback H.264 decode
	WebAssembly (pixman+xkbcommon)	libs/compositor-wasm (emsdk 3.1.46)	pixel ops + XKB keymap math
	SharedArrayBuffer + Web Workers	browser-native (COOP/COEP required)	async compositor, H.264 NAL decoder workers
	ImageBitmap	browser-native	web-bitmapbuf-unstable-v1 backing store
Shell UI (out of scope)	Preact	^10.22.0	compositor-shell SPA
	Tailwind CSS	^3.4.4	compositor-shell styling
	@preact/signals	^1.2.3	reactivity

Package inventory

Browser side

packages/compositor (@gfld/compositor, AGPL-3.0) — The browser-side Wayland compositor library. Pure TypeScript + WebGL + WebCodecs + WASM, no 3rd-party runtime deps. Exports createCompositorSession() and createAppLauncher(session, 'web' | 'remote'). Internally: Session.ts orchestrates, Compositor.ts/Surface.ts/Seat.ts/Pointer.ts/ Keyboard.ts/Output.ts implement Wayland semantics, render/ holds WebGL shaders (YUV/YUVA/RGB→RGBA conversion), desktop/ arranges surfaces (Floating vs AlwaysFullScreen), web/ speaks to same-origin-iframe JS/WASM clients, remote/ speaks to the native proxy (with kcp.ts, websocket-events.ts, webcodec-buffer-decoder.ts, wasm-buffer-decoder.ts, H264NALDecoder.worker.ts, xwayland/), browser/ handles dnd/selection/ input. Output is a WebGL texture inside an HTML5 canvas. Async — a slow client never blocks another.

libs/client-protocol (@gfld/client-protocol, MIT) — Browser-side generated Wayland protocol bindings (wayland.xml + xdg-shell.xml + web-bitmapbuf-unstable-v1.xml). Used by JS/WASM webapps via @gfld/common for the wire format.

libs/compositor-protocol (@gfld/compositor-protocol, MIT) — Node-side generated Wayland protocol bindings (same 3 XMLs). Used by @gfld/compositor for the in-browser server-side marshalling of Wayland messages.

libs/common (@gfld/common, MIT) — Shared client/server Wayland wire-format runtime. Two source files: Connection.ts (message framing + argument codec) + index.ts. Tiny core that both client-protocol and compositor-protocol build on.

libs/compositor-wasm (@gfld/compositor-wasm, MIT) — pixman 0.42.2 + xkbcommon 1.5.0 compiled to WASM via emsdk 3.1.46. Exposes src/index.ts (init function). Used by @gfld/compositor for pixel-region math and XKB keyboard layout state.

libs/compositor-ffmpeg-h264 (@gfld/compositor-ffmpeg-h264, Apache-2.0) — FFmpeg n5.1.2 compiled to a single WASM module. TypeScript wrapper in src/{H264Decoder.ts, H264Worker.ts, index.ts} runs in a Web Worker. Used as the software fallback H.264 decoder when the browser lacks WebCodecs.

Native side

packages/compositor-proxy (@gfld/compositor-proxy, AGPL-3.0, Linux-only, Node ≥18) — Native Wayland compositor that bridges Linux apps to the browser. Has BOTH a TypeScript layer (src/Session.ts, NativeWaylandClientSession.ts, NativeWaylandCompositorSession.ts, NativeAppContext.ts, SurfaceBufferEncoding.ts, XWaylandSession.ts, kcp.ts, Channel.ts, FrameFeedback.ts, wayland-server.ts) AND a C layer (native/) built with cmake -G Ninja. The C layer produces:

• wayland-server.so — patched libwayland-server (vendored at native/wayland/src/wayland-server/, with changes.diff and generate_wayland_server_patch.sh)
• westfield.so — Greenfield's westfield-* helpers (buffer, dmabuf, egl, xwayland, fdutils) + wlr_drm.c + wlr_linux_dmabuf_v1.c + drm-protocol.c + linux-dmabuf-unstable-v1-protocol.c
• proxy-encoding.so — GStreamer frame encoder (gst_frame_encoder.c + gst_main_loop.c)
• proxy-encoding-addon.node + proxy-poll-addon.node + wayland-server-addon.node — Node-API addons (vendored node-api headers in native/node-api/) loaded by src/addons/
• poll.c — epoll wrapper backed by libuv

CMake deps: glib-2.0, gstreamer-1.0/app/video/gl/allocators ≥1.18, graphene-1.0, egl ≥1.5, opengl ≥3.3, libffi, gbm, libdrm. Runtime also needs libudev, mesa, NVIDIA EGL wayland/gbm drivers, xwayland. Implements wl_drm

• linux-dmabuf-v1 for zero-copy GPU buffer transfer to GStreamer encoder pipeline (x264 software or nvh264 hardware). Connects to browser via WebSocket; has TypeScript KCP implementation ready for the future WebTransport switch. Supports direct peer-to-peer copy-paste between two proxy instances to avoid browser round-trip.

packages/compositor-proxy-cli (@gfld/compositor-proxy-cli, AGPL-3.0, Linux-only, Node ≥18) — CLI binary that wraps @gfld/compositor-proxy. src/main.ts runs an HTTP server + ws WebSocketServer on --bind-port (default 8081), and on first connection per compositorSessionId query param, fork()s a SessionProcess (src/SessionProcess.ts) — one child per active browser session. CLI args: --applications=<json> (URL→executable map), --encoder=x264|nvh264, --render-device=/dev/dri/renderD128, --base-url, --allow-origin, --bind-ip, --bind-port, --basic-auth. The --applications JSON maps paths like /gtk4-demo, /kwrite, /xterm to executables + args + env. Packaged with @vercel/pkg (Brotli, target node18-linux) into a single static binary.

libs/compositor-proxy-generator (@gfld/compositor-proxy-generator, MIT) — CLI that converts protocol/{wayland,xdg-shell,wayland-drm}.xml into TypeScript source under packages/compositor-proxy/src/protocol/. Used at build time only (yarn generate script).

Shell side (OUT OF SCOPE for user)

packages/compositor-shell (@gfld/compositor-shell, AGPL-3.0) — Preact

• Tailwind single-page-app shell built with Vite. Depends only on @gfld/compositor + preact at runtime. Provides application launcher, keyboard config, window management UI. Explicitly out of scope for the user per task brief. The protocol-level integration point the user needs to know about is UserShellApi (see 08-app-lifecycle.md).

Generators (3 sibling CLIs)

All three generators share the same dependency set (camelcase, uppercamelcase, meow, xml2js) and similar layout (ProtocolParser.mjs + ProtocolArguments.mjs + index.mjs). They consume the same protocol/*.xml files but emit slightly different TypeScript flavors for each side:

• libs/client-generator — emits browser-side client bindings into libs/client-protocol/src/protocol/.
• libs/compositor-generator — emits Node.js-side server bindings into libs/compositor-protocol/src/protocol/.
• libs/compositor-proxy-generator — emits proxy-side bindings into packages/compositor-proxy/src/protocol/. Different file layout: src/EndpointProtocolArguments.mjs + EndpointProtocolParser.mjs + index.mjs.

XWayland support

libs/xtsb (@gfld/xtsb, MIT, repo: github.com/udevbe/xtsb) — Vendored TypeScript X11/XCB bindings. Builds from xcbgen/ts_client.py against 5 XCB protocol XMLs in libs/xtsb/protocol/: xproto.xml, shape.xml, xfixes.xml, render.xml, composite.xml. Dual main/module entries: dist/main/index.node.js (Node) + dist/module/index.web.js (browser). Used by BOTH @gfld/compositor (browser side, packages/compositor/src/remote/xwayland/) AND @gfld/compositor-proxy (native side, src/XWaylandSession.ts) — same TS API on both ends of the wire. On the native C side, packages/compositor-proxy/native/wayland/src/westfield-xwayland.c actually launches XWayland and bridges it.

Wire between browser and native

• Transport today: WebSocket over TCP. Browser connects to ws(s)://<proxy-host>:<port>/<app-path>?compositorSessionId=<uuid>. compositor-proxy-cli uses ws ^8.17.1 with perMessageDeflate: false (compression off for latency). The compositorSessionId is generated by the browser and used by the CLI to route the upgrade to the correct per-session child process.
• Transport future: KCP-over-WebTransport (reliable UDP). The codebase already has a full TS KCP implementation in both packages/compositor-proxy/src/kcp.ts (~1100 lines) and packages/compositor/src/remote/kcp.ts. docs/pages/design/index.md explicitly calls out that WebSockets-over-TCP is ill-suited for real-time, that WebRTC DataChannels' SCTP congestion control is too slow, and that they're waiting for WebTransport + will pair it with KCP + forward-error-correction.
• Out-of-band data: copy-paste between two proxy instances goes peer-to-peer (proxy → proxy) without browser round-trip, configured via --base-url so each proxy can advertise itself to peers.

SDK (sdk/) — for browser-resident WASM apps, NOT native apps

The Greenfield SDK lets you compile existing native Wayland C/C++ apps to WebAssembly and run them directly in the browser compositor (no proxy needed). Based on Emscripten 3.1.60 with three Greenfield-specific extensions:

1. sdk/toolkit/bin/gfcc and gf++ — wrapper scripts around emcc/em++ that inject pthread pool size, asyncify, and the unix-socket fs shim.
2. sdk/toolkit/bin/wayland-scanner — symlink to the natively-built wayland-scanner inside build-sysroot/bin/wayland-scanner, used to generate C client headers/source from xdg-shell.xml during app builds.
3. sdk/sysrootlibs/ — 34 build scripts that cross-compile a full graphics stack (wayland, wayland-protocols, pixman, zlib, png, xml2, xkbcommon, icu, harfbuzz, freetype, fontconfig, pcre2, glib, jpeg, cairo, fribidi, tiff4, pango, gdk-pixbuf, egl-stub, glesv2-stub, epoxy-emscripten, graphene, sass, sassc, uapi-stub, upoll, cygepoll, expat, libffi; gtk4 currently broken) into sdk/sysroot/ for static linking.

sdk/build.sh runs everything end-to-end and writes meson-gf-toolchain.ini (with absolute _SDK_DIR). sdk/sdk_env.sh exports PKG_CONFIG_LIBDIR (pointing at sysroot/lib/pkgconfig + build-sysroot/lib/pkgconfig) and prepends toolkit/bin to PATH. Meson projects use --cross-file toolkit/meson-gf-cross.ini --cross-file toolkit/meson-gf-toolchain.ini (host_machine: cpu=mvp, cpu_family=wasm32, system=emscripten).

Important: the sdk/ is not for native apps. It's exclusively for WASM apps that run inside the browser via Emscripten. Native Linux apps on the proxy use the host's own system libraries — the Dockerfile installs them via apt-get install libffi8 libgbm1 libgstreamer1.0-plugins-base … xwayland xauth.

Versioning note

Every workspace package is 1.0.0-rc1. This signals:

1. Feature freeze for 1.0 — the architecture, protocol bindings, and public APIs are expected to be stable; no breaking changes anticipated before 1.0.0 final.
2. Not yet production-blessed — known issues remain (e.g. gtk4 WASM is "broken for now" per sdk/build.sh comment; WebTransport/KCP-over-UDP is designed but not yet active; Linux Kernel Library WASM + JuiceFS web FS are explicitly future work).
3. Yarn Berry semantics — workspace:^ resolution means all @gfld/* deps are pinned to rc1 across the tree; if the user forks and bumps, they bump the whole tree consistently.

For the user's purpose (building their own Wayland-to-HTML compositor for an Odoo 20 + JS/Python web OS), the rc1 status means they should treat Greenfield as a stable architectural reference but plan to pin to a specific commit if they fork it, since 1.0.0 final may land breaking-ish changes before the rc1 → 1.0.0 transition completes.

License of the source

The four packages/* (compositor, compositor-proxy, compositor-proxy-cli, compositor-shell) are AGPL-3.0-or-later. The lower-level libs/* are MIT or Apache-2.0. Anyone forking or building on the runtime packages must respect AGPL — see 11-pitfalls.md §License for implications.