Files

444 lines
27 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 01 — Layered Architecture Map
This chapter walks Greenfield's architecture layer by layer, from the lowest
abstraction (Wayland XML) up to the highest (the user-facing shell). Each layer
is described in terms of: **responsibility**, **key abstractions**,
**dependencies**, and **files of interest**. The chapter ends with an ASCII
diagram showing how the layers stack and where data flows between them.
## The eight layers
```
┌──────────────────────────────────────────────────────────────────────────┐
│ Layer 8: USER SHELL (Odoo 20 OWL addon, supplied by user) │
│ Hooks: UserShellApi (events + actions) │
│ ─ NOT covered in this chapter (see 08-app-lifecycle.md) │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 7: DESKTOP / LAYOUT POLICY │
│ Floating vs AlwaysFullScreen, view stacking, focus activation refcount │
│ packages/compositor/src/desktop/Desktop.ts │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 6: SURFACE LIFECYCLE / ROLES │
│ XdgWmBase, XdgSurface, XdgToplevel, XdgPopup, XdgPositioner, Shell, │
│ ShellSurface, Subcompositor, Subsurface, SurfaceRole │
│ packages/compositor/src/{Xdg*, Shell*, Sub*, SurfaceRole, SurfaceChild} │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 5: RENDERING PIPELINE │
│ Surface (state stack), View, Output (WebGL canvas), Renderer, Scene, │
│ Shaders (YUV→RGBA, YUVA→RGBA, RGB+A→RGBA), Loop (RAF), Callback │
│ packages/compositor/src/{Surface, View, Output, Loop, Callback, Region} │
│ packages/compositor/src/render/ │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 4: TRANSPORT (browser side) │
│ Channel (ARQ + Simple), ReconnectingWebSocket, RemoteAppLauncher, │
│ StreamingBuffer, BufferStream, EncodedFrame, EncoderFeedback, decoders │
│ (WebCodecs + WASM fallback) │
│ packages/compositor/src/remote/ │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 4: TRANSPORT (native side) │
│ Channel (ARQ + Simple), SessionController, kcp.ts, NativeAppContext, │
│ FrameFeedback, SurfaceBufferEncoding, wayland-server.ts │
│ packages/compositor-proxy/src/{Channel, kcp, Session*, Native*, ...} │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 3: WAYLAND RUNTIME (browser side) │
│ Connection (wire framing), Client, WlObject registry, marshallers │
│ libs/common/src/Connection.ts │
│ libs/compositor-protocol/src/{westfield-runtime-server.ts, generated/*} │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 3: WAYLAND RUNTIME (native side) │
│ wayland-server.ts + wayland-server-addon.node (N-API → libwayland-server)│
│ MessageInterceptor (per-resource request/event routing) │
│ packages/compositor-proxy/src/{wayland-server.ts, protocol/*_interceptor}│
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 2: PROTOCOL GENERATORS │
│ client-generator, compositor-generator, compositor-proxy-generator │
│ libs/{client,compositor,compositor-proxy}-generator/ │
├──────────────────────────────────────────────────────────────────────────┤
│ Layer 1: WAYLAND PROTOCOL XML │
│ protocol/{wayland, xdg-shell, wayland-drm, web-bitmapbuf-unstable-v1}.xml│
└──────────────────────────────────────────────────────────────────────────┘
```
## Layer 1 — Wayland Protocol XML
**Responsibility**: declare the protocol surface — what interfaces, requests,
events, enums, and bitfield types exist on the wire.
**Files of interest**:
- `protocol/wayland.xml` — standard upstream Wayland core. 22 interfaces,
`wl_compositor` v4, `wl_seat` v6, `wl_surface` v4 (roughly Wayland
1.20-ish). Kristian Høgsberg copyright. Used by all 3 generators.
- `protocol/xdg-shell.xml` — stable `xdg_shell` v1, 5 interfaces (xdg_wm_base,
xdg_surface, xdg_toplevel, xdg_popup, xdg_positioner). Used by all 3
generators.
- `protocol/wayland-drm.xml` — legacy `wl_drm` v2, used by native apps (Mesa,
XWayland) to advertise dmabuf buffers. **Only the proxy generator
processes this file** — so the proxy can intercept `create_buffer` /
`create_planar_buffer` / `create_prime_buffer` and turn dmabuf fds into
ProxyFDs for the browser's GStreamer encoding pipeline. Browser-side libs
do not generate `wl_drm` stubs.
- `protocol/web-bitmapbuf-unstable-v1.xml` — **Greenfield's custom
extension**. Single interface `web_bitmapbuf_factory` v1, single request
`create_buffer(id: new_id<wl_buffer>, bitmap: fd)`. Lets browser JS apps
hand a GPU-backed `ImageBitmap` directly to the compositor as a `wl_buffer`
— bypassing SHM copy, encoding, and dmabuf import. The `fd` arg is
marshalled via the same `fileDescriptor(...)` path as
`wl_shm.create_pool`'s fd, but holds an `ImageBitmap` (becomes a
transferable on the flush path). **Not processed by the proxy generator**
— native apps never use it.
**Dependencies**: none. This is the source of truth.
## Layer 2 — Protocol Generators
**Responsibility**: parse the XML with `xml2js`, walk the interface tree,
emit typed TypeScript bindings for each side.
**Key abstractions**: `ProtocolParser.mjs` (shared XML walker) +
`ProtocolArguments.mjs` (per-side argument-type dispatch table).
**Files of interest**:
- `libs/client-generator/{ProtocolParser, ProtocolArguments, index}.mjs` —
emits TS `*Proxy` classes for browser apps that want to be Wayland clients
of the browser compositor (via `MessagePort`).
- `libs/compositor-generator/{ProtocolParser, ProtocolArguments, index}.mjs`
— emits TS `*Resource` + `*Requests` classes for the browser-resident
compositor itself (receives requests, dispatches to `*Requests` impls,
emits events). Skips `wl_display`, `wl_registry` (hand-written in runtime).
- `libs/compositor-proxy-generator/{EndpointProtocolParser,
EndpointProtocolArguments, index}.mjs` — emits CommonJS
`*_interface.js` (wl_interface struct definitions for the N-API addon)
+ `*_interceptor.js` (per-opcode interceptors that decide routing) for
the Node.js native proxy.
**Why three generators?** Because each side has a fundamentally different job:
1. **client-generator** → browser apps that want to be Wayland clients of the
browser compositor (via `MessagePort`).
2. **compositor-generator** → the browser-resident compositor itself (receives
requests, dispatches to `*Requests` impls, emits events).
3. **compositor-proxy-generator** → the Node.js native proxy bridging real
libwayland-client apps to the browser. Emits `wl_interface` struct
definitions for the N-API addon + per-opcode interceptors that decide
routing.
**Dependencies**: `xml2js`, `camelcase`, `uppercamelcase`, `meow`.
## Layer 3 — Wayland Runtime
**Responsibility**: marshal and unmarshal the actual wire bytes. Implement
object lifecycle (creation, dispatch, destruction), new-id tracking, listener
dispatch, file descriptor handling.
**Browser side** (`libs/common/src/Connection.ts`) — the shared runtime for
both `@gfld/client-protocol` and `@gfld/compositor-protocol`:
- Wire format is **standard upstream Wayland**, byte-for-byte. 8-byte
header: `u32 object_id | u16 opcode | u16 size-incl-header`, then per-arg
bytes. `u/i/f/n/o` are 4 bytes; `s` and `a` are length-prefixed u32 +
payload padded to 4-byte alignment; `h` (FD) takes zero bytes — FDs travel
out-of-band in `wireMsg.fds[]`.
- Reading: `wlObject[opcode](message)` — the generated classes use
computed-property-key methods `[0](message) { ... }`, `[1](message) { ... }`
so opcode is the array index.
- Client side: `Proxy.marshallConstructor` calls `display.generateNextId()`
(1-indexed, recycles ids from `wl_display.delete_id`), constructs the new
Proxy (self-registers in constructor), patches the `new_id` arg's `value`,
then flushes.
- Server side: `Client.marshallConstructor` allocates from
`SERVER_OBJECT_ID_BASE = 0xff000000` (or recycles locally). When a
client-allocated id is destroyed, the server sends `wl_display.delete_id`
so the client can recycle.
- Connection registry: `wlObjects: { [id]: WlObject }`; `registerWlObject`
throws on collision.
- `Connection.message` dispatches by literal `wlObject[opcode](message)`.
Each handler returns void or a Promise; `Connection` awaits the promise
before reading the next message — this preserves wire-order while still
allowing async implementations. Per-client Connections mean a slow client
doesn't block others.
**Native side** (`packages/compositor-proxy/src/wayland-server.ts`) — a TS
wrapper around `wayland-server-addon.node` (the C N-API addon that wraps a
**patched** `libwayland-server`):
- `MessageInterceptor` — per-resource request/event dispatch. Each
`wl_*_interceptor.js` file declares `R<opcode>(...args)` and
`E<opcode>(...args)` functions that decide whether a message is routed
`{native: true}` (handle in the proxy), `{browser: true}` (forward to
browser compositor), or both.
- Special cases: `wl_display.sync` → both native and browser (must complete
before `done` event), `wl_display.get_registry` → both, `wl_registry.bind`
→ native if id is in `nativeGlobalNames[]` (e.g. `wl_drm`, `wl_shm`),
else browser. Fast-sync optimization skips browser round trip when no
prior browser-bound request occurred.
**File descriptor handling — the Greenfield innovation**:
`type FD = unknown` — it's intentionally opaque. Browser-side FDs are
**transferable JS objects**:
- `ArrayBufferView` for `wl_shm.create_pool`'s `fd` (SHM backing memory lives
in browser, zero-copy transferred via `MessagePort.postMessage(...,
transferList)`).
- `ImageBitmap` for `web_bitmapbuf_factory.create_buffer`'s `bitmap` arg
(GPU-backed texture, zero-copy).
- `MessagePort` for `wl_data_offer.receive`'s `fd` (in-browser pipe).
Native-side (proxy) FDs are real POSIX ints.
`NativeWaylandClientSession.onWireMessageEnd` converts each to a JSON
`ProxyFD = {handle, type, host}` (host = proxy's HTTP base URL),
length-prefixed + 4-byte aligned, prepended to the wire message before
sending over WebSocket/KCP. Browser's `webFS` resolves them by fetching
data over HTTP.
**Dependencies**: Layer 2's generated bindings + `@gfld/common`.
## Layer 4 — Transport
**Responsibility**: carry wire bytes between the browser compositor and the
native proxy. Multiplex logical channels. Handle reconnection, backpressure,
and latency.
**Browser side** (`packages/compositor/src/remote/`):
- `Channel.ts` — defines `ARQChannel` (KCP-wrapped, reliable, low-latency)
and `SimpleChannel` (raw WebSocket).
- `ReconnectingWebSocket` — vendored from `pladaria/reconnecting-websocket`
with `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s`, `minUptime=5s`,
`connTimeout=4s`, `retries=∞`.
- `RemoteAppLauncher.ts` — sets up the 4 logical channels per app client
(PROTOCOL, FRAME, XWM, FEEDBACK) and routes incoming messages.
- `RemoteInputOutput.ts` — implements the `InputOutput` interface for native
apps, with `mkfifo`, `mkstempMmap`, `readFd`, `writeFdAsStream`, `closeFd`
over separate WebSocket endpoints.
- `StreamingBuffer.ts`, `BufferStream.ts`, `EncodedFrame.ts`,
`EncoderFeedback.ts` — decode pipeline for incoming H.264 / PNG frames.
- `webcodec-buffer-decoder.ts`, `wasm-buffer-decoder.ts`,
`H264NALDecoder.worker.ts` — the actual decoders.
**Native side** (`packages/compositor-proxy/src/`):
- `Channel.ts` — mirror image of the browser side. `ARQChannel` and
`SimpleChannel`.
- `kcp.ts` — ~1100-line TypeScript port of the canonical C `ikcp.c`.
- `SessionController.ts` — WS server with 9 endpoint paths:
`/signal`, `/channel`, `/mkfifo`, `/mkstemp-mmap`, `/read-fd`,
`/read-fd-as-stream`, `/write-fd-as-stream`, `/close-fd`, `/request-keyframe`.
- `Session.ts`, `SessionProcess.ts`, `SessionController.ts` — process model:
one main process, one forked child per browser session.
- `NativeAppContext.ts` — per-app state, signaling WS, lifecycle (10-min
SIGHUP grace on disconnect, SIGTERM→SIGKILL escalation).
- `FrameFeedback.ts` — receives 4-byte feedback messages from the browser,
uses them to pace `wl_frame.done` callbacks.
- `SurfaceBufferEncoding.ts`, `ProxyBuffer.ts` — encoding pipeline.
- `io/` subdirectory — WebSocket server glue.
- `encoding/` subdirectory — high-level encoding orchestration.
**Wire format**: see `04-transport.md` for the full byte-level breakdown of
PROTOCOL, FRAME, XWM, and FEEDBACK channels.
**Dependencies**: Layer 3 + `ws` (Node) + browser-native WebSocket + `ReconnectingWebSocket`.
## Layer 5 — Rendering Pipeline
**Responsibility**: take incoming buffer frames (SHM or encoded), turn them
into pixels on a browser canvas/WebGL context. Manage surface state, damage
tracking, frame callbacks, and the compositing loop.
**Key abstractions**:
- `Surface.ts` — the `wl_surface` implementation. Double-buffered state
(`state` vs `pendingState`), atomic merge in `commitPending()`. Subsurface
sync mode cascades through the ancestor chain.
- `SurfaceChild.ts`, `SurfaceRole.ts`, `Subsurface.ts`, `Subcompositor.ts` —
surface tree / role system.
- `View.ts` — represents a surface's position on an output, used for
compositing.
- `Output.ts` — the `wl_output` implementation; how output is rendered to a
WebGL canvas.
- `Shm.ts` — `wl_shm` / shm pool handling. How shared memory buffers arrive
in the browser.
- `BufferContents.ts`, `BufferImplementation.ts`, `ImageBitmapBuffer.ts` —
buffer abstraction hierarchy.
- `Region.ts` — `wl_region`, damage regions (pixman-WASM backed).
- `Callback.ts` — frame callback mechanism.
- `Loop.ts` — the compositor main loop / `requestAnimationFrame` integration.
- `Compositor.ts` — the `wl_compositor` global; how it creates surfaces and
regions.
- `Session.ts` — top-level browser-side session.
- `Globals.ts` — how Wayland globals are registered.
- `render/` subdirectory — the WebGL renderer: `Renderer.ts`, `Scene.ts`,
`SceneShader.ts`, `Texture.ts`, `RenderState.ts`, `Program.ts`,
`ShaderCompiler.ts`, `ShaderSources.ts`, `YUVA2RGBA.ts`, `YUVA2RGBAShader.ts`,
`YUV2RGBShader.ts`, `RGBXandAToRGBA.ts`, `RGBandA2RGBAShader.ts`.
- `browser/` subdirectory — browser integration glue (canvas, RAF,
ResizeObserver, dnd, selection, input).
- `math/` subdirectory — matrix/region math (`Mat4.ts`, `Rect.ts`).
**Pipeline**: see `03-html5-rendering.md` for the full
commit → damage → render → present pipeline.
**Dependencies**: Layer 4 + WebGL + WebCodecs + `@gfld/compositor-wasm` (pixman + xkbcommon).
## Layer 6 — Surface Lifecycle / Roles
**Responsibility**: implement the role-specific surface state machines
(xdg-toplevel, xdg-popup, xdg-positioner, wl_shell_surface, subsurface).
Manage configure/ack-configure round-trips, popup grabs, window state.
**Files of interest**:
- `XdgWmBase.ts` — the `xdg_wm_base` global. How clients advertise they're
XDG clients. Includes ping-based unresponsiveness detection.
- `XdgSurface.ts` — `xdg_surface`. The role wrapper around a `wl_surface`.
- `XdgToplevel.ts` — `xdg_toplevel`. Top-level windows. State (maximized,
fullscreen, minimized, activated, resizing). Configure/ack-configure
round-trip.
- `XdgPopup.ts` — `xdg_popup`. Transient windows, menus, tooltips. Grab
semantics.
- `XdgPositioner.ts` — `xdg_positioner`. How popups are positioned relative
to parent.
- `Shell.ts`, `ShellSurface.ts` — the legacy `wl_shell` (deprecated but still
implemented for old clients).
- `Subcompositor.ts`, `Subsurface.ts`, `SurfaceChild.ts`, `SurfaceRole.ts`
— subsurface compositing, the surface tree.
**State machines**: see `08-app-lifecycle.md` for the full
XdgToplevel lifecycle, configure/ack round-trip semantics, popup
positioner math, and subsurface sync/desync mode.
**Dependencies**: Layer 5 + Layer 3 (Wayland runtime).
## Layer 7 — Desktop / Layout Policy
**Responsibility**: arrange surfaces on the output. Decide stacking order,
focus policy, and the high-level window management mode (Floating vs
AlwaysFullScreen).
**Files of interest**:
- `desktop/Desktop.ts` — implements both `FloatingDesktopSurface` and
`AlwaysFullScreenDesktopSurface`, chosen by `session.config.mode`.
Floating lets the user move/resize windows freely; AlwaysFullScreen
enforces one window per output (kiosk mode).
**Dependencies**: Layer 6 + Layer 5 (for View rendering).
## Layer 8 — User Shell
**Responsibility**: the actual user-facing UI. The taskbar, window
decorations, app launcher, keyboard layout picker, etc.
**Out of scope for the user** — they will build their own shell as an Odoo 20
OWL addon. The integration point is `UserShellApi.ts` (see
`08-app-lifecycle.md`).
**Greenfield's reference shell** (`packages/compositor-shell`) is a Preact +
Tailwind SPA, 562 lines of TypeScript total. Mentioned here only for
orientation; not analyzed in this bundle.
## How data flows across the layers
```
NATIVE APP (e.g. gtk4-demo, X11 via XWayland)
│
│ libwayland-client → UNIX socket + SCM_RIGHTS FDs
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ NATIVE PROXY (Node.js + C N-API addons) │
│ │
│ Layer 3 (native): wayland-server.ts + wayland-server-addon.node │
│ ↳ patched libwayland-server parses wire bytes │
│ ↳ MessageInterceptor routes each request to {native, browser} │
│ ↳ FDs wrapped as ProxyFD JSON │
│ │
│ Layer 4 (native): Channel.ts + kcp.ts + SessionController │
│ ↳ 4 logical channels: PROTOCOL, FRAME, XWM, FEEDBACK │
│ ↳ KCP-over-WebSocket for ARQ channels │
│ ↳ SimpleChannel for FEEDBACK │
│ │
│ Layer 5 (encoding): SurfaceBufferEncoding + proxy-encoding-addon │
│ ↳ GStreamer H.264 (x264 / nvh264 / vaapih264) on dedicated thread │
│ ↳ PNG fallback for ≤256×256 surfaces │
│ ↳ Zero-copy dmabuf via EGLImage → GstGLMemoryEGL │
│ │
│ Layer 1 (XML) → Layer 2 (proxy-generator) feeds Layer 3 interceptors │
└─────────────────────────────────────────────────────────────────────────┘
│
│ WebSocket (TCP) + KCP-over-WebSocket
│ 9 endpoint paths per session
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ BROWSER COMPOSITOR (TypeScript + WebGL + WebCodecs + WASM) │
│ │
│ Layer 4 (browser): Channel.ts + RemoteAppLauncher + decoders │
│ ↳ ReconnectingWebSocket with exponential backoff │
│ ↳ ARQChannel + SimpleChannel mirror the native side │
│ ↳ WebCodecs VideoDecoder preferred, FFmpeg WASM fallback │
│ ↳ H264NALDecoder.worker.ts (2 workers: opaque + alpha) │
│ │
│ Layer 3 (browser): @gfld/common Connection + compositor-protocol │
│ ↳ Standard Wayland wire format, byte-for-byte │
│ ↳ WlObject registry, opcode dispatch via wlObject[opcode](msg) │
│ ↳ Per-client Connections (slow client never blocks another) │
│ │
│ Layer 5: Surface.ts + View.ts + Output.ts + render/ │
│ ↳ Double-buffered state (state vs pendingState) │
│ ↳ Damage tracking via pixman-WASM regions │
│ ↳ WebGL1 with FBOs for YUV→RGBA, RGB+Alpha→RGBA conversion │
│ ↳ desynchronized:true canvas for low-latency presentation │
│ ↳ Frame callbacks fired on requestAnimationFrame vsync │
│ │
│ Layer 6: XdgWmBase + XdgSurface + XdgToplevel + XdgPopup + ... │
│ ↳ Configure/ack-configure round-trips │
│ ↳ Popup positioner math (anchor + gravity + constraints) │
│ ↳ Subsurface sync/desync mode cascading │
│ │
│ Layer 7: desktop/Desktop.ts │
│ ↳ Floating vs AlwaysFullScreen mode │
│ ↳ View stacking, focus refcount │
│ │
│ Layer 8: UserShellApi (events + actions) │
│ ↳ Events: clientCreated, surfaceCreated, surfaceTitleUpdated, │
│ surfaceActivationUpdated, ... │
│ ↳ Actions: initScene, refreshScene, destroyScene, │
│ closeClient, activateSurface, setUserConfiguration │
│ ↳ THE INTEGRATION SEAM — Odoo 20 OWL shell hooks here │
└─────────────────────────────────────────────────────────────────────────┘
│
│ UserShellApi.events.* fired by compositor
│ UserShellApi.actions.* called by shell
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ USER SHELL (Odoo 20 OWL addon — supplied by user) │
│ │
│ - Application launcher (lists apps from --applications JSON) │
│ - Window management UI (taskbar, decorations, minimize/maximize) │
│ - Keyboard layout picker (RMLVO) │
│ - Per-output canvas (HTMLCanvasElement, one per Output) │
│ - Trigger app launches via fetch('http://proxy:8081/<path>') │
│ - Open signalURL WebSocket to receive CONNECT_CHANNEL messages │
│ │
│ Not analyzed in this bundle (out of scope). │
└─────────────────────────────────────────────────────────────────────────┘
```
## Reading guide for the rest of the bundle
Now that you've seen the layer map, the remaining chapters drill into each
layer in detail:
- `02-wayland-protocol.md` — Layers 1, 2, 3 (XML → generators → runtime)
- `03-html5-rendering.md` — Layer 5 (rendering pipeline)
- `04-transport.md` — Layer 4 (transport, both sides)
- `05-input-handling.md` — Layer 5 input subsystem (cross-cuts Layers 3, 4, 5)
- `06-data-clipboard.md` — Layer 5 data device (cross-cuts Layers 3, 4, 5)
- `07-native-app-bridge.md` — Layer 4 native + Layer 3 native + Layer 1 XML native (proxy-only `wl_drm`)
- `08-app-lifecycle.md` — Layer 6 + Layer 8 (UserShellApi seam)
- `09-hardware-accel.md` — Layer 5 encoding + Layer 4 frame channel + Layer 3 WASM
- `10-porting-to-odoo-js-python.md` — how to rebuild all 8 layers in your stack
- `11-pitfalls.md` — what Greenfield does wrong, layer by layer
- `12-appendix.md` — glossary, file index, external references