Initial commit: Greenfield architectural analysis (14 files, ~80 pages, 10 parallel research agents)

This commit is contained in:
1b-agent committed 2026-10-08 08:48:25 +00:00
commit c4b45cf218
15 files changed
+5444

No files matched your search

+404
View File
@@ -0,0 +1,404 @@
# 07 — Native App Bridge (How Real Linux Apps Get Connected)
This chapter covers Greenfield's native app bridge: how real Linux
applications get launched, connected to the proxy, and routed to the
browser. This is the bridge that makes a Linux Wayland app appear inside
the browser. **This is the most important chapter for the user** — they
want to run real Linux apps in their Odoo 20 web app.
## Two app models — don't conflate them
Greenfield has **two app models**:
### Model 1: WASM web apps (browser-resident)
Built with the Greenfield SDK (Emscripten), run inside the browser via
`MessageChannel`. The `sdk/` directory serves only this model. Apps are
compiled to WASM, run in the same browser tab as the compositor, and
speak Wayland over a `MessagePort` instead of a UNIX socket.
### Model 2: Native Linux apps (the focus of this chapter)
Real ELF binaries (`gtk4-demo`, `kwrite`, `xterm`) running as child
processes of the Node.js proxy. Apps use the host's own system libraries
— **no sysroot, no chroot, no container**.
## Process model — three layers
### Layer 1: CLI Front Door (`compositor-proxy-cli/src/main.ts`)
A single Node.js process listens on `:8081`. Per browser session, it
`fork()`s one `SessionProcess` child — **one child per `compositorSessionId`**,
not per app. Multiple apps in the same browser tab share one child.
WebSocket upgrades are handed off to the matching child via
`childProcess.send(wsUpgrade, socket)`.
CLI args:
- `--bind-port=8081` (default)
- `--applications=<json>` (URL → executable map)
- `--encoder=x264|nvh264|vaapih264`
- `--render-device=/dev/dri/renderD128`
- `--base-url`, `--allow-origin`, `--bind-ip`, `--basic-auth`
### Layer 2: Session Process (`SessionProcess.ts`)
The forked child. Receives IPC messages:
- `start` — creates `Session` + `SessionController`
- `launchApp` — spawns the actual app binary
- `wsUpgrade` — hands WS to the controller
On `launchApp` it calls `launchApplication()` from `@gfld/compositor-proxy`,
which `spawn()`s the executable, captures `pid`, replies to main with
`{pid, key}`. On `session.close()` the process exits.
### Layer 3: Proxy Core (`compositor-proxy/src/`)
- **`NativeWaylandCompositorSession`** — creates a real `wl_display` via
`createDisplay()`, calls `addSocketAuto()` which creates the UNIX socket
at `$XDG_RUNTIME_DIR/wayland-N` and returns the name `wayland-N` (this
becomes `WAYLAND_DISPLAY`). Calls `initShm()` (adds YUV420/NV12/YUV444/
NV21 formats) and `initDrm()` (opens `/dev/dri/renderD128` via EGL/GBM,
registers `zwp_linux_dmabuf_v1` + `wl_drm` globals). Integrates the
wl_display FD into Node's libuv loop via `startPoll()`.
- **`NativeWaylandClientSession`** — per-client wire-message interceptor.
Uses `MessageInterceptor` + auto-generated per-resource interceptor
modules (`src/protocol/wl_*_interceptor.js`). Each request returns
`{native, browser, neverReplies?}` routing decision. Handles
`wl_display.sync` fast-path (skips browser round-trip when no reply
expected). Marshalls FDs as `ProxyFD` JSON.
- **`NativeAppContext`** — per-app state: `pid`, `key` (16-hex random
signaling auth), signaling WS, client sessions, data channels. Lifecycle:
10-min SIGHUP grace on browser disconnect, SIGTERM→SIGKILL escalation
(10 s) on force-quit, auto-destroy on app exit.
- **`SessionController`** — WS demultiplexer by pathname: `/signal`
(per-app control), `/channel` (multiplexed data channels), `/mkfifo`,
`/mkstemp-mmap`, `/read-fd`, `/write-fd-as-stream`, `/request-keyframe`.
- **`XWaylandSession`** — lazy Xwayland fork (only on first X client
connect). Xwayland itself becomes a Wayland client of the proxy. The
`wm_fd` is bridged via xtsb to the browser, which implements the actual
XWM logic.
- **`Channel`** — `ARQChannel` (KCP-over-WebSocket, reliable, low-latency)
for PROTOCOL/FRAME/XWM; `SimpleChannel` for FEEDBACK.
## App launch flow (definitive answer)
1. Browser sends `GET http://proxy:8081/<path>` with
`X-Compositor-Session-Id: <uuid>` header.
2. Main process forks a `SessionProcess` if none exists for that session
ID, sends `start` IPC.
3. Main sends `launchApp` IPC with `{name, executable, args, env, serial}`.
4. `SessionProcess` calls `launchApplication()`:
- `appEnv = {...process.env, ...env, WAYLAND_DISPLAY:
session.nativeWaylandCompositorSession.waylandDisplay}`
- `child_process.spawn(executable, args, {env: appEnv})` — **plain
`spawn`, no chroot/container**
5. On `spawn` event, captures `pid`, creates `NativeAppContext`, registers
`exit` listener.
6. Replies to main with `{pid, key}` → main returns HTTP 201 with
`{baseURL, signalURL, key, pid, name}`.
7. Browser opens `signalURL` as WebSocket → control channel established.
8. App connects to `$XDG_RUNTIME_DIR/wayland-N` → libwayland fires
`on_client_created` → proxy looks up PID via `getCredentials()`, walks
`/proc/<pid>/status` PPid chain to match a `NativeAppContext`.
9. For each new `wl_client`, creates a `PROTOCOL` ARQ channel and sends
`CONNECT_CHANNEL` over the signaling WS; browser opens a new WS at
`/channel?id=…`.
10. Wayland wire messages now flow: app → libwayland →
`setWireMessageCallback` → JS interceptor → routed to `{native,
browser}`.
## Native addon architecture
**Important**: The "native" code is **C, not C++** (one GLSL shader string +
GObject-style). Uses Node-API (N-API).
### `wayland-server-addon.node` (1231 lines C)
Wraps a **patched** `libwayland-server` (`libwayland-server.so.0` with
Greenfield additions in `native/wayland/changes.diff`):
- `wl_client_set_wire_message_cb`
- `wl_client_set_registry_created_cb`
- `wl_client_set_sync_done_cb`
- `wl_client_add_resource_created_listener`
- `wl_display_set_global_created_cb`
- `wl_resource_destroy_silently`
- `wl_get_server_object_ids_batch`
- `wl_connection_copy_fds_in`
Plus the `westfield` shared lib (EGL/GBM init, `wlr_linux_dmabuf_v1`,
`wlr_drm`, `westfield-xwayland`). **31 N-API functions** exposed. This is
what makes the proxy a real Wayland compositor.
### `proxy-poll-addon.node` (127 lines C)
Tiny wrapper around `uv_poll_t` (libuv, not raw epoll — though libuv uses
epoll under the hood). `startPoll(fd, cb)` integrates the wl_display
event-loop FD into Node's libuv loop so wayland events fire as Node
callbacks without blocking.
### `proxy-encoding-addon.node` (290 lines C) + `libproxy-encoding.so` (~1400 lines C)
GStreamer H.264/PNG encoder. Runs on a **dedicated GMainLoop thread** with
an `eventfd`-based message queue (`SyncSource` GSource). Two pipelines per
surface (opaque + alpha) for translucent windows. Supports `x264`
(software), `nvh264` (Nvidia NVENC), `vaapih264` (Intel, experimental).
Zero-copy DMA-BUF path via EGLImage → GStreamer GL. Uses
`napi_threadsafe_function` to marshal encoded samples back to Node.
## XWayland bridge
### Native side (`westfield-xwayland.c`, 466 lines C)
1. Creates `/tmp/.X11-unix/X<N>` listener socket
2. `setenv("DISPLAY", ":<N>", 1)`
3. Waits for X client connect
4. On connect: `socketpair(AF_UNIX)` × 3 (wayland / wm / display)
5. `wl_client_create(wl_display, sv[0])` — **Xwayland itself becomes a
Wayland client of the proxy!**
6. `fork()` + `execlp("Xwayland", "Xwayland", ":<N>", "-ac", "-rootless",
"-listen", "<unix_fd>", "-displayfd", "<display_fd>", "-wm", "<wm_fd>",
NULL)`
### xtsb (X11 TypeScript Bindings)
`libs/xtsb/`: pure-TS X11 client, no libxcb. Dual-environment (Node
`net.Socket` + browser WebSocket). Auto-generated from XCB XML protocol
files via `xcbgen/ts_client.py`. Handles MIT-MAGIC-COOKIE-1 auth from
`~/.Xauthority`. The browser-side xtsb implements the actual XWM logic
(translates X `CreateNotify` events → `xdg_toplevel` Wayland surfaces).
`XWaylandSession.ts` uses `nodeFDConnectionSetup(wmFd)` to wrap the wm_fd
as a `net.Socket`, performs X11 auth, and pipes the X11 wire protocol
through the XWM data channel to the browser.
## SDK misconception corrected
The `sdk/` directory is **NOT for native apps**. It's exclusively for
**WASM apps** that run inside the browser via Emscripten. The
`sysrootlibs/` cross-compiles wayland/gtk/cairo/etc. to wasm32. Native
Linux apps on the proxy use the host's own system libraries — the
Dockerfile installs them via `apt-get install libffi8 libgbm1
libgstreamer1.0-plugins-base … xwayland xauth`. The proxy's `pkg` config
only bundles three libs as assets (`libproxy-encoding.so`,
`libwayland-server.so.0`, `libwestfield.so`) — these are the proxy's OWN
addon dependencies, not the apps'.
## ASCII diagram — native app launch & connection
```
Browser (compositor-shell) Proxy host (Linux) App host (often same as proxy)
========================== ==================== ==============================
[1] User clicks app in shell $ compositor-proxy-cli \
│ └─── main.ts listens on :8081
│ sessionProcesses: Record<id, ChildProcess>
│
│ HTTP GET /gtk4-demo
│ Header: X-Compositor-Session-Id: <uuid>
├──────────────────────────────► │
│ │ if no child for <uuid>: fork(SessionProcess.ts)
│ │ └── IPC: {type:'start', payload:{config, sessionId}}
│ │ └── createSession() → NativeWaylandCompositorSession
│ │ ├── createDisplay()
│ │ │ └── wl_display_create() (libwayland-server)
│ │ ├── addSocketAuto()
│ │ │ └── creates $XDG_RUNTIME_DIR/wayland-N
│ │ │ returns "wayland-N" ◄── this becomes WAYLAND_DISPLAY
│ │ ├── initShm() + YUV420/NV12/YUV444/NV21
│ │ ├── initDrm("/dev/dri/renderD128")
│ │ │ └── westfield_egl_new() → EGLDisplay via GBM
│ │ │ wlr_linux_dmabuf_v1 global (zero-copy GPU buffers)
│ │ │ wlr_drm global (legacy dmabuf)
│ │ └── startPoll(wlDisplayFd, dispatchRequests)
│ │ └── uv_poll_t on EPOLLIN
│ │ (proxy-poll-addon.node)
│ │
│ │ IPC: {type:'launchApp', payload:{name, executable, args, env, serial}}
│ │ └── launchApplication() (NativeAppContext.ts)
│ │ ├── appEnv = {...process.env, ...env,
│ │ │ WAYLAND_DISPLAY: "wayland-N"}
│ │ ├── child_process.spawn("gtk4-demo", [], {env: appEnv})
│ │ │ ◄── posix_spawn/execvp, NO chroot, NO container
│ │ │ ──────────────────────────────► [2] gtk4-demo process starts
│ │ │ │
│ │ │ │ libwayland-client.so
│ │ │ │ connects to
│ │ │ │ $XDG_RUNTIME_DIR/wayland-N
│ │ │ │
│ │ │ ▼
│ │ │ ┌──────────────────────────┐
│ │ │ │ wl_display event loop │
│ │ │ │ (in proxy, libwayland- │
│ │ │ │ server.so.0) │
│ │ │ │ │
│ │ │ ◄── uv_poll fires, │ on_client_created → │
│ │ │ dispatchRequests() │ getCredentials(pid) │
│ │ │ wl_event_loop_dispatch() │ findMatchingNativeApp │
│ │ │ │ Context(pid) by PPID │
│ │ │ │ walk │
│ │ │ │ │
│ │ │ │ setWireMessageCallback │
│ │ │ │ setBufferCreated... │
│ │ │ │ setSyncDoneCallback │
│ │ │ │ setClientDestroyed... │
│ │ │ │ │
│ │ │ │ NativeWaylandClientSession
│ │ │ │ .messageInterceptor │
│ │ │ │ dispatches R<opcode> │
│ │ │ │ → {native, browser} │
│ │ │ └──────────────────────────┘
│ │ │
│ │ ├── child.on('spawn') → capture pid
│ │ │ createNativeAppContext(pid, name, external:false)
│ │ │ nativeAppContext.onDisconnect() // start 10-min SIGHUP timer
│ │ │
│ │ └── child.on('exit') → nativeAppContext.onExit({exitCode|signal})
│ │
│ │ IPC reply: {type:'launchAppSuccess', payload:{pid, key}}
│
│ HTTP 201 Created
│ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, internal:false }
◄────────────────────────────── │
│ │
│ Open WS: ws://…/signal?key=<16-hex>
├──────────────────────────────► │
│ │ SessionController demux /signal → nativeAppContext.onConnect(ws)
│ │ clears 10-min SIGHUP timer
│ │
│ │ For each new wl_client (one per wayland connection):
│ │ createProtocolChannel() → ARQChannel (KCP over WS)
│ │ sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc}
│ ◄──── CONNECT_CHANNEL JSON ───┤
│ │
│ Open WS: ws://…/channel?id=…&key=…
├──────────────────────────────► │ binds WS to the ARQChannel
│ │
│ Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval):
│
│ App → wl_surface.commit ──── proxy intercepts R1 (attach) + R6 (commit) ────► browser compositor
│ App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄────── FrameFeedback paces callback
│ App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected)
│ App ← wl_keyboard.enter ─── browser → proxy → sendEvents(wlClient, …) → flush(wlClient)
│
│ Buffer path (per wl_surface):
│ - App submits wl_buffer (dmabuf or shm)
│ - setBufferCreatedCallback → ProxyBuffer with creationSerial
│ - On commit: wl_surface_interceptor.R6 → Encoder.encodeFrame()
│ └── proxy-encoding-addon.encodeFrame(bufferId, contentSerial, creationSerial)
│ └── GStreamer pipeline (x264/nvh264/vaapih264) on dedicated GMainLoop thread
│ └── encoded H.264 sample → napi_threadsafe_function → Node main thread
│ └── sent over the surface's FRAME data channel (separate WS, also CONNECT_CHANNEL'd)
│
│ X11 path (xterm etc.):
│ - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by westfield_xwayland.c)
│ - First X connect → spawn_xserver() forks: execlp("Xwayland", …, "-rootless", "-wm", wm_fd)
│ - Xwayland itself becomes a Wayland client of the proxy (wl_client_create)
│ - When Xwayland ready (displayFd readable):
│ └── xtsb.nodeFDConnectionSetup(wmFd) → wraps as net.Socket
│ └── pipes through XWM data channel (4th channel type) to browser
│ └── browser-side xtsb speaks X11 → translates X windows → xdg_toplevel surfaces
│
│ Lifecycle:
│ - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP')
│ - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM')
│ └── if not exited in 10s → SIGKILL
│ - App crashes → child 'exit' event → nativeAppContext.onExit() → APP_TERMINATED msg → browser UI updates
│ - All clients gone for >1s → session.close() → process.exit() in SessionProcess.ts → main deletes entry
│
│ Native addon role summary:
│ - wayland-server-addon.node: full Wayland compositor (libwayland-server fork + westfield EGL/GBM/dmabuf/XWayland)
│ - proxy-poll-addon.node: libuv uv_poll_t hook so wayland FD events fire as Node callbacks
│ - proxy-encoding-addon.node: GStreamer H.264/PNG encoder on a dedicated GMainLoop thread, threadsafe_function back to Node
```
## File reference
### CLI (`packages/compositor-proxy-cli/src/`)
| File | Role |
|---|---|
| `main.ts` | HTTP + WS hub, forks per session |
| `main-controller.ts` | CLI arg parsing |
| `main-args.ts` | CLI arg definitions |
| `SessionProcess.ts` | Per-session child, IPC handler |
| `example-applications.json` | Pre-configured apps |
| `wait-until-ready.sh` | Startup readiness check |
### Proxy TS layer (`packages/compositor-proxy/src/`)
| File | Role |
|---|---|
| `index.ts` | Exports `createSession()`, `launchApplication()` |
| `Session.ts` | Top-level proxy session |
| `SessionController.ts` | WS server with 9 endpoint paths |
| `NativeAppContext.ts` | Per-app state, signaling WS, kill timers |
| `NativeWaylandClientSession.ts` | Per-client message interceptor |
| `NativeWaylandCompositorSession.ts` | wl_display creation, initShm, initDrm |
| `Channel.ts` | ARQChannel + SimpleChannel |
| `kcp.ts` | ~1100-line KCP implementation |
| `FrameFeedback.ts` | Receives 4-byte decode feedback, paces wl_frame.done |
| `ProxyBuffer.ts` | Proxy-side buffer representation |
| `SurfaceBufferEncoding.ts` | Encoding pipeline orchestration |
| `XWaylandSession.ts` | XWayland spawn + bridge |
| `wayland-server.ts` | TS wrapper around wayland-server-addon.node |
| `io/` | WebSocket server glue |
| `encoding/` | High-level encoding orchestration |
| `protocol/` | Generated interceptor modules |
| `addons/` | .node file shims (Linux-x64) |
| `config.ts` | Encoding-related configuration |
### Proxy C layer (`packages/compositor-proxy/native/`)
| Path | Role |
|---|---|
| `CMakeLists.txt` | Build config |
| `wayland/src/addon.c` | Node-API addon: wayland-server-addon.node |
| `wayland/src/westfield-buffer.c` | Buffer helpers |
| `wayland/src/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM |
| `wayland/src/westfield-egl.c` | EGL zero-copy setup |
| `wayland/src/westfield-xwayland.c` | XWayland spawn + bridge |
| `wayland/src/wlr_drm.c` | wl_drm global impl |
| `wayland/src/wlr_linux_dmabuf_v1.c` | zwp_linux_dmabuf_v1 global impl |
| `wayland/src/drm-protocol.c` | DRM protocol helpers |
| `wayland/src/linux-dmabuf-unstable-v1-protocol.c` | linux-dmabuf-v1 protocol stubs |
| `wayland/src/wayland-server/` | Vendored + patched libwayland-server (with changes.diff) |
| `encoding/src/gst_frame_encoder.c` | GStreamer H.264/PNG encoder |
| `encoding/src/gst_main_loop.c` | Dedicated GLib main loop for encoding |
| `encoding/src/node_encoder.c` | Node-API addon: proxy-encoding-addon.node |
| `poll/src/poll.c` | libuv uv_poll_t wrapper |
| `node-api/` | Vendored Node-API headers |
### XWayland support (`libs/xtsb/`)
| File | Role |
|---|---|
| `README.md` | Project intro |
| `package.json` | @gfld/xtsb package config |
| `protocol/xproto.xml` | X11 core protocol XML (5800+ lines) |
| `protocol/shape.xml` | Shape extension XML |
| `protocol/xfixes.xml` | XFIXES extension XML |
| `protocol/render.xml` | Render extension XML |
| `protocol/composite.xml` | Composite extension XML |
| `src/` | Generated TS bindings |
## Next actions for user (Odoo 20 integration)
1. Build the proxy on a Linux host (or use `udevbe/compositor-proxy-cli`
Docker image).
2. Write an `applications.json` mapping desired apps to HTTP paths.
3. Start proxy with `--allow-origin=https://your-odoo.example.com`.
4. Embed the Greenfield `compositor-shell` Preact SPA in Odoo (iframe or
QWeb template) — OR replace it with your Odoo 20 OWL shell (see
`10-porting-to-odoo-js-python.md`).
5. Trigger app launches from Odoo via
`fetch('http://<proxy>:8081/<path>', { headers: { 'X-Compositor-Session-Id': sessionId }})`
— Odoo frontend JS uses the returned `signalURL` to open the per-app
control WS.
6. For multi-tenant isolation: one `SessionProcess` per browser session is
automatic, but file-system isolation between tenants requires separate
proxy instances or containers per tenant.
7. For GPU acceleration: pass `--device /dev/dri:/dev/dri` to Docker; use
`--encoder=nvh264|vaapih264|x264` based on GPU.