one-os-dev-wayland-html5-compositor-greenfield-analysis/07-native-app-bridge.md

07 — Native App Bridge (How Real Linux Apps Get Connected)

This chapter covers Greenfield's native app bridge: how real Linux applications get launched, connected to the proxy, and routed to the browser. This is the bridge that makes a Linux Wayland app appear inside the browser. This is the most important chapter for the user — they want to run real Linux apps in their Odoo 20 web app.

Two app models — don't conflate them

Greenfield has two app models:

Model 1: WASM web apps (browser-resident)

Built with the Greenfield SDK (Emscripten), run inside the browser via MessageChannel. The sdk/ directory serves only this model. Apps are compiled to WASM, run in the same browser tab as the compositor, and speak Wayland over a MessagePort instead of a UNIX socket.

Model 2: Native Linux apps (the focus of this chapter)

Real ELF binaries (gtk4-demo, kwrite, xterm) running as child processes of the Node.js proxy. Apps use the host's own system libraries — no sysroot, no chroot, no container.

Process model — three layers

Layer 1: CLI Front Door (compositor-proxy-cli/src/main.ts)

A single Node.js process listens on :8081. Per browser session, it fork()s one SessionProcess child — one child per compositorSessionId, not per app. Multiple apps in the same browser tab share one child. WebSocket upgrades are handed off to the matching child via childProcess.send(wsUpgrade, socket).

CLI args:

• --bind-port=8081 (default)
• --applications=<json> (URL → executable map)
• --encoder=x264|nvh264|vaapih264
• --render-device=/dev/dri/renderD128
• --base-url, --allow-origin, --bind-ip, --basic-auth

Layer 2: Session Process (SessionProcess.ts)

The forked child. Receives IPC messages:

• start — creates Session + SessionController
• launchApp — spawns the actual app binary
• wsUpgrade — hands WS to the controller

On launchApp it calls launchApplication() from @gfld/compositor-proxy, which spawn()s the executable, captures pid, replies to main with {pid, key}. On session.close() the process exits.

Layer 3: Proxy Core (compositor-proxy/src/)

• NativeWaylandCompositorSession — creates a real wl_display via createDisplay(), calls addSocketAuto() which creates the UNIX socket at $XDG_RUNTIME_DIR/wayland-N and returns the name wayland-N (this becomes WAYLAND_DISPLAY). Calls initShm() (adds YUV420/NV12/YUV444/ NV21 formats) and initDrm() (opens /dev/dri/renderD128 via EGL/GBM, registers zwp_linux_dmabuf_v1 + wl_drm globals). Integrates the wl_display FD into Node's libuv loop via startPoll().
• NativeWaylandClientSession — per-client wire-message interceptor. Uses MessageInterceptor + auto-generated per-resource interceptor modules (src/protocol/wl_*_interceptor.js). Each request returns {native, browser, neverReplies?} routing decision. Handles wl_display.sync fast-path (skips browser round-trip when no reply expected). Marshalls FDs as ProxyFD JSON.
• NativeAppContext — per-app state: pid, key (16-hex random signaling auth), signaling WS, client sessions, data channels. Lifecycle: 10-min SIGHUP grace on browser disconnect, SIGTERM→SIGKILL escalation (10 s) on force-quit, auto-destroy on app exit.
• SessionController — WS demultiplexer by pathname: /signal (per-app control), /channel (multiplexed data channels), /mkfifo, /mkstemp-mmap, /read-fd, /write-fd-as-stream, /request-keyframe.
• XWaylandSession — lazy Xwayland fork (only on first X client connect). Xwayland itself becomes a Wayland client of the proxy. The wm_fd is bridged via xtsb to the browser, which implements the actual XWM logic.
• Channel — ARQChannel (KCP-over-WebSocket, reliable, low-latency) for PROTOCOL/FRAME/XWM; SimpleChannel for FEEDBACK.

App launch flow (definitive answer)

1. Browser sends GET http://proxy:8081/<path> with X-Compositor-Session-Id: <uuid> header.
2. Main process forks a SessionProcess if none exists for that session ID, sends start IPC.
3. Main sends launchApp IPC with {name, executable, args, env, serial}.
4. SessionProcess calls launchApplication():
   • appEnv = {...process.env, ...env, WAYLAND_DISPLAY: session.nativeWaylandCompositorSession.waylandDisplay}
   • child_process.spawn(executable, args, {env: appEnv}) — plain spawn, no chroot/container
5. On spawn event, captures pid, creates NativeAppContext, registers exit listener.
6. Replies to main with {pid, key} → main returns HTTP 201 with {baseURL, signalURL, key, pid, name}.
7. Browser opens signalURL as WebSocket → control channel established.
8. App connects to $XDG_RUNTIME_DIR/wayland-N → libwayland fires on_client_created → proxy looks up PID via getCredentials(), walks /proc/<pid>/status PPid chain to match a NativeAppContext.
9. For each new wl_client, creates a PROTOCOL ARQ channel and sends CONNECT_CHANNEL over the signaling WS; browser opens a new WS at /channel?id=….
10. Wayland wire messages now flow: app → libwayland → setWireMessageCallback → JS interceptor → routed to {native, browser}.

Native addon architecture

Important: The "native" code is C, not C++ (one GLSL shader string + GObject-style). Uses Node-API (N-API).

wayland-server-addon.node (1231 lines C)

Wraps a patched libwayland-server (libwayland-server.so.0 with Greenfield additions in native/wayland/changes.diff):

• wl_client_set_wire_message_cb
• wl_client_set_registry_created_cb
• wl_client_set_sync_done_cb
• wl_client_add_resource_created_listener
• wl_display_set_global_created_cb
• wl_resource_destroy_silently
• wl_get_server_object_ids_batch
• wl_connection_copy_fds_in

Plus the westfield shared lib (EGL/GBM init, wlr_linux_dmabuf_v1, wlr_drm, westfield-xwayland). 31 N-API functions exposed. This is what makes the proxy a real Wayland compositor.

proxy-poll-addon.node (127 lines C)

Tiny wrapper around uv_poll_t (libuv, not raw epoll — though libuv uses epoll under the hood). startPoll(fd, cb) integrates the wl_display event-loop FD into Node's libuv loop so wayland events fire as Node callbacks without blocking.

proxy-encoding-addon.node (290 lines C) + libproxy-encoding.so (~1400 lines C)

GStreamer H.264/PNG encoder. Runs on a dedicated GMainLoop thread with an eventfd-based message queue (SyncSource GSource). Two pipelines per surface (opaque + alpha) for translucent windows. Supports x264 (software), nvh264 (Nvidia NVENC), vaapih264 (Intel, experimental). Zero-copy DMA-BUF path via EGLImage → GStreamer GL. Uses napi_threadsafe_function to marshal encoded samples back to Node.

XWayland bridge

Native side (westfield-xwayland.c, 466 lines C)

1. Creates /tmp/.X11-unix/X<N> listener socket
2. setenv("DISPLAY", ":<N>", 1)
3. Waits for X client connect
4. On connect: socketpair(AF_UNIX) × 3 (wayland / wm / display)
5. wl_client_create(wl_display, sv[0]) — Xwayland itself becomes a Wayland client of the proxy!
6. fork() + execlp("Xwayland", "Xwayland", ":<N>", "-ac", "-rootless", "-listen", "<unix_fd>", "-displayfd", "<display_fd>", "-wm", "<wm_fd>", NULL)

xtsb (X11 TypeScript Bindings)

libs/xtsb/: pure-TS X11 client, no libxcb. Dual-environment (Node net.Socket + browser WebSocket). Auto-generated from XCB XML protocol files via xcbgen/ts_client.py. Handles MIT-MAGIC-COOKIE-1 auth from ~/.Xauthority. The browser-side xtsb implements the actual XWM logic (translates X CreateNotify events → xdg_toplevel Wayland surfaces).

XWaylandSession.ts uses nodeFDConnectionSetup(wmFd) to wrap the wm_fd as a net.Socket, performs X11 auth, and pipes the X11 wire protocol through the XWM data channel to the browser.

SDK misconception corrected

The sdk/ directory is NOT for native apps. It's exclusively for WASM apps that run inside the browser via Emscripten. The sysrootlibs/ cross-compiles wayland/gtk/cairo/etc. to wasm32. Native Linux apps on the proxy use the host's own system libraries — the Dockerfile installs them via apt-get install libffi8 libgbm1 libgstreamer1.0-plugins-base … xwayland xauth. The proxy's pkg config only bundles three libs as assets (libproxy-encoding.so, libwayland-server.so.0, libwestfield.so) — these are the proxy's OWN addon dependencies, not the apps'.

ASCII diagram — native app launch & connection

Browser (compositor-shell)              Proxy host (Linux)                           App host (often same as proxy)
==========================              ====================                          ==============================

[1] User clicks app in shell            $ compositor-proxy-cli \
        │                                └─── main.ts listens on :8081
        │                                     sessionProcesses: Record<id, ChildProcess>
        │
        │ HTTP GET /gtk4-demo
        │ Header: X-Compositor-Session-Id: <uuid>
        ├──────────────────────────────► │
        │                                │ if no child for <uuid>: fork(SessionProcess.ts)
        │                                │     └── IPC: {type:'start', payload:{config, sessionId}}
        │                                │         └── createSession() → NativeWaylandCompositorSession
        │                                │              ├── createDisplay()
        │                                │              │   └── wl_display_create()  (libwayland-server)
        │                                │              ├── addSocketAuto()
        │                                │              │   └── creates $XDG_RUNTIME_DIR/wayland-N
        │                                │              │       returns "wayland-N"  ◄── this becomes WAYLAND_DISPLAY
        │                                │              ├── initShm()  + YUV420/NV12/YUV444/NV21
        │                                │              ├── initDrm("/dev/dri/renderD128")
        │                                │              │   └── westfield_egl_new() → EGLDisplay via GBM
        │                                │              │       wlr_linux_dmabuf_v1 global  (zero-copy GPU buffers)
        │                                │              │       wlr_drm global              (legacy dmabuf)
        │                                │              └── startPoll(wlDisplayFd, dispatchRequests)
        │                                │                   └── uv_poll_t on EPOLLIN
        │                                │                        (proxy-poll-addon.node)
        │                                │
        │                                │ IPC: {type:'launchApp', payload:{name, executable, args, env, serial}}
        │                                │  └── launchApplication()  (NativeAppContext.ts)
        │                                │       ├── appEnv = {...process.env, ...env,
        │                                │       │              WAYLAND_DISPLAY: "wayland-N"}
        │                                │       ├── child_process.spawn("gtk4-demo", [], {env: appEnv})
        │                                │       │   ◄── posix_spawn/execvp, NO chroot, NO container
        │                                │       │       ──────────────────────────────►  [2] gtk4-demo process starts
        │                                │       │                                              │
        │                                │       │                                              │ libwayland-client.so
        │                                │       │                                              │ connects to
        │                                │       │                                              │ $XDG_RUNTIME_DIR/wayland-N
        │                                │       │                                              │
        │                                │       │                                              ▼
        │                                │       │                                  ┌──────────────────────────┐
        │                                │       │                                  │ wl_display event loop   │
        │                                │       │                                  │ (in proxy, libwayland-  │
        │                                │       │                                  │  server.so.0)            │
        │                                │       │                                  │                          │
        │                                │       │   ◄── uv_poll fires,             │  on_client_created →    │
        │                                │       │       dispatchRequests()         │  getCredentials(pid)    │
        │                                │       │       wl_event_loop_dispatch()   │  findMatchingNativeApp  │
        │                                │       │                                  │  Context(pid) by PPID    │
        │                                │       │                                  │  walk                   │
        │                                │       │                                  │                          │
        │                                │       │                                  │  setWireMessageCallback │
        │                                │       │                                  │  setBufferCreated...    │
        │                                │       │                                  │  setSyncDoneCallback     │
        │                                │       │                                  │  setClientDestroyed...   │
        │                                │       │                                  │                          │
        │                                │       │                                  │  NativeWaylandClientSession
        │                                │       │                                  │   .messageInterceptor    │
        │                                │       │                                  │   dispatches R<opcode>   │
        │                                │       │                                  │   → {native, browser}    │
        │                                │       │                                  └──────────────────────────┘
        │                                │       │
        │                                │       ├── child.on('spawn') → capture pid
        │                                │       │     createNativeAppContext(pid, name, external:false)
        │                                │       │     nativeAppContext.onDisconnect()  // start 10-min SIGHUP timer
        │                                │       │
        │                                │       └── child.on('exit') → nativeAppContext.onExit({exitCode|signal})
        │                                │
        │                                │ IPC reply: {type:'launchAppSuccess', payload:{pid, key}}
        │
        │ HTTP 201 Created
        │ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, internal:false }
        ◄──────────────────────────────  │
        │                                │
        │ Open WS: ws://…/signal?key=<16-hex>
        ├──────────────────────────────► │
        │                                │  SessionController demux /signal → nativeAppContext.onConnect(ws)
        │                                │  clears 10-min SIGHUP timer
        │                                │
        │                                │  For each new wl_client (one per wayland connection):
        │                                │    createProtocolChannel() → ARQChannel (KCP over WS)
        │                                │    sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc}
        │  ◄──── CONNECT_CHANNEL JSON ───┤
        │                                │
        │ Open WS: ws://…/channel?id=…&key=…
        ├──────────────────────────────► │  binds WS to the ARQChannel
        │                                │
        │  Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval):
        │
        │   App → wl_surface.commit ──── proxy intercepts R1 (attach) + R6 (commit) ────► browser compositor
        │   App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄────── FrameFeedback paces callback
        │   App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected)
        │   App ← wl_keyboard.enter ─── browser → proxy → sendEvents(wlClient, …) → flush(wlClient)
        │
        │  Buffer path (per wl_surface):
        │   - App submits wl_buffer (dmabuf or shm)
        │   - setBufferCreatedCallback → ProxyBuffer with creationSerial
        │   - On commit: wl_surface_interceptor.R6 → Encoder.encodeFrame()
        │     └── proxy-encoding-addon.encodeFrame(bufferId, contentSerial, creationSerial)
        │         └── GStreamer pipeline (x264/nvh264/vaapih264) on dedicated GMainLoop thread
        │             └── encoded H.264 sample → napi_threadsafe_function → Node main thread
        │                 └── sent over the surface's FRAME data channel (separate WS, also CONNECT_CHANNEL'd)
        │
        │  X11 path (xterm etc.):
        │   - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by westfield_xwayland.c)
        │   - First X connect → spawn_xserver() forks: execlp("Xwayland", …, "-rootless", "-wm", wm_fd)
        │   - Xwayland itself becomes a Wayland client of the proxy (wl_client_create)
        │   - When Xwayland ready (displayFd readable):
        │     └── xtsb.nodeFDConnectionSetup(wmFd) → wraps as net.Socket
        │         └── pipes through XWM data channel (4th channel type) to browser
        │             └── browser-side xtsb speaks X11 → translates X windows → xdg_toplevel surfaces
        │
        │  Lifecycle:
        │   - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP')
        │   - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM')
        │       └── if not exited in 10s → SIGKILL
        │   - App crashes → child 'exit' event → nativeAppContext.onExit() → APP_TERMINATED msg → browser UI updates
        │   - All clients gone for >1s → session.close() → process.exit() in SessionProcess.ts → main deletes entry
        │
        │  Native addon role summary:
        │   - wayland-server-addon.node: full Wayland compositor (libwayland-server fork + westfield EGL/GBM/dmabuf/XWayland)
        │   - proxy-poll-addon.node:    libuv uv_poll_t hook so wayland FD events fire as Node callbacks
        │   - proxy-encoding-addon.node: GStreamer H.264/PNG encoder on a dedicated GMainLoop thread, threadsafe_function back to Node

File reference

CLI (packages/compositor-proxy-cli/src/)

File	Role
main.ts	HTTP + WS hub, forks per session
main-controller.ts	CLI arg parsing
main-args.ts	CLI arg definitions
SessionProcess.ts	Per-session child, IPC handler
example-applications.json	Pre-configured apps
wait-until-ready.sh	Startup readiness check

Proxy TS layer (packages/compositor-proxy/src/)

File	Role
index.ts	Exports createSession(), launchApplication()
Session.ts	Top-level proxy session
SessionController.ts	WS server with 9 endpoint paths
NativeAppContext.ts	Per-app state, signaling WS, kill timers
NativeWaylandClientSession.ts	Per-client message interceptor
NativeWaylandCompositorSession.ts	wl_display creation, initShm, initDrm
Channel.ts	ARQChannel + SimpleChannel
kcp.ts	~1100-line KCP implementation
FrameFeedback.ts	Receives 4-byte decode feedback, paces wl_frame.done
ProxyBuffer.ts	Proxy-side buffer representation
SurfaceBufferEncoding.ts	Encoding pipeline orchestration
XWaylandSession.ts	XWayland spawn + bridge
wayland-server.ts	TS wrapper around wayland-server-addon.node
io/	WebSocket server glue
encoding/	High-level encoding orchestration
protocol/	Generated interceptor modules
addons/	.node file shims (Linux-x64)
config.ts	Encoding-related configuration

Proxy C layer (packages/compositor-proxy/native/)

Path	Role
CMakeLists.txt	Build config
wayland/src/addon.c	Node-API addon: wayland-server-addon.node
wayland/src/westfield-buffer.c	Buffer helpers
wayland/src/westfield-dmabuf.c	DMA-BUF import via EGL/GBM
wayland/src/westfield-egl.c	EGL zero-copy setup
wayland/src/westfield-xwayland.c	XWayland spawn + bridge
wayland/src/wlr_drm.c	wl_drm global impl
wayland/src/wlr_linux_dmabuf_v1.c	zwp_linux_dmabuf_v1 global impl
wayland/src/drm-protocol.c	DRM protocol helpers
wayland/src/linux-dmabuf-unstable-v1-protocol.c	linux-dmabuf-v1 protocol stubs
wayland/src/wayland-server/	Vendored + patched libwayland-server (with changes.diff)
encoding/src/gst_frame_encoder.c	GStreamer H.264/PNG encoder
encoding/src/gst_main_loop.c	Dedicated GLib main loop for encoding
encoding/src/node_encoder.c	Node-API addon: proxy-encoding-addon.node
poll/src/poll.c	libuv uv_poll_t wrapper
node-api/	Vendored Node-API headers

XWayland support (libs/xtsb/)

File	Role
README.md	Project intro
package.json	@gfld/xtsb package config
protocol/xproto.xml	X11 core protocol XML (5800+ lines)
protocol/shape.xml	Shape extension XML
protocol/xfixes.xml	XFIXES extension XML
protocol/render.xml	Render extension XML
protocol/composite.xml	Composite extension XML
src/	Generated TS bindings

Next actions for user (Odoo 20 integration)

1. Build the proxy on a Linux host (or use udevbe/compositor-proxy-cli Docker image).
2. Write an applications.json mapping desired apps to HTTP paths.
3. Start proxy with --allow-origin=https://your-odoo.example.com.
4. Embed the Greenfield compositor-shell Preact SPA in Odoo (iframe or QWeb template) — OR replace it with your Odoo 20 OWL shell (see 10-porting-to-odoo-js-python.md).
5. Trigger app launches from Odoo via fetch('http://<proxy>:8081/<path>', { headers: { 'X-Compositor-Session-Id': sessionId }}) — Odoo frontend JS uses the returned signalURL to open the per-app control WS.
6. For multi-tenant isolation: one SessionProcess per browser session is automatic, but file-system isolation between tenants requires separate proxy instances or containers per tenant.
7. For GPU acceleration: pass --device /dev/dri:/dev/dri to Docker; use --encoder=nvh264|vaapih264|x264 based on GPU.