Files

405 lines
24 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 07 — Native App Bridge (How Real Linux Apps Get Connected)
This chapter covers Greenfield's native app bridge: how real Linux
applications get launched, connected to the proxy, and routed to the
browser. This is the bridge that makes a Linux Wayland app appear inside
the browser. **This is the most important chapter for the user** — they
want to run real Linux apps in their Odoo 20 web app.
## Two app models — don't conflate them
Greenfield has **two app models**:
### Model 1: WASM web apps (browser-resident)
Built with the Greenfield SDK (Emscripten), run inside the browser via
`MessageChannel`. The `sdk/` directory serves only this model. Apps are
compiled to WASM, run in the same browser tab as the compositor, and
speak Wayland over a `MessagePort` instead of a UNIX socket.
### Model 2: Native Linux apps (the focus of this chapter)
Real ELF binaries (`gtk4-demo`, `kwrite`, `xterm`) running as child
processes of the Node.js proxy. Apps use the host's own system libraries
— **no sysroot, no chroot, no container**.
## Process model — three layers
### Layer 1: CLI Front Door (`compositor-proxy-cli/src/main.ts`)
A single Node.js process listens on `:8081`. Per browser session, it
`fork()`s one `SessionProcess` child — **one child per `compositorSessionId`**,
not per app. Multiple apps in the same browser tab share one child.
WebSocket upgrades are handed off to the matching child via
`childProcess.send(wsUpgrade, socket)`.
CLI args:
- `--bind-port=8081` (default)
- `--applications=<json>` (URL → executable map)
- `--encoder=x264|nvh264|vaapih264`
- `--render-device=/dev/dri/renderD128`
- `--base-url`, `--allow-origin`, `--bind-ip`, `--basic-auth`
### Layer 2: Session Process (`SessionProcess.ts`)
The forked child. Receives IPC messages:
- `start` — creates `Session` + `SessionController`
- `launchApp` — spawns the actual app binary
- `wsUpgrade` — hands WS to the controller
On `launchApp` it calls `launchApplication()` from `@gfld/compositor-proxy`,
which `spawn()`s the executable, captures `pid`, replies to main with
`{pid, key}`. On `session.close()` the process exits.
### Layer 3: Proxy Core (`compositor-proxy/src/`)
- **`NativeWaylandCompositorSession`** — creates a real `wl_display` via
`createDisplay()`, calls `addSocketAuto()` which creates the UNIX socket
at `$XDG_RUNTIME_DIR/wayland-N` and returns the name `wayland-N` (this
becomes `WAYLAND_DISPLAY`). Calls `initShm()` (adds YUV420/NV12/YUV444/
NV21 formats) and `initDrm()` (opens `/dev/dri/renderD128` via EGL/GBM,
registers `zwp_linux_dmabuf_v1` + `wl_drm` globals). Integrates the
wl_display FD into Node's libuv loop via `startPoll()`.
- **`NativeWaylandClientSession`** — per-client wire-message interceptor.
Uses `MessageInterceptor` + auto-generated per-resource interceptor
modules (`src/protocol/wl_*_interceptor.js`). Each request returns
`{native, browser, neverReplies?}` routing decision. Handles
`wl_display.sync` fast-path (skips browser round-trip when no reply
expected). Marshalls FDs as `ProxyFD` JSON.
- **`NativeAppContext`** — per-app state: `pid`, `key` (16-hex random
signaling auth), signaling WS, client sessions, data channels. Lifecycle:
10-min SIGHUP grace on browser disconnect, SIGTERM→SIGKILL escalation
(10 s) on force-quit, auto-destroy on app exit.
- **`SessionController`** — WS demultiplexer by pathname: `/signal`
(per-app control), `/channel` (multiplexed data channels), `/mkfifo`,
`/mkstemp-mmap`, `/read-fd`, `/write-fd-as-stream`, `/request-keyframe`.
- **`XWaylandSession`** — lazy Xwayland fork (only on first X client
connect). Xwayland itself becomes a Wayland client of the proxy. The
`wm_fd` is bridged via xtsb to the browser, which implements the actual
XWM logic.
- **`Channel`** — `ARQChannel` (KCP-over-WebSocket, reliable, low-latency)
for PROTOCOL/FRAME/XWM; `SimpleChannel` for FEEDBACK.
## App launch flow (definitive answer)
1. Browser sends `GET http://proxy:8081/<path>` with
`X-Compositor-Session-Id: <uuid>` header.
2. Main process forks a `SessionProcess` if none exists for that session
ID, sends `start` IPC.
3. Main sends `launchApp` IPC with `{name, executable, args, env, serial}`.
4. `SessionProcess` calls `launchApplication()`:
- `appEnv = {...process.env, ...env, WAYLAND_DISPLAY:
session.nativeWaylandCompositorSession.waylandDisplay}`
- `child_process.spawn(executable, args, {env: appEnv})` — **plain
`spawn`, no chroot/container**
5. On `spawn` event, captures `pid`, creates `NativeAppContext`, registers
`exit` listener.
6. Replies to main with `{pid, key}` → main returns HTTP 201 with
`{baseURL, signalURL, key, pid, name}`.
7. Browser opens `signalURL` as WebSocket → control channel established.
8. App connects to `$XDG_RUNTIME_DIR/wayland-N` → libwayland fires
`on_client_created` → proxy looks up PID via `getCredentials()`, walks
`/proc/<pid>/status` PPid chain to match a `NativeAppContext`.
9. For each new `wl_client`, creates a `PROTOCOL` ARQ channel and sends
`CONNECT_CHANNEL` over the signaling WS; browser opens a new WS at
`/channel?id=…`.
10. Wayland wire messages now flow: app → libwayland →
`setWireMessageCallback` → JS interceptor → routed to `{native,
browser}`.
## Native addon architecture
**Important**: The "native" code is **C, not C++** (one GLSL shader string +
GObject-style). Uses Node-API (N-API).
### `wayland-server-addon.node` (1231 lines C)
Wraps a **patched** `libwayland-server` (`libwayland-server.so.0` with
Greenfield additions in `native/wayland/changes.diff`):
- `wl_client_set_wire_message_cb`
- `wl_client_set_registry_created_cb`
- `wl_client_set_sync_done_cb`
- `wl_client_add_resource_created_listener`
- `wl_display_set_global_created_cb`
- `wl_resource_destroy_silently`
- `wl_get_server_object_ids_batch`
- `wl_connection_copy_fds_in`
Plus the `westfield` shared lib (EGL/GBM init, `wlr_linux_dmabuf_v1`,
`wlr_drm`, `westfield-xwayland`). **31 N-API functions** exposed. This is
what makes the proxy a real Wayland compositor.
### `proxy-poll-addon.node` (127 lines C)
Tiny wrapper around `uv_poll_t` (libuv, not raw epoll — though libuv uses
epoll under the hood). `startPoll(fd, cb)` integrates the wl_display
event-loop FD into Node's libuv loop so wayland events fire as Node
callbacks without blocking.
### `proxy-encoding-addon.node` (290 lines C) + `libproxy-encoding.so` (~1400 lines C)
GStreamer H.264/PNG encoder. Runs on a **dedicated GMainLoop thread** with
an `eventfd`-based message queue (`SyncSource` GSource). Two pipelines per
surface (opaque + alpha) for translucent windows. Supports `x264`
(software), `nvh264` (Nvidia NVENC), `vaapih264` (Intel, experimental).
Zero-copy DMA-BUF path via EGLImage → GStreamer GL. Uses
`napi_threadsafe_function` to marshal encoded samples back to Node.
## XWayland bridge
### Native side (`westfield-xwayland.c`, 466 lines C)
1. Creates `/tmp/.X11-unix/X<N>` listener socket
2. `setenv("DISPLAY", ":<N>", 1)`
3. Waits for X client connect
4. On connect: `socketpair(AF_UNIX)` × 3 (wayland / wm / display)
5. `wl_client_create(wl_display, sv[0])` — **Xwayland itself becomes a
Wayland client of the proxy!**
6. `fork()` + `execlp("Xwayland", "Xwayland", ":<N>", "-ac", "-rootless",
"-listen", "<unix_fd>", "-displayfd", "<display_fd>", "-wm", "<wm_fd>",
NULL)`
### xtsb (X11 TypeScript Bindings)
`libs/xtsb/`: pure-TS X11 client, no libxcb. Dual-environment (Node
`net.Socket` + browser WebSocket). Auto-generated from XCB XML protocol
files via `xcbgen/ts_client.py`. Handles MIT-MAGIC-COOKIE-1 auth from
`~/.Xauthority`. The browser-side xtsb implements the actual XWM logic
(translates X `CreateNotify` events → `xdg_toplevel` Wayland surfaces).
`XWaylandSession.ts` uses `nodeFDConnectionSetup(wmFd)` to wrap the wm_fd
as a `net.Socket`, performs X11 auth, and pipes the X11 wire protocol
through the XWM data channel to the browser.
## SDK misconception corrected
The `sdk/` directory is **NOT for native apps**. It's exclusively for
**WASM apps** that run inside the browser via Emscripten. The
`sysrootlibs/` cross-compiles wayland/gtk/cairo/etc. to wasm32. Native
Linux apps on the proxy use the host's own system libraries — the
Dockerfile installs them via `apt-get install libffi8 libgbm1
libgstreamer1.0-plugins-base … xwayland xauth`. The proxy's `pkg` config
only bundles three libs as assets (`libproxy-encoding.so`,
`libwayland-server.so.0`, `libwestfield.so`) — these are the proxy's OWN
addon dependencies, not the apps'.
## ASCII diagram — native app launch & connection
```
Browser (compositor-shell) Proxy host (Linux) App host (often same as proxy)
========================== ==================== ==============================
[1] User clicks app in shell $ compositor-proxy-cli \
│ └─── main.ts listens on :8081
│ sessionProcesses: Record<id, ChildProcess>
│
│ HTTP GET /gtk4-demo
│ Header: X-Compositor-Session-Id: <uuid>
├──────────────────────────────► │
│ │ if no child for <uuid>: fork(SessionProcess.ts)
│ │ └── IPC: {type:'start', payload:{config, sessionId}}
│ │ └── createSession() → NativeWaylandCompositorSession
│ │ ├── createDisplay()
│ │ │ └── wl_display_create() (libwayland-server)
│ │ ├── addSocketAuto()
│ │ │ └── creates $XDG_RUNTIME_DIR/wayland-N
│ │ │ returns "wayland-N" ◄── this becomes WAYLAND_DISPLAY
│ │ ├── initShm() + YUV420/NV12/YUV444/NV21
│ │ ├── initDrm("/dev/dri/renderD128")
│ │ │ └── westfield_egl_new() → EGLDisplay via GBM
│ │ │ wlr_linux_dmabuf_v1 global (zero-copy GPU buffers)
│ │ │ wlr_drm global (legacy dmabuf)
│ │ └── startPoll(wlDisplayFd, dispatchRequests)
│ │ └── uv_poll_t on EPOLLIN
│ │ (proxy-poll-addon.node)
│ │
│ │ IPC: {type:'launchApp', payload:{name, executable, args, env, serial}}
│ │ └── launchApplication() (NativeAppContext.ts)
│ │ ├── appEnv = {...process.env, ...env,
│ │ │ WAYLAND_DISPLAY: "wayland-N"}
│ │ ├── child_process.spawn("gtk4-demo", [], {env: appEnv})
│ │ │ ◄── posix_spawn/execvp, NO chroot, NO container
│ │ │ ──────────────────────────────► [2] gtk4-demo process starts
│ │ │ │
│ │ │ │ libwayland-client.so
│ │ │ │ connects to
│ │ │ │ $XDG_RUNTIME_DIR/wayland-N
│ │ │ │
│ │ │ ▼
│ │ │ ┌──────────────────────────┐
│ │ │ │ wl_display event loop │
│ │ │ │ (in proxy, libwayland- │
│ │ │ │ server.so.0) │
│ │ │ │ │
│ │ │ ◄── uv_poll fires, │ on_client_created → │
│ │ │ dispatchRequests() │ getCredentials(pid) │
│ │ │ wl_event_loop_dispatch() │ findMatchingNativeApp │
│ │ │ │ Context(pid) by PPID │
│ │ │ │ walk │
│ │ │ │ │
│ │ │ │ setWireMessageCallback │
│ │ │ │ setBufferCreated... │
│ │ │ │ setSyncDoneCallback │
│ │ │ │ setClientDestroyed... │
│ │ │ │ │
│ │ │ │ NativeWaylandClientSession
│ │ │ │ .messageInterceptor │
│ │ │ │ dispatches R<opcode> │
│ │ │ │ → {native, browser} │
│ │ │ └──────────────────────────┘
│ │ │
│ │ ├── child.on('spawn') → capture pid
│ │ │ createNativeAppContext(pid, name, external:false)
│ │ │ nativeAppContext.onDisconnect() // start 10-min SIGHUP timer
│ │ │
│ │ └── child.on('exit') → nativeAppContext.onExit({exitCode|signal})
│ │
│ │ IPC reply: {type:'launchAppSuccess', payload:{pid, key}}
│
│ HTTP 201 Created
│ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, internal:false }
◄────────────────────────────── │
│ │
│ Open WS: ws://…/signal?key=<16-hex>
├──────────────────────────────► │
│ │ SessionController demux /signal → nativeAppContext.onConnect(ws)
│ │ clears 10-min SIGHUP timer
│ │
│ │ For each new wl_client (one per wayland connection):
│ │ createProtocolChannel() → ARQChannel (KCP over WS)
│ │ sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc}
│ ◄──── CONNECT_CHANNEL JSON ───┤
│ │
│ Open WS: ws://…/channel?id=…&key=…
├──────────────────────────────► │ binds WS to the ARQChannel
│ │
│ Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval):
│
│ App → wl_surface.commit ──── proxy intercepts R1 (attach) + R6 (commit) ────► browser compositor
│ App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄────── FrameFeedback paces callback
│ App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected)
│ App ← wl_keyboard.enter ─── browser → proxy → sendEvents(wlClient, …) → flush(wlClient)
│
│ Buffer path (per wl_surface):
│ - App submits wl_buffer (dmabuf or shm)
│ - setBufferCreatedCallback → ProxyBuffer with creationSerial
│ - On commit: wl_surface_interceptor.R6 → Encoder.encodeFrame()
│ └── proxy-encoding-addon.encodeFrame(bufferId, contentSerial, creationSerial)
│ └── GStreamer pipeline (x264/nvh264/vaapih264) on dedicated GMainLoop thread
│ └── encoded H.264 sample → napi_threadsafe_function → Node main thread
│ └── sent over the surface's FRAME data channel (separate WS, also CONNECT_CHANNEL'd)
│
│ X11 path (xterm etc.):
│ - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by westfield_xwayland.c)
│ - First X connect → spawn_xserver() forks: execlp("Xwayland", …, "-rootless", "-wm", wm_fd)
│ - Xwayland itself becomes a Wayland client of the proxy (wl_client_create)
│ - When Xwayland ready (displayFd readable):
│ └── xtsb.nodeFDConnectionSetup(wmFd) → wraps as net.Socket
│ └── pipes through XWM data channel (4th channel type) to browser
│ └── browser-side xtsb speaks X11 → translates X windows → xdg_toplevel surfaces
│
│ Lifecycle:
│ - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP')
│ - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM')
│ └── if not exited in 10s → SIGKILL
│ - App crashes → child 'exit' event → nativeAppContext.onExit() → APP_TERMINATED msg → browser UI updates
│ - All clients gone for >1s → session.close() → process.exit() in SessionProcess.ts → main deletes entry
│
│ Native addon role summary:
│ - wayland-server-addon.node: full Wayland compositor (libwayland-server fork + westfield EGL/GBM/dmabuf/XWayland)
│ - proxy-poll-addon.node: libuv uv_poll_t hook so wayland FD events fire as Node callbacks
│ - proxy-encoding-addon.node: GStreamer H.264/PNG encoder on a dedicated GMainLoop thread, threadsafe_function back to Node
```
## File reference
### CLI (`packages/compositor-proxy-cli/src/`)
| File | Role |
|---|---|
| `main.ts` | HTTP + WS hub, forks per session |
| `main-controller.ts` | CLI arg parsing |
| `main-args.ts` | CLI arg definitions |
| `SessionProcess.ts` | Per-session child, IPC handler |
| `example-applications.json` | Pre-configured apps |
| `wait-until-ready.sh` | Startup readiness check |
### Proxy TS layer (`packages/compositor-proxy/src/`)
| File | Role |
|---|---|
| `index.ts` | Exports `createSession()`, `launchApplication()` |
| `Session.ts` | Top-level proxy session |
| `SessionController.ts` | WS server with 9 endpoint paths |
| `NativeAppContext.ts` | Per-app state, signaling WS, kill timers |
| `NativeWaylandClientSession.ts` | Per-client message interceptor |
| `NativeWaylandCompositorSession.ts` | wl_display creation, initShm, initDrm |
| `Channel.ts` | ARQChannel + SimpleChannel |
| `kcp.ts` | ~1100-line KCP implementation |
| `FrameFeedback.ts` | Receives 4-byte decode feedback, paces wl_frame.done |
| `ProxyBuffer.ts` | Proxy-side buffer representation |
| `SurfaceBufferEncoding.ts` | Encoding pipeline orchestration |
| `XWaylandSession.ts` | XWayland spawn + bridge |
| `wayland-server.ts` | TS wrapper around wayland-server-addon.node |
| `io/` | WebSocket server glue |
| `encoding/` | High-level encoding orchestration |
| `protocol/` | Generated interceptor modules |
| `addons/` | .node file shims (Linux-x64) |
| `config.ts` | Encoding-related configuration |
### Proxy C layer (`packages/compositor-proxy/native/`)
| Path | Role |
|---|---|
| `CMakeLists.txt` | Build config |
| `wayland/src/addon.c` | Node-API addon: wayland-server-addon.node |
| `wayland/src/westfield-buffer.c` | Buffer helpers |
| `wayland/src/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM |
| `wayland/src/westfield-egl.c` | EGL zero-copy setup |
| `wayland/src/westfield-xwayland.c` | XWayland spawn + bridge |
| `wayland/src/wlr_drm.c` | wl_drm global impl |
| `wayland/src/wlr_linux_dmabuf_v1.c` | zwp_linux_dmabuf_v1 global impl |
| `wayland/src/drm-protocol.c` | DRM protocol helpers |
| `wayland/src/linux-dmabuf-unstable-v1-protocol.c` | linux-dmabuf-v1 protocol stubs |
| `wayland/src/wayland-server/` | Vendored + patched libwayland-server (with changes.diff) |
| `encoding/src/gst_frame_encoder.c` | GStreamer H.264/PNG encoder |
| `encoding/src/gst_main_loop.c` | Dedicated GLib main loop for encoding |
| `encoding/src/node_encoder.c` | Node-API addon: proxy-encoding-addon.node |
| `poll/src/poll.c` | libuv uv_poll_t wrapper |
| `node-api/` | Vendored Node-API headers |
### XWayland support (`libs/xtsb/`)
| File | Role |
|---|---|
| `README.md` | Project intro |
| `package.json` | @gfld/xtsb package config |
| `protocol/xproto.xml` | X11 core protocol XML (5800+ lines) |
| `protocol/shape.xml` | Shape extension XML |
| `protocol/xfixes.xml` | XFIXES extension XML |
| `protocol/render.xml` | Render extension XML |
| `protocol/composite.xml` | Composite extension XML |
| `src/` | Generated TS bindings |
## Next actions for user (Odoo 20 integration)
1. Build the proxy on a Linux host (or use `udevbe/compositor-proxy-cli`
Docker image).
2. Write an `applications.json` mapping desired apps to HTTP paths.
3. Start proxy with `--allow-origin=https://your-odoo.example.com`.
4. Embed the Greenfield `compositor-shell` Preact SPA in Odoo (iframe or
QWeb template) — OR replace it with your Odoo 20 OWL shell (see
`10-porting-to-odoo-js-python.md`).
5. Trigger app launches from Odoo via
`fetch('http://<proxy>:8081/<path>', { headers: { 'X-Compositor-Session-Id': sessionId }})`
— Odoo frontend JS uses the returned `signalURL` to open the per-app
control WS.
6. For multi-tenant isolation: one `SessionProcess` per browser session is
automatic, but file-system isolation between tenants requires separate
proxy instances or containers per tenant.
7. For GPU acceleration: pass `--device /dev/dri:/dev/dri` to Docker; use
`--encoder=nvh264|vaapih264|x264` based on GPU.