Initial commit: MIT-licensed compositor architecture & build plan (14 files, ~80 pages, 10 parallel research agents)

This commit is contained in:
1b-agent committed 2026-10-08 08:48:28 +00:00
commit 582e61e942
15 files changed
+4970

No files matched your search

+332
View File
@@ -0,0 +1,332 @@
# 04 — Transport Layer (Python aiohttp + KCP, browser side clean-room)
This chapter covers the transport layer for the MIT approach: how Wayland
wire traffic flows between the Python proxy and the browser compositor.
## Recommended Python transport stack
| Layer | Choice | Why |
|---|---|---|
| **WebSocket framework** | **aiohttp.web** (v3.10+) | Best binary perf (300-600 Mbps on CPython, 2× on PyPy), native backpressure via `await ws.drain()`, single dependency, asyncio-native — eliminates the ASGI indirection tax of Starlette/uvicorn and the framework overhead of FastAPI |
| **KCP library** | **`kcp` PyPI package** (RealistikDash/kcp.py, v0.1.6, MIT, Cython wrapper over upstream `ikcp.c`) | API (`enqueue`/`receive`/`get_all_received`/`update`/`outbound_handler`) maps cleanly to Greenfield's `Kcp` class. ~1500 LoC port of Greenfield's TS KCP would buy nothing — the C-backed package is faster and battle-tested |
| **libwayland binding** | **pywayland 0.4.19** (Apache-2.0, CFFI) + **cffi escape hatch for `wl_event_loop_get_fd`** | pywayland's `EventLoop` wrapper omits `wl_event_loop_get_fd()`. Fix: call `pywayland.lib.wl_event_loop_get_fd(loop._ptr)` directly via cffi, then `asyncio.get_running_loop().add_reader(wl_fd, dispatch_wl)`. Replaces Greenfield's entire `proxy-poll-addon.node` C addon with ~10 lines of Python |
| **Multi-process model** | **`multiprocessing.Process` (fork start method) per Odoo session** | Matches Greenfield's per-`compositorSessionId` forking. FD passing of WS upgrade socket via `multiprocessing.reduction.send_handle`/`recv_handle` (uses SCM_RIGHTS on Linux). Preserves the SIGHUP/SIGKILL escalation semantics |
| **Encoder threading** | **`threading.Thread` + GMainLoop + `loop.call_soon_threadsafe`** | Mirrors Greenfield's `napi_threadsafe_function` pattern. GIL released during C encoding calls (GStreamer). Reuse Greenfield's MIT C files for dmabuf/egl via cffi — preserves all EGL/GBM/GStreamer pipelines |
| **Binary message format** | **Keep Greenfield's verbatim** | Browser-side TS unchanged; no need for a version field until v2. KCP gives message boundaries |
| **ReconnectingWebSocket** | **Reuse Greenfield's vendored copy** (Pedro Ladaria, MIT, 514 lines TS) | Browser-side; copy with attribution into Odoo OWL shell assets |
## Channel types (same as Greenfield)
| Type | Class | KCP? | Purpose |
|------|-------|------|---------|
| PROTOCOL | ARQChannel | Yes | Wayland wire protocol, per `wl_client` |
| FRAME | ARQChannel | Yes | H.264/PNG encoded surface content, per `wl_surface` |
| XWM | ARQChannel | Yes | XWayland window manager protocol |
| FEEDBACK | SimpleChannel | No | 4-byte telemetry (refreshInterval + decode avg), 1 Hz |
## WebSocket endpoints (9 paths, same as Greenfield)
```
/signal # JSON control plane (per-app)
/channel # binary data channels (multiplexed)
/mkfifo # create a POSIX pipe, return 2 ProxyFDs
/mkstemp-mmap # create a memfd-backed shm, return 1 ProxyFD
/read-fd # read bytes from a ProxyFD (one-shot)
/read-fd-as-stream # read bytes from a ProxyFD (streaming)
/write-fd-as-stream # write bytes to a ProxyFD (streaming)
/close-fd # close a ProxyFD
/request-keyframe # ask the encoder for a keyframe
```
All endpoints require JWT auth (HS256, validated by `pyjwt`).
## asyncio + libwayland integration (the riskiest piece)
**Verified working** in the sandbox:
```python
import asyncio, cffi
from pywayland.server import Display
extras_ffi = cffi.FFI()
extras_ffi.cdef("""
int wl_event_loop_get_fd(void *loop);
void *wl_display_get_event_loop(void *display);
int wl_event_loop_dispatch(void *loop, int timeout);
void wl_display_flush_clients(void *display);
""")
_lib = extras_ffi.dlopen("libwayland-server.so.0") # or pywayland's bundled
display = Display()
display.add_socket()
display.init_shm()
_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr))
_display_voidp = extras_ffi.cast('void *', _ptr_int)
_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp)
_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp)
async def main():
loop = asyncio.get_running_loop()
def on_ready():
_lib.wl_event_loop_dispatch(_loop_voidp, 0)
display.flush_clients()
loop.add_reader(_wl_fd, on_ready)
await asyncio.Future() # run forever
asyncio.run(main())
```
This pattern is used in production by **qtile's Wayland backend**
(`libqtile/backend/wayland/core.py`). qtile is GPL-2.0 so don't copy code,
but the architectural pattern is documented and reusable.
## Process model — three layers (same as Greenfield)
### Layer 1: Main process (HTTP + WS hub)
Single Python process listens on `:8081`. Per Odoo session, it forks
one `SessionProcess` child — one child per `compositor_session_id`, **not**
per app. Multiple apps in the same browser tab share one child.
WebSocket upgrades are handed off to the matching child via
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
### Layer 2: Session process
The forked child. Receives IPC messages:
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
- `launch_app` — spawns the actual app binary via `subprocess.Popen` + bubblewrap
- `ws_upgrade` — hands WS to the dispatcher
On `session.close()` the process exits.
### Layer 3: Per-client session
For each new `wl_client` (one per Wayland connection), creates a
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
JSON. Routes requests via per-resource dispatchers.
## FD virtualization (same approach as Greenfield)
`ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }`
In Python:
- `os.pipe()` for pipes
- `os.memfd_create(name, flags=0)` for shared memory (Python 3.8+ builtin,
replaces Greenfield's `createMemoryMappedFile` native addon)
- `asyncio.StreamReader` / `StreamWriter` for streaming reads/writes over
separate WebSocket endpoints
## Reconnection & lifecycle (same as Greenfield)
### Browser side
Vendored `ReconnectingWebSocket` (MIT):
- `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s`
- `minUptime=5s`, `connTimeout=4s`, `retries=∞`
- Close codes 4000-4999 tear down KCP; lower codes auto-reconnect
### Python side
No auto-reconnect (it's the server). Channel object lives on, waiting for
new WS upgrade with matching `id` + `key`. Outbound messages queue and
flush on reconnect.
### App kill timers
When browser disconnects from `/signal` WebSocket:
- 10-minute SIGHUP grace period
- After 10 min: SIGTERM, then SIGKILL after another 10s
- Close code 1001 → immediate SIGHUP (intentional tab close)
- Close code 4001 → immediate SIGTERM (force quit)
## JWT auth (NEW vs Greenfield)
Every WebSocket connection must include a valid JWT (HS256, 15-min expiry):
```
Authorization: Bearer <jwt>
```
Or as a query param: `?token=<jwt>`
JWT payload:
```json
{
"iss": "odoo.example.com",
"aud": "compositor-proxy",
"sub": 42, // Odoo user ID
"db": "production", // Odoo database name
"company_ids": [1, 3],
"scope": "compositor",
"compositor_session_id": "uuid-here",
"iat": 1736000000,
"exp": 1736000900
}
```
Validated by the proxy using `compositor.jwt_secret` from Odoo's
`ir.config_parameter` (shared via env var at proxy startup).
## ASCII diagram
```
PYTHON HOST (Linux) BROWSER (Odoo OWL shell)
========================== ========================
┌─────────────────────────┐ ┌──────────────────────────────┐
│ Native app (gtk4-demo) │ │ Odoo web client │
│ (sandboxed in bwrap) │ │ (Odoo 20 OWL shell addon) │
└───────────┬─────────────┘ └──────────────┬───────────────┘
│ WAYLAND_DISPLAY=wayland-0 │
│ Unix socket + SCM_RIGHTS FD passing │ DOM events → input
▼ │ WebSocket API
┌──────────────────────────────────────────┐ │
│ SessionProcess (fork'd Python child) │ │
│ ┌────────────────────────────────────┐ │ │
│ │ pywayland (Apache-2.0) │ │ │
│ │ + 5-line cffi shim │ │ │
│ │ - wl_display + wl_client │ │ │
│ │ - asyncio.add_reader for wl_fd │ │ │
│ └─────────────┬──────────────────────┘ │ │
│ │ wire msg + FDs │ │
│ ▼ │ │
│ ┌─────────────────────────────────────┐ │ │
│ │ Per-resource dispatchers │ │ │
│ │ (auto-generated by pywayland) │ │ │
│ │ - route to {native, browser} │ │ │
│ │ - serialize FDs as ProxyFD JSON │ │ │
│ │ - fast-sync optimization │ │ │
│ └─────────────┬───────────────────────┘ │ │
│ │ outbound buffer │ │
│ ▼ │ │
│ ┌─────────────────────────────────────┐ │ │
│ │ ARQChannel (KCP-wrapped) │ │ │
│ │ - kcp PyPI package (MIT) │ │ │
│ │ - MTU=64000, snd_wnd=128, rcv=1024 │ │ │
│ │ - kcp.update() every 100ms │ │ │
│ │ - PROTOCOL (per wl_client) │ │ │
│ │ - FRAME (per wl_surface) │ │ │
│ │ - XWM (per XWayland) │ │ │
│ └─────────────┬───────────────────────┘ │ │
│ ┌─────────────┴───────────────────────┐ │ │
│ │ SimpleChannel (FEEDBACK, no KCP) │ │ │
│ │ 4 bytes: refreshIntervalMs + │ │ │
│ │ avgDecodeDurationMs │ │ │
│ └─────────────┬───────────────────────┘ │ │
│ ┌─────────────▼───────────────────────┐ │ │
│ │ NativeAppContext │ │ │
│ │ - signaling WebSocket (JSON) │ │ │
│ │ - kill timers: SIGHUP@10min, │ │ │
│ │ SIGKILL@+10s │ │ │
│ │ - bubblewrap + seccomp sandbox │ │ │
│ └─────────────┬───────────────────────┘ │ │
│ ┌─────────────▼───────────────────────┐ │ │
│ │ SessionController │ │ │
│ │ - aiohttp.web.Application │ │ │
│ │ - 9 endpoints │ │ │
│ │ - JWT validation (pyjwt) │ │ │
│ │ - perMessageDeflate: false │ │ │
│ └─────────────┬───────────────────────┘ │ │
│ ┌─────────────▼───────────────────────┐ │ │
│ │ Encoder thread (GStreamer + GLib │ │ │
│ │ MainLoop in separate thread) │ │ │
│ │ - vaapih264enc / nvh264enc / │ │ │
│ │ openh264enc (NOT x264enc — GPL) │ │ │
│ │ - asyncio.run_coroutine_threadsafe │ │ │
│ │ for cross-thread marshalling │ │ │
│ └─────────────┬───────────────────────┘ │ │
└────────────────┼─────────────────────────────┘ │
│ │
│ ┌──────────────────────────┐ │
│ │ main.py (HTTP+WS hub) │ │
│ │ - socket.pause() + IPC │ │
│ │ send to session proc │ │
│ │ - multiprocessing.fork │ │
│ │ - JWT issuance delegated │ │
│ │ to Odoo backend │ │
│ └────────────┬─────────────┘ │
│ │ │
│ ◄──── 1 WebSocket per channel ────► │
│ ◄── 1 signaling WS per app ──► │
│ ◄── ad-hoc WebFD streams ─► │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────┐ │
│ │ ReconnectingWebSocket (vendored MIT) │ │
│ │ min 1-5s, factor 1.3, max 10s │ │
│ │ minUptime 5s, connTimeout 4s │ │
│ │ + Authorization: Bearer <jwt> │ │
│ └────────────┬───────────────────────────┘ │
│ ▼ │
│ ┌────────────────────────────────────────────┐ │
│ │ ARQChannel / SimpleChannel (clean-room TS) │ │
│ │ snd_wnd=1024, rcv_wnd=128 │ │
│ │ kcp.input → peekSize/recv → onMessage │ │
│ └────────────┬───────────────────────────────┘ │
│ ▼ │
│ ┌────────────────────────────────────────────┐ │
│ │ Custom Browser Compositor (clean-room TS) │ │
│ │ - WebCodecs VideoDecoder (HW-accelerated)│ │
│ │ - WebGL2 + OffscreenCanvas in Worker │ │
│ │ - Surface/View/Output/Seat/Keyboard/... │ │
│ └────────────┬───────────────────────────────┘ │
│ │ input events flow back via the │
│ │ PROTOCOL channel as ordinary │
│ │ Wayland requests (same framing) │
└──────────────────┘ │
Browser DOM ◄─┘
```
## Effort estimate (Python transport)
| Component | LoC | Dev-days | Risk |
|---|---|---|---|
| aiohttp WS server (9 endpoints + HTTP launcher) | 600 | 3 | Low |
| `Channel`/`ARQChannel`/`SimpleChannel` port | 250 | 1.5 | Low |
| `kcp.py` integration adapter | 100 | 0.5 | Low |
| `NativeAppContext` port (signaling, escalation) | 350 | 2 | Low |
| `ProxyInputOutput` port (FD workaround) | 300 | 2 | Medium |
| libwayland event-loop integration | 80 | 1 | **Medium-High** (under-documented) |
| Multiprocessing scaffolding + FD passing | 200 | 1.5 | Medium |
| GStreamer encoder thread + marshalling | 250 | 2 | Medium |
| Reconnection & lifecycle | 150 | 1 | Low |
| Tests (pytest + pytest-asyncio) | 600 | 3 | Low |
| **Total** | **2880** | **~17** | |
## Top risks & mitigations
1. **Highest risk**: libwayland ↔ asyncio event-loop integration (under-documented).
**Mitigation**: Spend 1 day on a standalone prototype (the 10-line script
above) before committing.
2. **Medium risk**: FD passing via `multiprocessing.reduction`. Fallback:
single-process with asyncio concurrency.
3. **Medium risk**: GStreamer threading. Mitigation: reuse Greenfield's
MIT C files via cffi — only the marshalling needs Python.
## Dependency manifest
```
# requirements.txt
aiohttp>=3.10,<4 # WebSocket server + HTTP
kcp>=0.1.6 # Cython KCP bindings (RealistikDash)
pywayland>=0.4.19 # libwayland CFFI wrapper (Apache-2.0)
cffi>=1.16 # for wl_event_loop_get_fd escape hatch
pygobject>=3.48 # GStreamer Python bindings
pyjwt>=2.8 # JWT validation
```
System packages (Debian):
```
libwayland-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
libgstreamer-plugins-bad1.0-dev libgbm-dev libdrm-dev libegl-dev
gstreamer1.0-vaapi # Intel/AMD hardware encoding
libnvidia-encode1 # NVIDIA hardware encoding (if applicable)
bubblewrap libseccomp-dev
python3-dev build-essential pkg-config
```
**DO NOT INSTALL**: `gstreamer1.0-plugins-ugly` (provides x264enc, GPL) or
`gstreamer1.0-libav` (LGPL source but GPL binary on Debian).