Initial commit: MIT-licensed compositor architecture & build plan (14 files, ~80 pages, 10 parallel research agents)
This commit is contained in:
commit
582e61e942
15 files changed
+4970
No files matched your search
+332
@@ -0,0 +1,332 @@
|
||||
# 04 — Transport Layer (Python aiohttp + KCP, browser side clean-room)
|
||||
|
||||
This chapter covers the transport layer for the MIT approach: how Wayland
|
||||
wire traffic flows between the Python proxy and the browser compositor.
|
||||
|
||||
## Recommended Python transport stack
|
||||
|
||||
| Layer | Choice | Why |
|
||||
|---|---|---|
|
||||
| **WebSocket framework** | **aiohttp.web** (v3.10+) | Best binary perf (300-600 Mbps on CPython, 2× on PyPy), native backpressure via `await ws.drain()`, single dependency, asyncio-native — eliminates the ASGI indirection tax of Starlette/uvicorn and the framework overhead of FastAPI |
|
||||
| **KCP library** | **`kcp` PyPI package** (RealistikDash/kcp.py, v0.1.6, MIT, Cython wrapper over upstream `ikcp.c`) | API (`enqueue`/`receive`/`get_all_received`/`update`/`outbound_handler`) maps cleanly to Greenfield's `Kcp` class. ~1500 LoC port of Greenfield's TS KCP would buy nothing — the C-backed package is faster and battle-tested |
|
||||
| **libwayland binding** | **pywayland 0.4.19** (Apache-2.0, CFFI) + **cffi escape hatch for `wl_event_loop_get_fd`** | pywayland's `EventLoop` wrapper omits `wl_event_loop_get_fd()`. Fix: call `pywayland.lib.wl_event_loop_get_fd(loop._ptr)` directly via cffi, then `asyncio.get_running_loop().add_reader(wl_fd, dispatch_wl)`. Replaces Greenfield's entire `proxy-poll-addon.node` C addon with ~10 lines of Python |
|
||||
| **Multi-process model** | **`multiprocessing.Process` (fork start method) per Odoo session** | Matches Greenfield's per-`compositorSessionId` forking. FD passing of WS upgrade socket via `multiprocessing.reduction.send_handle`/`recv_handle` (uses SCM_RIGHTS on Linux). Preserves the SIGHUP/SIGKILL escalation semantics |
|
||||
| **Encoder threading** | **`threading.Thread` + GMainLoop + `loop.call_soon_threadsafe`** | Mirrors Greenfield's `napi_threadsafe_function` pattern. GIL released during C encoding calls (GStreamer). Reuse Greenfield's MIT C files for dmabuf/egl via cffi — preserves all EGL/GBM/GStreamer pipelines |
|
||||
| **Binary message format** | **Keep Greenfield's verbatim** | Browser-side TS unchanged; no need for a version field until v2. KCP gives message boundaries |
|
||||
| **ReconnectingWebSocket** | **Reuse Greenfield's vendored copy** (Pedro Ladaria, MIT, 514 lines TS) | Browser-side; copy with attribution into Odoo OWL shell assets |
|
||||
|
||||
## Channel types (same as Greenfield)
|
||||
|
||||
| Type | Class | KCP? | Purpose |
|
||||
|------|-------|------|---------|
|
||||
| PROTOCOL | ARQChannel | Yes | Wayland wire protocol, per `wl_client` |
|
||||
| FRAME | ARQChannel | Yes | H.264/PNG encoded surface content, per `wl_surface` |
|
||||
| XWM | ARQChannel | Yes | XWayland window manager protocol |
|
||||
| FEEDBACK | SimpleChannel | No | 4-byte telemetry (refreshInterval + decode avg), 1 Hz |
|
||||
|
||||
## WebSocket endpoints (9 paths, same as Greenfield)
|
||||
|
||||
```
|
||||
/signal # JSON control plane (per-app)
|
||||
/channel # binary data channels (multiplexed)
|
||||
/mkfifo # create a POSIX pipe, return 2 ProxyFDs
|
||||
/mkstemp-mmap # create a memfd-backed shm, return 1 ProxyFD
|
||||
/read-fd # read bytes from a ProxyFD (one-shot)
|
||||
/read-fd-as-stream # read bytes from a ProxyFD (streaming)
|
||||
/write-fd-as-stream # write bytes to a ProxyFD (streaming)
|
||||
/close-fd # close a ProxyFD
|
||||
/request-keyframe # ask the encoder for a keyframe
|
||||
```
|
||||
|
||||
All endpoints require JWT auth (HS256, validated by `pyjwt`).
|
||||
|
||||
## asyncio + libwayland integration (the riskiest piece)
|
||||
|
||||
**Verified working** in the sandbox:
|
||||
|
||||
```python
|
||||
import asyncio, cffi
|
||||
from pywayland.server import Display
|
||||
|
||||
extras_ffi = cffi.FFI()
|
||||
extras_ffi.cdef("""
|
||||
int wl_event_loop_get_fd(void *loop);
|
||||
void *wl_display_get_event_loop(void *display);
|
||||
int wl_event_loop_dispatch(void *loop, int timeout);
|
||||
void wl_display_flush_clients(void *display);
|
||||
""")
|
||||
_lib = extras_ffi.dlopen("libwayland-server.so.0") # or pywayland's bundled
|
||||
|
||||
display = Display()
|
||||
display.add_socket()
|
||||
display.init_shm()
|
||||
|
||||
_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr))
|
||||
_display_voidp = extras_ffi.cast('void *', _ptr_int)
|
||||
_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp)
|
||||
_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp)
|
||||
|
||||
async def main():
|
||||
loop = asyncio.get_running_loop()
|
||||
def on_ready():
|
||||
_lib.wl_event_loop_dispatch(_loop_voidp, 0)
|
||||
display.flush_clients()
|
||||
loop.add_reader(_wl_fd, on_ready)
|
||||
await asyncio.Future() # run forever
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
This pattern is used in production by **qtile's Wayland backend**
|
||||
(`libqtile/backend/wayland/core.py`). qtile is GPL-2.0 so don't copy code,
|
||||
but the architectural pattern is documented and reusable.
|
||||
|
||||
## Process model — three layers (same as Greenfield)
|
||||
|
||||
### Layer 1: Main process (HTTP + WS hub)
|
||||
|
||||
Single Python process listens on `:8081`. Per Odoo session, it forks
|
||||
one `SessionProcess` child — one child per `compositor_session_id`, **not**
|
||||
per app. Multiple apps in the same browser tab share one child.
|
||||
|
||||
WebSocket upgrades are handed off to the matching child via
|
||||
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
|
||||
|
||||
### Layer 2: Session process
|
||||
|
||||
The forked child. Receives IPC messages:
|
||||
|
||||
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
|
||||
- `launch_app` — spawns the actual app binary via `subprocess.Popen` + bubblewrap
|
||||
- `ws_upgrade` — hands WS to the dispatcher
|
||||
|
||||
On `session.close()` the process exits.
|
||||
|
||||
### Layer 3: Per-client session
|
||||
|
||||
For each new `wl_client` (one per Wayland connection), creates a
|
||||
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
|
||||
JSON. Routes requests via per-resource dispatchers.
|
||||
|
||||
## FD virtualization (same approach as Greenfield)
|
||||
|
||||
`ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }`
|
||||
|
||||
In Python:
|
||||
- `os.pipe()` for pipes
|
||||
- `os.memfd_create(name, flags=0)` for shared memory (Python 3.8+ builtin,
|
||||
replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
- `asyncio.StreamReader` / `StreamWriter` for streaming reads/writes over
|
||||
separate WebSocket endpoints
|
||||
|
||||
## Reconnection & lifecycle (same as Greenfield)
|
||||
|
||||
### Browser side
|
||||
|
||||
Vendored `ReconnectingWebSocket` (MIT):
|
||||
- `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s`
|
||||
- `minUptime=5s`, `connTimeout=4s`, `retries=∞`
|
||||
- Close codes 4000-4999 tear down KCP; lower codes auto-reconnect
|
||||
|
||||
### Python side
|
||||
|
||||
No auto-reconnect (it's the server). Channel object lives on, waiting for
|
||||
new WS upgrade with matching `id` + `key`. Outbound messages queue and
|
||||
flush on reconnect.
|
||||
|
||||
### App kill timers
|
||||
|
||||
When browser disconnects from `/signal` WebSocket:
|
||||
- 10-minute SIGHUP grace period
|
||||
- After 10 min: SIGTERM, then SIGKILL after another 10s
|
||||
- Close code 1001 → immediate SIGHUP (intentional tab close)
|
||||
- Close code 4001 → immediate SIGTERM (force quit)
|
||||
|
||||
## JWT auth (NEW vs Greenfield)
|
||||
|
||||
Every WebSocket connection must include a valid JWT (HS256, 15-min expiry):
|
||||
|
||||
```
|
||||
Authorization: Bearer <jwt>
|
||||
```
|
||||
|
||||
Or as a query param: `?token=<jwt>`
|
||||
|
||||
JWT payload:
|
||||
```json
|
||||
{
|
||||
"iss": "odoo.example.com",
|
||||
"aud": "compositor-proxy",
|
||||
"sub": 42, // Odoo user ID
|
||||
"db": "production", // Odoo database name
|
||||
"company_ids": [1, 3],
|
||||
"scope": "compositor",
|
||||
"compositor_session_id": "uuid-here",
|
||||
"iat": 1736000000,
|
||||
"exp": 1736000900
|
||||
}
|
||||
```
|
||||
|
||||
Validated by the proxy using `compositor.jwt_secret` from Odoo's
|
||||
`ir.config_parameter` (shared via env var at proxy startup).
|
||||
|
||||
## ASCII diagram
|
||||
|
||||
```
|
||||
PYTHON HOST (Linux) BROWSER (Odoo OWL shell)
|
||||
========================== ========================
|
||||
|
||||
┌─────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ Native app (gtk4-demo) │ │ Odoo web client │
|
||||
│ (sandboxed in bwrap) │ │ (Odoo 20 OWL shell addon) │
|
||||
└───────────┬─────────────┘ └──────────────┬───────────────┘
|
||||
│ WAYLAND_DISPLAY=wayland-0 │
|
||||
│ Unix socket + SCM_RIGHTS FD passing │ DOM events → input
|
||||
▼ │ WebSocket API
|
||||
┌──────────────────────────────────────────┐ │
|
||||
│ SessionProcess (fork'd Python child) │ │
|
||||
│ ┌────────────────────────────────────┐ │ │
|
||||
│ │ pywayland (Apache-2.0) │ │ │
|
||||
│ │ + 5-line cffi shim │ │ │
|
||||
│ │ - wl_display + wl_client │ │ │
|
||||
│ │ - asyncio.add_reader for wl_fd │ │ │
|
||||
│ └─────────────┬──────────────────────┘ │ │
|
||||
│ │ wire msg + FDs │ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ Per-resource dispatchers │ │ │
|
||||
│ │ (auto-generated by pywayland) │ │ │
|
||||
│ │ - route to {native, browser} │ │ │
|
||||
│ │ - serialize FDs as ProxyFD JSON │ │ │
|
||||
│ │ - fast-sync optimization │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ │ outbound buffer │ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ ARQChannel (KCP-wrapped) │ │ │
|
||||
│ │ - kcp PyPI package (MIT) │ │ │
|
||||
│ │ - MTU=64000, snd_wnd=128, rcv=1024 │ │ │
|
||||
│ │ - kcp.update() every 100ms │ │ │
|
||||
│ │ - PROTOCOL (per wl_client) │ │ │
|
||||
│ │ - FRAME (per wl_surface) │ │ │
|
||||
│ │ - XWM (per XWayland) │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────┴───────────────────────┐ │ │
|
||||
│ │ SimpleChannel (FEEDBACK, no KCP) │ │ │
|
||||
│ │ 4 bytes: refreshIntervalMs + │ │ │
|
||||
│ │ avgDecodeDurationMs │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ NativeAppContext │ │ │
|
||||
│ │ - signaling WebSocket (JSON) │ │ │
|
||||
│ │ - kill timers: SIGHUP@10min, │ │ │
|
||||
│ │ SIGKILL@+10s │ │ │
|
||||
│ │ - bubblewrap + seccomp sandbox │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ SessionController │ │ │
|
||||
│ │ - aiohttp.web.Application │ │ │
|
||||
│ │ - 9 endpoints │ │ │
|
||||
│ │ - JWT validation (pyjwt) │ │ │
|
||||
│ │ - perMessageDeflate: false │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ Encoder thread (GStreamer + GLib │ │ │
|
||||
│ │ MainLoop in separate thread) │ │ │
|
||||
│ │ - vaapih264enc / nvh264enc / │ │ │
|
||||
│ │ openh264enc (NOT x264enc — GPL) │ │ │
|
||||
│ │ - asyncio.run_coroutine_threadsafe │ │ │
|
||||
│ │ for cross-thread marshalling │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
└────────────────┼─────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ main.py (HTTP+WS hub) │ │
|
||||
│ │ - socket.pause() + IPC │ │
|
||||
│ │ send to session proc │ │
|
||||
│ │ - multiprocessing.fork │ │
|
||||
│ │ - JWT issuance delegated │ │
|
||||
│ │ to Odoo backend │ │
|
||||
│ └────────────┬─────────────┘ │
|
||||
│ │ │
|
||||
│ ◄──── 1 WebSocket per channel ────► │
|
||||
│ ◄── 1 signaling WS per app ──► │
|
||||
│ ◄── ad-hoc WebFD streams ─► │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ ReconnectingWebSocket (vendored MIT) │ │
|
||||
│ │ min 1-5s, factor 1.3, max 10s │ │
|
||||
│ │ minUptime 5s, connTimeout 4s │ │
|
||||
│ │ + Authorization: Bearer <jwt> │ │
|
||||
│ └────────────┬───────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ ARQChannel / SimpleChannel (clean-room TS) │ │
|
||||
│ │ snd_wnd=1024, rcv_wnd=128 │ │
|
||||
│ │ kcp.input → peekSize/recv → onMessage │ │
|
||||
│ └────────────┬───────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Custom Browser Compositor (clean-room TS) │ │
|
||||
│ │ - WebCodecs VideoDecoder (HW-accelerated)│ │
|
||||
│ │ - WebGL2 + OffscreenCanvas in Worker │ │
|
||||
│ │ - Surface/View/Output/Seat/Keyboard/... │ │
|
||||
│ └────────────┬───────────────────────────────┘ │
|
||||
│ │ input events flow back via the │
|
||||
│ │ PROTOCOL channel as ordinary │
|
||||
│ │ Wayland requests (same framing) │
|
||||
└──────────────────┘ │
|
||||
Browser DOM ◄─┘
|
||||
```
|
||||
|
||||
## Effort estimate (Python transport)
|
||||
|
||||
| Component | LoC | Dev-days | Risk |
|
||||
|---|---|---|---|
|
||||
| aiohttp WS server (9 endpoints + HTTP launcher) | 600 | 3 | Low |
|
||||
| `Channel`/`ARQChannel`/`SimpleChannel` port | 250 | 1.5 | Low |
|
||||
| `kcp.py` integration adapter | 100 | 0.5 | Low |
|
||||
| `NativeAppContext` port (signaling, escalation) | 350 | 2 | Low |
|
||||
| `ProxyInputOutput` port (FD workaround) | 300 | 2 | Medium |
|
||||
| libwayland event-loop integration | 80 | 1 | **Medium-High** (under-documented) |
|
||||
| Multiprocessing scaffolding + FD passing | 200 | 1.5 | Medium |
|
||||
| GStreamer encoder thread + marshalling | 250 | 2 | Medium |
|
||||
| Reconnection & lifecycle | 150 | 1 | Low |
|
||||
| Tests (pytest + pytest-asyncio) | 600 | 3 | Low |
|
||||
| **Total** | **2880** | **~17** | |
|
||||
|
||||
## Top risks & mitigations
|
||||
|
||||
1. **Highest risk**: libwayland ↔ asyncio event-loop integration (under-documented).
|
||||
**Mitigation**: Spend 1 day on a standalone prototype (the 10-line script
|
||||
above) before committing.
|
||||
2. **Medium risk**: FD passing via `multiprocessing.reduction`. Fallback:
|
||||
single-process with asyncio concurrency.
|
||||
3. **Medium risk**: GStreamer threading. Mitigation: reuse Greenfield's
|
||||
MIT C files via cffi — only the marshalling needs Python.
|
||||
|
||||
## Dependency manifest
|
||||
|
||||
```
|
||||
# requirements.txt
|
||||
aiohttp>=3.10,<4 # WebSocket server + HTTP
|
||||
kcp>=0.1.6 # Cython KCP bindings (RealistikDash)
|
||||
pywayland>=0.4.19 # libwayland CFFI wrapper (Apache-2.0)
|
||||
cffi>=1.16 # for wl_event_loop_get_fd escape hatch
|
||||
pygobject>=3.48 # GStreamer Python bindings
|
||||
pyjwt>=2.8 # JWT validation
|
||||
```
|
||||
|
||||
System packages (Debian):
|
||||
```
|
||||
libwayland-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
|
||||
libgstreamer-plugins-bad1.0-dev libgbm-dev libdrm-dev libegl-dev
|
||||
gstreamer1.0-vaapi # Intel/AMD hardware encoding
|
||||
libnvidia-encode1 # NVIDIA hardware encoding (if applicable)
|
||||
bubblewrap libseccomp-dev
|
||||
python3-dev build-essential pkg-config
|
||||
```
|
||||
|
||||
**DO NOT INSTALL**: `gstreamer1.0-plugins-ugly` (provides x264enc, GPL) or
|
||||
`gstreamer1.0-libav` (LGPL source but GPL binary on Debian).
|
||||
Reference in new issue
Block a user