333 lines
20 KiB
Markdown
333 lines
20 KiB
Markdown
# 04 — Transport Layer (Python aiohttp + KCP, browser side clean-room)
|
||
|
||
This chapter covers the transport layer for the MIT approach: how Wayland
|
||
wire traffic flows between the Python proxy and the browser compositor.
|
||
|
||
## Recommended Python transport stack
|
||
|
||
| Layer | Choice | Why |
|
||
|---|---|---|
|
||
| **WebSocket framework** | **aiohttp.web** (v3.10+) | Best binary perf (300-600 Mbps on CPython, 2× on PyPy), native backpressure via `await ws.drain()`, single dependency, asyncio-native — eliminates the ASGI indirection tax of Starlette/uvicorn and the framework overhead of FastAPI |
|
||
| **KCP library** | **`kcp` PyPI package** (RealistikDash/kcp.py, v0.1.6, MIT, Cython wrapper over upstream `ikcp.c`) | API (`enqueue`/`receive`/`get_all_received`/`update`/`outbound_handler`) maps cleanly to Greenfield's `Kcp` class. ~1500 LoC port of Greenfield's TS KCP would buy nothing — the C-backed package is faster and battle-tested |
|
||
| **libwayland binding** | **pywayland 0.4.19** (Apache-2.0, CFFI) + **cffi escape hatch for `wl_event_loop_get_fd`** | pywayland's `EventLoop` wrapper omits `wl_event_loop_get_fd()`. Fix: call `pywayland.lib.wl_event_loop_get_fd(loop._ptr)` directly via cffi, then `asyncio.get_running_loop().add_reader(wl_fd, dispatch_wl)`. Replaces Greenfield's entire `proxy-poll-addon.node` C addon with ~10 lines of Python |
|
||
| **Multi-process model** | **`multiprocessing.Process` (fork start method) per Odoo session** | Matches Greenfield's per-`compositorSessionId` forking. FD passing of WS upgrade socket via `multiprocessing.reduction.send_handle`/`recv_handle` (uses SCM_RIGHTS on Linux). Preserves the SIGHUP/SIGKILL escalation semantics |
|
||
| **Encoder threading** | **`threading.Thread` + GMainLoop + `loop.call_soon_threadsafe`** | Mirrors Greenfield's `napi_threadsafe_function` pattern. GIL released during C encoding calls (GStreamer). Reuse Greenfield's MIT C files for dmabuf/egl via cffi — preserves all EGL/GBM/GStreamer pipelines |
|
||
| **Binary message format** | **Keep Greenfield's verbatim** | Browser-side TS unchanged; no need for a version field until v2. KCP gives message boundaries |
|
||
| **ReconnectingWebSocket** | **Reuse Greenfield's vendored copy** (Pedro Ladaria, MIT, 514 lines TS) | Browser-side; copy with attribution into Odoo OWL shell assets |
|
||
|
||
## Channel types (same as Greenfield)
|
||
|
||
| Type | Class | KCP? | Purpose |
|
||
|------|-------|------|---------|
|
||
| PROTOCOL | ARQChannel | Yes | Wayland wire protocol, per `wl_client` |
|
||
| FRAME | ARQChannel | Yes | H.264/PNG encoded surface content, per `wl_surface` |
|
||
| XWM | ARQChannel | Yes | XWayland window manager protocol |
|
||
| FEEDBACK | SimpleChannel | No | 4-byte telemetry (refreshInterval + decode avg), 1 Hz |
|
||
|
||
## WebSocket endpoints (9 paths, same as Greenfield)
|
||
|
||
```
|
||
/signal # JSON control plane (per-app)
|
||
/channel # binary data channels (multiplexed)
|
||
/mkfifo # create a POSIX pipe, return 2 ProxyFDs
|
||
/mkstemp-mmap # create a memfd-backed shm, return 1 ProxyFD
|
||
/read-fd # read bytes from a ProxyFD (one-shot)
|
||
/read-fd-as-stream # read bytes from a ProxyFD (streaming)
|
||
/write-fd-as-stream # write bytes to a ProxyFD (streaming)
|
||
/close-fd # close a ProxyFD
|
||
/request-keyframe # ask the encoder for a keyframe
|
||
```
|
||
|
||
All endpoints require JWT auth (HS256, validated by `pyjwt`).
|
||
|
||
## asyncio + libwayland integration (the riskiest piece)
|
||
|
||
**Verified working** in the sandbox:
|
||
|
||
```python
|
||
import asyncio, cffi
|
||
from pywayland.server import Display
|
||
|
||
extras_ffi = cffi.FFI()
|
||
extras_ffi.cdef("""
|
||
int wl_event_loop_get_fd(void *loop);
|
||
void *wl_display_get_event_loop(void *display);
|
||
int wl_event_loop_dispatch(void *loop, int timeout);
|
||
void wl_display_flush_clients(void *display);
|
||
""")
|
||
_lib = extras_ffi.dlopen("libwayland-server.so.0") # or pywayland's bundled
|
||
|
||
display = Display()
|
||
display.add_socket()
|
||
display.init_shm()
|
||
|
||
_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr))
|
||
_display_voidp = extras_ffi.cast('void *', _ptr_int)
|
||
_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp)
|
||
_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp)
|
||
|
||
async def main():
|
||
loop = asyncio.get_running_loop()
|
||
def on_ready():
|
||
_lib.wl_event_loop_dispatch(_loop_voidp, 0)
|
||
display.flush_clients()
|
||
loop.add_reader(_wl_fd, on_ready)
|
||
await asyncio.Future() # run forever
|
||
|
||
asyncio.run(main())
|
||
```
|
||
|
||
This pattern is used in production by **qtile's Wayland backend**
|
||
(`libqtile/backend/wayland/core.py`). qtile is GPL-2.0 so don't copy code,
|
||
but the architectural pattern is documented and reusable.
|
||
|
||
## Process model — three layers (same as Greenfield)
|
||
|
||
### Layer 1: Main process (HTTP + WS hub)
|
||
|
||
Single Python process listens on `:8081`. Per Odoo session, it forks
|
||
one `SessionProcess` child — one child per `compositor_session_id`, **not**
|
||
per app. Multiple apps in the same browser tab share one child.
|
||
|
||
WebSocket upgrades are handed off to the matching child via
|
||
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
|
||
|
||
### Layer 2: Session process
|
||
|
||
The forked child. Receives IPC messages:
|
||
|
||
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
|
||
- `launch_app` — spawns the actual app binary via `subprocess.Popen` + bubblewrap
|
||
- `ws_upgrade` — hands WS to the dispatcher
|
||
|
||
On `session.close()` the process exits.
|
||
|
||
### Layer 3: Per-client session
|
||
|
||
For each new `wl_client` (one per Wayland connection), creates a
|
||
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
|
||
JSON. Routes requests via per-resource dispatchers.
|
||
|
||
## FD virtualization (same approach as Greenfield)
|
||
|
||
`ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }`
|
||
|
||
In Python:
|
||
- `os.pipe()` for pipes
|
||
- `os.memfd_create(name, flags=0)` for shared memory (Python 3.8+ builtin,
|
||
replaces Greenfield's `createMemoryMappedFile` native addon)
|
||
- `asyncio.StreamReader` / `StreamWriter` for streaming reads/writes over
|
||
separate WebSocket endpoints
|
||
|
||
## Reconnection & lifecycle (same as Greenfield)
|
||
|
||
### Browser side
|
||
|
||
Vendored `ReconnectingWebSocket` (MIT):
|
||
- `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s`
|
||
- `minUptime=5s`, `connTimeout=4s`, `retries=∞`
|
||
- Close codes 4000-4999 tear down KCP; lower codes auto-reconnect
|
||
|
||
### Python side
|
||
|
||
No auto-reconnect (it's the server). Channel object lives on, waiting for
|
||
new WS upgrade with matching `id` + `key`. Outbound messages queue and
|
||
flush on reconnect.
|
||
|
||
### App kill timers
|
||
|
||
When browser disconnects from `/signal` WebSocket:
|
||
- 10-minute SIGHUP grace period
|
||
- After 10 min: SIGTERM, then SIGKILL after another 10s
|
||
- Close code 1001 → immediate SIGHUP (intentional tab close)
|
||
- Close code 4001 → immediate SIGTERM (force quit)
|
||
|
||
## JWT auth (NEW vs Greenfield)
|
||
|
||
Every WebSocket connection must include a valid JWT (HS256, 15-min expiry):
|
||
|
||
```
|
||
Authorization: Bearer <jwt>
|
||
```
|
||
|
||
Or as a query param: `?token=<jwt>`
|
||
|
||
JWT payload:
|
||
```json
|
||
{
|
||
"iss": "odoo.example.com",
|
||
"aud": "compositor-proxy",
|
||
"sub": 42, // Odoo user ID
|
||
"db": "production", // Odoo database name
|
||
"company_ids": [1, 3],
|
||
"scope": "compositor",
|
||
"compositor_session_id": "uuid-here",
|
||
"iat": 1736000000,
|
||
"exp": 1736000900
|
||
}
|
||
```
|
||
|
||
Validated by the proxy using `compositor.jwt_secret` from Odoo's
|
||
`ir.config_parameter` (shared via env var at proxy startup).
|
||
|
||
## ASCII diagram
|
||
|
||
```
|
||
PYTHON HOST (Linux) BROWSER (Odoo OWL shell)
|
||
========================== ========================
|
||
|
||
┌─────────────────────────┐ ┌──────────────────────────────┐
|
||
│ Native app (gtk4-demo) │ │ Odoo web client │
|
||
│ (sandboxed in bwrap) │ │ (Odoo 20 OWL shell addon) │
|
||
└───────────┬─────────────┘ └──────────────┬───────────────┘
|
||
│ WAYLAND_DISPLAY=wayland-0 │
|
||
│ Unix socket + SCM_RIGHTS FD passing │ DOM events → input
|
||
▼ │ WebSocket API
|
||
┌──────────────────────────────────────────┐ │
|
||
│ SessionProcess (fork'd Python child) │ │
|
||
│ ┌────────────────────────────────────┐ │ │
|
||
│ │ pywayland (Apache-2.0) │ │ │
|
||
│ │ + 5-line cffi shim │ │ │
|
||
│ │ - wl_display + wl_client │ │ │
|
||
│ │ - asyncio.add_reader for wl_fd │ │ │
|
||
│ └─────────────┬──────────────────────┘ │ │
|
||
│ │ wire msg + FDs │ │
|
||
│ ▼ │ │
|
||
│ ┌─────────────────────────────────────┐ │ │
|
||
│ │ Per-resource dispatchers │ │ │
|
||
│ │ (auto-generated by pywayland) │ │ │
|
||
│ │ - route to {native, browser} │ │ │
|
||
│ │ - serialize FDs as ProxyFD JSON │ │ │
|
||
│ │ - fast-sync optimization │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
│ │ outbound buffer │ │
|
||
│ ▼ │ │
|
||
│ ┌─────────────────────────────────────┐ │ │
|
||
│ │ ARQChannel (KCP-wrapped) │ │ │
|
||
│ │ - kcp PyPI package (MIT) │ │ │
|
||
│ │ - MTU=64000, snd_wnd=128, rcv=1024 │ │ │
|
||
│ │ - kcp.update() every 100ms │ │ │
|
||
│ │ - PROTOCOL (per wl_client) │ │ │
|
||
│ │ - FRAME (per wl_surface) │ │ │
|
||
│ │ - XWM (per XWayland) │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
│ ┌─────────────┴───────────────────────┐ │ │
|
||
│ │ SimpleChannel (FEEDBACK, no KCP) │ │ │
|
||
│ │ 4 bytes: refreshIntervalMs + │ │ │
|
||
│ │ avgDecodeDurationMs │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
│ ┌─────────────▼───────────────────────┐ │ │
|
||
│ │ NativeAppContext │ │ │
|
||
│ │ - signaling WebSocket (JSON) │ │ │
|
||
│ │ - kill timers: SIGHUP@10min, │ │ │
|
||
│ │ SIGKILL@+10s │ │ │
|
||
│ │ - bubblewrap + seccomp sandbox │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
│ ┌─────────────▼───────────────────────┐ │ │
|
||
│ │ SessionController │ │ │
|
||
│ │ - aiohttp.web.Application │ │ │
|
||
│ │ - 9 endpoints │ │ │
|
||
│ │ - JWT validation (pyjwt) │ │ │
|
||
│ │ - perMessageDeflate: false │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
│ ┌─────────────▼───────────────────────┐ │ │
|
||
│ │ Encoder thread (GStreamer + GLib │ │ │
|
||
│ │ MainLoop in separate thread) │ │ │
|
||
│ │ - vaapih264enc / nvh264enc / │ │ │
|
||
│ │ openh264enc (NOT x264enc — GPL) │ │ │
|
||
│ │ - asyncio.run_coroutine_threadsafe │ │ │
|
||
│ │ for cross-thread marshalling │ │ │
|
||
│ └─────────────┬───────────────────────┘ │ │
|
||
└────────────────┼─────────────────────────────┘ │
|
||
│ │
|
||
│ ┌──────────────────────────┐ │
|
||
│ │ main.py (HTTP+WS hub) │ │
|
||
│ │ - socket.pause() + IPC │ │
|
||
│ │ send to session proc │ │
|
||
│ │ - multiprocessing.fork │ │
|
||
│ │ - JWT issuance delegated │ │
|
||
│ │ to Odoo backend │ │
|
||
│ └────────────┬─────────────┘ │
|
||
│ │ │
|
||
│ ◄──── 1 WebSocket per channel ────► │
|
||
│ ◄── 1 signaling WS per app ──► │
|
||
│ ◄── ad-hoc WebFD streams ─► │
|
||
│ │ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────┐ │
|
||
│ │ ReconnectingWebSocket (vendored MIT) │ │
|
||
│ │ min 1-5s, factor 1.3, max 10s │ │
|
||
│ │ minUptime 5s, connTimeout 4s │ │
|
||
│ │ + Authorization: Bearer <jwt> │ │
|
||
│ └────────────┬───────────────────────────┘ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ ARQChannel / SimpleChannel (clean-room TS) │ │
|
||
│ │ snd_wnd=1024, rcv_wnd=128 │ │
|
||
│ │ kcp.input → peekSize/recv → onMessage │ │
|
||
│ └────────────┬───────────────────────────────┘ │
|
||
│ ▼ │
|
||
│ ┌────────────────────────────────────────────┐ │
|
||
│ │ Custom Browser Compositor (clean-room TS) │ │
|
||
│ │ - WebCodecs VideoDecoder (HW-accelerated)│ │
|
||
│ │ - WebGL2 + OffscreenCanvas in Worker │ │
|
||
│ │ - Surface/View/Output/Seat/Keyboard/... │ │
|
||
│ └────────────┬───────────────────────────────┘ │
|
||
│ │ input events flow back via the │
|
||
│ │ PROTOCOL channel as ordinary │
|
||
│ │ Wayland requests (same framing) │
|
||
└──────────────────┘ │
|
||
Browser DOM ◄─┘
|
||
```
|
||
|
||
## Effort estimate (Python transport)
|
||
|
||
| Component | LoC | Dev-days | Risk |
|
||
|---|---|---|---|
|
||
| aiohttp WS server (9 endpoints + HTTP launcher) | 600 | 3 | Low |
|
||
| `Channel`/`ARQChannel`/`SimpleChannel` port | 250 | 1.5 | Low |
|
||
| `kcp.py` integration adapter | 100 | 0.5 | Low |
|
||
| `NativeAppContext` port (signaling, escalation) | 350 | 2 | Low |
|
||
| `ProxyInputOutput` port (FD workaround) | 300 | 2 | Medium |
|
||
| libwayland event-loop integration | 80 | 1 | **Medium-High** (under-documented) |
|
||
| Multiprocessing scaffolding + FD passing | 200 | 1.5 | Medium |
|
||
| GStreamer encoder thread + marshalling | 250 | 2 | Medium |
|
||
| Reconnection & lifecycle | 150 | 1 | Low |
|
||
| Tests (pytest + pytest-asyncio) | 600 | 3 | Low |
|
||
| **Total** | **2880** | **~17** | |
|
||
|
||
## Top risks & mitigations
|
||
|
||
1. **Highest risk**: libwayland ↔ asyncio event-loop integration (under-documented).
|
||
**Mitigation**: Spend 1 day on a standalone prototype (the 10-line script
|
||
above) before committing.
|
||
2. **Medium risk**: FD passing via `multiprocessing.reduction`. Fallback:
|
||
single-process with asyncio concurrency.
|
||
3. **Medium risk**: GStreamer threading. Mitigation: reuse Greenfield's
|
||
MIT C files via cffi — only the marshalling needs Python.
|
||
|
||
## Dependency manifest
|
||
|
||
```
|
||
# requirements.txt
|
||
aiohttp>=3.10,<4 # WebSocket server + HTTP
|
||
kcp>=0.1.6 # Cython KCP bindings (RealistikDash)
|
||
pywayland>=0.4.19 # libwayland CFFI wrapper (Apache-2.0)
|
||
cffi>=1.16 # for wl_event_loop_get_fd escape hatch
|
||
pygobject>=3.48 # GStreamer Python bindings
|
||
pyjwt>=2.8 # JWT validation
|
||
```
|
||
|
||
System packages (Debian):
|
||
```
|
||
libwayland-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
|
||
libgstreamer-plugins-bad1.0-dev libgbm-dev libdrm-dev libegl-dev
|
||
gstreamer1.0-vaapi # Intel/AMD hardware encoding
|
||
libnvidia-encode1 # NVIDIA hardware encoding (if applicable)
|
||
bubblewrap libseccomp-dev
|
||
python3-dev build-essential pkg-config
|
||
```
|
||
|
||
**DO NOT INSTALL**: `gstreamer1.0-plugins-ugly` (provides x264enc, GPL) or
|
||
`gstreamer1.0-libav` (LGPL source but GPL binary on Debian).
|