1b-agent/one-os-dev-wayland-html5-compositor-greenfield-analysis
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c4b45cf2180d382e2d47e6665ea28c19bb891f75
one-os-dev-wayland-html5-co…/12-appendix.md
T

23 KiB
Raw Blame History

12 — Appendix: Glossary, File Index, References

Glossary of Greenfield terms

Wayland concepts

Term Meaning
wl_display The Wayland display connection — top-level object for both clients and the compositor.
wl_registry Per-client registry of available globals (compositor, seat, shm, output, etc.).
wl_compositor Global that creates wl_surface and wl_region objects.
wl_surface A 2D rectangular pixel buffer with state stack — the basic drawable.
wl_seat Represents a set of input devices (keyboard, pointer, touch).
wl_keyboard Per-seat keyboard interface — sends keymap, key events, modifiers.
wl_pointer Per-seat pointer interface — sends enter/leave/motion/button/axis events.
wl_touch Per-seat touch interface (declared but inert in Greenfield).
wl_output A monitor/screen — surfaces are positioned relative to outputs.
wl_shm Shared memory buffer mechanism — client provides raw pixel buffer.
wl_shm_pool A pool of shared memory that can be sliced into multiple wl_buffers.
wl_buffer A pixel buffer that can be attached to a wl_surface.
wl_callback One-shot frame callback — fired when the compositor presents a frame.
wl_data_device_manager Global for clipboard / DnD.
wl_data_device Per-seat-per-client data device (selection, DnD).
wl_data_source The provider side of a selection (the app that has data to offer).
wl_data_offer The receiver side of a selection (the app that wants to receive data).
wl_drm Legacy protocol for DMA-BUF buffers (used by Mesa, XWayland).
wl_subcompositor Global for subsurfaces (nested surfaces).
wl_subsurface A surface nested inside another surface (sync or desync).
wl_region A region (set of rectangles) used for input/opaque/damage.

xdg-shell concepts

Term Meaning
xdg_wm_base The xdg-shell global — clients use it to register as desktop apps.
xdg_surface A wrapper around wl_surface for xdg-shell semantics.
xdg_toplevel A top-level window (with title, app_id, state, parent).
xdg_popup A transient popup window (menus, tooltips, dropdowns).
xdg_positioner A specification for how to position a popup relative to its parent.
configure Compositor → client event telling the client what state/size to apply.
ack_configure Client → compositor request acknowledging a configure with a serial.

Greenfield-specific terms

Term Meaning
Session Top-level browser-side compositor session. Created by createCompositorSession().
Channel Transport abstraction. ARQChannel (KCP-wrapped) for reliable channels, SimpleChannel for raw WebSocket.
ARQChannel Automatic Repeat reQuest channel — KCP-over-WebSocket, reliable, low-latency.
SimpleChannel Raw WebSocket channel, no KCP. Used for FEEDBACK (4-byte messages).
Globals Registry of Wayland globals on the compositor side (Globals.ts).
Seat Browser-side wl_seat implementation. Aggregates keyboard/pointer/touch + focus management.
Surface Browser-side wl_surface implementation. Double-buffered state (state vs pendingState).
View A surface's position on an output, used for compositing. One view per surface per output.
Output Browser-side wl_output implementation. Wraps an HTML <canvas> with WebGL context.
RenderState Per-view-per-scene texture + transform state.
Scene One per output/canvas. Manages the view stack and WebGL textures.
BufferContents Abstraction over a buffer's content (mimeType + payload).
BufferImplementation Interface for buffer types: SHM, ImageBitmap, StreamingBuffer.
StreamingBuffer Per-surface buffer that receives encoded frames over the FRAME channel.
BufferStream Manages serial → frame promise resolution.
EncodedFrame Wire format parser for encoded frames (H.264 / PNG).
EncoderFeedback 4-byte feedback messages sent from browser to proxy at 1 Hz (refreshInterval + decode duration).
FrameFeedback Proxy-side receiver of EncoderFeedback messages. Used to pace wl_frame.done callbacks.
ProxyFD JSON descriptor for an FD over WebSocket: {handle, type, host}.
webFS Browser-side resolver for ProxyFD references.
InputOutput FD abstraction (browser pipes for in-browser apps, WebSocket endpoints for native apps).
RemoteInputOutput Native-app FD operations via WebSocket endpoints.
webInputOutput In-browser-app FD operations via MessageChannel ports.
UserShellApi The integration seam — {events, actions} interface between compositor and shell.
CompositorClient Opaque handle for a Wayland client (browser-side).
CompositorSurface Opaque handle for a Wayland surface (browser-side).
DesktopSurface Floating or AlwaysFullScreen mode wrapper. Emits userShell.events.*.
FloatingDesktopSurface Default floating window mode.
AlwaysFullScreenDesktopSurface Kiosk mode — one window per output.
XdgWmBase xdg_wm_base global, with ping-based unresponsiveness detection.
XdgSurface xdg_surface role wrapper.
XdgToplevel xdg_toplevel role — top-level windows.
XdgPopup xdg_popup role — transient windows.
XdgPositioner xdg_positioner — popup positioning math.
web-bitmapbuf-unstable-v1 Greenfield's custom Wayland protocol extension for browser JS apps.
wayland-server-addon C N-API addon wrapping patched libwayland-server.
proxy-encoding-addon C N-API addon wrapping GStreamer H.264/PNG encoder.
proxy-poll-addon C N-API addon wrapping libuv uv_poll_t for wl_display FD polling.
xtsb X11 TypeScript Bindings — pure-TS X11 client (no libxcb dependency).
westfield-* Greenfield's C helpers: westfield-buffer.c, westfield-dmabuf.c, westfield-egl.c, westfield-xwayland.c.
wlr_drm wl_drm global implementation (in wlr_drm.c).
wlr_linux_dmabuf_v1 zwp_linux_dmabuf_v1 global implementation.

Encoding / rendering terms

Term Meaning
dmabuf Linux DMA-BUF — a buffer that can be zero-copied between processes and GPUs.
EGLImage EGL wrapper around a dmabuf, used by GStreamer GL.
GstGLMemoryEGL GStreamer GL memory backed by an EGLImage — zero-copy GPU buffer.
I420 YUV 4:2:0 planar format (Y, U, V as separate planes).
NV12 YUV 4:2:0 semi-planar format (Y plane + interleaved UV plane).
BT.601 Color space conversion standard used by Greenfield's YUV→RGB shaders.
NAL H.264 Network Abstraction Layer unit — a single encoded video frame or part thereof.
AU H.264 Access Unit — a complete frame.
keyframe H.264 I-frame — independently decodable, used to start a stream.
WebCodecs Browser API for native video decode/encode (VideoDecoder, VideoEncoder).
WebTransport Future browser API for UDP-like transport over QUIC. Greenfield plans to use it.
KCP Reliable UDP protocol implemented in user-space. Greenfield runs it over WebSocket.

File-to-feature index

Browser side (packages/compositor/src/)

File Implements / Role
index.ts Public API: createCompositorSession(), createAppLauncher()
Session.ts Top-level browser session, decoder setup
Compositor.ts wl_compositor global
Globals.ts Wayland globals registry
Surface.ts wl_surface impl, double-buffered state, commit semantics
SurfaceChild.ts Surface tree child abstraction
SurfaceRole.ts SurfaceRole interface
Subsurface.ts wl_subsurface role, sync/desync mode
Subcompositor.ts wl_subcompositor global
View.ts Positioned surface for compositing
Output.ts wl_output, WebGL canvas binding
Shm.ts wl_shm / shm pool, BMP-header hack
BufferContents.ts Buffer abstraction (with mimeType)
BufferImplementation.ts Buffer implementation interface
ImageBitmapBuffer.ts ImageBitmap-backed buffer
Region.ts wl_region, pixman-WASM damage regions
Callback.ts Frame callback (wl_callback)
Loop.ts Compositor main loop, RAF integration
Seat.ts wl_seat global, multi-seat support, focus management
Keyboard.ts wl_keyboard impl, keymap, modifiers, key events
Pointer.ts wl_pointer impl, motion, buttons, axis
Touch.ts wl_touch (46-line stub, inert)
Xkb.ts xkbcommon WASM wrapper
KeyEvent.ts Key event wrapper
ButtonEvent.ts Pointer button event wrapper
AxisEvent.ts Pointer axis (scroll) event wrapper
InputQueue.ts Event coalescing + RAF dispatch
InputOutput.ts FD abstraction (browser pipes / WebSocket endpoints)
DataDeviceManager.ts wl_data_device_manager global
DataSource.ts wl_data_source impl, the provider side
DataOffer.ts wl_data_offer impl, the receiver side
WaylandDataSource.ts Wrapper around native wl_data_source events
Shell.ts wl_shell global (legacy)
ShellSurface.ts wl_shell_surface role
XdgWmBase.ts xdg_wm_base global, ping-based unresponsiveness detection
XdgSurface.ts xdg_surface role, configure/ack round-trip
XdgToplevel.ts xdg_toplevel role, window state
XdgPopup.ts xdg_popup role, transient windows, grabs
XdgPositioner.ts xdg_positioner, popup positioning math
UserShellApi.ts The integration seam — events + actions
render/Renderer.ts 4-phase render pipeline
render/Scene.ts One per output, manages view stack + textures
render/SceneShader.ts Final pass: textured quads to canvas
render/Texture.ts WebGL texture wrapper
render/RenderState.ts Per-view-per-scene texture + transform
render/Program.ts WebGL program wrapper
render/ShaderCompiler.ts Shader compile/link helper
render/ShaderSources.ts GLSL source strings
render/YUVA2RGBA.ts YUVA→RGBA FBO pass
render/YUVA2RGBAShader.ts BT.601 YUVA→RGBA shader
render/YUV2RGBShader.ts YUV→RGB shader (no alpha)
render/RGBXandA2RGBA.ts RGB+Alpha → RGBA FBO pass (WebCodecs output)
render/RGBandA2RGBAShader.ts Shader for above
remote/Channel.ts ARQChannel + SimpleChannel
remote/kcp.ts ~1100-line KCP implementation
remote/RemoteAppLauncher.ts Channel setup per app, frame delivery
remote/RemoteOutOfBandChannel.ts OOB opcode dispatch
remote/RemoteInputOutput.ts FD-over-WebSocket for native apps
remote/EncoderFeedback.ts Sends 4-byte decode feedback at 1 Hz
remote/StreamingBuffer.ts Per-surface encoded-frame buffer
remote/BufferStream.ts Manages serial → frame promise resolution
remote/EncodedFrame.ts Wire format parser for encoded frames
remote/buffer-decoder.ts Decoder interface
remote/webcodec-buffer-decoder.ts WebCodecs VideoDecoder path
remote/wasm-buffer-decoder.ts FFmpeg Wasm fallback path
remote/H264NALDecoder.worker.ts Worker entry, runs FFmpeg Wasm decoder
remote/XWindowManager.ts X11 CLIPBOARD atom watcher (XWayland)
remote/XDataSource.ts X11 clipboard source (XWayland apps)
remote/XDnDDataSource.ts X11 DnD source
remote/api.ts Public exports
browser/input.ts DOM keyboard → wl_keyboard events
browser/pointer.ts DOM pointer → wl_pointer events
browser/dnd.ts DnD bridge
browser/selection.ts Clipboard bridge
browser/text-selection.ts Firefox fallback (text-only)
browser/capabilities.ts Browser feature detection
desktop/Desktop.ts Floating vs AlwaysFullScreen modes
math/Mat4.ts 4×4 matrix math
math/Rect.ts Rectangle math
web/WebBuffersFactory.ts Browser-side impl of web_bitmapbuf_factory
web/WebInputOutput.ts In-browser-app FD operations
web/WebConnectionHandler.ts In-browser-app connection handling
polyfill/createImageBitmap.js ImageBitmap polyfill

Native side

File Implements / Role
packages/compositor-proxy/src/index.ts Exports createSession(), launchApplication()
packages/compositor-proxy/src/Session.ts Top-level proxy session
packages/compositor-proxy/src/SessionController.ts WS server with 9 endpoint paths
packages/compositor-proxy/src/NativeAppContext.ts Per-app state, signaling WS, kill timers
packages/compositor-proxy/src/NativeWaylandClientSession.ts Per-client message interceptor
packages/compositor-proxy/src/NativeWaylandCompositorSession.ts wl_display creation, initShm, initDrm
packages/compositor-proxy/src/Channel.ts ARQChannel + SimpleChannel
packages/compositor-proxy/src/kcp.ts ~1100-line KCP implementation
packages/compositor-proxy/src/FrameFeedback.ts Receives 4-byte decode feedback, paces wl_frame.done
packages/compositor-proxy/src/ProxyBuffer.ts Proxy-side buffer representation
packages/compositor-proxy/src/SurfaceBufferEncoding.ts Encoding pipeline orchestration
packages/compositor-proxy/src/XWaylandSession.ts XWayland spawn + bridge
packages/compositor-proxy/src/wayland-server.ts TS wrapper around wayland-server-addon.node
packages/compositor-proxy/src/io/ WebSocket server glue
packages/compositor-proxy/src/encoding/ High-level encoding orchestration
packages/compositor-proxy/src/protocol/ Generated interceptor modules
packages/compositor-proxy/src/addons/ .node file shims (Linux-x64)
packages/compositor-proxy/src/config.ts Encoding-related configuration
packages/compositor-proxy/native/CMakeLists.txt Build config
packages/compositor-proxy/native/wayland/src/addon.c Node-API addon: wayland-server-addon.node
packages/compositor-proxy/native/wayland/src/westfield-buffer.c Buffer helpers
packages/compositor-proxy/native/wayland/src/westfield-dmabuf.c DMA-BUF import via EGL/GBM
packages/compositor-proxy/native/wayland/src/westfield-egl.c EGL zero-copy setup
packages/compositor-proxy/native/wayland/src/westfield-xwayland.c XWayland spawn + bridge
packages/compositor-proxy/native/wayland/src/wlr_drm.c wl_drm global impl
packages/compositor-proxy/native/wayland/src/wlr_linux_dmabuf_v1.c zwp_linux_dmabuf_v1 global impl
packages/compositor-proxy/native/wayland/src/drm-protocol.c DRM protocol helpers
packages/compositor-proxy/native/wayland/src/linux-dmabuf-unstable-v1-protocol.c linux-dmabuf-v1 protocol stubs
packages/compositor-proxy/native/wayland/src/wayland-server/ Vendored + patched libwayland-server
packages/compositor-proxy/native/encoding/src/gst_frame_encoder.c GStreamer H.264/PNG encoder
packages/compositor-proxy/native/encoding/src/gst_main_loop.c Dedicated GLib main loop for encoding
packages/compositor-proxy/native/encoding/src/node_encoder.c Node-API addon: proxy-encoding-addon.node
packages/compositor-proxy/native/poll/src/poll.c libuv uv_poll_t wrapper
packages/compositor-proxy/native/node-api/ Vendored Node-API headers
packages/compositor-proxy-cli/src/main.ts HTTP + WS hub, forks per session
packages/compositor-proxy-cli/src/main-controller.ts CLI arg parsing
packages/compositor-proxy-cli/src/main-args.ts CLI arg definitions
packages/compositor-proxy-cli/src/SessionProcess.ts Per-session child, IPC handler
packages/compositor-proxy-cli/example-applications.json Pre-configured apps

Libs

File Role
libs/common/src/Connection.ts Shared wire format codec
libs/common/src/index.ts Public exports
libs/client-generator/{ProtocolParser,ProtocolArguments,index}.mjs Browser client bindings generator
libs/compositor-generator/{ProtocolParser,ProtocolArguments,index}.mjs Browser compositor bindings generator
libs/compositor-proxy-generator/src/{EndpointProtocolParser,EndpointProtocolArguments,index}.mjs Proxy interceptor generator
libs/client-protocol/src/westfield-runtime-client.ts Client-side runtime
libs/client-protocol/src/protocol/ Generated client bindings
libs/compositor-protocol/src/westfield-runtime-server.ts Server-side runtime
libs/compositor-protocol/src/protocol/ Generated server bindings
libs/compositor-wasm/build_wasm.sh pixman + xkbcommon WASM build
libs/compositor-wasm/xkbcommon/build.sh xkbcommon 1.5.0 build
libs/compositor-wasm/pixman/build.sh pixman 0.42.2 build
libs/compositor-wasm/src/index.ts TS wrapper, init function
libs/compositor-ffmpeg-h264/build_wasm.sh FFmpeg n5.1.2 → WASM build
libs/compositor-ffmpeg-h264/native/decoder.c 30-line wrapper exposing decode API
libs/compositor-ffmpeg-h264/src/H264Decoder.ts TS wrapper around the Wasm module
libs/compositor-ffmpeg-h264/src/H264Worker.ts Worker entry point
libs/xtsb/protocol/xproto.xml X11 core protocol XML (5800+ lines)
libs/xtsb/protocol/shape.xml Shape extension XML
libs/xtsb/protocol/xfixes.xml XFIXES extension XML
libs/xtsb/protocol/render.xml Render extension XML
libs/xtsb/protocol/composite.xml Composite extension XML
libs/xtsb/src/ Generated TS X11 bindings

Protocol XML

File Role
protocol/wayland.xml Standard upstream Wayland core protocol
protocol/xdg-shell.xml Stable xdg-shell protocol
protocol/wayland-drm.xml Legacy wl_drm for DMA-BUF (proxy-only)
protocol/web-bitmapbuf-unstable-v1.xml Greenfield's custom extension for browser JS apps

External references

Wayland protocol

libwayland

xkbcommon

GStreamer

FFmpeg Wasm

WebCodecs

WebTransport

KCP

Emscripten

X11 / XCB

XWayland

Greenfield project

Odoo 20

Pointers to the user's next steps

  1. Read chapters in order: 00 → 01 → 02 → 07 → 08 → 10 → 11. This gives you the architecture map, the protocol layer, the native app bridge (your primary use case), the UserShellApi integration seam, the porting blueprint, and the pitfalls to avoid.

  2. Start with Phase 1 of the porting blueprint (10-porting-to-odoo-js-python.md): Run Greenfield's compositor-proxy-cli as a Docker container, build a minimal Odoo 20 OWL addon that creates a <canvas>, and verify a single Linux app displays.

  3. Decide on the AGPL question early: Will you accept AGPL on the browser compositor + native proxy (and release your modifications), or will you reimplement them from scratch (and license them however you want)?

  4. Choose your Python stack: aiohttp vs FastAPI for the WebSocket server. pywayland for the Wayland bindings. python-xkbcommon for keymaps. gi.repository.Gst for GStreamer encoding. bubblewrap + seccomp for sandboxing.

  5. Plan for sandboxing from day one: Greenfield has none. If you skip this, your Odoo users can run arbitrary Linux code on your server. That's a security hole you can't ship.

  6. Build the OWL shell with UserShellApi.events callbacks: This is the clean integration seam. Don't try to import Greenfield internals into your Odoo shell — use the public UserShellApi interface.

  7. Budget for the Python port: ~4-8 weeks for a working Python proxy. ~8-16 weeks for production quality (sandboxing, multi-tenant, adaptive bitrate, clipboard sync, touch input, primary selection).

  8. Track upstream Greenfield commits: Pin to a specific commit hash if you fork it. Watch for 1.0 final (may break APIs).

Good luck with your Wayland-to-HTML compositor build!

Reference in new issue View Git Blame Copy Permalink