Initial commit: MIT-licensed compositor architecture & build plan (14 files, ~80 pages, 10 parallel research agents)
This commit is contained in:
commit
582e61e942
15 files changed
+4970
No files matched your search
@@ -0,0 +1,276 @@
|
||||
# 06 — Data Device, Selection, DnD (`wl_data_device`)
|
||||
|
||||
This chapter covers the data device / clipboard / drag-and-drop layer in
|
||||
the MIT approach. Same architecture as Greenfield (clean-room), but with
|
||||
**primary selection** added (which Greenfield lacks).
|
||||
|
||||
## Faithful Wayland model + primary selection (NEW)
|
||||
|
||||
`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per
|
||||
seat-per-client and the implementation lives directly on `Seat`.
|
||||
`WaylandDataSource` / `DataOffer` are the standard source/offer pair.
|
||||
|
||||
**NEW**: Implement `zwp_primary_selection_device_manager_v1` for middle-click
|
||||
paste, which Greenfield lacks entirely (grep returns zero hits).
|
||||
|
||||
## FD-over-WebSocket workaround (REUSED approach)
|
||||
|
||||
Same `ProxyFD` JSON descriptor as Greenfield:
|
||||
```
|
||||
ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }
|
||||
```
|
||||
|
||||
Three concrete `InputOutput` / `InputOutputFD` implementations:
|
||||
|
||||
### `webInputOutput` — for in-browser web apps
|
||||
|
||||
Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s.
|
||||
|
||||
### `RemoteInputOutput` — for native apps
|
||||
|
||||
All fd operations go over a secondary WebSocket to endpoints: `/mkfifo`,
|
||||
`/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`, `/read-fd`,
|
||||
`/close-fd`.
|
||||
|
||||
### `ProxyInputOutput` (Python side) — owns the real OS pipes and memfds
|
||||
|
||||
In Python:
|
||||
- `os.pipe()` for pipes
|
||||
- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
|
||||
`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD
|
||||
has `host === proxy.baseURL`.
|
||||
|
||||
**Crucial consequence**: For native→native clipboard transfers (user
|
||||
copies in one Linux app, pastes in another), the browser compositor never
|
||||
sees a single byte of the data — it only relays the fd reference. The
|
||||
actual content stays inside the proxy host's OS kernel pipe.
|
||||
|
||||
## MIME-type negotiation (IMPROVED over Greenfield)
|
||||
|
||||
Same hard-coded allow-list initially: `['text/plain', 'text/html',
|
||||
'image/png']`. **IMPROVEMENT**: use exact prefix matching, not substring
|
||||
matching (Greenfield's substring match can collide —
|
||||
`text/plain;charset=iso-8859-1` silently becomes `text/plain`).
|
||||
|
||||
### Native → browser
|
||||
|
||||
`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is
|
||||
matched against the allow-list (exact prefix match), survivors pulled as
|
||||
separate blobs via `mkfifo + send + readBlob`, then
|
||||
`navigator.clipboard.write([new ClipboardItem(...)])`.
|
||||
|
||||
### Browser → native
|
||||
|
||||
`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types,
|
||||
plus a synthetic `text/plain;charset=utf-8` alias appended (Linux apps
|
||||
expect the charset form). On `send`, the alias is normalised back to
|
||||
`text/plain` for the lookup.
|
||||
|
||||
## Clipboard mirroring (REUSED approach)
|
||||
|
||||
The browser has no `clipboardchange` event (Chromium issue 933608 cited in
|
||||
Greenfield's code). MIT approach polls `navigator.clipboard.read()`:
|
||||
|
||||
- on init
|
||||
- on every `window.focus` event
|
||||
|
||||
De-duplicates offers by mime-type set + total blob size, then calls
|
||||
`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`.
|
||||
|
||||
The feedback loop is broken (same as Greenfield) because
|
||||
`handleWaylandDataSourceUpdate` returns early when `dataSource instanceof
|
||||
BrowserDataSource`. Watch for the `clipboardchange` event in modern
|
||||
browsers (currently behind flags in Chrome) and switch to event-driven
|
||||
when available.
|
||||
|
||||
### Firefox fallback (REUSED approach)
|
||||
|
||||
`text-selection.ts` uses only `readText`/`writeText` and a
|
||||
`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images
|
||||
or HTML.
|
||||
|
||||
## Drag-and-drop (REUSED approach + NEW)
|
||||
|
||||
### Wayland-side (Seat.startDrag)
|
||||
|
||||
1. Validates the implicit pointer grab.
|
||||
2. Attaches `DragIconRole` to the optional icon surface.
|
||||
3. Installs a `PointerDrag` grab on both pointer and keyboard.
|
||||
|
||||
`PointerDrag` tracks the focused data device, sends
|
||||
`wl_data_device.enter`/`leave`/`motion`/`drop`, creates fresh
|
||||
`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation.
|
||||
|
||||
### Browser-side (`BrowserDnD` class)
|
||||
|
||||
Creates a hidden full-canvas `dragHelper` div with `draggable=true` when
|
||||
the Wayland drag-icon surface commits, so the browser shows the Wayland
|
||||
icon via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the
|
||||
user drags, `dragover` events on the dragHelper are converted back to
|
||||
`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`,
|
||||
keeping `PointerDrag` fed.
|
||||
|
||||
**NEW: Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD`
|
||||
so a Wayland drag can deliver bytes to a real DOM drop target. Greenfield
|
||||
has a FIXME admitting this isn't implemented.
|
||||
|
||||
## XWayland bridge (REUSED approach, IF XWayland enabled)
|
||||
|
||||
`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and
|
||||
builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the
|
||||
receiver pulls data:
|
||||
|
||||
1. `XDataSource.send` calls `XConvertSelection` to have the X server push
|
||||
the bytes into a property on `selectionWindow`.
|
||||
2. `writeProperty` copies them into the receiver's `InputOutputFD`.
|
||||
3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`).
|
||||
|
||||
Uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings.
|
||||
|
||||
## Primary selection (NEW — Greenfield lacks this)
|
||||
|
||||
Implement `zwp_primary_selection_device_manager_v1` (or the older
|
||||
`gtk_primary_selection`) on the compositor side. Forward `PRIMARY`
|
||||
notifications in `XWindowManager`. This unlocks middle-click paste from
|
||||
native apps.
|
||||
|
||||
## ASCII diagram — clipboard flow in both directions
|
||||
|
||||
```
|
||||
CLIPBOARD / SELECTION FLOW
|
||||
|
||||
=================================================================
|
||||
A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app)
|
||||
=================================================================
|
||||
|
||||
┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐
|
||||
│ Native app │ ─────────────────────────────────────────────► │ compositor │
|
||||
│ (sandboxed │ │ (browser, in DOM) │
|
||||
│ in bwrap) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │
|
||||
└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘
|
||||
│ selectionListeners fire
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ handleWaylandDataSource │
|
||||
│ Update() (selection.ts)│
|
||||
└──────────┬──────────────┘
|
||||
mkfifo() ──────────────────────────► │ Python /mkfifo
|
||||
ProxyFD read+write (os.pipe) ◄───── returns 2 ProxyFDs
|
||||
▼
|
||||
dataSource.send("text/plain", writeFD)
|
||||
│
|
||||
▼
|
||||
wl_data_source.send("text/plain", writeFDProxyFD)
|
||||
─────────────────────────────────────────► │ Native source
|
||||
│ receives real write fd
|
||||
│ (proxyFDtoNativeFD
|
||||
│ is a passthrough)
|
||||
▼
|
||||
Native app writes bytes
|
||||
──────────────► OS pipe ───┐
|
||||
│
|
||||
readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘
|
||||
▼ (proxy streams over WS)
|
||||
Blob {type:"text/plain"}
|
||||
▼
|
||||
navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
|
||||
▼
|
||||
┌────────────────────────────┐
|
||||
│ DOM clipboard populated │
|
||||
│ (Ctrl+V anywhere now works) │
|
||||
└────────────────────────────┘
|
||||
|
||||
|
||||
=================================================================
|
||||
B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM)
|
||||
=================================================================
|
||||
|
||||
┌─────────────────────────────┐
|
||||
│ User copies in DOM │
|
||||
│ (browser tab, textarea...) │
|
||||
└──────────────┬──────────────┘
|
||||
│ on window 'focus' event (or initial load)
|
||||
▼
|
||||
navigator.clipboard.read() ──► ClipboardItems[]
|
||||
▼
|
||||
handleBrowserDataSourceUpdate() — de-dup vs previous offers
|
||||
▼
|
||||
createBrowserDataSource(offers)
|
||||
.mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
|
||||
▼
|
||||
seat.setSelectionInternal(browserDataSource, serial)
|
||||
│
|
||||
│ 1) selectionListeners fire — handleWaylandDataSourceUpdate
|
||||
│ returns early (instanceof BrowserDataSource), breaking loop
|
||||
│
|
||||
│ 2) sendSelection(focusedClient)
|
||||
▼
|
||||
┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐
|
||||
│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │
|
||||
│ (browser, in DOM) │ │ (focused) │
|
||||
└───────────┬────────────┘ └──────┬───────┘
|
||||
│ │
|
||||
│ When the native app calls wl_data_offer.receive("text/plain", fd):
|
||||
│ │
|
||||
│ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
|
||||
▼
|
||||
BrowserDataSource.send(mimeType, ioFD)
|
||||
│
|
||||
▼
|
||||
matchingOffer.getType(mimeType) ─► Promise<Blob> (from ClipboardItem)
|
||||
│
|
||||
▼
|
||||
ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle> ─► Python proxy
|
||||
│
|
||||
▼
|
||||
asyncio.StreamWriter ──► native write fd
|
||||
│
|
||||
▼
|
||||
Bytes flow through OS pipe into native app
|
||||
│
|
||||
▼
|
||||
ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0
|
||||
|
||||
|
||||
=================================================================
|
||||
C) PRIMARY SELECTION (NEW — middle-click paste) ──► same flow but via
|
||||
zwp_primary_selection_device_manager_v1 instead of wl_data_device
|
||||
=================================================================
|
||||
|
||||
Native app selects text ──► zwp_primary_selection_source.set_selection
|
||||
│
|
||||
▼
|
||||
Browser receives primary selection offer ──► stores in seat.primarySelectionDataSource
|
||||
│
|
||||
▼
|
||||
User middle-clicks on a surface ──► seat.notifyPrimarySelectionRequest
|
||||
│
|
||||
▼
|
||||
primarySelectionSource.send(mimeType, writeFD) ──► same FD-over-WS flow as A/B
|
||||
```
|
||||
|
||||
## File reference
|
||||
|
||||
### Custom (clean-room TS)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` global |
|
||||
| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` impl |
|
||||
| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` impl |
|
||||
| `packages/compositor/src/data/WaylandDataSource.ts` | Native source wrapper |
|
||||
| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** |
|
||||
| `packages/compositor/src/data/PrimarySelectionSource.ts` | **NEW: middle-click paste source** |
|
||||
| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge |
|
||||
| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback |
|
||||
| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD implemented) |
|
||||
| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
||||
|
||||
### Python side
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| `os.pipe()` | Create pipes (replaces Greenfield's `makePipe` native addon) |
|
||||
| `os.memfd_create()` | Create shared memory (replaces Greenfield's `createMemoryMappedFile`) |
|
||||
| `asyncio.StreamReader` / `StreamWriter` | Streaming reads/writes over WebSocket endpoints |
|
||||
Reference in new issue
Block a user