277 lines
14 KiB
Markdown
277 lines
14 KiB
Markdown
# 06 — Data Device, Selection, DnD (`wl_data_device`)
|
|
|
|
This chapter covers the data device / clipboard / drag-and-drop layer in
|
|
the MIT approach. Same architecture as Greenfield (clean-room), but with
|
|
**primary selection** added (which Greenfield lacks).
|
|
|
|
## Faithful Wayland model + primary selection (NEW)
|
|
|
|
`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per
|
|
seat-per-client and the implementation lives directly on `Seat`.
|
|
`WaylandDataSource` / `DataOffer` are the standard source/offer pair.
|
|
|
|
**NEW**: Implement `zwp_primary_selection_device_manager_v1` for middle-click
|
|
paste, which Greenfield lacks entirely (grep returns zero hits).
|
|
|
|
## FD-over-WebSocket workaround (REUSED approach)
|
|
|
|
Same `ProxyFD` JSON descriptor as Greenfield:
|
|
```
|
|
ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }
|
|
```
|
|
|
|
Three concrete `InputOutput` / `InputOutputFD` implementations:
|
|
|
|
### `webInputOutput` — for in-browser web apps
|
|
|
|
Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s.
|
|
|
|
### `RemoteInputOutput` — for native apps
|
|
|
|
All fd operations go over a secondary WebSocket to endpoints: `/mkfifo`,
|
|
`/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`, `/read-fd`,
|
|
`/close-fd`.
|
|
|
|
### `ProxyInputOutput` (Python side) — owns the real OS pipes and memfds
|
|
|
|
In Python:
|
|
- `os.pipe()` for pipes
|
|
- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon)
|
|
|
|
`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD
|
|
has `host === proxy.baseURL`.
|
|
|
|
**Crucial consequence**: For native→native clipboard transfers (user
|
|
copies in one Linux app, pastes in another), the browser compositor never
|
|
sees a single byte of the data — it only relays the fd reference. The
|
|
actual content stays inside the proxy host's OS kernel pipe.
|
|
|
|
## MIME-type negotiation (IMPROVED over Greenfield)
|
|
|
|
Same hard-coded allow-list initially: `['text/plain', 'text/html',
|
|
'image/png']`. **IMPROVEMENT**: use exact prefix matching, not substring
|
|
matching (Greenfield's substring match can collide —
|
|
`text/plain;charset=iso-8859-1` silently becomes `text/plain`).
|
|
|
|
### Native → browser
|
|
|
|
`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is
|
|
matched against the allow-list (exact prefix match), survivors pulled as
|
|
separate blobs via `mkfifo + send + readBlob`, then
|
|
`navigator.clipboard.write([new ClipboardItem(...)])`.
|
|
|
|
### Browser → native
|
|
|
|
`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types,
|
|
plus a synthetic `text/plain;charset=utf-8` alias appended (Linux apps
|
|
expect the charset form). On `send`, the alias is normalised back to
|
|
`text/plain` for the lookup.
|
|
|
|
## Clipboard mirroring (REUSED approach)
|
|
|
|
The browser has no `clipboardchange` event (Chromium issue 933608 cited in
|
|
Greenfield's code). MIT approach polls `navigator.clipboard.read()`:
|
|
|
|
- on init
|
|
- on every `window.focus` event
|
|
|
|
De-duplicates offers by mime-type set + total blob size, then calls
|
|
`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`.
|
|
|
|
The feedback loop is broken (same as Greenfield) because
|
|
`handleWaylandDataSourceUpdate` returns early when `dataSource instanceof
|
|
BrowserDataSource`. Watch for the `clipboardchange` event in modern
|
|
browsers (currently behind flags in Chrome) and switch to event-driven
|
|
when available.
|
|
|
|
### Firefox fallback (REUSED approach)
|
|
|
|
`text-selection.ts` uses only `readText`/`writeText` and a
|
|
`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images
|
|
or HTML.
|
|
|
|
## Drag-and-drop (REUSED approach + NEW)
|
|
|
|
### Wayland-side (Seat.startDrag)
|
|
|
|
1. Validates the implicit pointer grab.
|
|
2. Attaches `DragIconRole` to the optional icon surface.
|
|
3. Installs a `PointerDrag` grab on both pointer and keyboard.
|
|
|
|
`PointerDrag` tracks the focused data device, sends
|
|
`wl_data_device.enter`/`leave`/`motion`/`drop`, creates fresh
|
|
`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation.
|
|
|
|
### Browser-side (`BrowserDnD` class)
|
|
|
|
Creates a hidden full-canvas `dragHelper` div with `draggable=true` when
|
|
the Wayland drag-icon surface commits, so the browser shows the Wayland
|
|
icon via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the
|
|
user drags, `dragover` events on the dragHelper are converted back to
|
|
`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`,
|
|
keeping `PointerDrag` fed.
|
|
|
|
**NEW: Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD`
|
|
so a Wayland drag can deliver bytes to a real DOM drop target. Greenfield
|
|
has a FIXME admitting this isn't implemented.
|
|
|
|
## XWayland bridge (REUSED approach, IF XWayland enabled)
|
|
|
|
`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and
|
|
builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the
|
|
receiver pulls data:
|
|
|
|
1. `XDataSource.send` calls `XConvertSelection` to have the X server push
|
|
the bytes into a property on `selectionWindow`.
|
|
2. `writeProperty` copies them into the receiver's `InputOutputFD`.
|
|
3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`).
|
|
|
|
Uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings.
|
|
|
|
## Primary selection (NEW — Greenfield lacks this)
|
|
|
|
Implement `zwp_primary_selection_device_manager_v1` (or the older
|
|
`gtk_primary_selection`) on the compositor side. Forward `PRIMARY`
|
|
notifications in `XWindowManager`. This unlocks middle-click paste from
|
|
native apps.
|
|
|
|
## ASCII diagram — clipboard flow in both directions
|
|
|
|
```
|
|
CLIPBOARD / SELECTION FLOW
|
|
|
|
=================================================================
|
|
A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app)
|
|
=================================================================
|
|
|
|
┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐
|
|
│ Native app │ ─────────────────────────────────────────────► │ compositor │
|
|
│ (sandboxed │ │ (browser, in DOM) │
|
|
│ in bwrap) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │
|
|
└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘
|
|
│ selectionListeners fire
|
|
▼
|
|
┌──────────────────────────┐
|
|
│ handleWaylandDataSource │
|
|
│ Update() (selection.ts)│
|
|
└──────────┬──────────────┘
|
|
mkfifo() ──────────────────────────► │ Python /mkfifo
|
|
ProxyFD read+write (os.pipe) ◄───── returns 2 ProxyFDs
|
|
▼
|
|
dataSource.send("text/plain", writeFD)
|
|
│
|
|
▼
|
|
wl_data_source.send("text/plain", writeFDProxyFD)
|
|
─────────────────────────────────────────► │ Native source
|
|
│ receives real write fd
|
|
│ (proxyFDtoNativeFD
|
|
│ is a passthrough)
|
|
▼
|
|
Native app writes bytes
|
|
──────────────► OS pipe ───┐
|
|
│
|
|
readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘
|
|
▼ (proxy streams over WS)
|
|
Blob {type:"text/plain"}
|
|
▼
|
|
navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
|
|
▼
|
|
┌────────────────────────────┐
|
|
│ DOM clipboard populated │
|
|
│ (Ctrl+V anywhere now works) │
|
|
└────────────────────────────┘
|
|
|
|
|
|
=================================================================
|
|
B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM)
|
|
=================================================================
|
|
|
|
┌─────────────────────────────┐
|
|
│ User copies in DOM │
|
|
│ (browser tab, textarea...) │
|
|
└──────────────┬──────────────┘
|
|
│ on window 'focus' event (or initial load)
|
|
▼
|
|
navigator.clipboard.read() ──► ClipboardItems[]
|
|
▼
|
|
handleBrowserDataSourceUpdate() — de-dup vs previous offers
|
|
▼
|
|
createBrowserDataSource(offers)
|
|
.mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
|
|
▼
|
|
seat.setSelectionInternal(browserDataSource, serial)
|
|
│
|
|
│ 1) selectionListeners fire — handleWaylandDataSourceUpdate
|
|
│ returns early (instanceof BrowserDataSource), breaking loop
|
|
│
|
|
│ 2) sendSelection(focusedClient)
|
|
▼
|
|
┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐
|
|
│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │
|
|
│ (browser, in DOM) │ │ (focused) │
|
|
└───────────┬────────────┘ └──────┬───────┘
|
|
│ │
|
|
│ When the native app calls wl_data_offer.receive("text/plain", fd):
|
|
│ │
|
|
│ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
|
|
▼
|
|
BrowserDataSource.send(mimeType, ioFD)
|
|
│
|
|
▼
|
|
matchingOffer.getType(mimeType) ─► Promise<Blob> (from ClipboardItem)
|
|
│
|
|
▼
|
|
ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle> ─► Python proxy
|
|
│
|
|
▼
|
|
asyncio.StreamWriter ──► native write fd
|
|
│
|
|
▼
|
|
Bytes flow through OS pipe into native app
|
|
│
|
|
▼
|
|
ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0
|
|
|
|
|
|
=================================================================
|
|
C) PRIMARY SELECTION (NEW — middle-click paste) ──► same flow but via
|
|
zwp_primary_selection_device_manager_v1 instead of wl_data_device
|
|
=================================================================
|
|
|
|
Native app selects text ──► zwp_primary_selection_source.set_selection
|
|
│
|
|
▼
|
|
Browser receives primary selection offer ──► stores in seat.primarySelectionDataSource
|
|
│
|
|
▼
|
|
User middle-clicks on a surface ──► seat.notifyPrimarySelectionRequest
|
|
│
|
|
▼
|
|
primarySelectionSource.send(mimeType, writeFD) ──► same FD-over-WS flow as A/B
|
|
```
|
|
|
|
## File reference
|
|
|
|
### Custom (clean-room TS)
|
|
|
|
| File | Role |
|
|
|---|---|
|
|
| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` global |
|
|
| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` impl |
|
|
| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` impl |
|
|
| `packages/compositor/src/data/WaylandDataSource.ts` | Native source wrapper |
|
|
| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** |
|
|
| `packages/compositor/src/data/PrimarySelectionSource.ts` | **NEW: middle-click paste source** |
|
|
| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge |
|
|
| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback |
|
|
| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD implemented) |
|
|
| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
|
|
|
### Python side
|
|
|
|
| Component | Role |
|
|
|---|---|
|
|
| `os.pipe()` | Create pipes (replaces Greenfield's `makePipe` native addon) |
|
|
| `os.memfd_create()` | Create shared memory (replaces Greenfield's `createMemoryMappedFile`) |
|
|
| `asyncio.StreamReader` / `StreamWriter` | Streaming reads/writes over WebSocket endpoints |
|