315 lines
16 KiB
Markdown
315 lines
16 KiB
Markdown
# 06 — Data Device, Selection, DnD (`wl_data_device`)
|
|
|
|
This chapter covers Greenfield's data device / clipboard / drag-and-drop
|
|
implementation: how `wl_data_device_manager`, `wl_data_source`,
|
|
`wl_data_offer`, `wl_data_device` are implemented, how clipboard contents
|
|
are transferred between browser and native apps, and how DnD is coordinated.
|
|
|
|
## Faithful Wayland model
|
|
|
|
`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per
|
|
seat-per-client and the implementation lives directly on `Seat` (which
|
|
implements `WlDataDeviceRequests`). `WaylandDataSource` / `DataOffer` are
|
|
the standard source/offer pair, ported almost verbatim from weston,
|
|
including the `chooseAction()` bit-trick for DnD action negotiation.
|
|
|
|
## The FD-over-WebSocket problem
|
|
|
|
Wayland's `wl_data_offer.receive(mimeType, fd)` /
|
|
`wl_data_source.send(mimeType, fd)` carry a real UNIX fd. Greenfield solves
|
|
this with a **`ProxyFD` JSON descriptor**:
|
|
|
|
```
|
|
ProxyFD = {
|
|
handle: number,
|
|
type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown',
|
|
host: string // proxy's HTTP base URL
|
|
}
|
|
```
|
|
|
|
Three concrete `InputOutput` / `InputOutputFD` implementations:
|
|
|
|
### `webInputOutput` — for in-browser web apps
|
|
|
|
Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s. All
|
|
"fd" operations are in-browser zero-copy transfers.
|
|
|
|
### `RemoteInputOutput` — for native apps
|
|
|
|
All fd operations go over a secondary WebSocket to endpoints:
|
|
`/mkfifo`, `/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`,
|
|
`/read-fd`, `/close-fd`.
|
|
|
|
### `ProxyInputOutput` (proxy-side) — owns the real OS pipes and memfd files
|
|
|
|
`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD
|
|
has `host === proxy.baseURL`.
|
|
|
|
The proxy interceptors for `wl_data_source` / `wl_data_offer` /
|
|
`wl_data_device` / `wl_data_device_manager` are auto-generated no-ops that
|
|
only handle resource construction and lifetime — all actual semantics live
|
|
in the browser compositor.
|
|
|
|
### Crucial consequence
|
|
|
|
For native→native clipboard transfers (e.g. user copies in one Linux app,
|
|
pastes in another), the browser compositor never sees a single byte of the
|
|
data — it only relays the fd reference. The actual content stays inside
|
|
the proxy host's OS kernel pipe.
|
|
|
|
## MIME-type negotiation
|
|
|
|
At the Wayland level, mime types are just relayed strings. Mapping to the
|
|
browser's hard-coded allow-list `['text/plain', 'text/html', 'image/png']`
|
|
happens only at the DOM boundary:
|
|
|
|
### Native → browser
|
|
|
|
`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is
|
|
substring-matched against the allow-list (`text/plain;charset=utf-8`
|
|
contains `text/plain`, so it maps), survivors are pulled as separate blobs
|
|
via `mkfifo + send + readBlob`, and finally
|
|
`navigator.clipboard.write([new ClipboardItem(...)])`.
|
|
|
|
### Browser → native
|
|
|
|
`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types,
|
|
plus a synthetic `text/plain;charset=utf-8` alias is appended (Linux apps
|
|
expect the charset form). On `send`, the alias is normalised back to
|
|
`text/plain` for the lookup.
|
|
|
|
### Caveat: substring matching is fragile
|
|
|
|
The substring match `dataSourceMimeType.indexOf(browserMimeType) !== -1`
|
|
can collide (`image/png` matching `image/png;base64` is fine, but
|
|
`text/plain;charset=iso-8859-1` silently becomes `text/plain`). Consider
|
|
an explicit prefix or exact-match policy in your implementation.
|
|
|
|
## Clipboard mirroring is pull-based
|
|
|
|
The browser has no `clipboardchange` event (Chromium issue 933608 is
|
|
cited in the code). Greenfield polls `navigator.clipboard.read()`:
|
|
|
|
- on init
|
|
- on every `window.focus` event
|
|
|
|
De-duplicates offers by mime-type set + total blob size, then calls
|
|
`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`.
|
|
|
|
The feedback loop is broken because `handleWaylandDataSourceUpdate` returns
|
|
early when `dataSource instanceof BrowserDataSource`.
|
|
|
|
### Firefox fallback
|
|
|
|
`text-selection.ts` uses only `readText` / `writeText` and a
|
|
`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images
|
|
or HTML.
|
|
|
|
## Drag-and-drop
|
|
|
|
### Wayland-side (Seat.startDrag)
|
|
|
|
1. Validates the implicit pointer grab.
|
|
2. Attaches `DragIconRole` to the optional icon surface.
|
|
3. Installs a `PointerDrag` grab on both pointer and keyboard.
|
|
|
|
`PointerDrag` tracks the focused data device, sends
|
|
`wl_data_device.enter` / `leave` / `motion` / `drop`, creates fresh
|
|
`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation.
|
|
|
|
### Browser-side (`BrowserDnD` class)
|
|
|
|
Creates a hidden full-canvas `dragHelper` div with `draggable=true` when
|
|
the Wayland drag-icon surface commits, so the browser shows the Wayland icon
|
|
via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the user
|
|
drags, `dragover` events on the dragHelper are converted back to
|
|
`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`, keeping
|
|
`PointerDrag` fed.
|
|
|
|
### Not implemented: dragging data FROM a Wayland app INTO a real DOM target
|
|
|
|
The FIXME in `dnd.ts` admits: dragging data FROM a Wayland app INTO a real
|
|
DOM drop target (file drag-out, etc.) is **not implemented**.
|
|
|
|
## XWayland bridge
|
|
|
|
`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and
|
|
builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the
|
|
receiver pulls data:
|
|
|
|
1. `XDataSource.send` calls `XConvertSelection` to have the X server push
|
|
the bytes into a property on `selectionWindow`.
|
|
2. `writeProperty` copies them into the receiver's `InputOutputFD`.
|
|
3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`).
|
|
|
|
## Primary selection is NOT implemented
|
|
|
|
A grep for `primary_selection`, `gtk_primary`, `zwp_primary_selection`
|
|
across the entire `packages/` tree returns zero hits. XWayland only
|
|
watches the `CLIPBOARD` atom, not `PRIMARY`. **Middle-click paste does
|
|
not work in Greenfield today.**
|
|
|
|
**For the user**: if your users expect Linux-style middle-click paste,
|
|
you'll need to implement `zwp_primary_selection_device_manager_v1` (or
|
|
`gtk_primary_selection`) on the compositor side and forward `PRIMARY`
|
|
notifications in `XWindowManager`.
|
|
|
|
## ASCII diagram — clipboard flow in both directions
|
|
|
|
```
|
|
CLIPBOARD / SELECTION FLOW
|
|
|
|
=================================================================
|
|
A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app)
|
|
=================================================================
|
|
|
|
┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐
|
|
│ Native app │ ─────────────────────────────────────────────► │ compositor │
|
|
│ (Wayland or │ │ (browser, in DOM) │
|
|
│ XWayland) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │
|
|
└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘
|
|
│ selectionListeners fire
|
|
▼
|
|
┌──────────────────────────┐
|
|
│ handleWaylandDataSource │
|
|
│ Update() (selection.ts)│
|
|
└──────────┬──────────────┘
|
|
mkfifo() ──────────────────────────► │ proxy /mkfifo
|
|
ProxyFD read+write (real OS pipe) ◄───── returns 2 ProxyFDs
|
|
▼
|
|
dataSource.send("text/plain", writeFD)
|
|
│
|
|
▼
|
|
wl_data_source.send("text/plain", writeFDProxyFD)
|
|
─────────────────────────────────────────► │ Native source
|
|
│ receives real write fd
|
|
│ (proxyFDtoNativeFD
|
|
│ is a passthrough)
|
|
▼
|
|
Native app writes bytes
|
|
──────────────► OS pipe ───┐
|
|
│
|
|
readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘
|
|
▼ (proxy streams over WS)
|
|
Blob {type:"text/plain"}
|
|
▼
|
|
navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
|
|
▼
|
|
┌────────────────────────────┐
|
|
│ DOM clipboard populated │
|
|
│ (Ctrl+V anywhere now works) │
|
|
└────────────────────────────┘
|
|
|
|
|
|
=================================================================
|
|
B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM)
|
|
=================================================================
|
|
|
|
┌─────────────────────────────┐
|
|
│ User copies in DOM │
|
|
│ (browser tab, textarea...) │
|
|
└──────────────┬──────────────┘
|
|
│ on window 'focus' event (or initial load)
|
|
▼
|
|
navigator.clipboard.read() ──► ClipboardItems[]
|
|
▼
|
|
handleBrowserDataSourceUpdate() — de-dup vs previous offers
|
|
▼
|
|
createBrowserDataSource(offers)
|
|
.mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
|
|
▼
|
|
seat.setSelectionInternal(browserDataSource, serial)
|
|
│
|
|
│ 1) selectionListeners fire — handleWaylandDataSourceUpdate
|
|
│ returns early (instanceof BrowserDataSource), breaking loop
|
|
│
|
|
│ 2) sendSelection(focusedClient)
|
|
▼
|
|
┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐
|
|
│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │
|
|
│ (browser, in DOM) │ │ (focused) │
|
|
└───────────┬────────────┘ └──────┬───────┘
|
|
│ │
|
|
│ When the native app calls wl_data_offer.receive("text/plain", fd):
|
|
│ │
|
|
│ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
|
|
│ (fd was created by the native app's libwayland; proxy
|
|
│ wrapped it as ProxyFD{host:proxy.baseURL, type:'unknown'}
|
|
│ then compositor's DataOffer.receive calls wrapFD → RemoteInputOutputFD)
|
|
▼
|
|
BrowserDataSource.send(mimeType, ioFD)
|
|
│
|
|
▼
|
|
matchingOffer.getType(mimeType) ─► Promise<Blob> (from ClipboardItem)
|
|
│
|
|
▼
|
|
ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle> ─► proxy
|
|
│
|
|
▼
|
|
createWriteStream({fd, autoClose:true})
|
|
ws stream ──► native write fd
|
|
│
|
|
▼
|
|
Bytes flow through OS pipe into native app
|
|
│
|
|
▼
|
|
ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0
|
|
|
|
|
|
=================================================================
|
|
PROXY FD TRANSLATION (works the same way in B and A)
|
|
=================================================================
|
|
|
|
Native fd (in proxy process)
|
|
│
|
|
│ wrap into ProxyFD JSON: {handle, type, host: proxy.baseURL}
|
|
│ JSON-encode, length-prefix, send over WS protocol channel
|
|
▼
|
|
┌─────────────────────────────────────────────────┐
|
|
│ Browser compositor │
|
|
│ client.userData.inputOutput.wrapFD(fd, type) │
|
|
│ → RemoteInputOutputFD (RemoteInputOutput) │
|
|
│ → WebPipeInputOutputFD (webInputOutput, │
|
|
│ for in-browser apps)│
|
|
└─────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## File reference
|
|
|
|
| File | Role |
|
|
|---|---|
|
|
| `packages/compositor/src/DataDeviceManager.ts` | `wl_data_device_manager` global |
|
|
| `packages/compositor/src/DataSource.ts` | `wl_data_source` impl, the provider side |
|
|
| `packages/compositor/src/DataOffer.ts` | `wl_data_offer` impl, the receiver side |
|
|
| `packages/compositor/src/WaylandDataSource.ts` | Wrapper around native `wl_data_source` events |
|
|
| `packages/compositor/src/Seat.ts` | `wl_data_device` impl (Seat implements `WlDataDeviceRequests`) |
|
|
| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge (native → DOM) |
|
|
| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback (text-only) |
|
|
| `packages/compositor/src/browser/dnd.ts` | Browser DnD bridge (Wayland → DOM) |
|
|
| `packages/compositor/src/InputOutput.ts` | FD abstraction interface |
|
|
| `packages/compositor/src/remote/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
|
| `packages/compositor/src/remote/XWindowManager.ts` | X11 CLIPBOARD atom watcher |
|
|
| `packages/compositor/src/remote/XDataSource.ts` | X11 clipboard source (XWayland apps) |
|
|
| `packages/compositor/src/remote/XDnDDataSource.ts` | X11 DnD source |
|
|
|
|
## Next actions / suggested improvements
|
|
|
|
1. **Primary selection** — implement `zwp_primary_selection_device_manager`
|
|
v1 (or `gtk_primary_selection`) on the compositor side and forward
|
|
`PRIMARY` notifications in `XWindowManager`. This unlocks middle-click
|
|
paste from native apps.
|
|
2. **`clipboardchange` event** — when browsers ship it, replace the
|
|
`window.focus`-based polling in `selection.ts` with a proper event-driven
|
|
subscription.
|
|
3. **Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD`
|
|
so a Wayland drag can deliver bytes to a real DOM drop target (the
|
|
`dnd.ts` FIXME).
|
|
4. **MIME allow-list discovery** — replace the hard-coded
|
|
`['text/plain', 'text/html', 'image/png']` with a runtime check once
|
|
browsers expose the supported types.
|
|
5. **MIME-type strict matching** — the substring match
|
|
`dataSourceMimeType.indexOf(browserMimeType) !== -1` can collide
|
|
(`image/png` matching `image/png;base64` is fine, but
|
|
`text/plain;charset=iso-8859-1` silently becomes `text/plain`).
|
|
Consider an explicit prefix or exact-match policy.
|