Initial commit: Greenfield architectural analysis (14 files, ~80 pages, 10 parallel research agents)
This commit is contained in:
1b-agent committed
commit
c4b45cf218
15 files changed
+5444
No files matched your search
@@ -0,0 +1,314 @@
|
||||
# 06 — Data Device, Selection, DnD (`wl_data_device`)
|
||||
|
||||
This chapter covers Greenfield's data device / clipboard / drag-and-drop
|
||||
implementation: how `wl_data_device_manager`, `wl_data_source`,
|
||||
`wl_data_offer`, `wl_data_device` are implemented, how clipboard contents
|
||||
are transferred between browser and native apps, and how DnD is coordinated.
|
||||
|
||||
## Faithful Wayland model
|
||||
|
||||
`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per
|
||||
seat-per-client and the implementation lives directly on `Seat` (which
|
||||
implements `WlDataDeviceRequests`). `WaylandDataSource` / `DataOffer` are
|
||||
the standard source/offer pair, ported almost verbatim from weston,
|
||||
including the `chooseAction()` bit-trick for DnD action negotiation.
|
||||
|
||||
## The FD-over-WebSocket problem
|
||||
|
||||
Wayland's `wl_data_offer.receive(mimeType, fd)` /
|
||||
`wl_data_source.send(mimeType, fd)` carry a real UNIX fd. Greenfield solves
|
||||
this with a **`ProxyFD` JSON descriptor**:
|
||||
|
||||
```
|
||||
ProxyFD = {
|
||||
handle: number,
|
||||
type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown',
|
||||
host: string // proxy's HTTP base URL
|
||||
}
|
||||
```
|
||||
|
||||
Three concrete `InputOutput` / `InputOutputFD` implementations:
|
||||
|
||||
### `webInputOutput` — for in-browser web apps
|
||||
|
||||
Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s. All
|
||||
"fd" operations are in-browser zero-copy transfers.
|
||||
|
||||
### `RemoteInputOutput` — for native apps
|
||||
|
||||
All fd operations go over a secondary WebSocket to endpoints:
|
||||
`/mkfifo`, `/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`,
|
||||
`/read-fd`, `/close-fd`.
|
||||
|
||||
### `ProxyInputOutput` (proxy-side) — owns the real OS pipes and memfd files
|
||||
|
||||
`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD
|
||||
has `host === proxy.baseURL`.
|
||||
|
||||
The proxy interceptors for `wl_data_source` / `wl_data_offer` /
|
||||
`wl_data_device` / `wl_data_device_manager` are auto-generated no-ops that
|
||||
only handle resource construction and lifetime — all actual semantics live
|
||||
in the browser compositor.
|
||||
|
||||
### Crucial consequence
|
||||
|
||||
For native→native clipboard transfers (e.g. user copies in one Linux app,
|
||||
pastes in another), the browser compositor never sees a single byte of the
|
||||
data — it only relays the fd reference. The actual content stays inside
|
||||
the proxy host's OS kernel pipe.
|
||||
|
||||
## MIME-type negotiation
|
||||
|
||||
At the Wayland level, mime types are just relayed strings. Mapping to the
|
||||
browser's hard-coded allow-list `['text/plain', 'text/html', 'image/png']`
|
||||
happens only at the DOM boundary:
|
||||
|
||||
### Native → browser
|
||||
|
||||
`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is
|
||||
substring-matched against the allow-list (`text/plain;charset=utf-8`
|
||||
contains `text/plain`, so it maps), survivors are pulled as separate blobs
|
||||
via `mkfifo + send + readBlob`, and finally
|
||||
`navigator.clipboard.write([new ClipboardItem(...)])`.
|
||||
|
||||
### Browser → native
|
||||
|
||||
`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types,
|
||||
plus a synthetic `text/plain;charset=utf-8` alias is appended (Linux apps
|
||||
expect the charset form). On `send`, the alias is normalised back to
|
||||
`text/plain` for the lookup.
|
||||
|
||||
### Caveat: substring matching is fragile
|
||||
|
||||
The substring match `dataSourceMimeType.indexOf(browserMimeType) !== -1`
|
||||
can collide (`image/png` matching `image/png;base64` is fine, but
|
||||
`text/plain;charset=iso-8859-1` silently becomes `text/plain`). Consider
|
||||
an explicit prefix or exact-match policy in your implementation.
|
||||
|
||||
## Clipboard mirroring is pull-based
|
||||
|
||||
The browser has no `clipboardchange` event (Chromium issue 933608 is
|
||||
cited in the code). Greenfield polls `navigator.clipboard.read()`:
|
||||
|
||||
- on init
|
||||
- on every `window.focus` event
|
||||
|
||||
De-duplicates offers by mime-type set + total blob size, then calls
|
||||
`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`.
|
||||
|
||||
The feedback loop is broken because `handleWaylandDataSourceUpdate` returns
|
||||
early when `dataSource instanceof BrowserDataSource`.
|
||||
|
||||
### Firefox fallback
|
||||
|
||||
`text-selection.ts` uses only `readText` / `writeText` and a
|
||||
`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images
|
||||
or HTML.
|
||||
|
||||
## Drag-and-drop
|
||||
|
||||
### Wayland-side (Seat.startDrag)
|
||||
|
||||
1. Validates the implicit pointer grab.
|
||||
2. Attaches `DragIconRole` to the optional icon surface.
|
||||
3. Installs a `PointerDrag` grab on both pointer and keyboard.
|
||||
|
||||
`PointerDrag` tracks the focused data device, sends
|
||||
`wl_data_device.enter` / `leave` / `motion` / `drop`, creates fresh
|
||||
`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation.
|
||||
|
||||
### Browser-side (`BrowserDnD` class)
|
||||
|
||||
Creates a hidden full-canvas `dragHelper` div with `draggable=true` when
|
||||
the Wayland drag-icon surface commits, so the browser shows the Wayland icon
|
||||
via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the user
|
||||
drags, `dragover` events on the dragHelper are converted back to
|
||||
`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`, keeping
|
||||
`PointerDrag` fed.
|
||||
|
||||
### Not implemented: dragging data FROM a Wayland app INTO a real DOM target
|
||||
|
||||
The FIXME in `dnd.ts` admits: dragging data FROM a Wayland app INTO a real
|
||||
DOM drop target (file drag-out, etc.) is **not implemented**.
|
||||
|
||||
## XWayland bridge
|
||||
|
||||
`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and
|
||||
builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the
|
||||
receiver pulls data:
|
||||
|
||||
1. `XDataSource.send` calls `XConvertSelection` to have the X server push
|
||||
the bytes into a property on `selectionWindow`.
|
||||
2. `writeProperty` copies them into the receiver's `InputOutputFD`.
|
||||
3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`).
|
||||
|
||||
## Primary selection is NOT implemented
|
||||
|
||||
A grep for `primary_selection`, `gtk_primary`, `zwp_primary_selection`
|
||||
across the entire `packages/` tree returns zero hits. XWayland only
|
||||
watches the `CLIPBOARD` atom, not `PRIMARY`. **Middle-click paste does
|
||||
not work in Greenfield today.**
|
||||
|
||||
**For the user**: if your users expect Linux-style middle-click paste,
|
||||
you'll need to implement `zwp_primary_selection_device_manager_v1` (or
|
||||
`gtk_primary_selection`) on the compositor side and forward `PRIMARY`
|
||||
notifications in `XWindowManager`.
|
||||
|
||||
## ASCII diagram — clipboard flow in both directions
|
||||
|
||||
```
|
||||
CLIPBOARD / SELECTION FLOW
|
||||
|
||||
=================================================================
|
||||
A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app)
|
||||
=================================================================
|
||||
|
||||
┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐
|
||||
│ Native app │ ─────────────────────────────────────────────► │ compositor │
|
||||
│ (Wayland or │ │ (browser, in DOM) │
|
||||
│ XWayland) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │
|
||||
└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘
|
||||
│ selectionListeners fire
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ handleWaylandDataSource │
|
||||
│ Update() (selection.ts)│
|
||||
└──────────┬──────────────┘
|
||||
mkfifo() ──────────────────────────► │ proxy /mkfifo
|
||||
ProxyFD read+write (real OS pipe) ◄───── returns 2 ProxyFDs
|
||||
▼
|
||||
dataSource.send("text/plain", writeFD)
|
||||
│
|
||||
▼
|
||||
wl_data_source.send("text/plain", writeFDProxyFD)
|
||||
─────────────────────────────────────────► │ Native source
|
||||
│ receives real write fd
|
||||
│ (proxyFDtoNativeFD
|
||||
│ is a passthrough)
|
||||
▼
|
||||
Native app writes bytes
|
||||
──────────────► OS pipe ───┐
|
||||
│
|
||||
readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘
|
||||
▼ (proxy streams over WS)
|
||||
Blob {type:"text/plain"}
|
||||
▼
|
||||
navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
|
||||
▼
|
||||
┌────────────────────────────┐
|
||||
│ DOM clipboard populated │
|
||||
│ (Ctrl+V anywhere now works) │
|
||||
└────────────────────────────┘
|
||||
|
||||
|
||||
=================================================================
|
||||
B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM)
|
||||
=================================================================
|
||||
|
||||
┌─────────────────────────────┐
|
||||
│ User copies in DOM │
|
||||
│ (browser tab, textarea...) │
|
||||
└──────────────┬──────────────┘
|
||||
│ on window 'focus' event (or initial load)
|
||||
▼
|
||||
navigator.clipboard.read() ──► ClipboardItems[]
|
||||
▼
|
||||
handleBrowserDataSourceUpdate() — de-dup vs previous offers
|
||||
▼
|
||||
createBrowserDataSource(offers)
|
||||
.mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
|
||||
▼
|
||||
seat.setSelectionInternal(browserDataSource, serial)
|
||||
│
|
||||
│ 1) selectionListeners fire — handleWaylandDataSourceUpdate
|
||||
│ returns early (instanceof BrowserDataSource), breaking loop
|
||||
│
|
||||
│ 2) sendSelection(focusedClient)
|
||||
▼
|
||||
┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐
|
||||
│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │
|
||||
│ (browser, in DOM) │ │ (focused) │
|
||||
└───────────┬────────────┘ └──────┬───────┘
|
||||
│ │
|
||||
│ When the native app calls wl_data_offer.receive("text/plain", fd):
|
||||
│ │
|
||||
│ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
|
||||
│ (fd was created by the native app's libwayland; proxy
|
||||
│ wrapped it as ProxyFD{host:proxy.baseURL, type:'unknown'}
|
||||
│ then compositor's DataOffer.receive calls wrapFD → RemoteInputOutputFD)
|
||||
▼
|
||||
BrowserDataSource.send(mimeType, ioFD)
|
||||
│
|
||||
▼
|
||||
matchingOffer.getType(mimeType) ─► Promise<Blob> (from ClipboardItem)
|
||||
│
|
||||
▼
|
||||
ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle> ─► proxy
|
||||
│
|
||||
▼
|
||||
createWriteStream({fd, autoClose:true})
|
||||
ws stream ──► native write fd
|
||||
│
|
||||
▼
|
||||
Bytes flow through OS pipe into native app
|
||||
│
|
||||
▼
|
||||
ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0
|
||||
|
||||
|
||||
=================================================================
|
||||
PROXY FD TRANSLATION (works the same way in B and A)
|
||||
=================================================================
|
||||
|
||||
Native fd (in proxy process)
|
||||
│
|
||||
│ wrap into ProxyFD JSON: {handle, type, host: proxy.baseURL}
|
||||
│ JSON-encode, length-prefix, send over WS protocol channel
|
||||
▼
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Browser compositor │
|
||||
│ client.userData.inputOutput.wrapFD(fd, type) │
|
||||
│ → RemoteInputOutputFD (RemoteInputOutput) │
|
||||
│ → WebPipeInputOutputFD (webInputOutput, │
|
||||
│ for in-browser apps)│
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## File reference
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/DataDeviceManager.ts` | `wl_data_device_manager` global |
|
||||
| `packages/compositor/src/DataSource.ts` | `wl_data_source` impl, the provider side |
|
||||
| `packages/compositor/src/DataOffer.ts` | `wl_data_offer` impl, the receiver side |
|
||||
| `packages/compositor/src/WaylandDataSource.ts` | Wrapper around native `wl_data_source` events |
|
||||
| `packages/compositor/src/Seat.ts` | `wl_data_device` impl (Seat implements `WlDataDeviceRequests`) |
|
||||
| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge (native → DOM) |
|
||||
| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback (text-only) |
|
||||
| `packages/compositor/src/browser/dnd.ts` | Browser DnD bridge (Wayland → DOM) |
|
||||
| `packages/compositor/src/InputOutput.ts` | FD abstraction interface |
|
||||
| `packages/compositor/src/remote/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
||||
| `packages/compositor/src/remote/XWindowManager.ts` | X11 CLIPBOARD atom watcher |
|
||||
| `packages/compositor/src/remote/XDataSource.ts` | X11 clipboard source (XWayland apps) |
|
||||
| `packages/compositor/src/remote/XDnDDataSource.ts` | X11 DnD source |
|
||||
|
||||
## Next actions / suggested improvements
|
||||
|
||||
1. **Primary selection** — implement `zwp_primary_selection_device_manager`
|
||||
v1 (or `gtk_primary_selection`) on the compositor side and forward
|
||||
`PRIMARY` notifications in `XWindowManager`. This unlocks middle-click
|
||||
paste from native apps.
|
||||
2. **`clipboardchange` event** — when browsers ship it, replace the
|
||||
`window.focus`-based polling in `selection.ts` with a proper event-driven
|
||||
subscription.
|
||||
3. **Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD`
|
||||
so a Wayland drag can deliver bytes to a real DOM drop target (the
|
||||
`dnd.ts` FIXME).
|
||||
4. **MIME allow-list discovery** — replace the hard-coded
|
||||
`['text/plain', 'text/html', 'image/png']` with a runtime check once
|
||||
browsers expose the supported types.
|
||||
5. **MIME-type strict matching** — the substring match
|
||||
`dataSourceMimeType.indexOf(browserMimeType) !== -1` can collide
|
||||
(`image/png` matching `image/png;base64` is fine, but
|
||||
`text/plain;charset=iso-8859-1` silently becomes `text/plain`).
|
||||
Consider an explicit prefix or exact-match policy.
|
||||
Reference in new issue
Block a user