one-os-dev-wayland-html5-compositor-greenfield-analysis/06-data-clipboard.md

06 — Data Device, Selection, DnD (wl_data_device)

This chapter covers Greenfield's data device / clipboard / drag-and-drop implementation: how wl_data_device_manager, wl_data_source, wl_data_offer, wl_data_device are implemented, how clipboard contents are transferred between browser and native apps, and how DnD is coordinated.

Faithful Wayland model

DataDeviceManager is a v3 global singleton. wl_data_device is per seat-per-client and the implementation lives directly on Seat (which implements WlDataDeviceRequests). WaylandDataSource / DataOffer are the standard source/offer pair, ported almost verbatim from weston, including the chooseAction() bit-trick for DnD action negotiation.

The FD-over-WebSocket problem

Wayland's wl_data_offer.receive(mimeType, fd) / wl_data_source.send(mimeType, fd) carry a real UNIX fd. Greenfield solves this with a ProxyFD JSON descriptor:

ProxyFD = {
  handle: number,
  type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown',
  host: string   // proxy's HTTP base URL
}

Three concrete InputOutput / InputOutputFD implementations:

webInputOutput — for in-browser web apps

Pipes are MessageChannel ports, transferred as ArrayBuffers. All "fd" operations are in-browser zero-copy transfers.

RemoteInputOutput — for native apps

All fd operations go over a secondary WebSocket to endpoints: /mkfifo, /mkstemp-mmap, /read-fd-as-stream, /write-fd-as-stream, /read-fd, /close-fd.

ProxyInputOutput (proxy-side) — owns the real OS pipes and memfd files

proxyFDtoNativeFD(proxyFD) returns the raw int fd because every ProxyFD has host === proxy.baseURL.

The proxy interceptors for wl_data_source / wl_data_offer / wl_data_device / wl_data_device_manager are auto-generated no-ops that only handle resource construction and lifetime — all actual semantics live in the browser compositor.

Crucial consequence

For native→native clipboard transfers (e.g. user copies in one Linux app, pastes in another), the browser compositor never sees a single byte of the data — it only relays the fd reference. The actual content stays inside the proxy host's OS kernel pipe.

MIME-type negotiation

At the Wayland level, mime types are just relayed strings. Mapping to the browser's hard-coded allow-list ['text/plain', 'text/html', 'image/png'] happens only at the DOM boundary:

Native → browser

selection.ts:handleWaylandDataSourceUpdate: each source mime type is substring-matched against the allow-list (text/plain;charset=utf-8 contains text/plain, so it maps), survivors are pulled as separate blobs via mkfifo + send + readBlob, and finally navigator.clipboard.write([new ClipboardItem(...)]).

Browser → native

BrowserDataSource.ts: ClipboardItem.types become the source mime types, plus a synthetic text/plain;charset=utf-8 alias is appended (Linux apps expect the charset form). On send, the alias is normalised back to text/plain for the lookup.

Caveat: substring matching is fragile

The substring match dataSourceMimeType.indexOf(browserMimeType) !== -1 can collide (image/png matching image/png;base64 is fine, but text/plain;charset=iso-8859-1 silently becomes text/plain). Consider an explicit prefix or exact-match policy in your implementation.

Clipboard mirroring is pull-based

The browser has no clipboardchange event (Chromium issue 933608 is cited in the code). Greenfield polls navigator.clipboard.read():

• on init
• on every window.focus event

De-duplicates offers by mime-type set + total blob size, then calls seat.setSelectionInternal(createBrowserDataSource(offers), serial).

The feedback loop is broken because handleWaylandDataSourceUpdate returns early when dataSource instanceof BrowserDataSource.

Firefox fallback

text-selection.ts uses only readText / writeText and a BrowserTextDataSource that serves text/plain;charset=utf-8 — no images or HTML.

Drag-and-drop

Wayland-side (Seat.startDrag)

1. Validates the implicit pointer grab.
2. Attaches DragIconRole to the optional icon surface.
3. Installs a PointerDrag grab on both pointer and keyboard.

PointerDrag tracks the focused data device, sends wl_data_device.enter / leave / motion / drop, creates fresh wl_data_offers via sendOffer, and runs the v3 action negotiation.

Browser-side (BrowserDnD class)

Creates a hidden full-canvas dragHelper div with draggable=true when the Wayland drag-icon surface commits, so the browser shows the Wayland icon via event.dataTransfer.setDragImage(cursorCanvas, ...). While the user drags, dragover events on the dragHelper are converted back to ButtonEvents and pushed through session.inputQueue.queueMotion, keeping PointerDrag fed.

Not implemented: dragging data FROM a Wayland app INTO a real DOM target

The FIXME in dnd.ts admits: dragging data FROM a Wayland app INTO a real DOM drop target (file drag-out, etc.) is not implemented.

XWayland bridge

XWindowManager listens on the X11 CLIPBOARD atom via XFixes and builds XDataSource (clipboard) or XDnDDataSource (XDnD). When the receiver pulls data:

1. XDataSource.send calls XConvertSelection to have the X server push the bytes into a property on selectionWindow.
2. writeProperty copies them into the receiver's InputOutputFD.
3. INCR transfers are handled (getIncrChunk, sendIncrChunk).

Primary selection is NOT implemented

A grep for primary_selection, gtk_primary, zwp_primary_selection across the entire packages/ tree returns zero hits. XWayland only watches the CLIPBOARD atom, not PRIMARY. Middle-click paste does not work in Greenfield today.

For the user: if your users expect Linux-style middle-click paste, you'll need to implement zwp_primary_selection_device_manager_v1 (or gtk_primary_selection) on the compositor side and forward PRIMARY notifications in XWindowManager.

ASCII diagram — clipboard flow in both directions

                                 CLIPBOARD / SELECTION FLOW

=================================================================
A) NATIVE APP COPIES  ──►  BROWSER DOM CLIPBOARD  (Ctrl+C in Linux app)
=================================================================

┌──────────────┐    wl_data_device.set_selection(src,serial)     ┌────────────────────────┐
│  Native app  │ ─────────────────────────────────────────────► │      compositor        │
│  (Wayland or │                                                │  (browser, in DOM)     │
│   XWayland)  │ ◄────── wl_data_offer.offer("text/plain") ───  │                        │
└──────────────┘            (only if this app has keyboard focus)└───────────┬────────────┘
                                                                         │ selectionListeners fire
                                                                         ▼
                                                          ┌──────────────────────────┐
                                                          │ handleWaylandDataSource  │
                                                          │   Update() (selection.ts)│
                                                          └──────────┬──────────────┘
                                  mkfifo()  ──────────────────────────► │ proxy /mkfifo
                                  ProxyFD read+write (real OS pipe) ◄───── returns 2 ProxyFDs
                                                                  ▼
                                          dataSource.send("text/plain", writeFD)
                                                                  │
                                                                  ▼
                       wl_data_source.send("text/plain", writeFDProxyFD)
                       ─────────────────────────────────────────► │ Native source
                                                                  │ receives real write fd
                                                                  │ (proxyFDtoNativeFD
                                                                  │  is a passthrough)
                                                                  ▼
                                                          Native app writes bytes
                                                          ──────────────► OS pipe ───┐
                                                                                       │
                       readFD.readBlob()  ◄──────────── read from /read-fd-as-stream ◄─┘
                                  ▼                                     (proxy streams over WS)
                          Blob {type:"text/plain"}
                                  ▼
                       navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
                                  ▼
                       ┌────────────────────────────┐
                       │   DOM clipboard populated  │
                       │ (Ctrl+V anywhere now works) │
                       └────────────────────────────┘


=================================================================
B) USER COPIES IN BROWSER  ──►  NATIVE APP GETS SELECTION  (Ctrl+C in DOM)
=================================================================

┌─────────────────────────────┐
│  User copies in DOM         │
│ (browser tab, textarea...)  │
└──────────────┬──────────────┘
               │   on window 'focus' event (or initial load)
               ▼
   navigator.clipboard.read()  ──►  ClipboardItems[]
               ▼
   handleBrowserDataSourceUpdate() — de-dup vs previous offers
               ▼
   createBrowserDataSource(offers)
       .mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
               ▼
   seat.setSelectionInternal(browserDataSource, serial)
               │
               │ 1) selectionListeners fire — handleWaylandDataSourceUpdate
               │    returns early (instanceof BrowserDataSource), breaking loop
               │
               │ 2) sendSelection(focusedClient)
               ▼
┌────────────────────────┐   wl_data_offer.offer("text/plain"),   ┌──────────────┐
│      compositor        │ ─ wl_data_device.selection(offer) ──► │ Native app  │
│  (browser, in DOM)     │                                       │ (focused)   │
└───────────┬────────────┘                                       └──────┬───────┘
            │                                                           │
            │  When the native app calls wl_data_offer.receive("text/plain", fd):
            │                                                           │
            │ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
            │     (fd was created by the native app's libwayland; proxy
            │      wrapped it as ProxyFD{host:proxy.baseURL, type:'unknown'}
            │      then compositor's DataOffer.receive calls wrapFD → RemoteInputOutputFD)
            ▼
   BrowserDataSource.send(mimeType, ioFD)
            │
            ▼
   matchingOffer.getType(mimeType) ─► Promise<Blob>  (from ClipboardItem)
            │
            ▼
   ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle>  ─► proxy
                                                              │
                                                              ▼
                                          createWriteStream({fd, autoClose:true})
                                          ws stream ──► native write fd
                                                              │
                                                              ▼
                                          Bytes flow through OS pipe into native app
            │
            ▼
   ioFD.close() ─► /close-fd  ─► EOF ─► native app's read returns 0


=================================================================
   PROXY FD TRANSLATION (works the same way in B and A)
=================================================================

   Native fd (in proxy process)
            │
            │ wrap into ProxyFD JSON: {handle, type, host: proxy.baseURL}
            │ JSON-encode, length-prefix, send over WS protocol channel
            ▼
   ┌─────────────────────────────────────────────────┐
   │  Browser compositor                             │
   │  client.userData.inputOutput.wrapFD(fd, type)   │
   │     → RemoteInputOutputFD (RemoteInputOutput)   │
   │     → WebPipeInputOutputFD (webInputOutput,     │
   │                              for in-browser apps)│
   └─────────────────────────────────────────────────┘

File reference

File	Role
packages/compositor/src/DataDeviceManager.ts	wl_data_device_manager global
packages/compositor/src/DataSource.ts	wl_data_source impl, the provider side
packages/compositor/src/DataOffer.ts	wl_data_offer impl, the receiver side
packages/compositor/src/WaylandDataSource.ts	Wrapper around native wl_data_source events
packages/compositor/src/Seat.ts	wl_data_device impl (Seat implements WlDataDeviceRequests)
packages/compositor/src/browser/selection.ts	Browser clipboard bridge (native → DOM)
packages/compositor/src/browser/text-selection.ts	Firefox fallback (text-only)
packages/compositor/src/browser/dnd.ts	Browser DnD bridge (Wayland → DOM)
packages/compositor/src/InputOutput.ts	FD abstraction interface
packages/compositor/src/remote/RemoteInputOutput.ts	FD-over-WebSocket for native apps
packages/compositor/src/remote/XWindowManager.ts	X11 CLIPBOARD atom watcher
packages/compositor/src/remote/XDataSource.ts	X11 clipboard source (XWayland apps)
packages/compositor/src/remote/XDnDDataSource.ts	X11 DnD source

Next actions / suggested improvements

1. Primary selection — implement zwp_primary_selection_device_manager v1 (or gtk_primary_selection) on the compositor side and forward PRIMARY notifications in XWindowManager. This unlocks middle-click paste from native apps.
2. clipboardchange event — when browsers ship it, replace the window.focus-based polling in selection.ts with a proper event-driven subscription.
3. Native→DOM DnD — implement the dataTransfer side of BrowserDnD so a Wayland drag can deliver bytes to a real DOM drop target (the dnd.ts FIXME).
4. MIME allow-list discovery — replace the hard-coded ['text/plain', 'text/html', 'image/png'] with a runtime check once browsers expose the supported types.
5. MIME-type strict matching — the substring match dataSourceMimeType.indexOf(browserMimeType) !== -1 can collide (image/png matching image/png;base64 is fine, but text/plain;charset=iso-8859-1 silently becomes text/plain). Consider an explicit prefix or exact-match policy.