winzik/docs/ide-screen.md
David Nurdin 59f56d9c36 JumPHP IDE 0.1.114: Screen on a phone (the computer's screen in a phone's browser, WebRTC) + the engine extension jumphp/webrtc
- extensions/webrtc (jumphp/webrtc): peer-to-peer data channels from PHP (browser: engine/web/jumphp-webrtc.js; Linux desktop: a GStreamer
  webrtcbin sidecar, native or Docker), Signaling::overNet(), the desktop's screen as H.264 + remote input; doc, tutorial, samples, TODO;
  games/webrtc-demo (tests + samples)
- IDE: the header button next to the menu, ide/src/Remote/ScreenHost.php, ide/ui/js/screen-share.js (consent per device, quality presets,
  QR code to https://play.jumphp.net/jumphp-screen/play/), 14 languages; docs/ide-screen.md
- code updates carry jumphp/webrtc in src/Bundled/webrtc (an overlay may not write vendor/), loaded when the executable lacks it

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-13 07:19:13 +01:00

36 lines
4.2 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Screen on a phone (JumPHP IDE): the computer's screen in a phone's browser
A VNC-like mode, but WebRTC: the IDE's header button **Screen on a phone** (next to the main menu ☰, icon `cast`) shares this computer's
**whole screen** as a live H.264 video, with the mouse and the keyboard, to **JumPHP Screen** (https://jumphp.net/jumphp-screen, its page
https://play.jumphp.net/jumphp-screen/play/). Separate from IDE Remote (`docs/ide-remote.md`, the screen icon: data, never pixels): its own button,
its own pairing, room and key; both can run at once.
```
IDE page (screen-share.js) ==E2E-encrypted signaling (channel.js, relay wss://jumphp.net/ide-relay/ws)==> JumPHP Screen (browser)
| screen.open / screen.signal (PHP commands) ^
ide/src/Remote/ScreenHost.php -> jumphp/webrtc Sidecar -> webrtc-gst.py (GStreamer) ==WebRTC: video + "input" data channel (P2P, LAN first)==+
```
- Why a sidecar: the IDE's webview (WebKitGTK) has no `RTCPeerConnection`. The peer is GStreamer `webrtcbin` (extensions/webrtc, `doc/sidecar.md`):
`ximagesrc` -> GPU encoder (VA-API / NVENC) or x264 zero-latency -> RTP -> WebRTC; latency 0 in webrtcbin, no B-frames, `alignment=au`.
Input (mouse, wheel, keys, Unicode text) is injected by the sidecar itself with XTest (never through PHP: latency).
- Runs natively when python3 + GStreamer webrtcbin + nice exist, else in Docker (`--network host --ipc host`, X11 socket, `/dev/dri`; the image is built
the first time, a few minutes; the dialog says so). Linux X11 only for now (Wayland, Windows, macOS: TODO of extensions/webrtc/doc/index.md).
- **What to install** (the dialog says what is missing): Linux with an X11 session, then either the native GStreamer packages (lowest latency;
Debian / Ubuntu: `sudo apt install python3-gi gir1.2-gst-plugins-bad-1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad
gstreamer1.0-plugins-ugly gstreamer1.0-nice gstreamer1.0-x libxtst6`, + `mesa-va-drivers` for AMD / Intel GPU encoding) or Docker usable without sudo.
Fedora, Arch, the check command and a symptom table: `extensions/webrtc/doc/index.md` "Desktop requirements". The phone: any current browser (Chrome,
Safari, Firefox), nothing to install. An IDE updated by a code update (not a full release) runs jumphp/webrtc from `src/Bundled/webrtc` (tools/overlay-build.php).
- Pairing: `https://play.jumphp.net/jumphp-screen/play/#r=<room>&k=<key>` (fragment only, never sent to a server; the app reads it from the framing page).
- Consent: every device is asked on the PC ("sees everything on this screen, not only the IDE"); Stop sharing closes every peer. Quality presets in the dialog
(Fast / Balanced / Sharp / Data saver); the phone changes the bitrate live.
- Messages, page -> IDE: `hello` {v:1, app:'screen', name}, `answer` {sdp}, `ice` {candidate, mline}, `restart`, `ping`, `bye`;
IDE -> page: `welcome`, `info` {encoder, width, height}, `offer` {sdp}, `ice`, `denied` {reason: refused | stopped | version}, `error` {message}.
- PHP commands (`ide/src/Ide.php`, only when jumphp/webrtc is installed: games/ide requires it): `screen.probe`, `screen.open` {id, fps, kbps, width, stun},
`screen.signal` {id, type+sdp | candidate+mline | kbps}, `screen.close` {id}, `screen.stop`; event `screen.event` (the sidecar's events).
- The viewer's controls: touchpad mode (slide, tap = click, 2 fingers = right click / scroll, 3 = middle, pinch = zoom, hold = drag) or touch mode,
⌨ (the phone's keyboard), Fn (Esc, Tab, sticky Ctrl / Alt / Shift / Super, arrows, F1-F12, shortcuts), Fit / 1:1, bitrate, stats, fullscreen.
- Files: `ide/ui/js/screen-share.js`, `ide/src/Remote/ScreenHost.php`, `extensions/webrtc/`, the app `/home/david/JumPHP-Projects/jumphpScreen/apps/jumphp-screen`
(repository https://git.jumphp.net/duchnoun/jumphp-screen; `ui/js/channel.js` = a copy of `ide/ui/js/remote/channel.js`).
- Checked end to end (2026-11-13): GStreamer peer -> Chrome, 1600×900 60 fps, RTT 1-3 ms, jitter buffer 5-7 ms, clicks and typed text reach the PC;
pitfalls found: XShm needs `--ipc host` in Docker (else black frames), a buffered stdin read in the sidecar lost commands, the browser frames the app.