docs/netplay-architecture.md
Issue: #2544 Status: Research / Pre-implementation Date: 2026-03-05
Provenance currently exposes RetroArch netplay only via the RetroArch in-game menu (pause → RA Menu → Network). There is no native Swift/SwiftUI netplay UI, no session discovery, and no support for netplay across native emulator cores (Dolphin, PPSSPP, Mednafen, etc.).
Key findings:
HAVE_NETWORKING, HAVE_NETPLAYDISCOVERY, and HAVE_NETPLAYDISCOVERY_NSNET but without HAVE_NETPLAY. LAN room discovery and NAT traversal code is present, but the core rollback/input-sync netplay engine is not enabled.HAVE_NETPLAY to BuildFlags.xcconfig is the single highest-leverage change to enable RetroArch netplay features through the RetroArch menu today.All RetroArch cores loaded by PVRetroArchCore share a single RetroArch runtime. Netplay is implemented at the RetroArch level (not per-core), making it the most accessible path.
| Feature | Current Status |
|---|---|
HAVE_NETWORKING | Enabled in BuildFlags.xcconfig |
HAVE_NETPLAYDISCOVERY | Enabled |
HAVE_NETPLAYDISCOVERY_NSNET | Enabled (uses NSNetService/Bonjour) |
HAVE_NETWORKGAMEPAD | Enabled |
HAVE_NETPLAY | MISSING — core netplay engine not compiled |
Implication: Adding -DHAVE_NETPLAY to OTHER_CFLAGS (and OTHER_DEBUG_CFLAGS) in CoresRetro/RetroArch/BuildFlags.xcconfig would enable RetroArch's full netplay system for all ~60 RetroArch cores. This includes rollback netplay, spectator mode, relay server support (RA.ME), and LAN discovery — all accessible via the RA in-game menu immediately.
RetroArch cores that are known-working with netplay upstream:
These cores have upstream netplay APIs that Provenance does not currently expose.
netplay.cpp is compiled in Cores/Mednafen/Package.swift (line 824)PVMednafenCoreBridge does not expose StartNetplay() / StopNetplay() or any networking hooks.Cores/Dolphin/PVDolphinCore/PVDolphinCore.mm has no netplay APIs surfaced. Dolphin's Core/NetPlay* classes exist in the dolphin-ios submodule but are not called.Cores/PPSSPP/PPSSPPGameCore.mmAdhocServer. The PSP's XLink Kai / Adhoc party emulation is partially supported.PPSSPPGameCore.mm.Cores/Mupen64Plus/Sources/mupen64plus-input-sdl's network mode. Limited and outdated; rarely used upstream.Cores/FCEU/fceux-netplay-server/fceux_netplay_server.h/.m exists but is an empty stub (just @implementation fceux_netplay_server @end). The FCEU-2.2.3 source has sdl-netplay.cpp / unix-netplay.cpp with a working SDL-based netplay implementation but it's SDL-specific and not adapted for Provenance.| Core | System(s) | Upstream Netplay | Built in Provenance | Native UI Needed |
|---|---|---|---|---|
| PVRetroArchCore | 60+ systems | Yes (RetroArch) | No (HAVE_NETPLAY missing) | Yes (or RA menu) |
| Mednafen | PS1, PCE, SNES, etc. | Yes (TCP, server) | Compiled, not exposed | Yes |
| Dolphin | GC, Wii | Yes (relay + LAN) | Not exposed | Yes |
| PPSSPP | PSP | Adhoc emulation | Not exposed | Yes |
| Mupen64Plus | N64 | Limited | Not exposed | Yes |
| FCEU native | NES | Stub only | Stub | N/A |
| TGBDual | GB/GBC | Link cable sim | Working locally | N/A |
RetroArch netplay is a rollback-based system (similar to GGPO) when latency is low, falling back to delay-based sync for high-latency connections.
Core mechanism:
Frame N:
- Player 1 sends input to Player 2 (and vice-versa)
- Both sides run the same frame with both inputs
- If inputs diverge due to latency, rollback to last confirmed frame and re-simulate
Key source files in PVCoreBridgeRetro/Sources/retro/:
tasks/task_netplay_lan_scan.c — broadcasts UDP query packets and collects LAN room responsestasks/task_netplay_nat_traversal.c — implements STUN-based NAT traversal for P2P WAN connectionstasks/task_netplay_find_content.c — matches game content hash with connected peerrunloop.c — netplay hooks in the main emulation loop (HAVE_NETPLAY guards)configuration.c / configuration.h — netplay_enable, netplay_server, netplay_port, netplay_delay_frames, etc.Discovery:
HAVE_NETPLAYDISCOVERYra.me — a publicly hosted TURN/STUN serverHAVE_NETPLAYDISCOVERY_NSNET (Bonjour/NSNetService) for LAN discovery — this is already enabledWhy HAVE_NETPLAY is missing:
Likely excluded to keep binary size down or due to past build failures. Enabling it requires also linking against mbedTLS (which is already present as HAVE_BUILTINMBEDTLS in the line flags) and may need HAVE_NETPLAY added to the linker/compiler search list that drives what .c files are compiled.
[PVRetroArchCoreBridge]
|
v
[RetroArch runloop.c] ←—HAVE_NETPLAY—→ [netplay.c]
| |
| [netplay_io.c] (input sync)
| [netplay_net.c] (TCP/UDP socket)
| [netplay_delta.c] (rollback)
v
[libretro core] ←——————retro_run()————————————
Once HAVE_NETPLAY is enabled:
What still needs custom implementation:
Framework: MultipeerConnectivity
Transport: Bluetooth + Wi-Fi (Infrastructure and peer-to-peer Wi-Fi)
Suitable for: LAN / local room discovery and session establishment
Pros:
Cons:
Best use in Provenance: Lobby discovery and initial handshake. Once peers find each other, hand off to RetroArch's own UDP netplay transport.
Framework: GameKit
Suitable for: WAN matchmaking, friend invites, turn-based and real-time matches
Pros:
GKMatch provides real-time data channel usable for input syncCons:
GKMatch has limited control over transport (no raw UDP)Best use in Provenance: Future WAN matchmaking for App Store builds. Not recommended for v1.
Framework: Network (available iOS 12+)
Transport: Direct TCP or UDP sockets with native Swift API
Pros:
Best use in Provenance: Used internally by RetroArch. Could be used for a Swift-side control plane (room management) while RetroArch handles the game data plane.
HAVE_NETPLAYDISCOVERY_NSNET is already compiled into Provenance's RetroArch build. This means RetroArch already uses Bonjour to advertise rooms on the LAN. We can consume this from Swift via NetServiceBrowser to list available rooms without any native core changes.
┌─────────────────────────────────────────────────────────────────┐
│ Provenance App (SwiftUI) │
│ ┌─────────────────┐ ┌───────────────────┐ ┌──────────────┐ │
│ │ NetplayLobby │ │ RoomBrowserView │ │ InGameHUD │ │
│ │ View │ │ (LAN / WAN) │ │ Overlay │ │
│ └────────┬────────┘ └────────┬──────────┘ └──────┬───────┘ │
│ │ │ │ │
│ ┌────────▼───────────────────▼─────────────────────▼────────┐ │
│ │ PVNetplayManager (Swift Actor) │ │
│ │ - Session lifecycle (host / join / spectate / leave) │ │
│ │ - Peer discovery (Bonjour + MultipeerConnectivity) │ │
│ │ - Core capability negotiation │ │
│ │ - ROM hash verification │ │
│ └──────────┬───────────────────────────────┬─────────────────┘ │
└─────────────┼───────────────────────────────┼───────────────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌──────────────────────────────┐
│ PVNetplayRetroArch │ │ PVNetplayNativeCore │
│ Bridge (ObjC/Swift) │ │ Bridge (future, per-core) │
│ │ │ │
│ - Calls RA command API │ │ - PVDolphinNetplayBridge │
│ to start/stop netplay│ │ - PVMednafenNetplayBridge │
│ - Reads RA config vars │ │ - PVPPSSPPAdhocBridge │
│ - Room status polling │ │ │
└─────────────┬───────────┘ └──────────────────────────────┘
│
▼
┌─────────────────────────────┐
│ RetroArch Runtime │
│ (PVRetroArchCoreBridge) │
│ │
│ - netplay.c / netplay_io.c │
│ - UDP input sync │
│ - Rollback state │
│ - RA.ME relay (WAN) │
│ - Bonjour LAN (NSNet) │
└─────────────────────────────┘
// PVNetplay/Sources/PVNetplay/PVNetplayManager.swift
public enum NetplayRole {
case host(port: UInt16)
case client(host: String, port: UInt16)
case spectator(host: String, port: UInt16)
}
public enum NetplayState {
case idle
case hosting(room: NetplayRoom)
case connecting(to: NetplayRoom)
case connected(session: NetplaySession)
case disconnected(reason: DisconnectReason)
}
public protocol PVNetplayCapable: AnyObject {
var supportsNetplay: Bool { get }
func startNetplay(role: NetplayRole, settings: NetplaySettings) async throws
func stopNetplay() async
var netplayState: NetplayState { get }
var netplayStatePublisher: AnyPublisher<NetplayState, Never> { get }
}
public struct NetplayRoom: Identifiable, Sendable {
public let id: UUID
public let hostName: String
public let gameName: String
public let gameHash: String // MD5 of ROM for verification
public let coreIdentifier: String
public let maxPlayers: Int
public let currentPlayers: Int
public let pingMS: Int?
public let isLAN: Bool
public let hostAddress: String
public let port: UInt16
}
public struct NetplaySettings: Sendable {
public var frameDelay: Int // 0 = rollback only, >0 = delay frames
public var maxSpectators: Int // 0-11
public var allowSpectators: Bool
public var relayServer: String? // nil = direct P2P, "ra.me" = relay
public var password: String?
public var playerIndex: Int // 0-based
}
RetroArch exposes a command interface (command.c) that can be called from ObjC. A thin bridge wraps this:
// PVNetplayRetroArchBridge.h
@interface PVNetplayRetroArchBridge : NSObject
/// Start hosting a netplay session using RetroArch's netplay engine
- (BOOL)startHosting:(NSString *)nickname
port:(uint16_t)port
frameDelay:(int)frameDelay
error:(NSError **)error;
/// Connect to a remote netplay host
- (BOOL)connectToHost:(NSString *)hostname
port:(uint16_t)port
nickname:(NSString *)nickname
error:(NSError **)error;
/// Stop current session
- (void)stopNetplay;
/// Query current session status (for HUD)
- (NSDictionary *)sessionStatus;
@end
Implementation calls into retroarch.c's command_event(CMD_EVENT_NETPLAY_INIT_DIRECT, ...) and related command events.
Since HAVE_NETPLAYDISCOVERY_NSNET is already compiled in, RetroArch hosts advertise themselves via Bonjour under the _retroarch._tcp service type. We can discover these from Swift:
// PVNetplayDiscovery.swift
final class PVNetplayDiscovery: NSObject, NetServiceBrowserDelegate, NetServiceDelegate {
private let browser = NetServiceBrowser()
private(set) var rooms: [NetplayRoom] = []
func startDiscovery() {
browser.delegate = self
browser.searchForServices(ofType: "_retroarch._tcp", inDomain: "local.")
}
func netServiceBrowser(_ browser: NetServiceBrowser,
didFind service: NetService,
moreComing: Bool) {
service.delegate = self
service.resolve(withTimeout: 5.0)
// Parse TXT record for game name, hash, player count
}
}
For MultipeerConnectivity (Bluetooth + P2P Wi-Fi), a separate PVNetplayMCDiscovery handles non-Wi-Fi scenarios.
New PVNetplay SPM package:
PVNetplay/
Package.swift
Sources/
PVNetplay/
PVNetplayManager.swift (actor, coordinates everything)
PVNetplaySession.swift (active session state)
PVNetplayRoom.swift (room model)
PVNetplaySettings.swift (user preferences)
Discovery/
PVNetplayBonjourDiscovery.swift
PVNetplayMCDiscovery.swift
Bridges/
PVNetplayRetroArchBridge.swift (wraps ObjC bridge)
PVNetplayNativeCoreBridge.swift (protocol for native cores)
PVUI adds:
PVUI/Sources/PVSwiftUI/Netplay/
NetplayLobbyView.swift
NetplayRoomBrowserView.swift
NetplayRoomCreateView.swift
NetplayInGameOverlay.swift
NetplaySettingsView.swift
Option A — From Game Library:
Game long-press context menu
└── "Netplay" → NetplayLobbyView
Option B — From In-Game pause menu:
Pause menu
└── "Network Play" → NetplayLobbyView
┌─────────────────────────────────┐
│ Network Play │
│ │
│ Playing: Super Mario World │
│ Core: RetroArch (snes9x) │
│ │
│ ┌──────────────────────────┐ │
│ │ Host a Room │ │
│ │ Invite friends to play │ │
│ └──────────────────────────┘ │
│ │
│ ┌──────────────────────────┐ │
│ │ Browse Rooms │ │
│ │ Join a game in progress│ │
│ └──────────────────────────┘ │
│ │
│ ┌──────────────────────────┐ │
│ │ Spectate │ │
│ │ Watch without playing │ │
│ └──────────────────────────┘ │
│ │
│ [?] Core support: Full │ ← badge: Full / Partial / None
└─────────────────────────────────┘
┌─────────────────────────────────┐
│ ← Browse Rooms [Refresh] │
│ │
│ LOCAL NETWORK ────────────── │
│ │
│ ┌──────────────────────────┐ │
│ │ JoeMatt's Room │ │
│ │ Chrono Trigger • SNES │ │
│ │ 1/2 players • LAN • 2ms │ │
│ └──────────────────────────┘ │
│ │
│ ┌──────────────────────────┐ │
│ │ RetroNight Session │ │
│ │ Street Fighter II • SNES │ │
│ │ 2/2 players • spectate │ │
│ └──────────────────────────┘ │
│ │
│ WAN / RELAY ──────────────── │
│ (requires RA.ME — future) │
│ │
└─────────────────────────────────┘
Step 1: Room Settings
┌─────────────────────────────────┐
│ ← Create Room │
│ │
│ Room Name: [JoeMatt's Room ] │
│ Max Players: [2] [3] [4] │
│ Frame Delay: [0 (rollback)] │
│ Allow Spectators: [ON] │
│ Password: [Optional ] │
│ Network: [LAN Only ▾] │
│ │
│ [Start Hosting] │
└─────────────────────────────────┘
Step 2: Waiting for Players
┌─────────────────────────────────┐
│ Hosting: JoeMatt's Room │
│ │
│ P1: JoeMatt (you) ✓ │
│ P2: Waiting... ○ │
│ │
│ Share link: [Copy] [AirDrop] │
│ │
│ [Start Game] (disabled) │
│ [Cancel] │
└─────────────────────────────────┘
Minimal non-intrusive overlay in corner:
[NET] P2 ████░ 18ms ⊠
Expands on tap:
┌──────────────────────┐
│ Network Play │
│ P1: You (local) │
│ P2: Friend 18ms │
│ Frame delay: 2 │
│ Rollbacks: 3 │
│ [Chat] [Disconnect] │
└──────────────────────┘
Tap room → ROM verification check
├── Hash matches → "Join as Player 2?" → confirm → connect → game starts
└── Hash mismatch → "ROM mismatch. Ensure you have the same ROM file."
Goal: Enable HAVE_NETPLAY in Provenance's RetroArch build, allowing all 60+ RA cores to use netplay through the existing RA in-game menu.
Changes:
-DHAVE_NETPLAY to OTHER_CFLAGS and OTHER_DEBUG_CFLAGS in CoresRetro/RetroArch/BuildFlags.xcconfigmbedTLS (already present as HAVE_BUILTINMBEDTLS) links correctlyRisk: Low. RetroArch netplay code is mature and already compiles on other Apple platforms.
Deliverable: All RetroArch cores gain netplay via RA menu. No native UI yet.
Goal: List available LAN netplay rooms in the native Provenance UI. No need to enter RA menu.
Changes:
PVNetplay SPM package (Tier 5-6)PVNetplayBonjourDiscovery using NetServiceBrowser to find _retroarch._tcp servicesNetplayRoomBrowserView to PVUI with room listingNo bridge changes required — we're just reading what RetroArch already advertises.
Deliverable: Users can see available LAN rooms in native UI; joining still opens RA menu.
Goal: Start and join netplay sessions from native SwiftUI without touching the RA menu.
Changes:
PVNetplayRetroArchBridge (ObjC) wrapping RA command APINetplayLobbyView, NetplayRoomCreateView in SwiftUIPVNetplayManager Swift actor for session lifecycleNetplayInGameOverlay) showing ping/statusDeliverable: Full native SwiftUI netplay UI for RetroArch cores on LAN.
Goal: Connect over the internet via relay servers.
Options:
Recommendation: Option A first (lowest cost, works with HAVE_NETPLAY). Option C long-term for App Store distribution.
Priority order based on popularity and implementation effort:
Each requires:
PVNetplayNativeCoreBridge protocol implementationPVNetplayManager| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
HAVE_NETPLAY causes build failures | Low | High | Gradual flag addition; test in CI on each PR |
| App Store rejection (network features) | Medium | High | Use only documented Apple APIs; no VPN-like behavior |
| RA.ME relay reliability | Medium | Medium | Allow direct P2P as primary path; relay as fallback |
| ROM piracy surface area (sharing ROM hashes) | Medium | High | Only verify hash, never transfer ROM data |
| Latency unplayable for high-speed games | High | Medium | Show estimated latency in room browser; let users choose |
| MultipeerConnectivity adds latency | Medium | Medium | Use MC only for discovery/handshake; hand off to UDP |
| Core state desync | Medium | High | Leverage RetroArch's mature desync detection; surface errors clearly in HUD |
| Phase 0 RA menu UX friction remains | High | Low | Acceptable; Phase 2 resolves this |
| GameKit requires App Store distribution | High | Low | Only use for App Store builds; degrade gracefully |
Port conflicts: RetroArch netplay defaults to port 55435. Are there firewall/carrier conflicts on iOS cellular? Consider making port configurable.
Background networking: Apple restricts background network activity. Netplay sessions must remain in-foreground. Should we warn users before starting?
Save state sync: RetroArch syncs save state at session start. For native cores, we need an equivalent. Serialize current state before connecting.
Chat: Text chat during netplay (RA supports it). Should we add voice via WebRTC or GameKit voice channels?
Spectator stream: RA supports up to 11 spectators. Should we add a "Watch live" mode in the room browser?
tvOS: MultipeerConnectivity works on tvOS. Netplay on tvOS (game room on TV) is a compelling use case. Should Phase 2 target tvOS simultaneously?
Controller assignment: When 2 players join, who is P1? Host is always P1 in RA. Should we allow swapping post-connect?
Relay infrastructure: If we host a Provenance relay, what's the operational cost? RA.ME handles ~10KB/s per session pair; even 1000 concurrent sessions is manageable.
PVCoreBridgeRetro/Sources/retro/tasks/task_netplay_*.cCoresRetro/RetroArch/BuildFlags.xcconfigCores/Mednafen/Package.swift line 824 (netplay.cpp compiled)Cores/FCEU/fceux-netplay-server/