docs/core/file-sync.mdx
<Note>This is provisional documentation describing the planned File Sync implementation.</Note>
File Sync orchestrates content synchronization between locations by leveraging Spacedrive's Virtual Distributed File System (VDFS) index. Unlike traditional sync tools that scan filesystems, File Sync operates entirely through index queries. This makes sync state resolution a simple database operation and ensures perfect consistency with your indexing rules.
File Sync builds on Spacedrive's core infrastructure. Each sync operation runs as a Job, providing state persistence and progress tracking. The system derives all sync decisions from the VDFS index, eliminating complex state management and ensuring files ignored during indexing remain excluded from sync.
The VDFS index serves as the single source of truth for File Sync. When you configure a sync relationship between two locations, the system:
This approach transforms sync from a complex distributed systems problem into straightforward database queries.
Index as Truth
If a file isn't indexed, it doesn't exist for sync purposes. This ensures consistency between what you see in Spacedrive and what gets synchronized.
Push-Based Transfer The device that has files pushes them to devices that need them. This works naturally with iOS background processing and simplifies the protocol.
Library Sync First File Sync requires a complete, up-to-date index via Library Sync before operating. This ensures all devices have the same view of what exists where.
Leverage Existing Jobs
File Sync orchestrates FileCopyJob and DeleteJob rather than implementing custom transfer logic. The existing job system already handles local vs remote routing, progress tracking, and error recovery.
File Sync offers four distinct modes, each designed for specific workflows. Configure these per-location through the sync settings panel.
Creates an exact copy of your source at the destination. Changes flow one direction only. Perfect for backup scenarios where you want a read-only copy of important data.
When you delete a file from the source, Mirror Mode removes it from the destination too. This keeps your backup lean and matches your working set exactly.
Keeps two locations identical across different devices. Changes made anywhere propagate everywhere. Ideal for working on the same projects from multiple computers.
The system uses last-write-wins for conflicts. When the same file changes in multiple places, the most recent modification takes precedence. Previous versions remain in your Spacedrive history.
Moves completed work to long-term storage. After verifying successful transfer, the source files are removed to free local space. The system performs cryptographic verification before any deletion.
Use this mode when finishing projects. Your files move to cheaper storage while maintaining full Spacedrive search and preview capabilities.
Intelligently manages limited local storage. Frequently accessed files stay local while older content moves to secondary storage. Files remain visible in Spacedrive and download on-demand when needed.
The algorithm considers access patterns, available space, and manual pins. You control which files always stay local by pinning them in the Spacedrive interface.
File Sync uses index queries to determine what needs synchronization. This eliminates filesystem scanning and complex state tracking.
Before File Sync can operate, Library Sync must complete:
impl FileSyncJob {
async fn can_start(&self) -> bool {
// Ensure Library Sync is complete
let library_sync_state = self.library.sync_state().await;
library_sync_state.is_fully_synced()
}
}
This guarantees all devices share the same index view, preventing inconsistent sync decisions.
The sync resolver calculates operations by comparing index entries:
// Find files that need copying (exist in source but not target)
let to_copy = source_entries
.filter(|e| !target_entries.contains(e.path))
.collect();
// Find files that need deletion (exist in target but not source)
let to_delete = target_entries
.filter(|e| !source_entries.contains(e.path))
.collect();
For bidirectional sync, the system uses timestamps to detect conflicts when both sides modified the same file.
For cases requiring full directory sync including normally-ignored files, sync conduits support an index mode override:
pub enum IndexingMode {
Standard, // Use normal exclusion rules
SyncComplete, // Index everything for this sync
}
File transfers use Spacedrive's existing P2P infrastructure over QUIC. The protocol supports:
Each sync session negotiates capabilities. Modern devices use parallel transfers while older systems fall back to sequential mode.
File Sync uses a push-based model where the device with files initiates transfers:
impl FileSyncJob {
async fn execute(&self, ctx: &JobContext) -> Result<()> {
let conduit = self.load_conduit().await?;
match conduit.sync_mode {
SyncMode::Mirror => {
// Source pushes all changes to target
if self.am_source(&conduit) {
let ops = self.calculate_mirror_push().await?;
self.push_changes(ops, conduit.target).await?;
}
}
SyncMode::Bidirectional => {
// Each device pushes its own changes
let my_changes = self.calculate_my_changes().await?;
if !my_changes.is_empty() {
self.push_changes(my_changes, other_device).await?;
}
}
}
}
}
When receiving files, devices validate against their local index:
impl FileTransferProtocol {
async fn handle_incoming_file(&self, metadata: FileMetadata) -> Result<()> {
// Check if we already have this content
let existing = self.db.content_identity()
.find_by_hash(&metadata.content_hash)
.await?;
if existing.is_some() {
return Ok(()); // Skip transfer, already have it
}
// Proceed with transfer
self.receive_file(metadata).await
}
}
This prevents duplicate transfers and ensures consistency.
File Sync introduces the SyncConduit entity to manage sync relationships between directories:
pub struct SyncConduit {
pub id: i32,
pub uuid: Uuid,
// Sync endpoints - must be directories
pub source_entry_id: i32, // FK to Entry
pub target_entry_id: i32, // FK to Entry
// Configuration
pub sync_mode: SyncMode,
pub enabled: bool,
pub schedule: SyncSchedule,
// Index integration
pub use_index_rules: bool, // Default: true
pub index_mode_override: Option<IndexingMode>,
// Performance
pub parallel_transfers: i32,
pub bandwidth_limit_mbps: Option<i32>,
// State tracking via index timestamps
pub last_sync_completed_at: Option<DateTime<Utc>>,
pub sync_generation: i64, // Increments each sync
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
SyncConduit creates a directed relationship between two Entry records. The source and target must be directories (kind=1). For bidirectional sync, the direction indicates the initial sync priority but changes flow both ways.
Track sync operations for conflict detection and debugging:
pub struct SyncGeneration {
pub id: i32,
pub conduit_id: i32,
pub generation: i64,
pub started_at: DateTime<Utc>,
pub completed_at: Option<DateTime<Utc>>,
// Operation summary
pub files_copied: i32,
pub files_deleted: i32,
pub conflicts_resolved: i32,
pub bytes_transferred: i64,
}
Each sync creates a new generation record. For bidirectional sync, this history helps detect when the same file changed on both sides since the last sync.
Create sync relationships through the Spacedrive interface by right-clicking any directory and selecting "Sync to...". The system creates a SyncConduit record linking your source and destination.
Sync Schedule: Controls when File Sync runs. Instant mode monitors filesystem events for immediate sync. Periodic modes batch changes for efficiency.
Bandwidth Limits: Prevents sync from saturating your connection. Applies per-conduit, so multiple syncs share available bandwidth intelligently.
Conflict Resolution: Determines automatic resolution strategy. The system maintains conflict history for manual review regardless of settings.
The sync status panel shows all active sync relationships. Monitor transfer progress, view sync history, and troubleshoot issues from a unified interface.
Key metrics displayed:
File Sync currently has these constraints:
Future updates will address these limitations based on user feedback and real-world usage patterns.
The index-driven architecture provides several advantages over traditional filesystem scanning approaches:
Performance
No directory traversal needed. Sync resolution becomes efficient database queries instead of recursive filesystem operations.
Consistency
Files ignored by indexing rules automatically stay out of sync. No divergence between what Spacedrive shows and what gets synchronized.
Simplicity
Push-based transfers eliminate complex negotiation protocols. The device with data initiates transfers, making the system predictable and debuggable.
Reliability
Library Sync ensures all devices share the same worldview before File Sync operates. Combined with receiver validation, this prevents inconsistent states.
While the current system transfers whole files, future versions will add block-level delta transfers for large file modifications:
// Future: Only transfer changed blocks
pub struct DeltaTransfer {
pub file_id: i32,
pub changed_blocks: Vec<BlockRange>,
pub block_size: u32,
}
The current DeleteJob lacks cross-device support. Future updates will add remote deletion capability to complete the sync feature set.
Advanced conflict handling will maintain version history and offer multiple resolution strategies beyond last-write-wins.
File Sync excels at personal file management workflows:
Project Folders: Keep active projects synchronized between laptop and desktop
Photo Libraries: Mirror photos from phone to NAS for backup
Document Archive: Move completed work to cloud storage automatically
Selective Cache: Keep recent files local while older content lives in cloud
The index-based design ensures these workflows remain fast and reliable while respecting your organization preferences.