.tasks/core/INDEX-008-nested-locations-support.md
Locations are currently implemented as isolated entry trees with orphan roots (parent_id: null). This prevents nested locations where a subdirectory of one location becomes its own location.
The design intent is for locations to be virtual organizational concepts that reference existing entries in the unified entry tree, not isolated silos. This would enable:
Filesystem:
/Users/jamespine/Documents/
├── Work/
│ ├── project.txt
│ └── notes.md
└── Personal/
User actions:
1. sd location add "/Users/jamespine/Documents"
2. sd location add "/Users/jamespine/Documents/Work"
Database result (BROKEN):
entries:
1: Documents (parent: null) ← Location A root
2: Work (parent: 1) ← Created by Location A indexing
3: Personal (parent: 1)
4: project.txt (parent: 2)
5: notes.md (parent: 2)
100: Work (parent: null) ← Location B root (DUPLICATE!)
101: project.txt (parent: 100) ← DUPLICATE!
102: notes.md (parent: 100) ← DUPLICATE!
locations:
Location A: entry_id = 1
Location B: entry_id = 100
Issues:
Entry duplication (Work, project.txt, notes.md exist twice)
Broken tree (Location B's root is orphaned)
Wasted storage (same data indexed twice)
Sync confusion (two UUIDs for same file)
Update conflicts (which entry to modify?)
Database result (CORRECT):
entries:
1: Documents (parent: null)
2: Work (parent: 1) ← Shared by both locations
3: Personal (parent: 1)
4: project.txt (parent: 2)
5: notes.md (parent: 2)
locations:
Location A: entry_id = 1 (points to Documents)
Location B: entry_id = 2 (points to EXISTING Work entry)
entry_closure:
(1, 1, 0) # Documents → Documents (self)
(1, 2, 1) # Documents → Work (child)
(1, 3, 1) # Documents → Personal (child)
(1, 4, 2) # Documents → project.txt (grandchild)
(1, 5, 2) # Documents → notes.md (grandchild)
(2, 2, 0) # Work → Work (self)
(2, 4, 1) # Work → project.txt (child)
(2, 5, 1) # Work → notes.md (child)
Benefits:
Single unified entry tree (no duplication)
Work entry has correct parent (Documents)
Both locations reference same physical entries
Changes to Work/ reflected in both locations
Storage efficient
Sync consistent (one UUID per file)
Locations are views over the entry tree, not owners of entries.
Each location defines:
Multiple locations can reference overlapping subtrees with different behaviors.
File: core/src/location/manager.rs:100-122
Current:
// Always creates new entry
let entry_model = entry::ActiveModel {
parent_id: Set(None), // Orphan root
...
};
let entry_record = entry_model.insert(&txn).await?;
Needed:
// Check if entry already exists at this path
let existing_entry = directory_paths::Entity::find()
.filter(directory_paths::Column::Path.eq(&path_str))
.one(&txn)
.await?;
let entry_id = match existing_entry {
Some(dir_path) => {
// REUSE existing entry for nested location
info!(
"Reusing existing entry {} for nested location at {}",
dir_path.entry_id,
path_str
);
dir_path.entry_id
}
None => {
// Create new root entry (parent location doesn't exist)
let entry_model = entry::ActiveModel {
parent_id: Set(None), // Will be orphan unless we detect parent location
...
};
let entry_record = entry_model.insert(&txn).await?;
// Create closure and directory_path entries...
entry_record.id
}
};
// Location points to existing or new entry
let location_model = location::ActiveModel {
entry_id: Set(Some(entry_id)),
...
};
File: core/src/location/manager.rs:~180
Current:
// Always spawns indexer job
let job = IndexerJob::from_location(location_id, sd_path, mode);
library.jobs().dispatch(job).await?;
Needed:
// Check if this entry is already indexed
let entry = entry::Entity::find_by_id(entry_id)
.one(db)
.await?
.ok_or(...)?;
if entry.indexed_at.is_some() {
info!(
"Location root already indexed at {}, skipping indexer job",
entry.indexed_at.unwrap()
);
// But we might still want to apply THIS location's index_mode
// if it's different from the parent location's mode
if should_reindex_with_different_mode(entry_id, mode, db).await? {
let job = IndexerJob::from_location(location_id, sd_path, mode);
library.jobs().dispatch(job).await?;
}
} else {
// Not yet indexed, spawn job as normal
let job = IndexerJob::from_location(location_id, sd_path, mode);
library.jobs().dispatch(job).await?;
}
File: core/src/service/watcher/mod.rs (new logic)
Problem: If both Location A and Location B watch overlapping paths, which one handles events?
Options:
Option A: All watchers trigger (simple but wasteful)
// Both Location A and B get notified for /Documents/Work/test.txt
// Both call responder
// Responder is idempotent, so duplicate processing is safe but inefficient
Option B: Innermost location wins (efficient)
// In the watcher event dispatch or routing:
async fn find_deepest_watching_location(
&self,
event_path: &Path,
library_id: Uuid,
db: &DatabaseConnection,
) -> Result<Option<Uuid>> {
// NOTE: All locations in watched_locations are already filtered to THIS device
// (INDEX-003 Phase 1 ensures only owned locations are watched)
let mut candidates = Vec::new();
for (location_id, watched_loc) in self.watched_locations.read().await.iter() {
// Get location's entry record to check tree relationship
let location_record = location::Entity::find()
.filter(location::Column::Uuid.eq(*location_id))
.one(db)
.await?;
if let Some(loc) = location_record {
if let Some(root_entry_id) = loc.entry_id {
// Check if event path is under this location's entry tree
// Use entry_closure and directory_paths, not path string matching
if is_path_in_entry_tree(event_path, root_entry_id, db).await? {
// Get depth of location's root in the overall entry tree
let depth = get_entry_depth(root_entry_id, db).await?;
candidates.push((*location_id, depth));
}
}
}
}
// Return location with deepest (highest depth value) root entry
// Deeper in tree = more nested = should take precedence
Ok(candidates
.into_iter()
.max_by_key(|(_, depth)| *depth)
.map(|(id, _)| id))
}
async fn is_path_in_entry_tree(
path: &Path,
root_entry_id: i32,
db: &DatabaseConnection,
) -> Result<bool> {
// Try to resolve the path within this entry tree
let path_str = path.to_string_lossy().to_string();
let result = db
.query_one(Statement::from_sql_and_values(
DbBackend::Sqlite,
r#"
SELECT 1
FROM directory_paths dp
INNER JOIN entry_closure ec ON ec.descendant_id = dp.entry_id
WHERE dp.path = ?
AND ec.ancestor_id = ?
LIMIT 1
"#,
vec![path_str.into(), root_entry_id.into()],
))
.await?;
Ok(result.is_some())
}
Device filtering note: Since INDEX-003 Phase 1 ensures only this device's locations are loaded into watched_locations, we don't need additional device_id filtering here. All locations in the HashMap are guaranteed to be owned by the current device.
Recommendation: Start with Option A (both trigger), optimize to Option B later.
File: core/src/location/manager.rs (delete method)
Problem: Deleting Location A shouldn't delete entries used by Location B
Solution:
async fn delete_location(&self, location_id: Uuid, db: &DatabaseConnection) -> Result<()> {
let location = location::Entity::find()
.filter(location::Column::Uuid.eq(location_id))
.one(db)
.await?
.ok_or(...)?;
// Check if other locations reference this entry or its descendants
if let Some(entry_id) = location.entry_id {
// Get all descendants of this location's root
let descendant_ids = entry_closure::Entity::find()
.filter(entry_closure::Column::AncestorId.eq(entry_id))
.all(db)
.await?
.into_iter()
.map(|ec| ec.descendant_id)
.collect::<Vec<_>>();
// Check if any other locations reference these entries
let other_locations = location::Entity::find()
.filter(location::Column::EntryId.is_in(descendant_ids.clone()))
.filter(location::Column::Id.ne(location.id))
.count(db)
.await?;
if other_locations > 0 {
warn!(
"Location shares entries with {} other location(s), preserving entry tree",
other_locations
);
// Just delete the location record, keep entries
} else {
// Safe to delete entire entry tree
delete_subtree(entry_id, db).await?;
}
}
// Delete location record
location::Entity::delete_by_id(location.id)
.exec(db)
.await?;
Ok(())
}
Challenge: How to sync nested locations across devices?
Scenario:
/Documents) and Location B (/Documents/Work)Current sync (no nesting support):
With nesting support:
Implementation: Location sync already uses entry_id reference, so this works automatically! Just need to ensure receiving device doesn't re-create entries.
Question: Who "owns" entry 2 (Work)?
Answer: The device owns it (through Location A's device_id), not the location itself.
Device A owns Location A (/Documents)
└─ Location A owns the indexing process for entry 1 and descendants
└─ Including entry 2 (Work)
Device A creates Location B (/Documents/Work)
└─ Location B is just a VIEW into entry 2's subtree
└─ Still owned by Device A
└─ No ownership conflict
Implication: Nested locations must be on the same device as their parent location's device.
Validation needed:
// When creating nested location, verify it's under a location on THIS device
if let Some(parent_location) = find_parent_location(&path, db).await? {
if parent_location.device_id != current_device_id {
return Err(LocationError::CannotNestAcrossDevices {
path: path.to_string(),
parent_location: parent_location.uuid,
parent_device: parent_location.device_id,
});
}
}
Files:
core/src/location/manager.rsTasks:
add_location() to check for existing entries at pathdirectory_paths only if entry was createdentry_closure only if entry was createdFiles:
core/src/location/manager.rsTasks:
Files:
core/src/service/watcher/mod.rscore/src/service/watcher/worker.rsTasks:
find_deepest_watching_location() helperFiles:
core/src/ops/locations/delete/action.rs (or manager)Tasks:
Files:
core/src/infra/db/entities/location.rsTasks:
File: core/tests/nested_locations_test.rs (new)
#[tokio::test]
async fn test_nested_location_reuses_entries() {
let device = setup_test_device().await;
// Create parent location
let location_a = create_location(device, "/Documents").await;
wait_for_index().await;
// Verify Work entry exists
let work_entry = find_entry_by_path("/Documents/Work").await.unwrap();
assert_eq!(work_entry.parent_id, Some(documents_entry_id));
// Create nested location at Work
let location_b = create_location(device, "/Documents/Work").await;
// Verify NO new entry created
let work_entry_after = find_entry_by_path("/Documents/Work").await.unwrap();
assert_eq!(work_entry.id, work_entry_after.id); // Same entry!
// Verify Location B points to existing entry
assert_eq!(location_b.entry_id, Some(work_entry.id));
// Verify no duplicate entries
let all_work_entries = entry::Entity::find()
.filter(entry::Column::Name.eq("Work"))
.all(db)
.await?;
assert_eq!(all_work_entries.len(), 1); // Only ONE Work entry
}
#[tokio::test]
async fn test_nested_location_watcher_precedence() {
let device = setup_test_device().await;
let location_a = create_location(device, "/Documents").await;
let location_b = create_location(device, "/Documents/Work").await;
// Create file in nested location
create_file("/Documents/Work/test.txt").await;
// Verify only Location B's worker processed it (innermost wins)
let worker_metrics_a = get_worker_metrics(location_a.id).await;
let worker_metrics_b = get_worker_metrics(location_b.id).await;
assert_eq!(worker_metrics_a.events_processed.load(), 0);
assert_eq!(worker_metrics_b.events_processed.load(), 1);
}
#[tokio::test]
async fn test_delete_parent_preserves_nested_location() {
let device = setup_test_device().await;
let location_a = create_location(device, "/Documents").await;
let location_b = create_location(device, "/Documents/Work").await;
wait_for_index().await;
let work_entry_id = location_b.entry_id.unwrap();
// Delete parent location
delete_location(location_a.id).await.unwrap();
// Verify Work entry still exists (referenced by Location B)
let work_entry = entry::Entity::find_by_id(work_entry_id)
.one(db)
.await?;
assert!(work_entry.is_some());
// Verify Location B still works
let location_b_after = location::Entity::find_by_id(location_b.id)
.one(db)
.await?;
assert!(location_b_after.is_some());
}
#[tokio::test]
async fn test_nested_location_sync() {
let (device_a, device_b) = setup_paired_devices().await;
// Device A creates nested locations
let location_a = create_location(device_a, "/Documents").await;
wait_for_index().await;
let location_b = create_location(device_a, "/Documents/Work").await;
// Sync to Device B
wait_for_sync().await;
// Verify Device B has both locations
let synced_location_a = find_location(device_b, location_a.uuid).await.unwrap();
let synced_location_b = find_location(device_b, location_b.uuid).await.unwrap();
// Verify they reference the same entry tree (no duplication)
let work_entries = entry::Entity::find()
.filter(entry::Column::Name.eq("Work"))
.all(device_b.db())
.await?;
assert_eq!(work_entries.len(), 1); // Only ONE Work entry
// Verify entry_id relationships preserved
assert_eq!(synced_location_b.entry_id, Some(work_entries[0].id));
}
#[tokio::test]
async fn test_cannot_nest_across_devices() {
let (device_a, device_b) = setup_paired_devices().await;
// Device A creates location
let location_a = create_location(device_a, "/Documents").await;
wait_for_sync().await;
// Device B tries to create nested location under Device A's location
let result = create_location(device_b, "/Documents/Work").await;
// Should fail - can't nest under another device's location
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), LocationError::CannotNestAcrossDevices { .. }));
}
Scenario:
/Documents) deleted/Documents/Work) still existsSolution:
// When deleting Location A:
// - Keep entry tree intact (Location B references it)
// - Entry 2's parent_id still points to entry 1
// - Entry 1 no longer has a location pointing to it
// - This is fine! Entry 1 exists as an unreferenced node
// - Or: Set entry 1's parent_id based on filesystem parent
Alternative: Prevent deleting parent locations if nested locations exist:
// Check for child locations before allowing deletion
let child_locations = find_locations_under_entry_subtree(entry_id, db).await?;
if !child_locations.is_empty() {
return Err(LocationError::HasNestedLocations {
location_id,
nested: child_locations,
});
}
Scenario:
# Move Work directory to Personal
mv /Documents/Work /Documents/Personal/Work
Current behavior:
entry_id still points to entry 2Solution: Update location path when root entry moves:
// After moving entry via responder:
// Check if any locations reference this entry
let locations_using_entry = location::Entity::find()
.filter(location::Column::EntryId.eq(moved_entry_id))
.all(db)
.await?;
for location in locations_using_entry {
// Rebuild location path from entry's new path
let new_path = PathResolver::get_full_path(db, moved_entry_id).await?;
location::ActiveModel {
id: Set(location.id),
// Update any path-related fields...
updated_at: Set(chrono::Utc::now()),
..Default::default()
}.update(db).await?;
}
Scenario:
/Documents) has mode: Shallow/Documents/Work) has mode: Deep/Documents/Work/test.pdf?Solution: Innermost location's mode wins:
// When indexing or processing:
fn get_effective_index_mode(path: &Path, db: &DatabaseConnection) -> IndexMode {
let all_containing_locations = find_locations_containing_path(path, db).await?;
// Find deepest location
let deepest = all_containing_locations
.into_iter()
.max_by_key(|loc| count_path_components(&loc.path));
deepest.map(|loc| loc.index_mode).unwrap_or(IndexMode::Shallow)
}
Problem: Location B references entry 2, but what if Location A hasn't synced yet?
Current sync order (from docs):
With nesting:
entry_id: 2Solution: Defer nested location sync until parent location syncs:
// In location::Model::apply_state_change()
if let Some(entry_id) = location_data.entry_id {
// Check if the entry exists
if entry::Entity::find_by_id(entry_id).one(db).await?.is_none() {
// Entry doesn't exist yet - parent location hasn't synced
// Defer this location until later
tracing::debug!(
"Deferring nested location {} - entry {} not yet synced",
location_uuid,
entry_id
);
return Ok(()); // Skip for now, will retry on next sync
}
}
Or better: Use the existing dependency system to ensure entries sync before locations that reference them.
No schema changes needed! The current schema already supports this:
CREATE TABLE locations (
id INTEGER PRIMARY KEY,
uuid TEXT UNIQUE,
device_id INTEGER,
entry_id INTEGER, ← Can point to ANY entry (not just orphans)
...
);
CREATE TABLE entries (
id INTEGER PRIMARY KEY,
parent_id INTEGER, ← Can be null (root) or reference parent
...
);
The flexibility is already built in!
Breaking change: No
Backwards compatibility: Yes - existing non-nested locations continue to work
Rollout:
Benefits:
Costs:
Net impact: Positive for users with many nested locations, neutral for simple use cases.
Location list view:
Documents (/Users/jamespine/Documents)
└─ Work (/Users/jamespine/Documents/Work) [nested]
Photos (/Users/jamespine/Pictures)
Considerations:
Modified files:
core/src/location/manager.rscore/src/service/watcher/mod.rscore/src/service/watcher/worker.rscore/src/ops/locations/delete/action.rsNew files:
core/tests/nested_locations_test.rscore/src/location/nesting.rs (helper functions)