.tasks/core/LSYNC-023-rebuild-closure-tables-on-sync.md
CRITICAL BUG: Closure tables (entry_closure and tag_closure) are NOT rebuilt when entries/tags are synced from other devices, leaving them with only self-references.
-- Device A (source) has full closure table:
SELECT * FROM entry_closure WHERE ancestor_id = 1;
(1, 1, 0) -- Desktop → Desktop (self)
(1, 2, 1) -- Desktop → Desk (child)
(1, 3, 1) -- Desktop → .localized (child)
(1, 4, 2) -- Desktop → file.txt (grandchild)
... 79 total relationships
-- Device B (after sync) has BROKEN closure table:
SELECT * FROM entry_closure;
(1, 1, 0) -- Desktop → Desktop (self only!)
(2, 2, 0) -- Desk → Desk (self only!)
(3, 3, 0) -- .localized → .localized (self only!)
... NO parent-child relationships!
Severity: CRITICAL - Breaks core functionality
Consequences:
WHERE ancestor_id = X) returns nothingentry_closure JOIN)Evidence:
Jam instance (synced):
- Entries: 1,987
- entry_closure records: 27 (all self-references)
- Missing: ~1,960 parent-child closure relationships
entry::Model::apply_state_change() (line 329-499) inserts/updates entry records but does NOT rebuild entry_closure.
When locally indexing, EntryProcessor::create_entry_in_conn() populates closure:
// line 289-309 in entry.rs
let self_closure = entry_closure::ActiveModel {
ancestor_id: Set(result.id),
descendant_id: Set(result.id),
depth: Set(0),
};
out_self_closures.push(self_closure);
// Copy parent's ancestors
if let Some(parent_id) = parent_id {
conn.execute_unprepared(&format!(
"INSERT INTO entry_closure (ancestor_id, descendant_id, depth)
SELECT ancestor_id, {}, depth + 1
FROM entry_closure
WHERE descendant_id = {}",
result.id, parent_id
))
...
}
But apply_state_change() doesn't do this!
File: core/src/infra/db/entities/entry.rs:~497
Add closure table population to apply_state_change():
pub async fn apply_state_change(data: serde_json::Value, db: &DatabaseConnection) -> Result<(), sea_orm::DbErr> {
// ... existing upsert logic ...
let entry_id = if let Some(existing_entry) = existing {
// Update ...
existing_entry.id
} else {
// Insert ...
inserted.id
};
// Rebuild entry_closure for this entry
rebuild_entry_closure(entry_id, parent_id, db).await?;
// If directory, update directory_paths ...
Ok(())
}
async fn rebuild_entry_closure(
entry_id: i32,
parent_id: Option<i32>,
db: &DatabaseConnection,
) -> Result<(), sea_orm::DbErr> {
use sea_orm::{ConnectionTrait, Set};
// Delete existing closure records for this entry
entry_closure::Entity::delete_many()
.filter(entry_closure::Column::DescendantId.eq(entry_id))
.exec(db)
.await?;
// Insert self-reference
let self_closure = entry_closure::ActiveModel {
ancestor_id: Set(entry_id),
descendant_id: Set(entry_id),
depth: Set(0),
};
self_closure.insert(db).await?;
// If there's a parent, copy all parent's ancestors
if let Some(parent_id) = parent_id {
db.execute(Statement::from_sql_and_values(
DbBackend::Sqlite,
r#"
INSERT INTO entry_closure (ancestor_id, descendant_id, depth)
SELECT ancestor_id, ?, depth + 1
FROM entry_closure
WHERE descendant_id = ?
"#,
vec![entry_id.into(), parent_id.into()],
))
.await?;
}
Ok(())
}
Pros:
Cons:
File: core/src/service/sync/backfill.rs:~140
Add closure rebuild after backfill completes:
// After Phase 3: backfill_device_owned_state
info!("Rebuilding closure tables after backfill...");
rebuild_all_entry_closures(db).await?;
rebuild_all_tag_closures(db).await?;
// Phase 4: Transition to ready
Implementation:
async fn rebuild_all_entry_closures(db: &DatabaseConnection) -> Result<()> {
use sea_orm::ConnectionTrait;
// Clear existing closure table
entry_closure::Entity::delete_many().exec(db).await?;
// 1. Insert all self-references
db.execute(Statement::from_sql_and_values(
DbBackend::Sqlite,
r#"
INSERT INTO entry_closure (ancestor_id, descendant_id, depth)
SELECT id, id, 0 FROM entries
"#,
vec![],
))
.await?;
// 2. Recursively build parent-child relationships
// Keep inserting until no new relationships found
let mut iteration = 0;
loop {
let result = db.execute(Statement::from_sql_and_values(
DbBackend::Sqlite,
r#"
INSERT OR IGNORE INTO entry_closure (ancestor_id, descendant_id, depth)
SELECT ec.ancestor_id, e.id, ec.depth + 1
FROM entries e
INNER JOIN entry_closure ec ON ec.descendant_id = e.parent_id
WHERE e.parent_id IS NOT NULL
AND NOT EXISTS (
SELECT 1 FROM entry_closure
WHERE ancestor_id = ec.ancestor_id
AND descendant_id = e.id
)
"#,
vec![],
))
.await?;
iteration += 1;
let rows_affected = result.rows_affected();
tracing::debug!(
iteration = iteration,
rows_inserted = rows_affected,
"entry_closure rebuild iteration"
);
if rows_affected == 0 {
break; // No more relationships to add
}
if iteration > 100 {
return Err(anyhow::anyhow!("entry_closure rebuild exceeded max iterations - possible cycle"));
}
}
info!("Rebuilt entry_closure table in {} iterations", iteration);
Ok(())
}
Pros:
Cons:
Combine both approaches:
Implement Option 1 (real-time rebuild) because:
Then add Option 2 as a safety measure to run after backfill in case of any missed entries.
File: core/src/infra/db/entities/entry.rs
rebuild_entry_closure() functionapply_state_change() after upsertFile: core/src/service/sync/backfill.rs
rebuild_all_entry_closures() functionbackfill_device_owned_state() completesFile: core/src/infra/db/entities/tag.rs (or sync code)
File: core/tests/sync_closure_rebuild_test.rs (new)
#[tokio::test]
async fn test_entry_closure_rebuilt_during_sync() {
let (device_a, device_b) = setup_paired_devices().await;
// Device A creates location with nested entries
create_location(device_a, "/Test").await;
create_file(device_a, "/Test/folder/subfolder/file.txt").await;
// Verify Device A has full closure table
let closure_a = count_closure_records(device_a).await;
assert!(closure_a > 10); // Self-refs + parent-child rels
// Sync to Device B
wait_for_sync().await;
// Verify Device B has FULL closure table (not just self-refs)
let closure_b = count_closure_records(device_b).await;
assert_eq!(closure_a, closure_b); // Should match!
// Verify can query descendants on Device B
let descendants = query_descendants(device_b, root_entry_id).await;
assert!(descendants.len() > 1); // Should find children!
}
entry_closure rebuilt in apply_state_change()