609. Find Duplicate File in System

Problem

Given a list paths of directory info, including the directory path, and all the files with contents in this directory, return all the duplicate files in the file system in terms of their paths. You may return the answer in any order.

A group of duplicate files consists of at least two files that have the same content.

A single directory info string in the input list has the following format:

• "root/d1/d2/.../dm f1.txt(f1_content) f2.txt(f2_content) ... fn.txt(fn_content)"

It means there are n files (f1.txt, f2.txt ... fn.txt) with content (f1_content, f2_content ... fn_content) respectively in the directory "root/d1/d2/.../dm". Note that n >= 1 and m >= 0. If m = 0, it means the directory is just the root directory.

The output is a list of groups of duplicate file paths. For each group, it contains all the file paths of the files that have the same content. A file path is a string that has the following format:

• "directory_path/file_name.txt"

Example 1:

Input: paths = ["root/a 1.txt(abcd) 2.txt(efgh)","root/c 3.txt(abcd)","root/c/d 4.txt(efgh)","root 4.txt(efgh)"]
Output: [["root/a/2.txt","root/c/d/4.txt","root/4.txt"],["root/a/1.txt","root/c/3.txt"]]

Example 2:

Input: paths = ["root/a 1.txt(abcd) 2.txt(efgh)","root/c 3.txt(abcd)","root/c/d 4.txt(efgh)"]
Output: [["root/a/2.txt","root/c/d/4.txt"],["root/a/1.txt","root/c/3.txt"]]

Constraints:

• 1 <= paths.length <= 2 * 104
• 1 <= paths[i].length <= 3000
• 1 <= sum(paths[i].length) <= 5 * 105
• paths[i] consist of English letters, digits, '/', '.', '(', ')', and ' '.
• You may assume no files or directories share the same name in the same directory.
• You may assume each given directory info represents a unique directory. A single blank space separates the directory path and file info.

Follow up:

• Imagine you are given a real file system, how will you search files? DFS or BFS?
• If the file content is very large (GB level), how will you modify your solution?
• If you can only read the file by 1kb each time, how will you modify your solution?
• What is the time complexity of your modified solution? What is the most time-consuming part and memory-consuming part of it? How to optimize?
• How to make sure the duplicated files you find are not false positive?

Problem Summary

Given a list of directory information, including directory paths and all files with contents in those directories, you need to find the paths of all duplicate file groups in the file system. A group of duplicate files includes at least two files with exactly the same content. A single directory info string in the input list has the following format: "root/d1/d2/.../dm f1.txt(f1_content) f2.txt(f2_content) ... fn.txt(fn_content)". This means there are n files (the contents of f1.txt, f2.txt ... fn.txt are respectively f1_content, f2_content ... fn_content) in the directory root/d1/d2/.../dm. Note: n>=1 and m>=0. If m=0, it means the directory is the root directory. The output is a list of duplicate file path groups. For each group, it contains all file paths of files with the same content. A file path is a string with the following format: "directory_path/file_name.txt"

Solution Approach

• This problem is considered easy and tests basic string operations and the use of a map. First, use string operations to obtain the directory path, file name, and file content. Then use a map to find duplicate files, where the key is the file content and the value is a list storing paths and file names. Traverse each file and add it to the map. Finally, traverse the map. If the length of the value list corresponding to a key is greater than 1, it means duplicate files have been found, and this list can be added to the final answer.
• The valuable part of this problem is in the Follow up. Interested readers can carefully think through the following questions:
  1. Suppose you have a real file system. How would you search the files? DFS or BFS?
  2. If the file content is very large (GB level), how would you modify your solution?
  3. If you can only read 1 kb of the file each time, how would you modify your solution?
  4. What is the time complexity of the modified solution? What are the most time-consuming part and the most memory-consuming part? How can they be optimized?
  5. How can you ensure that the duplicate files you find are not false positives?

Code

package leetcode

import "strings"

func findDuplicate(paths []string) [][]string {
	cache := make(map[string][]string)
	for _, path := range paths {
		parts := strings.Split(path, " ")
		dir := parts[0]
		for i := 1; i < len(parts); i++ {
			bracketPosition := strings.IndexByte(parts[i], '(')
			content := parts[i][bracketPosition+1 : len(parts[i])-1]
			cache[content] = append(cache[content], dir+"/"+parts[i][:bracketPosition])
		}
	}
	res := make([][]string, 0, len(cache))
	for _, group := range cache {
		if len(group) >= 2 {
			res = append(res, group)
		}
	}
	return res
}