docs/Project-Restructured.md
Based on discussion in https://github.com/danielmiessler/fabric/issues/1127
This plan synthesizes the proposal by ksylvan with key clarifications and additions from jaredmontoya, eugeis, and others in the thread. The goal is to reorganize the project to align with standard Go conventions, reduce root-level clutter, and improve overall clarity for developers.
Revision 2 Changes: Added support for additional binary tools (code_helper and to_pdf) in the cmd/ directory structure.
The current project structure mixes application code, web assets, scripts, and configuration files at the top level. This creates several challenges:
cmd/ and private, non-reusable package code in internal/.nix, Dockerfile) are intermingled, obscuring the separation of concerns.The proposed restructure addresses these issues by organizing the project by function, adhering to community best practices.
This is the high-level view of the proposed structure. It incorporates the core plan and ensures that technically required files like flake.nix remain in the root directory.
.
├── cmd
│ ├── fabric
│ │ └── main.go # Main application entrypoint
│ ├── code_helper
│ │ ├── main.go # Code analysis helper tool
│ │ └── code.go # Supporting code for code_helper
│ └── to_pdf
│ └── main.go # LaTeX to PDF conversion tool (renamed from to_pdf.go)
├── internal
│ ├── cli # All CLI-related code
│ ├── core # Core application logic (e.g., chatter)
│ ├── domain # Domain types, moved from 'common'
│ ├── patterns # Logic for loading/managing patterns
│ ├── plugins # All plugin logic (ai, db, etc.)
│ ├── server # The 'restapi' code, renamed for clarity
│ ├── tools # Non-binary tool utilities (converter, jina, youtube, etc.)
│ └── util # Specific, shared utilities (to be used sparingly)
├── data
│ ├── patterns/ # All pattern markdown files
│ └── strategies/ # All strategy json files
├── scripts
│ ├── docker
│ │ ├── Dockerfile
│ │ ├── docker-compose.yml
│ │ ├── start-docker.sh # Helper script to start docker-compose stack
│ │ └── README.md # Docker deployment documentation
│ ├── python_ui
│ │ ├── streamlit.py
│ │ └── requirements.txt
│ ├── pattern_generation
│ │ ├── extract_patterns.py
│ │ └── ...
│ └── setup_fabric.bat # Windows setup script
├── docs
│ ├── images/
│ ├── NOTES.md
│ └── Pattern_Descriptions/ # Documentation about patterns
├── web/ # (Svelte frontend, unchanged)
├── completions/ # (Shell completions, unchanged)
├── nix/ # (Nix environment, unchanged)
├── go.mod
├── go.sum
├── LICENSE
├── README.md
├── .gitignore
├── .envrc # (Must remain in root for direnv)
├── flake.nix # (Must remain in root for Nix)
└── flake.lock # (Must remain in root for Nix)
✅ Introduction of cmd/ Directory
cmd/ subdirectories:
cmd/fabric/main.go - Main application entrypointcmd/code_helper/ - Code analysis helper tool (moved from plugins/tools/code_helper/)cmd/to_pdf/ - LaTeX to PDF conversion tool (moved from plugins/tools/to_pdf/)✅ Introduction of internal/ Directory
cli, core, plugins, restapi, common) are moved under internal/.internal directory can only be imported by code within the same repository. This makes the application's core logic private and prevents other projects from creating unintended dependencies on it, clarifying that the project is an application, not a public library.✅ Reorganizing and Renaming Packages
restapi -> internal/server: The package is renamed to describe its function (providing an HTTP server) rather than its implementation detail (REST).common: The common package has been broken up. Core data structures moved to a dedicated internal/domain package. Utility functions moved closer to the packages that use them, with truly shared utilities placed in internal/util.patterns logic: Code for loading and managing patterns consolidated into internal/patterns.plugins/tools -> internal/tools: Non-binary tool utilities (converter, jina, youtube, etc.) moved to internal/tools while binary tools moved to cmd/.✅ Consolidating Data, Scripts, and Docs
data/: The patterns/ and strategies/ directories, which are data assets consumed by the application, moved into a data/ directory to distinguish them from source code.scripts/: Helper scripts (Python, shell, batch files, Docker, etc.) grouped under scripts/ to clarify their role as auxiliary tools:
scripts/docker/ - Docker deployment files and helper scriptsscripts/python_ui/ - Streamlit UI and Python dependenciesscripts/pattern_generation/ - Pattern extraction and generation toolsdocs/: Miscellaneous markdown files (NOTES.md, etc.) and related assets like images moved to docs/ for better organization.✅ Create New Directories: Create the new top-level directories: cmd/fabric, cmd/code_helper, cmd/to_pdf, internal, data, scripts, and docs.
✅ Move Binary Tools:
plugins/tools/code_helper/ to cmd/code_helper/plugins/tools/to_pdf/to_pdf.go to cmd/to_pdf/main.go (rename file)plugins/tools/ to internal/tools/✅ Move Go Packages: Move the existing Go package directories (cli, core, plugins, restapi) into the new internal/ directory.
✅ Refactor and Rename Go Packages:
internal/restapi to internal/server.common package, moving its contents into appropriate new locations like internal/domain and internal/util.✅ Move Main Entry Point: Move main.go to cmd/fabric/main.go.
✅ Update Go Imports: This is a critical step. Use an IDE or tools like goimports to update all import paths in all .go files to reflect the new structure:
.../fabric/cli becomes .../fabric/internal/cli.../fabric/plugins/tools/... becomes .../fabric/internal/tools/...fabric, code_helper, to_pdf)✅ Move Data Assets: Move the patterns/ and strategies/ directories into the data/ directory. Update the application code to read from these new paths.
✅ Move Scripts and Docs:
Dockerfile, docker-compose.yml) to scripts/docker/ and create helper scripts and documentationstreamlit.py and requirements.txt) to scripts/python_ui/extract_patterns.py, etc.) to scripts/pattern_generation/setup_fabric.bat) and other helper scripts into scripts/NOTES.md and the images directory into docs/✅ Update Build and CI/CD Processes:
scripts/docker/Dockerfile to reference new locations./cmd/fabric, ./cmd/code_helper, ./cmd/to_pdf✅ Test and Validate:
go build ./cmd/fabric to ensure the main application compiles correctly.go build ./cmd/code_helper to ensure the code helper tool compiles correctly.go build ./cmd/to_pdf to ensure the PDF tool compiles correctly.go test ./...../cmd/fabric instead of the root directory.go install github.com/danielmiessler/fabric/cmd/fabric@latest works for all three tools.Status: All major restructuring tasks have been completed successfully as of the current PR.
Summary of Achievements:
fabric, code_helper, to_pdf) compile successfullyinternal/common package successfully dissolved into internal/domain and internal/utilRemaining Tasks (⚠️):
go install command verification - requires publishing/taggingI have a draft PR ready here: https://github.com/Homebrew/homebrew-core/pull/229472
The current Homebrew formula at https://raw.githubusercontent.com/ksylvan/homebrew-core/refs/heads/main/Formula/f/fabric-ai.rb will need to be updated to work with the new project structure:
Current formula build command:
def install
system "go", "build", *std_go_args(ldflags: "-s -w")
end
Required update for new structure:
def install
system "go", "build", *std_go_args(ldflags: "-s -w"), "./cmd/fabric"
end
Additional considerations:
main.go (which no longer exists)./cmd/fabricfabric, code_helper, to_pdf) could potentially be packaged, but the main fabric binary is the primary targetgo install commands for new structure:
# Main fabric tool
go install github.com/danielmiessler/fabric/cmd/fabric@latest
# Additional tools (if desired)
go install github.com/danielmiessler/fabric/cmd/code_helper@latest
go install github.com/danielmiessler/fabric/cmd/to_pdf@latest
The project now follows standard Go conventions and is ready for review and merge.