Handling File

GoFr simplifies the complexity of working with different file stores by offering a uniform API. This allows developers to interact with different storage systems using the same set of methods, without needing to understand the underlying implementation details of each file store.

USAGE

By default, local file-store is initialized and user can access it from the context.

GoFr also supports FTP/SFTP file-store. Developers can also connect and use their cloud storage bucket as a file-store. Following cloud storage options are currently supported:

• AWS S3
• Google Cloud Storage (GCS)
• Azure File Storage

The file-store can be initialized as follows:

FTP file-store

package main

import (
	"gofr.dev/pkg/gofr"

	"gofr.dev/pkg/gofr/datasource/file/ftp"
)

func main() {
	app := gofr.New()

	app.AddFileStore(ftp.New(&ftp.Config{
		Host:      "127.0.0.1",
		User:      "user",
		Password:  "password",
		Port:      21,
		RemoteDir: "/ftp/user",
	}))

	app.Run()
}

SFTP file-store

package main

import (
	"gofr.dev/pkg/gofr"

	"gofr.dev/pkg/gofr/datasource/file/sftp"
)

func main() {
	app := gofr.New()

	app.AddFileStore(sftp.New(&sftp.Config{
		Host:     "127.0.0.1",
		User:     "user",
		Password: "password",
		Port:     22,
	}))

	app.Run()
}

AWS S3 Bucket as File-Store

To run S3 File-Store locally we can use localstack, docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

package main

import (
	"gofr.dev/pkg/gofr"

	"gofr.dev/pkg/gofr/datasource/file/s3"
)

func main() {
	app := gofr.New()

	// For temporary credentials (AWS STS), set Config.SessionToken.
	// BaseEndpoint is not necessary while connecting to AWS as it automatically resolves it on the basis of region.
	// However, in case we are using any other AWS compatible service, such like running or testing locally, then this needs to be set.
	// Note that locally, AccessKeyID & SecretAccessKey is not checked if we use localstack.
	app.AddFileStore(s3.New(&s3.Config{
		EndPoint:        "http://localhost:4566",
		BucketName:      "gofr-bucket-2",
		Region:          "us-east-1",
		AccessKeyID:     app.Config.Get("AWS_ACCESS_KEY_ID"),
		SecretAccessKey: app.Config.Get("AWS_SECRET_ACCESS_KEY"),
	}))

	app.Run()
}

Note: The current implementation supports handling only one bucket at a time, as shown in the example with gofr-bucket-2. Bucket switching mid-operation is not supported.

S3-Compatible Providers (Cloudflare R2, MinIO, and more)

The same s3 file store works with any S3-compatible object storage. These providers differ from AWS in a few details that matter for connecting and for signed URLs: the addressing style (path-style vs virtual-hosted), the region used to sign requests, and whether they accept the integrity checksums that the AWS SDK adds to uploads by default.

Set Config.Flavor to apply the right defaults for your provider:

Example — Cloudflare R2:

app.AddFileStore(s3.New(&s3.Config{
	Flavor:          s3.FlavorR2,
	EndPoint:        "https://<accountid>.r2.cloudflarestorage.com",
	BucketName:      "my-bucket",
	AccessKeyID:     app.Config.Get("R2_ACCESS_KEY_ID"),
	SecretAccessKey: app.Config.Get("R2_SECRET_ACCESS_KEY"),
}))

For a provider not listed above, use s3.FlavorGeneric together with the relevant overrides:

usePathStyle := true

app.AddFileStore(s3.New(&s3.Config{
	Flavor:          s3.FlavorGeneric,
	EndPoint:        "https://storage.example.com",
	BucketName:      "my-bucket",
	Region:          "us-east-1",
	UsePathStyle:    &usePathStyle, // overrides the flavor's addressing-style default
	AccessKeyID:     app.Config.Get("S3_ACCESS_KEY_ID"),
	SecretAccessKey: app.Config.Get("S3_SECRET_ACCESS_KEY"),
}))

Notes:

• Config.SessionToken is available for temporary credentials (AWS STS or scoped R2 API tokens).
• Config.UsePathStyle (a *bool) overrides the flavor's addressing default when set; leave it nil to use the flavor default.
• Signed URLs (see Cloud-Specific Operations) work across all flavors — the URL host follows the resolved addressing style so links resolve correctly.

Google Cloud Storage (GCS) Bucket as File-Store

Local Setup with fake-gcs-server:

1. Start fake-gcs-server with HTTP:

docker run -d --name fake-gcs-server -p 4443:4443 \
  fsouza/fake-gcs-server -scheme http -port 4443

2. Create a bucket:

curl -X POST http://localhost:4443/storage/v1/b?project=my-project-id \
  -H "Content-Type: application/json" \
  -d '{"name":"my-bucket"}'

3. Set environment variable in your configs/.env file:

STORAGE_EMULATOR_HOST=localhost:4443

4. Connect to GCS in your application:

package main

import (
	"gofr.dev/pkg/gofr"
	"gofr.dev/pkg/gofr/datasource/file/gcs"
)

func main() {
	app := gofr.New()

	// Local setup with fake-gcs-server (uses STORAGE_EMULATOR_HOST)
	app.AddFileStore(gcs.New(&gcs.Config{
		BucketName: "my-bucket",
		ProjectID:  "my-project-id",
	}))

	app.Run()
	app.Run()
}

Production Setup:

For production, authenticate using one of these methods:

// Option 1: Using GOOGLE_APPLICATION_CREDENTIALS environment variable
// Set: export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
app.AddFileStore(gcs.New(&gcs.Config{
	BucketName: "my-bucket",
	ProjectID:  "my-project-id",
}))

// Option 2: Using CredentialsJSON directly
credJSON, _ := os.ReadFile("gcs-credentials.json")
app.AddFileStore(gcs.New(&gcs.Config{
	BucketName:      "my-bucket",
	CredentialsJSON: string(credJSON),
	ProjectID:       "my-project-id",
}))

Note:

• When STORAGE_EMULATOR_HOST is set, the client automatically connects to the local emulator without authentication.
• For production, use either GOOGLE_APPLICATION_CREDENTIALS environment variable or CredentialsJSON config field
• Currently supports one bucket per file-store instance

Azure File Storage as File-Store

Azure File Storage provides fully managed file shares in the cloud. To use Azure File Storage with GoFr:

package main

import (
	"gofr.dev/pkg/gofr"
	"gofr.dev/pkg/gofr/datasource/file/azure"
)

func main() {
	app := gofr.New()

	// Create Azure File Storage filesystem
	fs, err := azure.New(&azure.Config{
		AccountName: "mystorageaccount",
		AccountKey:  "myaccountkey",
		ShareName:   "myshare",
		// Endpoint is optional, defaults to https://{AccountName}.file.core.windows.net
		// Endpoint: "https://custom-endpoint.file.core.windows.net",
	})

	if err != nil {
		app.Logger().Fatalf("Failed to initialize Azure File Storage: %v", err)
	}

	app.AddFileStore(fs)

	app.Run()
}

Note:

• Azure File Storage uses file shares (similar to S3 buckets or GCS buckets)
• Authentication requires both AccountName and AccountKey
• The Endpoint field is optional and defaults to https://{AccountName}.file.core.windows.net
• Currently supports one file share per file-store instance
• The implementation automatically retries connection if the initial connection fails
• Automatic parent directory creation: When creating files in nested paths (e.g., dir1/subdir/file.txt), parent directories are automatically created, matching local filesystem behavior
• Content type detection: Content types are automatically detected based on file extensions (e.g., .json → application/json, .txt → text/plain)

Cloud-Specific Operations

Beyond the standard filesystem interface, some cloud storage providers support richer capabilities — setting file metadata on upload and generating secure, time-limited download URLs. These are available through the CloudFileSystem interface.

Note: These operations are currently supported by Google Cloud Storage (GCS) and AWS S3 (including S3-compatible providers such as Cloudflare R2, MinIO, DigitalOcean Spaces, and Backblaze B2 — see S3-Compatible Providers). Other cloud providers may gain support in future releases.

Checking Cloud Support

Use file.AsCloud() to safely check whether the configured file store supports cloud-specific operations. This avoids a raw type assertion and returns a typed interface:

import "gofr.dev/pkg/gofr/datasource/file"

cfs, ok := file.AsCloud(c.File)
if !ok {
    return nil, file.ErrSignedURLsNotSupported
}

Uploading a File with Metadata

CreateWithOptions works like Create but lets you set a Content-Type, Content-Disposition, and arbitrary key-value metadata on the object at upload time:

f, err := cfs.CreateWithOptions(c, "reports/q1.csv", &file.FileOptions{
    ContentType:        "text/csv",
    ContentDisposition: `attachment; filename="q1.csv"`,
    Metadata: map[string]string{
        "uploaded-by":    "invoice-service",
        "report-quarter": "Q1-2026",
    },
})
if err != nil {
    return nil, err
}
defer f.Close()

_, err = f.Write(csvData)

Setting ContentDisposition ensures browsers download the file as an attachment rather than attempting to render it inline. Custom Metadata fields are stored on the object as user metadata (visible in the GCS console/gsutil output, or as x-amz-meta-* headers on S3 and S3-compatible providers).

Generating a Signed URL

GenerateSignedURL creates a time-limited, pre-authenticated URL that allows anyone with the link to download the file — no cloud credentials required on the client side:

url, err := cfs.GenerateSignedURL(c, "reports/q1.csv", 15*time.Minute, nil)
if err != nil {
    return nil, err
}

return url, nil

Pass FileOptions as the last argument to override the Content-Disposition header that the signed URL serves — useful when the object was uploaded without a disposition header but you want the browser to treat it as a download:

url, err := cfs.GenerateSignedURL(c, "reports/q1.csv", 1*time.Hour, &file.FileOptions{
    ContentDisposition: `attachment; filename="report.csv"`,
})

Note:

• On GCS, signed URLs use the parsed service-account key when CredentialsJSON is set, otherwise they fall back to IAM-based signing (which requires the iam.serviceAccounts.signBlob permission).
• On S3 and S3-compatible providers, the URL is signed locally with AWS Signature Version 4 using the configured access key — no extra permission is needed.
• The URL is pre-authenticated — anyone who has it can download the file until it expires.
• Expiry is measured from the moment GenerateSignedURL is called.
• file.AsCloud returns (nil, false) for local, FTP, and SFTP file stores — always check the ok result.

Creating Directory

To create a single directory

err := ctx.File.Mkdir("my_dir",os.ModePerm)

To create subdirectories as well

err := ctx.File.MkdirAll("my_dir/sub_dir", os.ModePerm)

Get current Directory

currentDir, err := ctx.File.Getwd()

Change current Directory

To switch to parent directory

currentDir, err := ctx.File.Chdir("..")

To switch to another directory in same parent directory

currentDir, err := ctx.File.Chdir("../my_dir2")

To switch to a subfolder of the current directory

currentDir, err := ctx.File.Chdir("sub_dir")

Note: This method attempts to change the directory, but S3's flat structure and fixed bucket make this operation inapplicable. Similarly, GCS uses a flat structure where directories are simulated through object prefixes. Azure File Storage supports directory operations natively, so Chdir works as expected.

Read a Directory

The ReadDir function reads the specified directory and returns a sorted list of its entries as FileInfo objects. Each FileInfo object provides access to its associated methods, eliminating the need for additional stat calls.

If an error occurs during the read operation, ReadDir returns the successfully read entries up to the point of the error along with the error itself. Passing "." as the directory argument returns the entries for the current directory.

entries, err := ctx.File.ReadDir("../testdir")

for _, entry := range entries {
    entryType := "File"

    if entry.IsDir() {
        entryType = "Dir"
    }

    fmt.Printf("%v: %v Size: %v Last Modified Time : %v\n", entryType, entry.Name(), entry.Size(), entry.ModTime())
}

Note: In S3 and GCS, directories are represented as prefixes of file keys/object names. This method retrieves file entries only from the immediate level within the specified directory. Azure File Storage supports native directory structures, so ReadDir works with actual directories.

Creating and Save a File with Content

file, _ := ctx.File.Create("my_file.text")

_, _ = file.Write([]byte("Hello World!"))

// Closes and saves the file.
file.Close()

Note for Azure File Storage:

• Files can be created in nested directories (e.g., dir1/subdir/file.txt). Parent directories are automatically created if they don't exist
• Content types are automatically detected based on file extensions (e.g., .json, .txt, .csv, .xml, .html, .pdf)
• This behavior matches local filesystem operations for consistency

Reading file as CSV/JSON/TEXT

GoFr support reading CSV/JSON/TEXT files line by line.

reader, err := file.ReadAll()

for reader.Next() {
	var b string

	// For reading CSV/TEXT files user need to pass pointer to string to SCAN.
	// In case of JSON user should pass structs with JSON tags as defined in encoding/json.
	err = reader.Scan(&b)

	fmt.Println(b)
}

Opening and Reading Content from a File

To open a file with default settings, use the Open command, which provides read and seek permissions only. For write permissions, use OpenFile with the appropriate file modes.

Note: In FTP, file permissions are not differentiated; both Open and OpenFile allow all file operations regardless of specified permissions.

csvFile, _ := ctx.File.Open("my_file.csv")

b := make([]byte, 200)

// Read reads up to len(b) bytes into b.
_, _ = file.Read(b)

csvFile.Close()

csvFile, err = ctx.File.OpenFile("my_file.csv", os.O_RDWR, os.ModePerm)

// WriteAt writes the buffer content at the specified offset.
_, err = csvFile.WriteAt([]byte("test content"), 4)
if err != nil {
     return nil, err
}

Getting Information of the file/directory

Stat retrieves details of a file or directory, including its name, size, last modified time, and type (such as whether it is a file or folder)

file, _ := ctx.File.Stat("my_file.text")
entryType := "File"

if entry.IsDir() {
     entryType = "Dir"
}

fmt.Printf("%v: %v Size: %v Last Modified Time : %v\n", entryType, entry.Name(), entry.Size(), entry.ModTime())

Note: In S3 and GCS:

• Names without a file extension are treated as directories by default.
• Names starting with "0" are interpreted as binary files, with the "0" prefix removed (S3 specific behavior).

For directories, the method calculates the total size of all contained objects and returns the most recent modification time. For files, it directly returns the file's size and last modified time.

Azure File Storage supports native file and directory structures, so Stat returns accurate metadata for both files and directories.

Rename/Move a File

To rename or move a file, provide source and destination fields. In case of renaming a file provide current name as source, new_name in destination. To move file from one location to another provide current location as source and new location as destination.

err := ctx.File.Rename("old_name.text", "new_name.text")

Deleting Files

Remove deletes a single file

Note: Currently, the S3 package supports the deletion of unversioned files from general-purpose buckets only. Directory buckets and versioned files are not supported for deletion by this method. GCS supports deletion of both files and empty directories. Azure File Storage supports deletion of both files and empty directories.

err := ctx.File.Remove("my_dir")

The RemoveAll command deletes all subdirectories as well. If you delete the current working directory, such as "../currentDir", the working directory will be reset to its parent directory.

Note: In S3, RemoveAll only supports deleting directories and will return an error if a file path (as indicated by a file extension) is provided for S3. GCS and Azure File Storage handle both files and directories.

err := ctx.File.RemoveAll("my_dir/my_text")

GoFr supports relative paths, allowing locations to be referenced relative to the current working directory. However, since S3 and GCS use a flat file structure, all methods require a full path relative to the bucket. Azure File Storage supports native directory structures, so relative paths work as expected with directory navigation.

Errors have been skipped in the example to focus on the core logic, it is recommended to handle all the errors.