ContextQMD
Back to Gitlabhq

Use a project as a Go package

doc/user/project/use_project_as_go_package.md

18.11.28.8 KB
Original Source

{{< details >}}

  • Tier: Free, Premium, Ultimate
  • Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

{{< /details >}}

{{< history >}}

  • Changed in GitLab 17.3 to return 404 errors for unauthorized go get requests.

{{< /history >}}

Prerequisites:

  • To use a private project in a subgroup as a Go package, you must authenticate Go requests. Go requests that are not authenticated cause go get to fail. You don't need to authenticate Go requests for projects that are not in subgroups.

To use a project as a Go package, use the go get and godoc.org discovery requests. You can use the meta tags:

[!note] If you make a go get request with invalid HTTP credentials, you receive a 404 error. You can find the HTTP credentials in ~/.netrc (MacOS and Linux) or ~/_netrc (Windows). In Go 1.24 and later, you can also use the GOAUTH environment variable to provide credentials. For more information, see authenticate with GOAUTH.

Authenticate Go requests to private projects

Prerequisites:

  • Your GitLab instance must be accessible with HTTPS.
  • You must have a personal access token with read_api scope.

Authenticate with GOAUTH

In Go 1.24 and later, use the GOAUTH environment variable to provide credentials with a custom command.

[!note] The git dir value for GOAUTH does not work for private projects in nested subgroups with a depth greater than 1. Use a custom command instead.

To authenticate with GOAUTH, create a custom command that adds an HTTP Basic authentication header to Go requests. The following example uses your Git over HTTPS credentials, returned by git credential fill, to authenticate requests to gitlab.com:

shell
#!/usr/bin/env bash
GITLAB_URL="https://gitlab.com"

creds=$(echo "url=${GITLAB_URL}" | git credential fill 2>&1) || {
  printf >&2 'error: git credential fill failed:\n%s\n' "$creds"
  exit 1
}

username=""
password=""
while IFS='=' read -r key value; do
  case "$key" in
    username) username="$value" ;;
    password) password="$value" ;;
  esac
done <<< "$creds"

if [ -z "$username" ] || [ -z "$password" ]; then
  printf >&2 'error: git credential fill did not return a username or password for %s\n' "$GITLAB_URL"
  exit 1
fi

encoded=$(printf '%s:%s' "$username" "$password" | base64 | tr -d '\n')

# Expected output format: https://pkg.go.dev/cmd/go@master#hdr-GOAUTH_environment_variable
printf '%s\n\nAuthorization: Basic %s\n\n' "$GITLAB_URL" "$encoded"

To use this script:

  1. Save the script to a file, for example gitlab_goauth.sh.

  2. Make the file executable:

    shell
    chmod +x gitlab_goauth.sh

  3. Set the GOAUTH environment variable to use your command:

    shell
    export GOAUTH="command <absolute_path_to_your_command>"

Alternatively, to use your existing .netrc file with GOAUTH:

shell
export GOAUTH="netrc"

Authenticate with .netrc

To authenticate Go requests with a .netrc file, create the file with the following information:

plaintext
machine gitlab.example.com
login <gitlab_user_name>
password <personal_access_token>

On Windows, Go reads ~/_netrc instead of ~/.netrc.

The go command does not transmit credentials over insecure connections. It authenticates HTTPS requests made by Go, but does not authenticate requests made through Git.

Authenticate Git requests

If Go cannot fetch a module from a proxy, it uses Git. Git uses a .netrc file to authenticate requests, but you can configure other authentication methods.

Configure Git to either:

  • Embed credentials in the request URL:

    shell
    git config --global url."https://${user}:${personal_access_token}@gitlab.example.com".insteadOf "https://gitlab.example.com"

  • Use SSH instead of HTTPS:

    shell
    git config --global url."[email protected]:".insteadOf "https://gitlab.example.com/"

Disable Go module fetching for private projects

To fetch modules or packages, Go uses the environment variables:

  • GOPRIVATE
  • GONOPROXY
  • GONOSUMDB

To disable fetching:

  1. Disable GOPRIVATE:
    • To disable queries for one project, disable GOPRIVATE=gitlab.example.com/my/private/project.
    • To disable queries for all projects on GitLab.com, disable GOPRIVATE=gitlab.example.com.
  2. Disable proxy queries in GONOPROXY.
  3. Disable checksum queries in GONOSUMDB.
  • If the module name or its prefix is in GOPRIVATE or GONOPROXY, Go does not query module proxies.
  • If the module name or its prefix is in GOPRIVATE or GONOSUMDB, Go does not query Checksum databases.

Authenticate Git requests to private subgroups

If the Go module is located under a private subgroup like gitlab.com/namespace/subgroup/go-module, then the Git authentication doesn't work. It happens, because go get makes an unauthenticated request to discover the repository path. Without authentication, GitLab responds with gitlab.com/namespace/subgroup.git to prevent a security risk of exposing the project's existence for unauthenticated users. As a result, the Go module cannot be downloaded.

You can configure Go authentication to download Go modules in private subgroups.

Workaround: use .git in the module name

There is a way to skip go get request and force Go to use a Git authentication directly, but it requires a modification of the module name. From Go documentation:

If the module path has a VCS qualifier (one of .bzr, .fossil, .git, .hg, .svn) at the end of a path component, the go command will use everything up to that path qualifier as the repository URL. For example, for the module example.com/foo.git/bar, the go command downloads the repository at example.com/foo.git using Git, expecting to find the module in the bar subdirectory.

  1. Go to go.mod of the Go module in a private subgroup.
  2. Add .git to the module name. For example, renamemodule gitlab.com/namespace/subgroup/go-module to module gitlab.com/namespace/subgroup/go-module.git.
  3. Commit and push this change.
  4. Visit Go projects that depend on this module and adjust their import calls. For example, import gitlab.com/namespace/subgroup/go-module.git.

The Go module should be correctly fetched after this change. For example, GOPRIVATE=gitlab.com/namespace/* go mod tidy.

Fetch Go modules from Geo secondary sites

Use Geo to access Git repositories that contain Go modules on secondary Geo servers.

You can use SSH or HTTP to access the Geo secondary server.

Use SSH to access the Geo secondary server

To access the Geo secondary server with SSH:

  1. Reconfigure Git on the client to send traffic for the primary to the secondary:

    shell
    git config --global url."[email protected]".insteadOf "https://gitlab.example.com"
git config --global url."[email protected]".insteadOf "http://gitlab.example.com"
    • For gitlab.example.com, use the primary site domain name.
    • For gitlab-secondary.example.com, use the secondary site domain name.

  2. Ensure the client is set up for SSH access to GitLab repositories. You can test this on the primary, and GitLab replicates the public key to the secondary.

The go get request generates HTTP traffic to the primary Geo server. When the module download starts, the insteadOf configuration sends the traffic to the secondary Geo server.

Use HTTP to access the Geo secondary

You must use persistent access tokens that replicate to the secondary server. You cannot use CI/CD job tokens to fetch Go modules with HTTP.

To access the Geo secondary server with HTTP:

  1. Add a Git insteadOf redirect on the client:

    shell
    git config --global url."https://gitlab-secondary.example.com".insteadOf "https://gitlab.example.com"
    • For gitlab.example.com, use the primary site domain name.
    • For gitlab-secondary.example.com, use the secondary site domain name.

  2. Generate a personal access token and add the credentials in the client's ~/.netrc file:

    shell
    machine gitlab.example.com login USERNAME password TOKEN
machine gitlab-secondary.example.com login USERNAME password TOKEN

The go get request generates HTTP traffic to the primary Geo server. When the module download starts, the insteadOf configuration sends the traffic to the secondary Geo server.