Summary

Plan build fails if a plan checks out a Bitbucket Server or Cloud repository that contains git submodules (if submodules are enabled in advanced repository settings in Bamboo).

Diagnosis

You will find authentication errors in plan build logs. If the submodule has been declared using an SSH Git URL.

Repository not found
The requested repository does not exist, or you do not have permission to access it.
fatal: Could not read from remote repository.

If the Git submodule has been added with HTTPS URL:

fatal: Authentication failed for 'https://bitbucket-server.com/scm/vpe/adat.git/'
fatal: clone of 'https://bitbucket-server.com/scm/scm/vpe/adat.git' 
into submodule path '/opt/data/bamboo/xml-data/build-dir/MTP-MT-JOB1/submodule' failed

Or if it shows that the task was canceled in the build log:

Task was canceled - 'Checkout gen-sw Repository(1)' of type com.atlassian.bamboo.plugins.vcs:task.vcs.checkout

Cause

Bamboo tries to checkout submodules but fails because it does not have username/password (for https URLs) or SSH key (for SSH Git URLs). In case of submodules  the clone happens outside of Bamboo SSH proxy.

One of the other causes may be the ~/.git-credentials file has corrupted entries for username or password, or the file itself is missing from the Bamboo server/agent.

Solution

There are three solutions to this problem . The first solution implies changes to .gitmodules file and adding SSH key to repository settings. The second one is all about configuring SSH keys (or passwordless access for https repositories) on all agent nodes.

The third one is to update the ~/.git-credentials file to refresh the username or password or create a new one if it is not present.

Option 1: Use Relative Paths

1. To force Bamboo to use SSH proxy, use relative paths instead of HTTPS or SSH Git URLs.

   The following .gitmodules

   [submodule "module1"]
    path = submodule1
    url = ssh://git@bitbucket-server.com:7999/project1/repo1.git
   [submodule "submodule2"]
    path = submodule2
    url = ssh://git@bitbucket-server.com:7999/project1/repo2.git

   Should become:

   [submodule "module1"]
    path = submodule1
    url = ../repo1.git
   [submodule "submodule2"]
    path = submodule2
    url = ../repo2.git

   In this particular example the relative path is ../repo1 and ../repo2, thus we assume that the main repo is in the same Bitbucket project. If Bitbucket projects are different, a relative path should look like this: ../../projectName/repo.git

2. Having replaced URLs with relative paths, a public SSH key needs to be added to submodule repositories in Bitbucket.
   1. Copy SSH key from the advanced settings of the main repository in Bamboo:
      
   2. Save the key as Access Token in Bitbucket server repository settings:
      
   3. For Bitbucket Cloud you may skip Step 2 if you have already uploaded private key to Bamboo and added a public key to your profile when adding the main repository to a plan.

Option #2: Use Absolute URLs

If for some reason changes to .gitmodules are not possible and either https or ssh URLS are used, agent environments need to be configured to let Git clone the repositories without password prompts. Depending on the URL and the OS instructions vary.

Configuration needs to be performed on all remote agent machines (and Bamboo server machine as well if local agents are used).

Linux: HTTPS

echo "https://admin:password@git.mybitbucket.com" > ~/.git-credentials

The downside of this approach is that credentials are store in a plan text file which isn't secure. Also, this command needs to be run on all new agent machines (unless the job that uses Git repo with submodules has dedicated agents)

The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.

Linux: SSH

Either generate or copy id_rsa to ~/.ssh. Make sure the .ssh directory permissions are 700 (drwx------) and the private key (id_rsa) is 600 (-rw-------). Do not forget to add a public key as an access token to Bitbucket repository settings or user profile.

Again, the downside here is that a private key is stored on agent machines, and it needs to be copied to any new agent machines (unless dedicated agents are used).

Windows: HTTPS

There are two options here:

• use _netrc file
• recommended - use Git Credential Manager. Make sure GCM_INTERACTIVE is set to Never.

The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.

Windows: SSH

Follow instructions for Linux users. File and directory permission issues are possible. You may need to manually clone the repository to troubleshoot such issues.

Option #3: Update/create  the .git-credentials file 

• As the user which is running Bamboo ,please perform a git clone of the main repository(one which calls the submodules) in the Bamboo remote agent.

git clone http://bitbucket:7990/scm/bam/submodule_demo.git

• Navigate to the repository

cd submodule_demo

• Perform an update of the submodule by running the below command 

git submodule update --init --recursive

This prompts for password, please provide the username and password for the Bitbucket user which has access to those submodule repositories.

The Bitbucket repository username and password is now updated in the .git-credentials file if present. If not its creates a new .git-credentials file under the home directory in the Bamboo remote agent.

The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.