SSH operations fail after upgrading Bitbucket Server/DC or Bamboo Server/DC
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
SSH operations fail after upgrading Bitbucket to 7.21.x or Bamboo 9.2.12 or later. For example, a git clone over SSH fails with the below error
git clone <SSH_CLONE_URL>
Cloning into 'repository'...
Connection reset by 10.138.15.76 port 7999
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Environment
- Bitbucket 7.21.18, 7.21.16, 8.9.7
- Bamboo 9.2.11 and later - with a Java SSH client such as JGIt or Apache-SSHD
- Java JDK version:
- < Java 8u251
- < Java 11.0.5
Diagnosis
Bitbucket
Enable SSH debug logging on the Bitbucket server and replicate the issue by cloning the repository over SSH:
curl -u <admin_user>:<admin_user_password> -v -X PUT -d "" -H "Content-Type: application/json" <BITBUCKET_BASE_URL>/rest/api/latest/logs/logger/org.apache.sshd/debugThe below log messages are seen in the Bitbucket server log (atlasssian-bitbucket.log)
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.sshd.common.io.nio2.Nio2Session handleReadCycleFailure(Nio2Session[local=/XX.XX.XX.XX:7999, remote=/10.133.0.31:51310]) NoSuchProviderException after 5950866 nanos at read cycle=2: JCE cannot authenticate the provider BC
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.sshd.common.io.nio2.Nio2Session exceptionCaught(Nio2Session[local=/XX.XX.XX.XX:7999, remote=/10.133.0.31:51310]) caught NoSuchProviderException[JCE cannot authenticate the provider BC] - calling handler
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.s.c.s.h.SessionTimeoutListener sessionException(ServerSessionImpl[null@/XX.XX.XX.XX:<PORT>]) NoSuchProviderException: JCE cannot authenticate the provider BC
To disable SSH debug logging:
curl -u <admin_user>:<admin_user_password> -v -X PUT -d "" -H "Content-Type: application/json" <BITBUCKET_BASE_URL>/rest/api/latest/logs/logger/org.apache.sshd/warnBamboo via a Checkout task
Bamboo may report an issue similar to the one reported on Bitbucket in the Server or Agent logs while running a Checkout task:
WARN [sshd-SshServer[5b523e53](port=0)-nio2-thread-3] [LoggingUtils] exceptionCaught(ServerSessionImpl[null@/127.0.0.1:63021])[state=Opened] NoSuchProviderException: JCE cannot authenticate the provider BCCause
Bitbucket started with an older JDK8 version. The issue is caused by JDK bug JDK-8214096 which is fixed in higher JDK versions.
Bamboo uses an Apache-SSHD process to proxy the Git authentication that is also affected by JDK bug JDK-8214096.
Solution
Upgrade the JDK version used by Bitbucket and Bamboo.
Fixed JDK versions as per JDK-8214096:
| Fixed Version | |
|---|---|
| Java 8 | >=JDK8u251b04 >=JDK8u261b01 |
| Java 11 | 11.0.5 b01 |