Troubleshooting Remote Agents
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Technical Overview of Remote Agent Connectivity Mechanism
- The remote agent and server communication is done through HTTP and JMS protocols, on top of separate unicast TCP connections.
- The remote agent and server communication is bi-directional, and the remote agent initiates both TCP connections.
- Remote agents use the HTTP protocol while bootstrapping and registering against Bamboo.
- Artifacts are also transferred using the HTTP protocol.
The JMS protocol is used for other communications (such as dispatching jobs, heartbeat messages, etc) together with the Apache ActiveMQ (AMQ) message broker.
This article discusses Troubleshooting Remote Agents (secure and unsecured). For information on how to secure remote agents, please go to Securing your remote agents.
HTTP Connection
The remote agent normally connects to Bamboo using the same URL used for general user access.
| Examples of addresses | Port | Context path (not mandatory) | Protocol |
|---|---|---|---|
| http://<bamboo.hostname>:8085/agentServer/ | 8085 | <empty> | HTTP |
| https://<bamboo.hostname>/bamboo/agentServer/ | 443 | bamboo | HTTPS |
| http://<bamboo.hostname>/bamboo/agentServer/ | 80 | bamboo | HTTP |
JMS Connection
This connection will be established to the address configured in the <bamboo-local-home>/bamboo.cfg.xml file. There are two properties defined here:
| Property | Explanation |
|---|---|
bamboo.jms.broker.client.uri | The broker client URI is the address distributed to the agents. This is the endpoint they will attempt to connect to. |
bamboo.jms.broker.uri | The broker URI is the address Bamboo will use to bind the primary JMS connector.
|
- Any changes to these properties will require a Bamboo restart.
- The address should NOT be set to localhost. Ideally, the agent should reach the server directly to its IP or hostname. If a reverse proxy address is set here, please ensure the proxy is correctly configured to redirect the broker TCP traffic.
- Bamboo also spins up two additional JMS connectors to handle SSL and elastic agent traffic. Bamboo looks at the primary JMS connector port number to determine the port numbers it will use for the additional connectors. This is the formula used by Bamboo:
- SSL JMS connector = primary JMS connector port number + 1. If the primary JMS connector is 54663, then Bamboo will use 54664 for the SSL JMS connector.
- Elastic agents JMS connector = primary JMS connector port number + 2. If the primary JMS connector is 54663, then Bamboo will use 54665 for the elastic agents JMS connector. This connector is bound to localhost, by default.
| Examples of addresses | Port | Protocol |
|---|---|---|
| tcp://<bamboo.hostname>:54663 | 54663 | TCP (JMS) |
| ssl://<bamboo.hostname>:54664 | 54664 | TCP (JMS - secure) |
| nio://<bamboo.hostname>:54663 | 54663 | TCP (JMS - buffered TCP) |
Diagnosis
The commands below can be used for primary connectivity checks and should succeed if the correct addresses and ports are used.
| Agent-side checks | Command | Examples | Related articles |
|---|---|---|---|
| ping <Server_Hostname> | ping my.bamboo.url | |
| telnet <Server_Hostname> <HTTP_Port> | telnet my.bamboo.url 8085 | |
| ping <Server_Hostname or IP_address> | ping 192.168.10.10 | |
| telnet <Server_Hostname or IP_address> <JMS_Port> | telnet 192.168.10.10 54663 | |
| Server-side checks | Command | Examples | |
| ping <Server_Hostname or IP_address> | ping bamboo.hostname | |
| Windows: netstat -an | findstr <HTTP_Port> | netstat -an | findstr 8085 netstat -an | grep 8085 | |
| Windows: netstat -an | findstr <Broker_Port> Unix Like: netstat -an | grep <Broker_Port> | netstat -an | findstr 54663 netstat -an | grep 54663 |
Increasing debug logging
You'll find instructions on how to enable debug logging for Bamboo and/or agents inside the following page:
Here's a list of classes that can be set to DEBUG inside Bamboo and/or agents to help troubleshoot the communication between Bamboo and your agents:
| Package | Agent / Server | What you will see in the logs |
|---|---|---|
com.atlassian.bamboo.trigger.quartz | SERVER | Log messages for all scheduled Plan triggers. |
com.atlassian.utils.process | AGENT SERVER | All external commands started by Bamboo including complete information about environment. |
com.atlassian.bamboo.v2.trigger.DependencyChainListener | SERVER | Will provide information about which plans will be considered to run as a result of a build |
com.atlassian.bamboo.plugins.stash.events | SERVER | All events coming from Stash to Bamboo (branch creation/deletion, commits). |
com.atlassian.bamboo.plan.PlanExecutionManagerImpl | SERVER | Additional information about progress of a build through a build pipeline. |
com.atlassian.bamboo.chains.ChainExecutionManagerImpl | SERVER | Additional information about progress of a build through a build pipeline. |
com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl | AGENT | Main agent loop. |
com.atlassian.bamboo.v2.build.task.InitializeBuild | AGENT | Activities run before regular tasks are run (creating / cleaning working directory) |
com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask | AGENT | Source checkout. |
com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask | AGENT | Everything before and after running a task. |
com.atlassian.bamboo.task.TaskExecutorImpl | AGENT | Running a task |
com.atlassian.bamboo.v2.build.queue.BambooMessageConverter | AGENT SERVER | Details about messages sent through JMS. |
com.atlassian.bamboo.v2.build.agent.BambooAgentMessageListener | SERVER | Deserialization of agent communication on the server side. |
com.atlassian.bamboo.plugins.git.GitCommandProcessor | AGENT SERVER | Output of selected git commands |
com.atlassian.bamboo.v2.trigger.DefaultChangeDetectionManager | SERVER | Details about change detection (times, etc) |
com.atlassian.bamboo.migration | SERVER | Export / import details |
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl | SERVER | Additional information about automatic plan branch management |
net.schmizz.sshj | AGENT SERVER | SSH/SCP task |
org.apache.sshd.client.session | AGENT SERVER | Bamboo -> git server (ie Stash) connection initialization |
org.apache.sshd.common.channel | AGENT SERVER | Bamboo -> git server communication (lots of data) |
com.atlassian.bamboo.build.expiry | SERVER | Build expiry activity |
com.atlassian.bamboo.deployments.expiry | SERVER | Deployment expiry |
com.atlassian.bamboo.plugins.ssh |
| SSH Proxy (set to DEBUG) |
org.apache.sshd.server.session |
| SSH Proxy (set to ALL).  Very, very verbose, so use with caution and as briefly as possible to avoid flooding the logs. |
Solution
Bamboo with a reverse proxy
If Bamboo is running behind a reverse proxy and the remote agent is having trouble connecting with Bamboo (in the first stages of connection - using HTTP protocol) or experiencing artifact transfer problems, it might be worth bypassing the proxy to check if it could be causing the issue. The following document can be used for this purpose:
Other common causes for connectivity issues
Some common reasons for port and socket connectivity issues include:
- Firewalls
- Antivirus
- Forward proxies
Please check the above to see if any are interfering or blocking required ports and connections.