Problem

Remote agent disconnected due to heartbeat timeout.

The following appears in the remote agent atlassian-bamboo.log

2018-07-19 01:23:44,249 FATAL [scheduler_Worker-10] [AgentContext] Agent encountered a fatal error: 'The last successful transmission was 11 minutes ago, the heartbeat timeout is 10 minutes, terminating agent'. Exiting.
java.util.concurrent.TimeoutException: The last successful transmission was 11 minutes ago, the heartbeat timeout is 10 minutes, terminating agent

Diagnosis

Check CPU and RAM usage in the machine hosting Bamboo server and remote/elastic agent.

Cause

1. Multiple or all agents disconnected at the same time
   1. Bamboo server running out of heap memory or CPU, causing it fail to connect to remote/elastic agents.
   2. There is a remote/elastic agent producing huge build log which take a lot of Bamboo server resources to process it causing other remote/elastic agents to be disconnected.
2. Remote/elastic agent disconnected when running build - CPU or RAM usage too high in the machine hosting remote/elastic agent causing it fail to connect to Bamboo server.

Resolution

1. Multiple or all agents disconnected at the same time
   
   1. Increase heap memory and make sure that machine specification meet requirements.
   2. Reduce the huge build log size for the related plan
      1. Windows - Check huge build log size in the machine hosting Bamboo <BAMBOO_HOME>\xml-data\builds\<plan-idNumber>\download-data\build_logs

      2. Linux - Run the following command in the machine hosting Bamboo: 

         find <BAMBOO_HOME>/xml-data/builds -type f -name "*.log" -mtime -30 -exec du -h --time {} \; | sort -rh | head -50

      3. From the idNumber, run the following SQL query to look for the related plan: 

         SELECT bd.BUILD_ID, pr.TITLE, bd.TITLE 
         FROM BUILD bd
         JOIN PROJECT pr ON pr.PROJECT_ID = bd.PROJECT_ID
         WHERE BUILD_ID = <idNumber>;

2. Remote/elastic agent disconnected when running build - increase the machine specification of the remote/elastic agent so that CPU or RAM usage is not fully utilised in the build.