HealthCheck: Cluster Time Synchronization

Still need help?

The Atlassian Community is here for you.

Ask the community

Overview

This check will confirm if the system clocks on each node are set within five seconds of each other. JIRA Data Center requires that the system time of each of the nodes are within five seconds for predictable behaviour. Environments lacking clock synchronisation may experience similar problems as described in the following KB: 

This scheduler instance (<nodename>) is still active but was recovered by another instance in the cluster

Understanding the Results

IconResultWhat this means
The health check passed successfully.All nodes report a System Time within 5 seconds
System clocks of nodes are not synchronized. Difference: milliseconds.The times between nodes are out of sync and require fixing.

Troubleshooting

ProblemSuggestion
A Network Time Protocol (NTP) server has been configured.

Ideally, when working with multi-node environments, it is best practices to set the system clock through the use of a NTP server. Consult your OS vendor and Sys Admin for more guidance on configuring an NTP server, examples below.


A NTP server cannot be configured.Manually set the time of nodes to be in-sync and use other methods of keeping the times in synchronisation.
Different NTP servers are being used.

Configure your environment to use the same NTP servers.



Database issues are causing slowness during the check.

The time stamp to compare node cluster times is databaseSystemTimeReaderFactory.getReader().getDatabaseSystemTimeMillis().In other words, the check is using the database server time for each node for the comparison and must make a database request to confirm from within each node. If there are database issues resulting in long execution times, the check will fail.

Cluster Time Health Check
    protected SupportHealthStatus doCheck() {
        CompositeHealthStatus healthStatus = new CompositeHealthStatus(Application.JIRA, getHelpUrl(HELP_PATH), clock);

        try {
            final long currentDatabaseTime = databaseSystemTimeReaderFactory.getReader().getDatabaseSystemTimeMillis();
            final Set<String> activeNodes = getActiveNodesKeys();







Providing Information to Support

In case you are unable to troubleshoot and fix the problem by yourself, please create a support ticket at support.atlassian.com and attach the following information to the ticket:

  • Take a Screenshot of the Health Check results.
  • Collect a Support ZIP from each of the Data Centre nodes.
  • Any collected information from the suggestions in this document.

Last modified on Oct 25, 2021

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.