Jira Service Management 10.3.x Long Term Support release performance report

We’re committed to continuously evolving and enhancing the quality of our performance reports. This report presents a comprehensive overview of the performance metrics for the most recent 10.3 Long Term Support (LTS) release of Jira Service Management Data Center.

Skip to:

• Summary
• Testing methodology
  • Testing methodology
  • Test environment
  • Test dataset
• Testing results
• Constant load test
• Multi-node comparison
• Assets archiving
• Further resources for scaling and optimizing

About Long Term Support releases

We recommend upgrading Jira Service Management regularly, but if your organization's process means you only upgrade about once a year, upgrading to a Long Term Support release is a good option. It provides continued access to critical security, stability, data integrity, and performance fixes until this version reaches end of life.

Summary

In the JSM 10.3 release, we’re showcasing the results of our performance testing through a new framework that focuses on Non-Functional Requirements (NFR) thresholds for essential user actions. These thresholds serve as benchmarks for reliability, responsiveness, and scalability, ensuring that our system adheres to the expected performance standards. We've successfully delivered features that significantly impact the product and consistently addressed performance regressions to pass all NFR tests. You can be confident that as the product evolves and you scale, we’ll remain dedicated to optimizing and maintaining the performance.

Testing methodology

Testing methodology

Our testing methodology incorporates both browser testing and REST API call assessments. Each test was scripted to execute actions based on the scenarios outlined above.

For the browser tests, we conducted 500 samples per test. The REST API tests were performed as soak tests over a duration of one hour. Upon completion of all browser and REST API tests, we gathered instance metrics and statistics for analysis.

Test environment

The performance tests were all run on a set of AWS EC2 instances, deployed in the ap-southeast-2 region. The tested instance was an out-of-the-box fresh installation setup as 5 nodes and without any additional configuration. This approach allowed us to ensure satisfying performance in a basic setup with more nodes adding additional performance gains if necessary.

Below, you can check the details of the environments used for Jira Service Management Data Center and the specifications of the EC2 instances.

Database

Load generator

Test dataset

Before we started testing, we needed to determine what size and shape of the dataset represents a typical extra large Jira Service Management instance. To achieve that, we created a new dataset that more accurately matches the instance profiles of Jira Service Management in very large organizations.

The data was collected based on the anonymous statistics received from real customer instances. A machine-learning (clustering) algorithm was used to group the data into small, medium, large, and extra-large instances data shapes. For our tests, we decided to use median values gathered from extra-large instances.

The following table presents the dataset we used in our tests. It simulates a data shape typical in our extra-large customer cohort.

Testing results

NFR tests

This year, we've introduced a new set of test results based on a framework that focuses on Non-Functional Requirements (NFR) thresholds for key user actions.

We've established a target threshold for each measured action. These thresholds, set according to the action type, serve as benchmarks for reliability, responsiveness, and scalability, ensuring that our product meets the expected performance standards. We're committed to ensuring that we don’t deliver performance regressions, maintaining the quality and reliability of our product.

The measured performance of the action was defined as the time from the beginning of an action until the action is performed and the crucial information is visible. Thanks to this approach, we can more closely measure the performance as it's perceived by the end user.

The following interpretations of the response times apply:

• 50th percentile - Gives a clear understanding of the average performance for most users in extra-large instances. It's less affected by extreme outliers, so it shows the central tendency of response times.

• 90th percentile - Highlights performance for worst-case scenarios, which may affect a smaller, but still noteworthy, portion of users in the extra-large instances.

Note that:

• We routinely carry out internal performance testing to ensure that all actions remain within the NFR thresholds. Our continuous monitoring enables us to optimize performance and effectively address any regressions.

• All actions had a 0% error rate, demonstrating strong system reliability and stability. For the list of bugs resolved since the previous 5.12 LTS, refer to the Jira Service Management 10.3 LTS release notes.

The results overall are as follows:

Constant load test

For this test, a constant load was sustained for one hour. The test started with 0 concurrent users, gradually increasing to a peak of approximately 160 users over a span of 15 minutes. The users were highly active, performing high-load actions above the standard level. All actions were executed concurrently, and later categorized into three groups: help seeker, agent, and assets, illustrating the expected performance for each persona. The results are as follows:

• No regressions were observed in overall performance or hardware metrics.

• No significant error detected during execution.

• Throughout one hour of sustained load, response times remained stable for all actions. Note that:

  • Spike in response time at the beginning are due to start of tests.

  • Small spikes observed in the latter half of the sustained load period are due to garbage collection activities. The system recovers quickly from these spikes which are considered acceptable and expected.

• CPU and memory usage consistently held steady across all actions.

Help seeker actions

Agent actions

Assets actions

Multi-node comparison

For this test, the number of concurrent users were gradually increased from 0 to 560 virtual users over a span of 15 minutes, followed by a sustained load for one hour, with environments with 3, 5, and 7 nodes.

The results indicate significant performance improvements between instances with 3 and 5 nodes. An instance with 5 nodes responds faster, uses less CPU while keeping the same throughput (hits/s). The occurrence of timeouts also notably decreases on the 5-node instance. However, the performance difference between 5 and 7 nodes is marginal. Therefore, for the scenario and instance sizes simulated, opting for a 5-node configuration is deemed more cost-effective.

Assets archiving

In Jira Service Management 5.15, we introduced the ability to archive Assets objects. You can now declutter your instance and improve the performance of object searches by archiving the assets you no longer actively need. Previously, you had to permanently delete objects when your available index memory was low to make room for new objects. Now, you can archive objects instead. More about archiving objects

Here are some of the observed effects of effectively using Assets archiving:

• Archiving objects can enhance the performance of operations involving all objects by approximately 20%. In a scenario where there are 2 million objects, with 600,000 of them archived, processes like Assets clean reindexing and resource-intensive AQL queries (for example the anyAttribute query) experience a 20% increase in speed.

• Average CPU usage is up to 30% less when reindexing and running AQL searches with archived objects.

• Total memory usage decreased by 35% when running Assets indexing with archived objects.

Further resources for scaling and optimizing

If you'd like to learn more about scaling and optimizing JSM, here are a few additional resources that you can refer to.

Jira Service Management guardrails

Product guardrails are data-type recommendations designed to help you identify potential risks and aid you in making decisions about the next steps in your instance optimization journey. Learn more about JSM guardrails

Jira Enterprise Services

For help with scaling JSM in your organization directly from experienced Atlassians, check out our Additional Support Services offering.

Atlassian Experts

The Atlassian Experts in your local area can also help you scale Jira in your own environment.