High CPU Usage caused by Ancestors Check in Confluence

Related content

  • No related content found

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

Corrupted entries in the application Ancestors Table (responsible for maintaining the relationship in between parent and descendant pages on Confluence) can cause View Page threads to run for a long period, triggering symptoms like High CPU Usage and intense GC Activity, which can eventually lead the application to an outage. Thread Dumps will usually show that the offending threads are executing the getInheritedContentPermissionSets (accountable for checking permissions) method being executed:

"http-nio-8090-exec-20 uri:/pages/viewpage.action username:" #274 daemon prio=5 os_prio=0 tid=0x00007f6484011000 nid=0x2e8b runnable [0x00007f61c350f000]
   java.lang.Thread.State: RUNNABLE
	at net.sf.hibernate.impl.AbstractVisitor.processValue(AbstractVisitor.java:67)
	at net.sf.hibernate.impl.AbstractVisitor.processValues(AbstractVisitor.java:36)
	at net.sf.hibernate.impl.SessionImpl.isUpdateNecessary(SessionImpl.java:2666)
	at net.sf.hibernate.impl.SessionImpl.flushEntity(SessionImpl.java:2581)
	at net.sf.hibernate.impl.SessionImpl.flushEntities(SessionImpl.java:2513)
	at net.sf.hibernate.impl.SessionImpl.flushEverything(SessionImpl.java:2308)
	at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1831)
	at net.sf.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:1594)
	at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1559)
	at net.sf.hibernate.impl.QueryImpl.list(QueryImpl.java:49)
	at com.atlassian.confluence.security.persistence.dao.hibernate.HibernateContentPermissionSetDao.lambda$getInheritedContentPermissionSets$0(HibernateContentPermissionSetDao.java:38)
	at com.atlassian.confluence.security.persistence.dao.hibernate.HibernateContentPermissionSetDao$$Lambda$766/775792763.doInHibernate(Unknown Source)
	at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
	at org.springframework.orm.hibernate.HibernateTemplate.executeFind(HibernateTemplate.java:341)
	at com.atlassian.confluence.security.persistence.dao.hibernate.HibernateContentPermissionSetDao.getInheritedContentPermissionSets(HibernateContentPermissionSetDao.java:25)

Diagnosis

This problem is triggered regardless of environmental variables, but its symptoms are usually the following:

  • High CPU Usage.
  • Garbage Collection Threads being heavily used.

Also, it is important to diagnose if there are corrupted entries in the Ancestors Table or not, since those are known to cause this issue.

Is my Ancestors Table corrupted?

In order to diagnose if there are indeed corrupted entries in the Ancestors Table, an administrator may run the query below at Confluence's database:

Is my Ancestors Table corrupted? 
SELECT A.ANCESTORID, A.DESCENDENTID, COUNT(A.DESCENDENTID)
FROM CONFANCESTORS A, CONTENT B
WHERE B.CONTENTID = A.DESCENDENTID
GROUP BY A.ANCESTORID, A.DESCENDENTID HAVING COUNT(A.DESCENDENTID) != 1
ORDER BY 3 DESC

In case it has returned any results, then it is corrupted indeed.

Cause

This is caused by corrupted entries in the application Ancestors Table - Responsible for maintaining the relationship in between parent and descendant pages on Confluence.

Workaround

The workaround consists of re-constructing the application Ancestors Table as described in Rebuilding the Ancestor Table documentation. Steps to do so have been listed below as well:

------------------------------------------------------------------------------------------------------------------

The ancestor table records the parent and descendant (child) relationship between pages. It is also used when determining whether a page will inherit view restrictions from a parent page. 

Occasionally records in the ancestor table can become corrupted. When this happens you will need to rebuild the ancestor table.

How to find out if the ancestor table needs to be rebuilt

Run the following query:

SELECT A.ANCESTORID, A.DESCENDENTID, COUNT(A.DESCENDENTID)
FROM CONFANCESTORS A, CONTENT B
WHERE B.CONTENTID = A.DESCENDENTID
GROUP BY A.ANCESTORID, A.DESCENDENTID HAVING COUNT(A.DESCENDENTID) != 1
ORDER BY 3 DESC

If this query returns any results, the ancestors table is broken and needs to be rebuilt.

Rebuild the ancestor table

To rebuild the ancestor table:

  1. Back up your database.
  2. Use the following URL to access the Page Level Permissions page 
    <your-site>/admin/permissions/pagepermsadmin.action

  3. Choose Rebuild ancestor table. 

  4. Go to  > General Configuration > Cache Management.
  5. Flush the Inherited Content Permissions cache. 
You may also need to rebuild the content index so that the permissions take effect in search results.

This process takes about ten minutes for an average size site, and you may not be able to save new content during this time. We recommend you take a full backup of your database and complete this process during a maintenance window.

Track progress of the rebuild

To track the progress of the ancestor table rebuild:

  1. Go to  > General Configuration > Logging & Profiling

  2. Add the following entry, setting the logging level to INFO

    com.atlassian.confluence.pages.ancestors.HibernatePageAncestorManager

    See Configuring Logging for more information. 

  3. Open <confluence_home>/logs/atlassian-confluence.log. You should now see additional ancestor table logging, similar to the example below:

    2018-07-25 20:48:56,202 INFO [http-nio-8666-exec-10] [confluence.pages.ancestors.HibernatePageAncestorManager] rebuild Started
2018-07-25 20:48:56,213 INFO [http-nio-8666-exec-10] [confluence.pages.ancestors.HibernatePageAncestorManager] clearHibernateCollectionCache Evicting the contents of Hibernate 'com.atlassian.confluence.pages.Page.ancestors' collection cache
2018-07-25 20:48:56,235 INFO [http-nio-8666-exec-10] [confluence.pages.ancestors.HibernatePageAncestorManager] apply Stored ancestors for child pages... 16/16
2018-07-25 20:48:56,236 INFO [http-nio-8666-exec-10] [confluence.pages.ancestors.HibernatePageAncestorManager] rebuild Ancestors persisted to database in 11 ms
2018-07-25 20:48:56,237 INFO [http-nio-8666-exec-10] [confluence.pages.ancestors.HibernatePageAncestorManager] rebuild Complete!

------------------------------------------------------------------------------------------------------------------

Resolution

There have been some suggestions to properly log and alert administrators when there are problems with the Ancestors Table, being the main one implemented already:

With proper logging, administrators can proactively rebuild the table upon need, however, this doesn't fix the cause behind why the table has been corrupted in first place and this is still pending a fix. Actions can be tracked using below bugs:


Last modified on Jan 30, 2025

Was this helpful?

Yes
No
Provide feedback about this article

Related content

  • No related content found
Powered by Confluence and Scroll Viewport.