Upgrading hangs when generating pre-upgrade recovery file
Problem
The following appears in the atlassian-confluence.log:
2014-05-02 12:28:46,220 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 5.5 [build 5501 based on commit hash a39771cd4bb7ab00a226a7ea8d6dd65e64226a7f]
2014-05-02 12:28:48,451 WARN [localhost-startStop-1] [atlassian.config.xml.AbstractDom4jXmlConfigurationPersister] saveDocumentAtomically Unable to move C:\Program Files\Atlassian\Application Data\Confluence\confluence.cfg.xml1232542810330726807tmp to C:\Program Files\Atlassian\Application Data\Confluence\confluence.cfg.xml. Falling back to non-atomic overwrite.
2014-05-02 12:28:48,482 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization started
2014-05-02 12:28:51,711 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] <init> Loading EhCache cache manager
2014-05-02 12:29:01,524 WARN [localhost-startStop-1] [org.springframework.beans.TypeConverterDelegate] findDefaultEditor PropertyEditor [com.sun.beans.editors.EnumEditor] found through deprecated global PropertyEditorManager fallback - consider using a more isolated form of registration, e.g. on the BeanWrapper/BeanFactory!
2014-05-02 12:29:08,575 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization completed in 20093 ms
2014-05-02 12:29:08,668 INFO [localhost-startStop-1] [confluence.upgrade.impl.DefaultUpgradeManager] beforeUpgrade Generating pre-upgrade recovery file...
2014-05-02 12:29:08,699 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] begin []
2014-05-02 12:29:08,699 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] begin DATABASE_INFORMATION: []
2014-05-02 12:29:08,715 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] end DATABASE_INFORMATION: []
...
...
...
2014-05-02 12:29:10,961 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] end TABLE_DATA: [os_user_group]
2014-05-02 12:29:10,961 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] begin TABLE_DATA: [pagetemplates]
2014-05-02 12:29:10,961 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] end TABLE_DATA: [pagetemplates]
2014-05-02 12:29:10,961 INFO [localhost-startStop-1] [confluence.upgrade.recovery.ConfluenceProgressMonitor] begin TABLE_DATA: [plugindata]
2014-05-02 12:35:55,099 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 5.5 [build 5501 based on commit hash a39771cd4bb7ab00a226a7ea8d6dd65e64226a7f] In the latest catalina logs, you'll also see:
SEVERE: A child container failed during start
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
...
Caused by: java.lang.OutOfMemoryError: Java heap space
...
at com.atlassian.dbexporter.node.stax.StaxStreamWriter$1.setContentAsBinary(StaxStreamWriter.java:131)
...
at com.atlassian.dbexporter.DbExporter.exportData(DbExporter.java:53)
at com.atlassian.confluence.upgrade.recovery.DefaultRecoveryFileGenerator.generate(DefaultRecoveryFileGenerator.java:76)
at com.atlassian.confluence.upgrade.impl.DefaultUpgradeManager.beforeUpgrade(DefaultUpgradeManager.java:237)
at com.atlassian.confluence.upgrade.AbstractUpgradeManager.upgrade(AbstractUpgradeManager.java:134)
Cause
As of 5.5, the first thing that occurs when an upgrade is triggered is the generation of a recovery file, which can be used to restore your instance to the state that it was in just right at the point that the upgrade was started. It appears that this process is currently being affected by: CONF-33554 - Getting issue details... STATUS
Resolution
There is no need to rollback the instance, as the upgrade has not actually started. Implement one of the fixes below and then restart your instance.
Method 1 (ideal for large instances)
If you are taking your own backups prior to attempting any upgrades then the best option is to disable this functionality, particularly if your Confluence instance is large or has many plugins installed.
To do so, please follow the instructions in Configuring System Properties to set -Dconfluence.upgrade.recovery.file.enabled=false.
Notice we need to add the system property above to the new version before the first startup, when the upgrade is actually performed. If you are using the installer, you need to add the property before accepting "Start Confluence now?" stage of the process.
For more information about the best way to backup Confluence, please see Production Backup Strategy
Method 2
If you'd prefer to let Confluence generate this file, you'll need to increase the maximum size of the heap. Fix java.lang.OutOfMemoryError in Confluence will guide you through this. Having increased this value, trigger the upgrade again and monitor for any memory errors.