Bamboo schedule backup fails when Expiry and Backup is scheduled at the same time
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Bamboo scheduled backup fails if expiry and Bamboo backup is scheduled to run at the same time.
Environment
6.10.6
Diagnosis
When you run the Bamboo backup during the time when Expiry is running, schedule backup may fail with the below error:
2021-04-13 00:00:00,014 INFO [scheduler_Worker-2] [ServerLifecycleManagerImpl] Server state changed to 'PAUSING' from 'RUNNING'
2021-04-13 00:00:00,019 INFO [scheduler_Worker-2] [XmlMigrator] Creating zip:/opt/atlassian/bamboo/application-data/bamboo-home/backups/bamboo_backup_2021_04_13.zip
2021-04-13 00:00:00,022 INFO [5-Server Lifecycle Manager:pool-11-thread-1] [ServerLifecycleManagerImpl] Server state changed to 'PAUSED' from 'PAUSING'
2021-04-13 00:00:00,022 INFO [scheduler_Worker-2] [XmlMigrator] Starting export of data...
....
2021-04-13 00:00:50,512 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Running cleanup task... checking for build expiry configuration
2021-04-13 00:00:50,628 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Found 1 build(s) to expire for PLAN-API
2021-04-13 00:00:50,722 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Affected 1 results in plan PLAN-API
2021-04-13 00:00:50,730 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Found 1 build(s) to expire for PLAN-APIREP
2021-04-13 00:00:50,784 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Affected 1 results in plan PLAN-APIREP
2021-04-13 00:00:50,853 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Found 5 build(s) to expire for BCOM-CCB2C
2021-04-13 00:00:51,081 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Affected 5 results in plan BCOM-CCB2C
2021-04-13 00:00:51,112 INFO [http-nio-8085-exec-8] [AuthenticatorImpl] Authenticated app 'jira:b4551014-12be-43b2-bc9e-46cc4c1ebf46' as user 'null' successfully
2021-04-13 00:00:51,112 INFO [http-nio-8085-exec-8] [JohnsonFilter] The application is still starting up, or there are errors. Redirecting request from '/rest/remote-link-aggregation/latest/aggregation' to '/error/errorEvents.action'
2021-04-13 00:00:51,214 INFO [http-nio-8085-exec-5] [AuthenticatorImpl] Authenticated app 'jira:b4551014-12be-43b2-bc9e-46cc4c1ebf46' as user 'null' successfully
2021-04-13 00:00:51,214 INFO [http-nio-8085-exec-5] [JohnsonFilter] The application is still starting up, or there are errors. Redirecting request from '/rest/remote-link-aggregation/latest/aggregation' to '/error/errorEvents.action'
2021-04-13 00:00:51,255 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Found 1 build(s) to expire for BCOM-SMET
2021-04-13 00:00:51,307 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Affected 1 results in plan BCOM-SMET
2021-04-13 00:00:51,328 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Found 1 build(s) to expire for BCOM-TEST1
2021-04-13 00:00:51,342 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Affected 1 results in plan BCOM-TEST1
2021-04-13 00:00:51,347 INFO [1-BuildExpiryBean:pool-4-thread-1] [BuildExpiryBeanImpl] Cleanup task completed in 835.1 ms, removed 9 builds
2021-04-13 00:00:51,703 INFO [scheduler_Worker-2] [XmlMigrator] Updating destination zip /opt/atlassian/bamboo/application-data/bamboo-home/backups/bamboo_backup_2021_04_26.zip
2021-04-13 00:00:51,735 INFO [scheduler_Worker-2] [XmlMigrator] Unmounted /opt/atlassian/bamboo/application-data/bamboo-home/backups/bamboo_backup_2021_04_26.zip
2021-04-13 00:00:51,742 INFO [scheduler_Worker-2] [ServerLifecycleManagerImpl] Server state changed to 'RUNNING' from 'PAUSED'
2021-04-13 00:00:51,742 ERROR [scheduler_Worker-2] [ScheduledBackupJob] java.io.IOException: /opt/atlassian/bamboo/application-data/bamboo-home/xml-data/builds/plan-38797438-STAG21B2C/download-data/build_logs/plan-38797438-STAG21B2C-441.log (missing file)
java.io.IOException: /opt/atlassian/bamboo/application-data/bamboo-home/xml-data/builds/plan-38797539-PROD21B2C/download-data/build_logs/plan-38797539-PROD21B2C-441.log (missing file)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:183)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:169)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:169)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:169)
at de.schlichtherle.truezip.file.TBIO.cp_r0(TBIO.java:169)
at de.schlichtherle.truezip.file.TBIO.cp_r(TBIO.java:138)
at de.schlichtherle.truezip.file.TFile.cp_rp(TFile.java:3301)
Cause
The cause of this failure is that the expiry thread which is running in the background, can be deleting some plans, log files, artifacts etc which may at the same time be backed up. The backup tries to backup these files which in fact may have already been deleted.
Solution
The solution for this is to rethink the approach of running expiry and backup at the same time. Either disable the expiry or move the scheduled backup to a different time.
Last modified on Jan 30, 2023
Powered by Confluence and Scroll Viewport.