Subversion repository indexing fails due to CorruptJournalException
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
Subversion repository indexing fails and the following appears in the atlassian-fisheye.log:
2020-04-28 10:52:34,468 ERROR [JOURNAL_FLUSHER ] persistit Log4JAdapter-log - [JOURNAL_FLUSHER] ERROR Journal write failure com.persistit.exception.CorruptJournalException: Journal file <FISHEYE_INST>/cache/pipeline/pipeline_journal.000000000000 size 263,574,087 does not match current address 263,576,031 in <FISHEYE_INST>/cache/pipeline/pipeline_journal.000000000000 at offset 263,576,031 (6,290,647 similar occurrences in 632,360 seconds)where <FISHEYE_INST> is Fisheye's data directory.
Cause
This problem is related to the pipelined indexing which uses the PersistIt library, which is a queue that keeps information on what to index. It stores its queue data on the disk in this pipeline_journal.000000000000
file. Unfortunately, the administration section for a repository does not give any insight on the queue status, and it also does not give a method to clean it up or repair.
Workarounds
- Not confirmed yet (as of May 2020):
Fisheye/Crucible development team believes that if the$FISHEYE_INST/cache/pipeline/pipeline_journal.000000000000file is deleted while Fisheye is not running the problem will be fixed upon restart.
Local tests confirm that thepipeline_journal.000000000000file does get re-created upon restart, but there is still no confirmation that this really resolves the error because we haven't been able to replicate theCorruptJournalExceptionlocally.
This workaround should be tried first though, not only to confirm if this really resolves the issue, but because the other workaround should be followed only as a last resource. - Confirmed workaround:
If deleting just thepipeline_journal.000000000000file and restarting the instance does not help (as suggested in the workaround above), the only alternative solution is to re-index the repository from scratch, all over again.
NOTE: The repository needs to be removed and re-added from Fisheye administration panel, and not just re-indexed from scratch.