Crucible upgrade guide

Database upgrade task

This release contains a database upgrade task which may take some time at startup. Upgrade time depends on the number of code reviews and number of files stored in them, and on speed of your database. The output is saved to: $FISHEYE_INST/var/log/atlassian-fisheye-upgrade-db-[nnn]-108.log file

and it looks like this:

...
2019-10-16 11:25:31,772 INFO  [main ] AddRevPathHashTask-afterUpgradeScript - Calculating hash of (source name, revision number and path) for existing revisions.
2019-10-16 11:27:52,638 INFO  [main ] AddRevPathHashTask-updateDatabase - Updated 100000 rows
2019-10-16 11:30:06,507 INFO  [main ] AddRevPathHashTask-updateDatabase - Updated 200000 rows
...
2019-10-16 11:50:08,886 INFO  [main ] AddRevPathHashTask-updateDatabase - Updated 1083447 rows
2019-10-16 11:50:08,958 INFO  [main ] BumpDbVersionUpgradeTask-bumpVersionNumberUp - Database upgrade from version 107 is successful

You can use this query to estimate the number of rows to be updated:

SELECT count(*) FROM cru_revision

On a mediocre instance (AWS m4.xlarge, 16GB RAM, 6GB JVM heap, RDS Postgres), it processes about 50'000 rows per minute. It took about 25 minutes to upgrade an instance with 20'000 reviews and 1'000'000 files in them.

Advance notice: end of support for Internet Explorer 11

We have decided to end support for Internet Explorer 11. Crucible 4.8 will be the last version to support Internet Explorer. See End of Support Announcements for Crucible for full details. 

Since Crucible 4.8, the Microsoft Edge web browser is supported.

No JRE in Windows installer

Windows installer no longer contains a bundled JRE. Install Java 8 JRE on your machine before upgrading. See Upgrading Fisheye on Windows for details.

Subversion server 1.5 and 1.6 are no longer supported

While we are not aware of any backward-incompatible changes in Crucible 4.8, we strongly recommend an upgrade to a newer version of Subversion.

REST API changes

GET /rest-service/reviews-v1/<review-id>/comments is deprecated.

Use /rest-service/reviews-v1/<review-id>/comments/general and /rest-service/reviews-v1/<review-id>/comments/versioned instead.

<permId> for comments

The deprecated <permId> attribute has been removed from comments (general comments, file comments, comment replies etc). Use the <permaId> attribute instead.

Java API changes

The Comments class is deprecated. Use GeneralComments and VersionedComments instead. The DetailedReviewData class uses these two in setters and getters.

Deprecated GeneralCommentData.Builder class has been removed. Use GeneralCommentBuilder instead.

Deprecated methods setFromLineRange/getFromLineRange from VersionedLineCommentData have been removed. Use VersionedLineCommentBuilder#addRange and VersionedLineCommentData#getLineRanges instead.

JVM system properties changes

Property fisheye.changeset.paths.limit minimum legal value was changed from undefined to 0.

Property fisheye.pipeline.batch.cslimit default value was changed from 10000 to 1000, and maximum legal value was changed from 50000 to unlimited.

Property fisheye.pipeline.batch.pathlimit default value was changed from 60000 to 100000, and maximum legal value was changed from 120000 to unlimited

Deprecation announcements

Old versions of Postgres, SQL Server, Git, Mercurial, Subversion and Perforce have been deprecated. While they are still supported by Crucible 4.8 we recommend their upgrade. See end of platform announcements for details.

Several old versions of databases, Git and Mercurial have been deprecated. While they are still supported by Crucible 4.7 we recommend their upgrade. See end of platform announcements for details.

In case you host Fisheye on 32-bit Windows, we recommend migration to the 64-bit version as Crucible 4.7 is the last version with the 32-bit installer.

In case you run Fisheye with MySQL database, we recommend migration to UTF8MB4 encoding in order to have support for full UTF8 character set.

In case you use SQL Server database with a bundled driver, Crucible 4.7.1 will automatically migrate from jTDS to Microsoft JDBC driver. The database URL will change from 'jdbc:jtds:sqlserver' to 'jdbc:sqlserver' scheme. In case you use any custom connection parameters, such as domain-based authentication, you have to change them to the ones supported by Microsoft JDBC (properties supported by jTDS and by Microsoft JDBC). You can check application logs for any upgrade-related warnings.

In case you have any third party plugins installed, please check their compatibility with Crucible 4.7, as underlying Jetty server has been upgraded (change of Jetty mainly affects JSP files). The HTTP PUT method is not allowed for JSP files - use POST instead. The HTTP GET shall not use the 'Content-encoding' property.

New JVM properties

Two new properties to control in-memory blame cache have been added:

• fisheye.blame.scm.fallback.cache.timeout.in.seconds=N (seconds since adding to cache)
• fisheye.blame.scm.fallback.cache.size=N (number of files)

REST API changes

All review-related endpoints (such as /rest-service/reviews-v1, /rest-service/search-v1 or /rest-service-fecru/recently-visited-v1) can handle multiple issue keys now. The <linkedIssues> is new property to link issues. The <jiraIssueKey> is deprecated, on GET it returns a first issue (alphabetically) from the <linkedIssues> list, on POST it links an issue with a review. 

Java API changes

ReviewData(String, String, String, UserData, UserData, UserData, String, String, ReviewData$State, Boolean, PermId, Date, Date, Date, int, String) has been removed

ReviewData(String, String, String, UserData, UserData, UserData, String, String, ReviewData$State, ReviewData$ReviewType, Boolean, PermId, Date, Date, Date, int, String) has been removed

com.atlassian.fisheye.plugin.web.helpers.Helper#getLinkedIssue method has been added

getWebPanelsForIssueInReview(HttpServletRequest, HttpServletResponse, String, String, String) has been added for handling issue transitions in a close review dialog - in case your plugin uses 'crucible.review.complete.dialog.jira.action' or 'crucible.review.close.dialog.jira.action' webPanelKey then it requires an update

Mercurial 4.5 and Git 2.16-2.18 are supported

Crucible 4.6 supports Mercurial up to version 4.5 and Git up to version 2.18.

 Git 2.16.x is incompatible with OpenSSH older than 6.8, more details in FE-7016.

Crucible 4.4 upgrade notes

Mercurial 4.1 and Git 2.12 are supported

Crucible 4.4 supports Mercurial 4.1 and  Git 2.12 clients

Mercurial 1.5 - 1.8.4 is no longer supported

As of Crucible 4.4, the oldest Mercurial version supported is 1.9.3 (released in 2011).  Before you upgrade to Crucible 4.4, upgrade Mercurial client binaries.

Crucible 4.2 upgrade notes

Support for repository renaming

For Crucible 4.2, and later versions, each repository is now identified by both a (display) name and a key. The name can be changed, even for existing repositories, while the key can never be changed. Previously, repositories were identified only by an immutable 'name' attribute (equivalent to the 'key' attribute in Crucible 4.2). See Renaming a repository for more details.

When upgrading to Crucible 4.2, each repository's 'name' will be used to populate both its (display) name and key.

Crucible 4.1 upgrade notes

Subversion 1.9 is supported

Crucible 4.1 supports Subversion 1.9 and the new FSFS format 7 introduced by this release. You may add new Subversion 1.9 repositories to Crucible as well as upgrade (with 'svnadmin upgrade' command) repositories that you've already added. Subversion 1.9 support covers both the bundled SVN client (SVNKit) as well as the Subversion 1.9 native library setup.

Please note however that during internal performance tests we've observed some indexing slowdowns when SVN 1.9 (FSFS format 7) repositories were accessed with the file:// protocol using the bundled SVN client (comparing to SVN 1.8 ones). Therefore, if indexing time is a priority for you, we recommend that you continue to use SVN 1.5 - 1.8 repositories, if possible. 

Subversion 1.1 - 1.4 is no longer supported

As of Crucible 4.1, the oldest Subversion version supported is 1.5 (released in 2008). This applies also to the repository format: the lowest supported is format 5 and FSFS format 3. Therefore, before you upgrade to Crucible 4.1, please upgrade both the Subversion client and server binaries as well as your repositories (using the 'svn upgrade' command). 

Subversion merges are supported

Crucible 4.1 supports merge operation in Subversion. You may merge branches in Subversion and see information about this in commit graph and on changeset page. More information is available on official documentation page.

Please note that merge information is processed only for new changesets. The repository history does not rescanned in order to expose the merge information for old changesets, however, if you need such historical information, you can perform a full repository reindex.

Crucible 4.0 upgrade notes

User directories migration

Crucible 4.0 includes new user directories administration. Please refer to Fisheye 4.0 user directories migration for details of how users-related settings are migrated.

API changes for plugin developers

Please note that if you're not a plugin developer, changes described in this section will not affect you.

PLUGINS SYSTEM 4.0 UPGRADE

Crucible 4.0 ships with an upgraded Plugins System. Please refer to the Atlassian Plugins 4.0 Upgrade Guide for a detailed list of changes introduced by the upgrade.

CRUCIBLE API CHANGES

Crucible API changes in Crucible 4.0 are mainly related to the introduction of user directories. Host-based authentication has been removed, and LDAP-based authentication is now handled by Crowd rather than directly by Crucible, resulting in the removal of the HostAuthSettings and LdapAuthSettings classes. You could implement a custom authenticator for Crucible or a custom directory connector for Crowd, if absolutely necessary, but we strongly recommend that you rely on existing authentication methods.

The UserService# getActiveUserCount method is deprecated - use the getLicensedUserCount instead.

You can no longer call UserService# setCrucibleEnabledForUsers to enable Crucible access for certain users, as this method has been removed. Instead, you should assign users to a proper group and set group permissions using the new GlobalPermissionService# setPermissionsForGroup method.

The REST API has minor changes related to user and group management. See the documentation for more details.

Dropping Host-based Authentication 

As already announced, host-based authentication is no longer supported by Crucible 4.0 and later versions.

Crucible 3.10 upgrade notes

Secure connections using self-signed certificates may fail

Crucible 3.10 uses an updated version of commons-httpclient that provides SNI (Server Name Indication) support. However, this version of commons-httpclient uses stricter domain name verification, so webhooks and application links that use a secure connection may stop working. This is only likely to happen when Crucible accesses an application using a secure connection verified by a self-signed certificate, where the application domain name (for example 'jira.company.com') does not match the certificate common name (such as 'company.com').

You may need to update the SSL certificates you use with secure connections between Crucible 3.10 and other applications – self-signed certificates may no longer work as before.

Uppercase project keys upgrade task

The project key upgrade task runs on startup after upgrading to Crucible 3.10; it should only take a few seconds to run.

This is required because  Crucible 3.10 now only allows uppercase project keys. We've made this change so that the DB project key column can be properly indexed, allowing much improved performance for a number of different method calls.

• All existing project keys will be converted to be all uppercase and all unique. For Crucible, this also includes the project key part of review keys. 
• If a project key conflict occurs, the project key will be renamed by adding an  UPGRADE[Number]  suffix. You can change renamed project keys manually after the upgrade if necessary.
• The upgrade task produces logs for project key changes. Look for logs starting with [projectKey.uppercase] 
• All operations using the project key as an argument are case sensitive, with the exception that  view operations in the browser are case insensitive and will upper case the project key automatically.
• Fisheye and Crucible track recently visited projects, reviews and snippets for every user. Any projects, reviews and snippets with renamed project keys however will not appear in the recently visited cache. 

If there are entity links between Jira Software and Crucible projects, the mapping will be automatically updated. You can check this by visiting

Administration > Add-ons > Fisheye configuration > Fisheye/Crucible entity mappings in Jira. If Crucible projects are not all uppercased, click Refresh cache to update the mapping. 

Note that Crucible will refuse to start if the DB is not case-sensitive with UTF-8 default encoding, to avoid potential data corruption during the upgrade task. In this case you'll see the following log message:

FeCru connecting to DB not using case-sensitive UTF8 encoding

LDAP synchronization

Crucible 3.10 now  supports paging (with a default page size of 1000) when requesting data from the LDAP server, and works seamlessly when the number of user accounts exceeds 1000 in Active Directory.  It  reverts to the previous behavior with servers that don't support LDAPv3.

The paging size can be controlled with the fisheye.ldap.sync.page.size system property. Setting it to 0 disables paging.

Crucible 3.9 upgrade notes

Lucene index upgrade task

A reindex of the Lucene index runs on startup after upgrading to Crucible 3.9. This is required because new fields have been added to the Lucene index to enable sorting of reviews by the Review column ( CRUC-6560 - Review dashboard can't be sorted by Review key CLOSED  ).  

Reindexing takes about 10 minutes for 15000 reviews. During reindex, Crucible is usable but some reviews may not be visible.

Smart Commits now use the Jira REST API

As of version 3.9.0, Crucible uses the Jira REST API instead of the Jira Fisheye Plugin for Jira Smart Commits. This change ensures that you will be able to use Smart Commits with future versions of Jira. Note that using Smart Commits with previous versions of Crucible (earlier than 3.9.0) and Jira 7 will cause an error notification via email. 

Git clone changes

As of version 3.9.0, Crucible turns Git garbage collection off when cloning a repository (by adding the gc.pruneExpire=never option) to prevent unreferenced objects being removed from local clones. Also, when cloning a repository, git config is run on each repository during instance startup. This change bumps the Git cache version ( CACHE_VERSION ) to 22.

Supported platform upgrades

• Oracle 12c is now supported.
• Git 2.4.6 and Mercurial 3.4.2 are now supported.
• Support for Java 7 has been removed from Crucible 3.9, as previously announced.
• Support for Internet Explorer 9 has been removed from Crucible 3.9, as previously announced.

Crucible 3.8 upgrade notes

New database index to improve Review Dashboard and Review Search pages load time

A database upgrade task is run on startup that adds a database index. This task should take no more than a few minutes, and should complete within seconds on most instances.

Improved Git indexing time for newly created branches

In order to allow faster Git indexing, a new field was added to the internal repository caches. The cache for each Git repository is automatically upgraded when the repository is started for the first time after upgrading. It is expected to take under a minute per repository, but may take slightly longer if repositories have thousands of branches.

Supported platform upgrades

• Java 7 is deprecated, as previously announced.

Crucible 3.7 upgrade notes

Supported platform upgrades

• Java 7 support is deprecated – please see the End of Support Announcements for Crucible. 

Crucible 3.6 upgrade notes

Supported platform upgrades

• Java 6 is not supported by Crucible 3.6, as previously announced.
• Java 7 support is deprecated – please see the  End of Support Announcements for Crucible . 

SSLv3 support disabled by default

As of Crucible 3.6, SSLv3 support is disabled by default. This shouldn't affect normal operations in supported browsers. If you need to re-enable SSLv3 support, please consult Configuring SSL cipher suites for Jetty.

Crucible 3.5 upgrade notes

Crucible 3.4 upgrade notes

Windows installer

We've produced 32-bit and 64-bit installers for Crucible on Windows. Each installer adds Crucible as a Windows service, and starts the service, automatically. The express install creates, by default, a Data directory and a separate install directory  in C:\Atlassian. The custom install mode allows you to choose different locations for the install and Data directories, with the restriction that the Data directory must not be contained in the install directory. The installer creates the FISHEYE_INST system environment variable.

See Installing Crucible on Windows for detailed installation instructions.

Download the Crucible installer here.

Crucible may now bind to a different IP address on Windows

Prior to Crucible 3.4, a bug in Crucible (  FE-4909 CLOSED ) meant that Crucible may not have correctly bound to the IP address you configured. This may have happened if you configured Crucible to bind to a single IP address on a network interface that has several IP addresses; Crucible may in fact have bound to a different IP address. For example, if you have an interface with the IP addresses 1.2.3.4 and 1.2.3.5, and you configured Crucible to use 1.2.3.5, it may have incorrectly bound to 1.2.3.4.

Now that the bug is fixed, Crucible 3.4, and later versions, will now correctly bind to the configured IP address, although this may now be different from the previously bound address.

v1 REST API resources deprecated

Note that the 'v1' REST API resources are deprecated and will be removed in a future release. See the Fisheye Crucible REST API.

Crucible 3.3 upgrade notes

The following platforms are no longer supported by Crucible 3.3:

• Internet Explorer 8
• MySQL 5.0
• PostgreSQL 8.2
• SQL Server 2005

Please read the End of Support Announcements for Crucible.

Supported platform upgrades

• SVN 1.8 is supported by Crucible 3.3.
• Microsoft Internet Explorer 11 is supported by Crucible 3.3.

See the Crucible Supported platforms.

Crucible 3.2 upgrade notes

REST endpoint change

For Crucible 3.2, the RestReviewService.remindIncompleteReviewers() ('/rest-service/reviews-v1/{reviewId}/remind') end point was changed from accepting 'application/x-www-form-urlencoded' content type with 'message' and 'recipient' params to 'application/json' content type with 'message' and 'recipients' JSON fields. See  https://docs.atlassian.com/fisheye-crucible/latest/wadl/crucible.html#d2e1881.

User data is moved from data0.bin to the SQL database

An upgrade task is run on startup that moves user data to the SQL database. We are doing this to mitigate the risk of data corruption or loss.

XSRF protection

Crucible 3.2.0 includes protection against XSRF attacks.

If you have implemented a LightSCM plugin, and have used the SimpleConfigurationServlet base class provided in the scmutils library, you will need to modify your administration page so that it performs 'delete' operations using an HTTP POST, not a GET.

You can have Crucible convert your anchor tags to form POSTs at runtime by giving them the class "anchor-post". For example, the anchor:

<a class="anchor-post" href="#" data-href="./fsscm?name=TEST&delete=true">Delete</a>

will be converted into a form which POSTs to ./fsscm with form parameters name=TEST and delete=true.

Internally managed Git repositories no longer supported

As previously announced, internally managed Git repositories are no longer supported by Crucible 3.2.

Please read the migration guide for information about options and procedures for migrating your internally managed Git repositories – note that we recommend that you upgrade to Crucible 3.2 before migrating any internally managed repositories.

Supported platform upgrades

• Communication with Jira versions older than 5.0 is no longer supported.
• Microsoft SQL Server 2012 is now supported (support for SQL Server 2005 is deprecated).
• Microsoft Internet Explorer 10 is now supported (support for IE 8 is deprecated).
• MySQL 5.0 is deprecated.
• PostgreSQL 8.2 is deprecated.
• The Atlassian AUI plugin has been upgraded to AUI 5.2.
• jQuery has been upgraded to 1.8.3.
• backbonejs has been upgraded to 1.0.

Crucible 3.1 upgrade notes

Crucible 3.1 Merge some per-repository Lucene indices into a global cross-repository Lucene index

Crucible 3.1 has greatly improved performance and scalability for QuickSearch and QuickNav. To achieve this, the per-repository 'METADATA' Lucene indices will be moved into a single global cross-repository Lucene index. This means Crucible is able to search across more repositories in less time because now only a single search index needs to be queried instead of the previous N. Merging these indices into the single cross-repository index can be refreshed in two ways:

1. Recommended: As an upgrade task that is run automatically when Crucible 3.1 is run for the first time.
2. As an offline process on a separate staging server.

During the automatic upgrade task, Crucible is fully usable and functional, although search results for files, commits and committers may be incomplete.

In our testing we have found that the automatic upgrade task took 4 hours to complete on a Crucible instance with 144 repositories of different kinds, with 58 GB of data in the FISHEYE_INST folder (excluding logs). We are confident that the automatic upgrade task is suitable for the majority of production Crucible installations. It is worth repeating that the instance was fully functional (reviews, Jira Integration, Activity Streams, Charts etc) apart from Quick Nav and Quick Search during this time.

Nevertheless, where required, we provide instructions for performing the reindex as an offline process on a separate staging server.

Plugin Settings will be moved from the config.xml to the SQL database

As of Crucible 3.1.0, plugin settings which were previously stored in the <properties> element inside config.xml will be stored in the SQL database. This includes settings for any bundled plugins such as ApplicationLinks, the UniversalPluginManager etc, and any 3rd party plugins. 

An upgrade task is run on startup which will first insert all the properties found in config.xml into a new table in the SQL database. Once successful, the properties are removed from config.xml. 

As part of this change, the RepositoryOptions.setProperties (Map<String, String>properties) and RepositoryOptions.getProperties() methods have been removed from our API. If you are using a plugin which uses either of these methods, you will need to update the plugin to a version which uses the Spring component PluginSettingsFactory. Plugins can use this to access the migrated global and per-repository properties that were previously available via the old RepositoryOptions API. 

Crucible 3.0 upgrade notes

Jetty 8

Crucible 3.0 now uses Jetty 8 as its web server and Java servelet container. This change should be completely transparent when you upgrade to Crucible 3.0. However, if you have customised either your jetty-web.xml file, or the maxFormContentSize system property, you will need to update those in the new version. See Enabling Access Logging in Fisheye and this Fisheye KB page for more information.

Infinity DB

Crucible 3.0 now uses the InfinityDB 3.0 database internally to provide improved performance for concurrent access to Crucible. This change is transparent to users in all respects.

Pipelined indexing

Crucible 3.0 introduces a new indexing approach that splits the repository indexing process into separate tasks that can be performed in a phased and concurrent way. Users will benefit from the way in which Crucible functionality, such as review creation, now becomes available as indexing progresses. This change is transparent to users in all other respects. See Pipelined indexing.

Improved handling of user preferences with session cookies 

Upgrading may result in some users losing their preferences.

SQL Server transaction isolation configuration

We recommend a configuration change for SQL Server to use snapshot mode for the transaction isolation level – see Migrating to SQL Server. This change avoids occasional database deadlocks, and prevents performance warning messages in the logs and admin screens.