Summary

Searching Bitbucket Data Center or Bitbucket Server fails with the following message in web interface:
Search is currently unavailable
Refresh the page and try again

Environment

This issue has been reproduced in Bitbucket version 7.21.9 and 8.9.0, but other versions may also be affected.

Diagnosis

• In Bitbucket's web interface, under Administration > Server Settings, clicking Test button in the Search section results with error:
  Access to the search server was denied. Check your settings.
  
  
  

• Executing a curl command, with the correct credentials and URL of Elasticsearch, returns error 401, eg.

  cUrl test

  $ curl -u <ES_User>:<ES_Password> https://<Elasticsearch_URL>:9200
  {"error":"unknown","status":401}

  
  
  

Cause

The account used by Elasticsearch with Buckler plugin has been locked because of too many failed attempts to authenticate.

Solution

• Check if there is a script, or another automated process, repeatedly sending requests directly to Bitbucket's Elasticsearch, with incorrect credentials.
  Stop that script or process, or configure the firewall so that only Bitbucket nodes can send queries to Elasticsearch.
• Allow Elasticsearch to run for at least 5 minutes without receiving any requests.
  This may require either stopping Bitbucket, or temporarily isolating it from Elasticsearch, using the firewall or /etc/hosts file on each node.
• After 5 minutes without receiving any requests Elasticsearch should reset its account and the configured credentials should start working again:

  • The curl command should return the correct response, including the cluster_name, Elasticsearch version information etc. - eg.

    cUrl test

    $ curl -u <ES_User>:<ES_Password> https://<Elasticsearch_URL>:9200
    {
      "name" : "uaz3Vt6",
      "cluster_name" : "elasticsearch",
      "cluster_uuid" : "QuvwbzFeRPGSTuIc4FnoLg",
      "version" : {
        "number" : "5.5.3",
        "build_hash" : "9305a5e",
        "build_date" : "2017-09-07T15:56:59.599Z",
        "build_snapshot" : false,
        "lucene_version" : "6.6.0"
      },
      "tagline" : "You Know, for Search"
    }

  • On the Server Settings page, the Test button should return success
  • In Bitbucket's web interface, the search function should work again