Confluence Mail Server Setup issue due to TLS
Platform Notice: Data Center and Cloud By Request - This article was written for the Atlassian data center platform but may also be useful for Atlassian Cloud customers. If completing instructions in this article would help you, please contact Atlassian Support and mention it.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
Confluence fails to connect to the mail server while connecting through a secure port(587) with TLS enabled. It throws below error:
An error has occurred with sending the test email:
com.atlassian.mail.MailException: javax.mail.MessagingException: Could not convert socket to TLS;
nested exception is:
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at com.atlassian.confluence.jmx.JmxSMTPMailServer.doSendWithMessageIdSupport(JmxSMTPMailServer.java:120)
at com.atlassian.confluence.jmx.JmxSMTPMailServer.send(JmxSMTPMailServer.java:79)
at com.atlassian.confluence.admin.actions.mail.SendTestEmailAction.execute(SendTestEmailAction.java:57)Diagnosis
Environment
- Confluence server running with Java 8
Diagnostic Steps
Enable mail debug and SSL debug logging as follows:
- Stop Confluence.
Enable debug logging by including -Dmail.debug=true and -Djavax.net.debug=ssl:handshake parameter to
CATALINA_OPTS. Check Configuring System Properties. These parameters will print out logging email protocol details incatalina.log.If you do not see the details in the
catalina.outlog, try removing any spaces that may be present with the parameter.Please do not include add between the parameters while adding in the property file.
- Restart Confluence.
Try to send the Test email again. The result would be something like below (printed inside
catalina.log):
STARTTLS
220 TLS go ahead
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
.
.
http-nio-17000-exec-7, handling exception: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
http-nio-17000-exec-7, SEND TLSv1.2 ALERT: fatal, description = unexpected_message
http-nio-17000-exec-7, WRITE: TLSv1.2 Alert, length = 2
http-nio-17000-exec-7, called closeSocket()The SSL Handshake between Confluence and the SMTP Mail Server fails in the above scenario. For a successful SSL handshake, the log entries should be like below in Catalina.out logs:
server Finished handshake message
Followed by..
client Finished handshake messageCause
The issue is caused due to incorrect TLS version being used. This can also be verified by running the below OpenSSL command using different TLS versions. For the correct TLS version, the public certificate of the mail server will be returned.
openssl s_client -connect mail.example.com:587 -tls1 -starttls smtp
openssl s_client -connect mail.example.com:587 -tls1_1 -starttls smtp
openssl s_client -connect mail.example.com:587 -tls1_2 -starttls smtpWorkaround
Once supported TLS version is identified, it is required to enforce the TLS version in Java being used.
E.g., To enforce TLSv1.2 protocol, add below parameter following Configuring System Properties.
CATALINA_OPTS="-Dmail.smtp.starttls.enable=true -Dmail.smtp.ssl.protocols=TLSv1.2 ${CATALINA_OPTS}"