Hi!
I was checking the source code for your question, and searching for MAIL_URL I found this:
I think the best you can get right now is a mix and match of the following options:
rocketchat:
...
environment:
- MAIL_URL=smtps://user:password@smtp_server:25&secure=false&ignoreTLS=true
One could work around this requirement by setting up an smtp relay that will use the provided certificate and provide a smtp server that will work with Rocket.Chat.