Opened 7 years ago

Closed 7 years ago

#4822 closed Bug/Something is broken (fixed)

Problem in Outlook with new certificate

Reported by: Allison Wolf Owned by: Jamie McClelland
Priority: Urgent Component: Tech
Keywords: x509 chain ca-cartel thawte microsoft windowsxp outlook imap smtp postfix courier apache2 intermediate Cc: lclement@…
Sensitive: no

Description

We are getting certificate error cannot verify certificate secure.critpath.org in Microsoft Outlook.

Attachments (2)

outlook certificate.png (18.4 KB) - added by Allison Wolf 7 years ago.
thawte certificates.JPG (69.4 KB) - added by Allison Wolf 7 years ago.

Download all attachments as: .zip

Change History (26)

comment:1 Changed 7 years ago by Jamie McClelland

Hi Allison, the certificate was install yesterday morning. Is this the first time this copy of Outlook has accessed their email since then? Or has it accessed it successfully since Thursday morning?

Do other Outlook users have this problem or is it just one outlook user?

jamie

Changed 7 years ago by Allison Wolf

Attachment: outlook certificate.png added

comment:2 Changed 7 years ago by Allison Wolf

It seems that this is happening to most folks. I thought that it would go away since you just updated the certificate and it would take time to verify. But we are still getting them and now Jane is also getting them.

I have attached the certificate error. Let me know if we need to do anything. Usually we get this error and they ask us to download the certificate. Let me know if there is anything else you need me to do?

comment:3 Changed 7 years ago by Daniel Kahn Gillmor

Owner: changed from jamie, to Jamie McClelland
Status: newassigned

making the assignment actually show an OpenID.

comment:4 Changed 7 years ago by Daniel Kahn Gillmor

Keywords: x509 chain ca-cartel thawte microsoft windowsxp outlook imap added

from my perspective on the network, it appears that the machine is offering an X.509 certificate chain with a fairly common trust anchor:

Certificate chain
 0 s:/C=US/ST=Pennsylvania/L=Philadelphia/O=Critical Path Project/OU= Secure Services Division/CN=secure.critpath.org
   i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
 1 s:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
   i:/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA

is it possible that the "thawte Primary Root CA" isn't listed as a trusted authority on your windows machine?

Doing a bit of digging, it appears that thawte has two variants of this certificate, one that is self-signed and one that is chained to their "Thawte Premium Server CA". (the two certificates use the same public key material)

awolf: is the machine that this was tried on up-to-date with the latest microsoft "Update for Root Certificates"? If it is not, would you consider trying applying that? if you are running Windows XP, and you are using Microsoft Update, you should be able to find it in the list of optional updates.

If your client already has that update installed, or you install it and still get this error, then let's talk about possible next steps (e.g. including the non-self-signed certificate as an additional intermediate cert).

comment:5 Changed 7 years ago by Allison Wolf

It is looking like the certificate error is not only in Outlook but also when folks go and look at their mail through the browser.

Several of our employees look at their mail through either Internet Explorer and Firefox. They are also getting the certificate error.

I just did a Windows update and downloaded a Microsoft Office Update, After the reboot I went back into Outlook and I received the same error but this time I was able to download the certificate.

This is going to be a pain if we have to walk folks through downloading certificates both through their browsers and Outlook. Is their anyway to prevent this? Of are we going to have to bite the bullet and walk everyone through updating their certificates. We have not had to do this in the past. Thanks

comment:6 Changed 7 years ago by Daniel Kahn Gillmor

Windows Update and Microsoft Office Update wouldn't be relevant to this. I was trying to ask specifically about the "Update for Root Certificates" -- were you able to get that update?

I note that you mentioned "i was able to download the certificate" -- what does this mean? Which certificate were you able to download? What mechanism did you use to download it?

comment:7 Changed 7 years ago by Allison Wolf

I apologize Microsoft already updates the Thawte. They are listed as a trusted authority. When the error appears in Outlook you are given the opportunity to view the certificate and when you do, the option to install the certificate is available. But as I just discovered that does not solve the issue.

But also folks are getting the error through IMP (when checking mail through both IE and Firefox) So the issue of the workstation and downloading the certificates in Outlook is not the issue any more. I believe the issue is more wide spread.

It seems to be happening to everyone. WHat is different about the certificate from last year and this year? Do we need to setup a conference call with Thawte?

comment:8 Changed 7 years ago by Daniel Kahn Gillmor

Owner: changed from Jamie McClelland to Daniel Kahn Gillmor

thawte has several "root" CA certificates with very similar names. They recently transitioned one of these CA certificates to be their new official baseline for all their offerings. However, older versions of un-updated operating systems might not have the most recent one. The one i'm interested in has a common name of "thawte Primary Root CA". You might see others with names like "Thawte Personal Basic CA", "Thawte Premium Server CA", etc. These others are older CAs, using weaker public keys than they have any business using (they're 1024-bit RSA keys, which are arguably in contravention of NIST's guidelines on public key length for 2010).

I think what changed was that sometime during 2010, Thawte started issuing all their certificates chained from the "thawte Primary Root CA", which uses a stronger key (2048-bit RSA). But older operating systems that are not updated may not have that CA certificate in their root certificate store, while they do have the other Thawte CA certificates.

Can you please check to see whether "thawte Primary Root CA" is in your winXP system's root certificate store, and whether that system has installed the latest "Update for Root Certificates"?

comment:9 Changed 7 years ago by Allison Wolf

I have checked my system and I run Windows 7. Yes I have thawte Primary Root CA and they have been updated. Windows 7 automatically checks for updates on certificates. Windows XP has an update that you download which updates the certificates.

But since I am having the issue and my certificates are up to date what else could the problem be. By the way I check the store in Internet Explorer under tools/content/certificates/trusted root certificate Authorities.

I will keep looking around for more infor.

comment:10 Changed 7 years ago by Daniel Kahn Gillmor

Ah, ok, i didn't know you were running windows 7, i was assuming from the screenshot that this was all on XP clients.

Is the thawte Primary Root CA set to be a trusted root authority for TLS connections, or is it just listed as an authority but not marked as "trusted"? Can you provide a screenshot or a text-based listing of the trusted root authorities? i'm sorry but i don't have windows machines to work from to test this out :/

Changed 7 years ago by Allison Wolf

Attachment: thawte certificates.JPG added

comment:11 Changed 7 years ago by Allison Wolf

I have attached the thawte certificates that are trusted in Internet explorer. I have gotten the certifiacte error in Outlook and IE but not in Firefox. At least not on my system. If you have any ideas that would be great. I will keep looking on this issue.

comment:12 Changed 7 years ago by Allison Wolf

Later this afternoon I will call thawte and see if they have an answer for this issue.

comment:13 Changed 7 years ago by Daniel Kahn Gillmor

Could you try pointing your web browser at https://secure.critpath.org/ again? I believe that the web configuration was missing the intermediate CA cert.

I've added that cert via a line in your https apache config

SSLCertificateChainFile /etc/ssl/thawte-intermediate-ca.pem

(after placing that intermediate certificate in the filesystem on didier).

I've got my hands on an up-to-date windows XP system now, and i can confirm that the web connection works with this machine.

I'm now turning my attention to outlook express (the machine i'm testing with doesn't have full-blown outlook)

comment:14 Changed 7 years ago by Daniel Kahn Gillmor

Keywords: smtp postfix courier apache2 added

I'm able to connect with imaps far enough for the imap daemon do reject my login (i don't have a legitimate account). so i don't think that imap is the problem either.

This leads me to suspect that your concern is with sending mail, instead of receiving mail.

I just tested for smtp, and see no intermediate certificates listed here:

echo | openssl s_client -starttls smtp -connect secure.critpath.org:submission

it produces only a single element in the "Certificate chain" section, when there should be two:

---
Certificate chain
 0 s:/C=US/ST=Pennsylvania/L=Philadelphia/O=Critical Path Project/OU= Secure Services Division/CN=secure.critpath.org
   i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA
---

so i think postfix is what needs to be configured with intermediate certificates

comment:15 Changed 7 years ago by Daniel Kahn Gillmor

Keywords: intermediate added

I've just added an intermediate certificate to postfix with the following steps:

cat /etc/ssl/didier.mayfirst.org.crt /etc/ssl/thawte-intermediate-ca.pem > /etc/ssl/didier.mayfirst.org.plus.intermediate.crt

and then re-set smtpd_tls_cert in /etc/postfix/main.cf:

smtpd_tls_cert_file=/etc/ssl/didier.mayfirst.org.plus.intermediate.crt

i can now get to the point where my authentication fails, instead of complaining that the certificate is bad.

does it work for you now?

comment:16 Changed 7 years ago by Allison Wolf

I believe you have got it!!!

Can we leave the ticket open for just one mor day to make sure this solves the issue.

thanks

comment:17 Changed 7 years ago by Daniel Kahn Gillmor

Owner: changed from Daniel Kahn Gillmor to Jamie McClelland

I'm actually going to reassign this to jamie to ask him to make sure that my changes to the postfix config doesn't get overwritten by our configuration management system. If there are more problems with your connectivity, though, feel free to re-assign it to me.

comment:18 Changed 7 years ago by Allison Wolf

Resolution: fixed
Status: assignedclosed

Issue is fixed Thank you

comment:19 Changed 7 years ago by Daniel Kahn Gillmor

Resolution: fixed
Status: closedassigned

#4910 suggests that this issue is still open.

comment:20 Changed 7 years ago by Daniel Kahn Gillmor

Priority: HighUrgent

The changes i had made to /etc/postfix/main.cf appear to have been reverted. I've re-added them, so you should no longer be getting the error messages.

However, i don't know what made them get overwritten. I suspect that a run of the configuration management system (puppet) clobbered the changes somehow.

Jamie, please look into this -- i'm not sure what the interactions are between puppet, postfix, courier, apache, and the server's certificates in /etc/ssl/.

In particular, Apache seems to want the host's key in one file (SSLCertificateKeyFile), EE cert in another (SSLCertificateFile), and the intermediate certificates in another file (SSLCertificateChainFile). But postfix want the key in one file (smtpd_tls_key_file), and EE and the intermediates concatenated into a separate single file (smtpd_tls_cert_file). Finally, Courier seems to want all three objects in a single file (TLS_CERTFILE).

Is there a standard way for MF/PL to approach this?

comment:21 Changed 7 years ago by Jamie McClelland

Thanks dkg for sorting this out (yet again).

I've just fully documented how it is supposed to work.

I haven't made any changes on didier (yet) pending review of the documentation. The documented method works with RapidSSL certificates (and intermediate certificate files). I'd like to ensure that nobody sees problems with this setup using Thawte certs.

Assuming no problems, I'll configure didier according to this setup by friday (and before running puppet on the server again). Once changed, I'll run puppet to ensure the configuration can survive a puppet run.

jamie

comment:22 Changed 7 years ago by Daniel Kahn Gillmor

Thanks for the docs, Jamie. I've just updated them for terminology. I'm minorly surprised (but pleased) to see that apache and postfix are OK with having certificates in their secret key file, and that apache is OK with having the EE cert in the SSLCertificateChainFile. I think your policy outline is great, and about as simple as we can make such a thing.

I'm happy to coordinate on IRC to make such a push to didier with appropriate testing to make sure things don't break.

comment:23 Changed 7 years ago by Jamie McClelland

Great - thanks dkg. I got a little swamped today. If you are around friday or saturday perhaps we could take a stab at it then. I'll try you then.

jamie

comment:24 Changed 7 years ago by Jamie McClelland

Resolution: fixed
Status: assignedclosed

I've updated all the /etc/ssl files so they match how those files are configured on all of our servers. I also re-ran puppet. I've tested https://secure.critpath.org/ and sending and receiving from a test email acount using both POP and IMAP with starttls and sending via 587 with starttls.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.