Hypolite Petovan

Since this morning, roughly 1 out of 25 #HTTPS requests to friendica.mrpetovan.com fails because of an invalid date? The #SSL certificate isn't expired and I don't understand why only a small fractions of the requests fail.

Can anyone please #help me troubleshoot the issue?

Adam

5 years ago

You'll have to find out why apache is loading this cert, it has:

Serial #: 03:29:D0:39:1E:C4:02:B5:D2:90:07:C7:A3:23:AB:12:2A:C5
Expires: June 19, 2019

SHA-256 Fingerprint: 40:B9:53:99:15:40:79:AA:7E:13:A5:B0:50:A2:E6:5E:A5:27:3C:1D:12:9B:12:F4:9B:4F:5A:00:3E:A9:99:72
SHA1 Fingerprint:35:F1:D3:F3:F4:DB:A2:03:34:57:91:91:BC:0B:CD:64:1C:56:25:98

(I'm not sure if you'd been able to grab that info along the way)

Find the cert file that has that, and I suspect you'll be able to start tracking down how it's getting used in Apache.

Hypolite Petovan

5 years ago

None of my current certificates have this SHA1 fingerprint:

/etc/letsencrypt/live# find . -name cert.pem -print -exec openssl x509 -fingerprint -in {} -noout \;
./kingu.reactoweb.com/cert.pem
SHA1 Fingerprint=08:80:1D:D7:D5:5E:85:3B:37:31:3C:12:A5:4D:B2:B1:6C:61:49:7D
./pixelfed.mrpetovan.com/cert.pem
SHA1 Fingerprint=48:86:4B:AC:F1:5F:72:80:3B:AC:48:4E:8D:23:F2:C9:8A:E9:3A:63
./aeriesguard.com/cert.pem
SHA1 Fingerprint=62:03:72:B6:8E:83:FE:7A:E9:39:89:53:52:96:01:E5:BF:BD:42:24
./searx.aeriesguard.com/cert.pem
SHA1 Fingerprint=B2:EE:41:38:F4:9E:CE:B6:15:7F:9C:D9:A3:94:70:02:1E:01:13:48
./test.aeriesguard.com/cert.pem
SHA1 Fingerprint=D8:48:91:76:1F:A0:12:10:D0:8C:F2:78:B4:11:00:89:6F:0E:0A:E1
./dir.friendica.mrpetovan.com/cert.pem
SHA1 Fingerprint=42:2B:57:42:D8:2B:3F:7D:26:5E:98:93:E5:F4:55:93:39:7A:BB:8A
./airship.mrpetovan.com/cert.pem
SHA1 Fingerprint=0D:2E:39:21:3D:E1:83:57:C0:29:05:5A:5B:5D:1E:A1:96:06:D3:A7
./mrpetovan.com-0001/cert.pem
SHA1 Fingerprint=4D:AC:83:61:CC:79:0B:52:E9:08:B2:5B:4C:2C:2E:21:B2:BE:33:92
./mrpetovan.com-0002/cert.pem
SH

None of my current certificates have this SHA1 fingerprint:

/etc/letsencrypt/live# find . -name cert.pem -print -exec openssl x509 -fingerprint -in {} -noout \;
./kingu.reactoweb.com/cert.pem
SHA1 Fingerprint=08:80:1D:D7:D5:5E:85:3B:37:31:3C:12:A5:4D:B2:B1:6C:61:49:7D
./pixelfed.mrpetovan.com/cert.pem
SHA1 Fingerprint=48:86:4B:AC:F1:5F:72:80:3B:AC:48:4E:8D:23:F2:C9:8A:E9:3A:63
./aeriesguard.com/cert.pem
SHA1 Fingerprint=62:03:72:B6:8E:83:FE:7A:E9:39:89:53:52:96:01:E5:BF:BD:42:24
./searx.aeriesguard.com/cert.pem
SHA1 Fingerprint=B2:EE:41:38:F4:9E:CE:B6:15:7F:9C:D9:A3:94:70:02:1E:01:13:48
./test.aeriesguard.com/cert.pem
SHA1 Fingerprint=D8:48:91:76:1F:A0:12:10:D0:8C:F2:78:B4:11:00:89:6F:0E:0A:E1
./dir.friendica.mrpetovan.com/cert.pem
SHA1 Fingerprint=42:2B:57:42:D8:2B:3F:7D:26:5E:98:93:E5:F4:55:93:39:7A:BB:8A
./airship.mrpetovan.com/cert.pem
SHA1 Fingerprint=0D:2E:39:21:3D:E1:83:57:C0:29:05:5A:5B:5D:1E:A1:96:06:D3:A7
./mrpetovan.com-0001/cert.pem
SHA1 Fingerprint=4D:AC:83:61:CC:79:0B:52:E9:08:B2:5B:4C:2C:2E:21:B2:BE:33:92
./mrpetovan.com-0002/cert.pem
SHA1 Fingerprint=CE:5D:9D:B2:01:D6:21:37:5B:2A:B0:B3:55:DC:EE:3A:F0:8E:D5:37
./blog.reactoweb.com/cert.pem
SHA1 Fingerprint=24:D0:BC:6D:8A:00:DA:DA:5A:5E:93:45:BF:09:25:37:E1:6E:B1:1C
./dir.friendica.social/cert.pem
SHA1 Fingerprint=11:9A:E4:D1:50:C7:EF:31:84:5F:8B:3A:22:34:56:25:EE:E7:2A:19
./nextcloud.mrpetovan.com/cert.pem
SHA1 Fingerprint=B6:CB:41:89:7E:34:87:51:D7:89:D3:79:CD:61:98:18:DE:2F:D1:E1
./forum.aeriesguard.com/cert.pem
SHA1 Fingerprint=01:48:A0:4C:E6:AF:F2:04:00:61:0E:CB:B4:B2:B8:D2:B2:60:68:0A
./diaspora.aeriesguard.com/cert.pem
SHA1 Fingerprint=57:F8:2E:F8:89:0E:CB:29:B7:D9:14:DD:61:A7:9E:E1:C5:44:D0:E5
./labs.reactoweb.com/cert.pem
SHA1 Fingerprint=61:D3:F0:38:5D:71:5C:B2:7A:44:03:31:FC:B9:05:0D:A7:82:AE:97
./piwik.mrpetovan.com/cert.pem
SHA1 Fingerprint=A3:FB:B8:D0:C5:4A:74:D4:F2:16:CE:A0:4F:6F:C6:18:71:8C:CD:72
./friendica.mrpetovan.com/cert.pem
SHA1 Fingerprint=72:5E:7B:F9:91:4E:8A:D2:81:AD:38:9B:3D:D0:F7:4F:9F:91:95:CC
./mrpetovan.com/cert.pem
SHA1 Fingerprint=7F:E9:CB:A6:3B:BD:87:CD:6C:CB:BE:07:FF:2F:38:15:FA:97:12:DF
./hubzilla.mrpetovan.com/cert.pem
SHA1 Fingerprint=0E:C5:96:90:BE:4E:0E:30:3A:13:37:1F:DD:D9:21:1D:0D:9C:97:2B
./gnusocial.mrpetovan.com/cert.pem
SHA1 Fingerprint=81:BE:8F:0B:C0:13:D3:54:79:AC:39:A9:99:45:E6:C9:49:F9:BA:F0

However, the working certificate fingerprint I can see in my browser matches the one corresponding to friendica.mrpetovan.com so it sounds like it's usingthe fingerprint from a previous certificate maybe?

Steffen K9 🐰

5 years ago from libranet.de

Your fingerprints are funny. 😁

Adam

5 years ago

That's interesting... would an extra add-on be doing that? No smileys are showing on my end (and I personally would find that a little annoying in a 'code' block)...

Steffen K9 🐰

5 years ago from libranet.de

I have two add-ons enabled that come to my mind 'unicode-smileys' and 'highlightjs'.

Hypolite Petovan

5 years ago

Can confirm the issue with Unicode Emojis.

Hypolite Petovan

5 years ago

See https://github.com/friendica/friendica/pull/7286

Adam

5 years ago

Heh, that's awesome that a Friendica issue was resolved out of this.

Hypolite Petovan

5 years ago

And the cakes goes to Dojo: https://serverfault.com/a/845338

The reason was it was an intermittent error came from lingering Apache processes, even after restart, even after complete stop. These processes would have had the previous certificate in memory while the newer processes would correctly serve the newer certificate. And Apache would randomly assign a process for each request, hence the 1 error out of 25 requests.

Adam

5 years ago

I don't know anything about Dojo - does it manage your Apache, or it's own?

I find it amazing that Apache would allow itself to start again (during a restart) when other conflicting processes would already be around. I don't recall ever running in to that situation myself, but I guess it's entirely possible.

I'm glad you found it - I hope you find a solution to hammer away old apache processes in the future too - that would be a situation where unexpected behaviour would drive me a little crazy!

Hypolite Petovan

5 years ago

Dojo is the ServerFault username of the person who described my problem in the link I provided.

Adam

5 years ago

lol! Oh, that Dojo!

I wound up finding https://dojotoolkit.org/ after searching "apache dojo".

Nevermind me, then! lol

⇧