Today, one of our MacOS X servers running Snow Leopard stopped sending push notifications.
The problem was a bit unusual, so I decided to share my notes on the issue.
I noticed that devices stopped receiving push notifications from this particular server and /var/log/system.log was stating connection errors:
3/9/13 5:20:52 PM push_notify notification server connect failed, will retry in 300 seconds
First, we need to check the physical connection to APNS servers:
telnet gateway.sandbox.push.apple.com 2195
Connected to gateway.sandbox.push-apple.com.akadns.net.
Escape character is '^]'.
If that is correct, we need to see if the certificates used by the server are correct.
Let’s look at what the server is using for push notifications:
$ sudo serveradmin settings notification:sslKeyFile
notification:sslKeyFile = "/etc/certificates/server.example.com.PMESMFPWKKB2K09I6I8BTKJERP5K152ADN239GDX8.concat.pem"
$ sudo serveradmin settings notification:sslCAFile
notification:sslCAFile = "/etc/certificates/server.example.com.PMESMFPWKKB2K09I6I8BTKJERP5K152ADN239GDX8.chain.pem"
Now, let’s look at the server’s certificates:
$ ls -l /etc/certificates/server.example.com.*
-rw-r--r-- 1 root wheel 1245 Mar 9 15:26 /etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.cert.pem
-rw-r--r-- 1 root wheel 1245 Mar 9 15:26 /etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.chain.pem
-rw-r----- 1 root certusers 2988 Mar 9 15:26 /etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.concat.pem
-rw-r----- 1 root certusers 1743 Mar 9 15:26 /etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.key.pem
As you can clearly see from the output – there is no such certificate in the storage.
All we have to do now is change the certificate to the correct one:
$ sudo serveradmin settings notification:sslKeyFile = "/etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.concat.pem"
$ sudo serveradmin settings notification:sslCAFile = "/etc/certificates/server.example.com.N3ZKL8CPVO96759Y38K22D682J3PNB3HA4DL9800J.chain.pem"
Then reboot, et voila! Push notification service works again!
P.S. If you have multiple certificates and don’t know which to select for APN service, you can find out the info about them using openssl:
find /etc/certificates -type f -name '*.cert.pem' | while read certificate; do openssl x509 -in $certificate -noout -text; done