POST Request exceedingly slow

[Ross]

Does anyone have an idea of why a POST request for a drupal 7 site would be ridiculously slow, but all other operations would function normally?.

This has to do with the secure part of saharareporters. It's not simply that the process is slow, but it takes in the neighborhood of 2 minutes for the connection to be established. The logs show zero activity for the entire two minutes after the browser has attempted to take an action to post a story. Eventually, the content does get posted. The same behavior occurs when attempting to delete a story. Normally I would expect mysql being overloaded when something like this occurred, but that is not the case in this situation.

I have tried everything I can think of and have ruled out traffic or server load being the culprit. I ran tcpdump on my local machine and saw similar behavior. Initially, a few connections were made with the server hosting saharareporters, but then for the next two minutes a bunch of connections were made with some third parties like fastly.com and amazon.

Finally, I used iptables to only allow my machine to communicate with the server. This improved nothing. I'm completely mystified by this behavior.

Any suggestions welcome.

[JaimeV]

Which server/site is this happening on? It sounds like you've checked the same things I would but let's get jamie's input.

[Ross]

Oh sorry this is on acholonu, secure.sahararreporters.com.

[Jamie McClelland]

Wow, this is mysterious. What about third party spam protection services (like askimet? Maybe when you post content it is sent out for verification? - although that wouldn't happen on deletions...). If you download the site and database to your local machine does it happen also? Anything change prior to the problem starting?

If the site is perfectly responsive for other actions, then it sounds like some kind of security protection gone awry. The flood table lists IPs that are blocked - maybe something there?

How about checking watchdog for any clues? And the apache error log might shed some light.

And of course restarting apache, php fpm and mysql when you get desperate to just make it work. And drush cc all if you haven't already.

[Ross]

Thanks Jamie, your suggestions helped me find the issue. The big change that happened over the weekend was that ngugi.mayfirst.org was moved to another colo in Tokyo, causing a change in it's ip address. While I changed the ip for cloudflare, I did not change it's dns entry.

The drupal site is configured to purge any page on update using the domain name ngugi.mayfirst.org, so the purging had to wait to time out before allowing the post process to continue.

Anyway, thanks for your help.

~/ross

[Jamie McClelland]

Nice sleuthing! Glad you figured it out. What a obscure one!