The only recent updates to the config have been yum updates (including a reboot to pick up the latest kernel). Php diagnostics on a few of the 12-second pages indicates that the pages are generated in less than a few 100 ms – so where are the other 11.+ seconds spent? The system is not using swap space and there is nothing in either the apache or vhost log files.
Anyone have any ideas on what is happening here?
For lack of any better idea, I removed mod_deflate but not using gzip compression seemed to have no effect on the response delay. I also tried explicitly flushing a couple of php pages – also no effect.
My only other ideas are to see if I can come up with the mix of page elements that triggers the problem and/or see if I can figure out how to capture the problem in a strace dump (not so nice on a production server).
Neither prospect looks like much fun 🙁