Is there a way to identify code that sends HTTP requests to a remote server in the background?

I noticed that when I have spotty reception on my development machine, many shops take a long time to load, even running on localhost.

I have a feeling that is because some remote servers take a long time to respond due to my spotty connection. I would like to identify these requests and remove them, because I fear they might also slow down the live installations, for example if the external server is down or slow...

  • 1
    Please note/check whether you're allowed to remove these calls. You porbably are allowed, but I have seen instances where it is one of the extensions requirements. – David Mulder Nov 5 '14 at 17:50

We do it with a command line (Bash) function on any respective web server in the stack,

monitor_outbound ()
  tcpflow -p -c -i bond0 '( port 80 or port 443 ) and ' $(ip addr list | grep -Eoh "([0-9]{1,3}\.){3}[0-9]+" | sort -u | awk '{print "src host " $1 " or"}' | tr '\n' ' ' | sed -E 's# or $##g') | grep --line-buffered -oE '(GET|POST|HEAD) .* HTTP/1.[01]|Host: .*' | sed --unbuffered "s/^/$(date "+%b %d %T"): /g"

Short of that, you can leverage the CURL/fopen methods in Magento (but that's assuming they bother to use core methods to do it), more often than not, you'll find custom modules use their open file_get_contents() or curl methods.

You can do a quick check with a simple grep (assuming the code isn't obfuscated)

grep -ilrE "curl|file_get_contents|fopen|Zend_Http_Client" app/code/local app/code/community

And as @Marius said below, have a look for obscuficated modules, that might hide their methods in base64 encrypted strings

grep -ilrE "eval[\t ]+\(" app/code/local app/code/community

The only problem with the CLI method above, is whilst extremely effective for outbound HTTP requests, its useless for HTTPS (given the traffic is encrypted), the best you'll get is an IP/hostname, but nothing further.

  • 2
    You may also want to look for eval and see what happens in those eval functions. Most of the extensions that phone home use eval just so you don't find their phone number easily. – Marius Nov 5 '14 at 11:52
  • 2
    Updated to include a regex for eval's too – Ben Lessani - Sonassi Nov 5 '14 at 11:58

Hard one to answer - if some third party extensions are sending requests to a remote server this could be happening in a multitude of ways depending on how the request has been built. Your best option to find such instances (although potentially long winded) is to actually study the extension code, there isn't unfortunately any one blanket method for identifying things like this.

Having no idea what extensions you have installed or how they have been built, I will add that if the extension has been developed well, any such communication should have been done in some a way that it will never affect frontend performance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.