SharePoint 2013 ignore code from search crawl or load balancer probes

Hi all,

Sometimes when your homepage has a control or a webpart that access external site (e.g. wordpress blog), it will create unnecessary traffic when the page is touched by a search crawl or load balancer probes.

If your probes are checking every few minutes it can create massive unwanted traffic that can increase your internet cost.

The solution is to check the UserAgent of your request header to see where the request is coming from, and skip your code if it's coming from either of them.