Online User Agent Switcher
Get webpage contents from while setting the User Agent. The URL is requested to the server and the server responds back like the request was sent via browser or crawler.
There are two alternate ways to set your default user agent using php ini_set function or via .htaccess file.
Set via PHP
ini_set('user_agent', 'Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16');
Set via .htaccess
php_value user_agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1
List of Search Engine Web Crawlers
| Googlebot (Desktop) | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Googlebot (Smartphone) | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Googlebot News | Googlebot-News |
| Googlebot Images | Googlebot-Image/1.0 |
| Googlebot Video | Googlebot-Video/1.0 |
| Google AdsBot | AdsBot-Google (+http://www.google.com/adsbot.html) |
| Google Apps | AdsBot-Google-Mobile-Apps |
| Google AdSense | Mediapartners-Google |
| Bingbot | Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) |
| MSNBot | msnbot/2.0b (+http://search.msn.com/msnbot.htm) |
| MSNBot-Media | msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) |
| AdIdxBot | Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm) |
| BingPreview | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b |
| YandexBot | Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) |
| Baiduspider | Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
| Yahoo! Slurp | Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| DuckDuckBot | DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html) |
