Online User Agent Switcher
Get webpage contents from while setting the User Agent. The URL is requested to the server and the server responds back like the request was sent via browser or crawler.
There are two alternate ways to set your default user agent using php ini_set function or via .htaccess file.
Set via PHP
ini_set('user_agent', 'Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16');
Set via .htaccess
php_value user_agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1
List of Search Engine Web Crawlers
| Googlebot (Desktop) | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Googlebot (Smartphone) | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Googlebot News | Googlebot-News |
| Googlebot Images | Googlebot-Image/1.0 |
| Googlebot Video | Googlebot-Video/1.0 |
| Google AdsBot | AdsBot-Google (+http://www.google.com/adsbot.html) |
| Google Apps | AdsBot-Google-Mobile-Apps |
| Google AdSense | Mediapartners-Google |
| Bingbot | Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) |
| MSNBot | msnbot/2.0b (+http://search.msn.com/msnbot.htm) |
| MSNBot-Media | msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) |
| AdIdxBot | Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm) |
| BingPreview | Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b |
| YandexBot | Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) |
| Baiduspider | Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
| Yahoo! Slurp | Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| DuckDuckBot | DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html) |
Joydeep Deb
Senior Digital Marketer & Project Manager
Joydeep Deb is a results-driven Senior Digital Marketer and Project Manager with deep expertise in Lead Generation and Online Brand Management. An IIM Calcutta Alumni with an MBA in Marketing, he specializes in SEO, SEM (PPC), and Web Technologies.
Based in Bangalore, Karnataka - India.