Many commercial tools emulate the behavior of search crawlers. But none beats data straight from the search engines themselves: Google Search Console and Bing Webmaster Tools.
Webmaster Tools’ “Keyword Research” report can uncover new keyword opportunities and competitors. Click any word or phrase in the list to see:
Site Explorer provides an overview of your site’s structure and a detailed review of each section or URL. It’s essentially a free crawler.
Google Search Console
Search Console discloses the pages Google has indexed and provides helpful tips on improving other metrics, such as click-through rates, page speed, and more. The platform also shows keywords that users search to find your site.
I listed last month six critical Search Console reports. But there are other valuable sections.
Google Search Console and Bing Webmaster Tools can help site owners improve organic traffic. Both provide info on their own search engine. Yet each offers a different angle, making it a good idea to use them concurrently.
Bing Webmaster Tools provides helpful (and free) SEO tools. Bing provides an easy way to import your data from Google Search Console into a Webmaster Tools account. Verification is near-instant. Populating all reports can take up to 48 hours, in my experience.
- The “Enhancements” tab addresses your rich snippets and whether Google has used them. The section shows only the snippets you quality for. For example, you won’t see “Product” in this section unless you’ve applied “product” markup on your site (using Schema.org or similar).
- “Removals” allow site owners to request the removal of any URL from Google’s index.
- “International targeting” (Legacy tools and reports > International targeting) allows site owners to specify a country to rank in.
- “Change of address” (Settings > Change of address) alerts Google of upcoming domain name changes to mitigate the impact.
Bing Webmaster Tools
Search Console has many useful reports for both beginners and experienced SEOs. The tool augments basic search info with specifics about how Google’s bot has interacted with the site.
Unlike Search Console, Webmaster Tools does not assess your site’s security. But it does disclose the status of a site’s copyright infringement claims, wherein a verified owner has filed with Bing a copyright infringement notice against another site. It’s an example of the two tools complementing each other.
Bing’s URL Inspection report includes optimization tips and missing elements.
To get started, verify your site with Search Console by connecting it to the Google Analytics account or using a plugin.
Webmaster Tool’s Keyword Research report can uncover new keyword opportunities and competitors, among other benefits.
- Related keywords,
- “Question keywords” those in a question format to include in your copy,
- Newly discovered and trending keywords.
For indexed URLs, Bing offers an enhancement report with optimization tips and missing elements.
Knowing how search engine crawlers interact with your site is crucial for organic rankings. The activity of crawlers reveals discovered web pages, followable links, accessible code, and more.