Answered: Should I allow google to Crawl everything ?

Post date: 2020-05-26 20:18:48
Views: 97

Sure, if your site is open to the public I don't see why not.

If you have any content you're not certain if it should or should not be indexed, you can block search indexing with 'noindex' 

Let's say you don't feel like sharing profile pages because it has too sensitive information about your users. To prevent all bots from indexing your User pages, you would open your  qa-theme.php  file and add the following code.

public function head_metas()
{
    if ($this->template === 'user') {
        $this->output('<meta name="robots" content="noindex">');
    }
    parent::head_metas();
}

This if you don't have a  head_metas()  function in your  qa-theme  already of course. If you have, you'd just paste that output in. If you only want to block Google from indexing, then you'd do:

<meta name="googlebot" content="noindex">

Read more about indexing here: https://support.google.com/webmasters/answer/93710?hl=en

Please click Here to read the full story.
 
Other Top and Latest Questions:
Informatica says it's not for sale, following Salesforce's reported interest in $10 billion deal
Crypto prices gain to start the week following first Bitcoin halving since 2020
Delta Air Lines gives staff another 5% raise, hikes starting wages to $19 an hour
Hi Grandma, Relax, We have a Newborn
How to start a high-paying side hustle: 'It has to be something that you're actually passionate about,' says expert
Shogun: A Dream of a Dream Show Only
Answered: Nested comments on replies
Movie: Hundreds of Beavers
Movie: National Lampoon's Van Wilder
Foot Locker debuts 'store of the future' as it looks to win back Wall Street's confidence