Robots.txt Setting in Blogger / Blogspot

Robots.txt Setting - Maybe my friend had often heard the term robots.txt. What is robots.txt? if necessary in the settings? what if I leave it alone? probably many other questions.

To better understand the meaning and how the robots.txt, to make it more understandable, I will make the answer to question as the Hummingbird algorithm Regarding the article.

What's robots.txt?

Robots.txt is a command for the search engine robots to explore or browse a page on our blog. Robots.txt is arguably filter our blog from search engines.

Does every blog has a robots.txt?

Yes, all the blogs already have a robots.txt given by bloggers. By default robots.txt on blogs like this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://namablog/feeds/posts/default?orderby=UPDATED

To view the default robots.txt, please type dibrowser http: // name blog.blogspot.com/robots.txt

What is the meaning of the above code?

User-agent: Mediapartners-Google
Google Adsense robot crawls blogs

Disallow:
Are not allowed no

User-agent: *
All Robot Search Engines / Search engine

Disallow: / search
Not allowed to crawl the search folder and setrusnya, like .... / search / label and ... search / search? Updated ...

Allow: /
Allow all pages to be crawled, except that written on Disallow above. Mark (/) or less means the blog name.

Sitemap: http: // namablog / feeds / posts / default? Orderby = UPDATED
sitemap or address our blog feed.

Why not label incorporated into search?

By writing the default code Disallow: / search, meaning that crawl to label such?

 http://nameblog.com/search/label/Tutorial address, will not be crawled by search engines because the label is not a real URL to the specific page.

How do I prevent the robot on a particular page?

To prevent google crawl certain pages on your blog, do not want to tell my friend about me pages indexed by search engines. For the URL about me on your blog eg http://nameblog.blogspot.com/p/about.html

So for robots.txt, please copy the code above defaut, add a page that is not allowed, the result is like this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Disallow: /p/about.html
Allow: /
Sitemap: http://namablog/feeds/posts/default?orderby=UPDATED

Where is the way to edit the robots.txt?

For adding or editing robots.txt, go to Settings> Preferences search, on Robots.txt click Edit and then click Yes, then write the desired robots.txt code> Save Changes

I never permutations robots.txt, how?

It's okay, your blog will still be crawled by search engine robots because as I mentioned before, every blog already possessed default robots.txt.

Be careful with the use of robots.txt, if one could be writing your blog are ignored by search engines.