What is Robots.txt File and How to Add in Blog – Full Guide

What is Robots.txt File and How to Add in Blog – Full Guide


If you are a blogger or you have just started blogging, then you must have heard about this in the blogger or wordpress, about the robots.txt file, you must know about the robots, the robot is a floor machine and this is our command This is the work that robots.txt file belongs to. Whatever command we will give to this file, it will sends these commands directly to the search engine as you know that search engine’s showing result to user according to there search.

What is a robots.txt file ?

As the name suggests .txt is an extension of a text file in which we can only write text. This is a robot.txt file, in which we write some text or message. In this message we will share the contents of our blog Let’s write about which parts to show in the search engine and not.

For example, if you do not want to display the page in your search engine from your blog or if you do not want the category or tags to be indexed in your blog, then you can snooze it all in a robots.txt file after that Even a search engine like google, yahoo, bing will start indexing the posts of your blog, then all these search engines will get a message from the robots.txt file, that all these things from the blog are indexed and All these things Do not index it, this is the advantage it makes your blog seo friendly and it makes the search engine easy to index which part of the index and no angle.

How to add a Robots.txt file to Blog.

In Blog, you can go to the option of the robots.txt file and go to the search preferences.
make blogs SEO friendly , then we have the option of these robots.txt file. If you leave this option in the blog¬†where the other people leave it, then the blogger automatically creates the best robots.txt file for you, but if you want to have certain parts hedged from your blog to the search engine Then you can do it, But if you are taking blogging seriously and later you have migrated your blogspot blog to wordpress then you should use this feature. Now let’s learn how to add this file to blogspot blog. Before that let us know about some things that you will use in this file.

Allow: If you allow something inside the text file, it means that you want this part to appear in the search engine
Disallow: If you disallow a post, it means that you do not want these parts to appear in search engines
User-agent: By using this option you can show your visitors what they want to appear on their site, such as the source of the image, saying this file is SAV etc.
User-agent: Mediapartners-Google: Using this, you can hide the adsense code from user if you disallow it, google adsense will not show ads in your site.

1. Go to the settings of the blogger and click on the search preferences and then click on robots.txt and then click on edit

Blogger, Setting, robot

2. Now you have to copy the code below and paste it inside the box and save it

User-agent: Mediapartners-Google



Disallow: /search

Allow: /

Sitemap: http://www.Chandigarhkeasri.com/sitemap.xml

Note: Place your sitemap on the place of the sitemap

Blog, Robot, blogger, Setting, wordpress,

This way, you can do some blogs and show on your blogspot blog by placing it in robots.txt file and if you are using a blogger then this feature will not be good for you and if you use the wordpress use If you are using this feature, please use it.

How to Add Robots.txt File in WordPress
1. If you use wordpress then you must have installed Yoast SEO, then you have to go to yoast seo and click Tools, then click on the file editor.

Blogger, Blog, Setting, Robot, Yoast, Seo

2. Now copy the code below and paste it inside the robots.txt file and then save changes to robots.txt and save it.

User-agent: *
Disallow: /wp-admin/
Disallow: /cgi-bin/
Disallow: /comments/feed/
Disallow: /trackback/
Disallow: /xmlrpc.php
Allow: /wp-admin/admin-ajax.php

User-agent: Mediapartners-Google*
Allow: /
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Mobile
Allow: /

Sitemap: https://catchhow.com/sitemap.xml

Note: Place your sitemap on the place of the sitemap

blog, blogger, setting, robot.txt, seoTips: If you use blogspot blog then you do not add a robots.txt file, Google automatically creates the best robots file for you, and if you use wordpress then you should use this robot file properly If you have any questions then you can ask in the comment box below

Leave a Reply

Close Menu