what is robots txt for seo and how to create one

With the robots file you have more control of your website pages, and what you want the search engine to see or not see..

Hello folks thank you for joining my course in this video we're going to learn about What is the robots.txt for SEO and how to create one.

HOW TO ACCESS A ROBOTS.TXT FILE

The robots.txt is a simple file with directives that can be manipulated by the crawlers and read by the search engine. However, in technical SEO using those directives, allow you to choose if your content can go live on google search or not.

You can access your robots.txt file by adding /robots.txt at the end of your domain for example :


http://example.com/robots.txt

Grab your FREE MASTER CHECKLIST and learn how to do SEO TO YOUR NEXT CLIENT.

LOCAL-SEO-AUDIT V0.1 -2022-

If you try this out for your website and suddenly you've been taken to a 404 error page that means it's time to create a robot file.


To create robots file simple way >go to your desk >right click >create new file txt>copy the code down below>save it as txt file>name it “robots” no capitals>upload into the root directory of your website >

User-Agent: *

Allow: /

https://example.com/sitemap.xml


The star plays as a wild card (all the web crawlers type bing, yahoo)

Allow: / (index yes /)

For WordPress users > install Yoast plugin it's a software all-in-one SEO >go to settings robots.txt >click on generate the file and it's done.



Watch The Video Course.