| s.txt is possibly the most miss understood file that a | | | | Many online marketers who provide ebooks or other |
| website can contain. | | | | digital products for instant download will list their |
| Many people think that by using a robots.txt file on their | | | | download thank you pages in the robots.txt file |
| website they are protecting pages and folders from | | | | because they obviously don't want those pages |
| thieves and hackers. In fact it is totally the opposite! | | | | indexed in search engines. |
| robots.txt opens up an enormous security hole that | | | | By using robots.txt this way though, you will be opening |
| hackers and theives will use to easily gain access to | | | | up your product to anyone who has a slight bit of |
| the parts of your website that you don't want them to. | | | | knowledge about how the file works.robots.txt is easily |
| What is robots.txt?robots.txt is a file that you create | | | | readable by any human that opens a browser and |
| and upload to your websites root directory that is used | | | | types in and if you have listed your thank you pages, |
| by search engine spiders to determine which parts of | | | | all they have to do is go to that url and take your |
| your website they should index and which folders | | | | product(s)! |
| pages that you, the website owner,don't want listed in | | | | It's that easy! |
| search engine indexes. | | | | And I'm living proof that this works, as this is exactly |
| Why would not want pages indexed? | | | | what happened to me. I had listed my thank you |
| There are many reasons why you might not want | | | | pages in robots.txt and thought that they were safe |
| search engines to index pages on your website, such | | | | from hackers and thieves, then one day I was |
| as private membership pages or exclusive training | | | | checking my web site stats and BAM, someone had |
| pages and such like. | | | | been to every single thank you page, and taken |
| If you are an Internet Marketer selling your own ebook | | | | everything. |
| or other digital product, you wouldn't want your thank | | | | The moral is, don't list any URL in robots.txt that you |
| you pages indexed either! | | | | don't want humans to have free access to. Use |
| And this is where the misunderstanding comes to the | | | | robots.txt with great caution and secure your thank |
| fore, and robots.txt becomes your foe. | | | | you pages using dedicated software. |