Is it possible to use wildcards in the Robots.txt file? I have heard that you can but only some search engines will pay attention to them. Is it bad to use wildcards, such as * and ?
Well, as far as i know yes you can use it BUT with robot files you need to try to think ahead…so you block a page to google that maybe you want yahoo to see…it just depends on what your focus is. Each bot is different so it depends on what the site wants to do
Add a New Topic
Reply to Post
Login
Register
Search 
Forums
Quote and Reply
Topic RSS 