prashantidealittechno / robots.txt Goto Github PK
View Code? Open in Web Editor NEWThis project forked from jonasjacek/robots.txt
Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.
Home Page: https://github.com/jonasjacek/robots.txt
License: Other