Init
This commit is contained in:
11
Work/Random/Ai Bots.md
Normal file
11
Work/Random/Ai Bots.md
Normal file
@@ -0,0 +1,11 @@
|
||||
https://github.com/ai-robots-txt/ai.robots.txt
|
||||
|
||||
https://github.com/ai-robots-txt/ai.robots.txt/blob/main/robots.txt
|
||||
|
||||
### Using .htaccess
|
||||
``` .htaccess
|
||||
# Block bot traffic from certain user agents
|
||||
RewriteEngine On
|
||||
RewriteCond %{HTTP_USER_AGENT} (Amazonbot|SemrushBot|meta-externalagent|Bytespider) [NC]
|
||||
RewriteRule .* - [R=503,L]
|
||||
```
|
||||
Reference in New Issue
Block a user