This commit is contained in:
2025-10-25 20:11:21 +02:00
commit fd37421245
700 changed files with 211892 additions and 0 deletions

11
Work/Random/Ai Bots.md Normal file
View File

@@ -0,0 +1,11 @@
https://github.com/ai-robots-txt/ai.robots.txt
https://github.com/ai-robots-txt/ai.robots.txt/blob/main/robots.txt
### Using .htaccess
``` .htaccess
# Block bot traffic from certain user agents
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Amazonbot|SemrushBot|meta-externalagent|Bytespider) [NC]
RewriteRule .* - [R=503,L]
```