Feat: Let search engines crawl free. and raise the burst limit to ~20 req/10s

free.* now serves a robots.txt that allows everything, so free images embedded in
pages (e.g. apophenia.news articles) can be indexed and used for previews. The
main site and ai.* keep the homepage-only robots.txt. The WAF rate limit rule is
now 20 req/10s and skips known bots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-23 22:11:39 +00:00
co-authored by Claude Opus 5.5
parent 3181eda2a2
commit cd604838bd
5 changed files with 11 additions and 5 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
{
"version": 1,
"include": ["/*"],
"exclude": ["/", "/index.html", "/robots.txt", "/assets/*"]
"exclude": ["/", "/index.html", "/assets/*"]
}