Feat: Let search engines crawl free. and raise the burst limit to ~20 req/10s

free.* now serves a robots.txt that allows everything, so free images embedded in
pages (e.g. apophenia.news articles) can be indexed and used for previews. The
main site and ai.* keep the homepage-only robots.txt. The WAF rate limit rule is
now 20 req/10s and skips known bots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-23 22:11:39 +00:00
co-authored by Claude Opus 5.5
parent 3181eda2a2
commit cd604838bd
5 changed files with 11 additions and 5 deletions
+2 -1
View File
@@ -108,6 +108,7 @@ Need images you can publish? `free.direct-img.link` only serves images marked **
- `i` above `20` serves `max-i.webp`, which lists each site's range. Other invalid values serve `bad.webp`
- Only free images are served: if there's no image (or fewer than `i` working ones), `bad.webp` is served (remembered for 24h) instead of an AI image. With a `src`, only that source's images are searched
- Each `src` + `i` + query combination is cached separately. Like the main site, other `i` values within an hour of a search reuse its results and don't count as searches
- Search engines may crawl `free.` (its robots.txt allows everything), so free images embedded in your pages can show up in image search and link previews. The main site and `ai.` allow only the homepage
**Sources:**
@@ -251,7 +252,7 @@ Create in your Cloudflare dashboard:
**Security → WAF → Rate limiting rules:**
1. **Rate Limit** — 15 req/10s per IP → Block 10s
1. **Rate Limit** — 20 req/10s per IP → Block 10s, skipping known bots (`and not cf.client.bot`) so search engines can crawl
### 8. Deploy