mirror of
https://github.com/direct-img/direct-img.link.git
synced 2026-10-29 07:24:52 +00:00
Feat: Promote every ?i= and ?src=, measured by a WAF logging rule
Analytics has no query string, so a WAF custom rule logs image requests with theirs and promote.js splits each path's views across its variants. The rewrite names static files after the param values (/orange%2Bcat/i2-), since the static file server redirects names with = or &. Drops the legacy /free/5/x paths and the /1/ segment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -270,7 +270,7 @@ Fork this repo and add these **GitHub Actions secrets** (Settings → Secrets an
|
||||
|
||||
Change the three `routes` in `wrangler.jsonc` to your domain, `free.<your-domain>` and `ai.<your-domain>` (each needs a proxied DNS record; with nothing behind it, use `AAAA 100::`). Any hostname starting with `free.` serves free images, and any starting with `ai.` serves AI images; everything else serves regular search.
|
||||
|
||||
Then add the rules that let static images work (needs a token with Zone: Transform Rules Edit):
|
||||
Then add the rules that let static images work (needs a token with Zone: Transform Rules Edit and Zone WAF Edit):
|
||||
|
||||
```bash
|
||||
CLOUDFLARE_API_TOKEN=... node scripts/rules.js <your-domain> free.<your-domain> ai.<your-domain>
|
||||
@@ -299,24 +299,25 @@ Add an object lifecycle rule in **R2 → direct-img-store → Settings** that ap
|
||||
|
||||
### Static Images
|
||||
|
||||
Every request runs the Worker, which counts toward the free plan's 100,000 requests a day, except requests for files in `public/`: those are free, unlimited and keep working past the limit. So before each deploy, `scripts/promote.js` copies the most viewed images from R2 into `public/`:
|
||||
Every request runs the Worker, which counts toward the free plan's 100,000 requests a day, except requests for files in `public/`: those are free, unlimited and keep working past the limit. So before each deploy, `scripts/promote.js` copies the most viewed images (any `?i=` and `?src=`) from R2 into `public/`:
|
||||
|
||||
- Views come from Cloudflare's zone analytics over the last 7 days, which count static files too, so a promoted image that stops being viewed drops out
|
||||
- Views come from the last 7 days of Cloudflare's zone analytics, which count static files too, so a promoted image that stops being viewed drops out. Analytics has no query string, so each query's views are split across its `?i=` and `?src=` by a WAF custom rule (`scripts/rules.js`) that logs every image request with its query string. The job reads up to 1,000 logged requests per hour, enough to estimate the split on a busy day. These show up as **Skip** events in **Security → Events**
|
||||
- The top 19,900 with at least `MIN_VIEWS` (20) are promoted, up to `MAX_BYTES` (4 GB) in total. The free plan allows 20,000 static files per deploy
|
||||
- Only JPEG, PNG, GIF and WebP are promoted, and only queries of up to 8 words (see below). Static files get their type from their extension, which these have none of, so `public/_headers` declares them all `image/jpeg`, and browsers detect these four formats from the bytes anyway
|
||||
- Images are cached between runs (Actions cache), so only newly promoted ones download from R2
|
||||
- An image is dropped once it's 90 days old, the same as the R2 lifecycle rule, so a popular query (e.g. `current us president`) is still searched again instead of staying static forever. It's promoted again the next day if it's still popular
|
||||
|
||||
Static files ignore query strings and are shared by all three hostnames, so URL Rewrite Rules (`scripts/rules.js`) turn each bare query into its static file's path, under its hostname, first. The visitor's URL doesn't change:
|
||||
Static files ignore query strings and are shared by all three hostnames, so URL Rewrite Rules (`scripts/rules.js`) turn each image request into its static file's path first: under its hostname, named after its `?i=` and `?src=`. The visitor's URL doesn't change:
|
||||
|
||||
| Request | Looked up as |
|
||||
|---|---|
|
||||
| `direct-img.link/orange+cat` | `/direct-img.link/1/orange%2Bcat` (lowercased, `+` encoded like the file's name) |
|
||||
| `free.direct-img.link/Orange+Cat` | `/free.direct-img.link/1/orange%2Bcat` |
|
||||
| `direct-img.link/orange+cat` | `/direct-img.link/orange%2Bcat/i-` (lowercased, `+` encoded like the file's name) |
|
||||
| `direct-img.link/Orange+Cat?i=2` | `/direct-img.link/orange%2Bcat/i2-` |
|
||||
| `free.direct-img.link/orange+cat?src=wikimedia&i=2` | `/free.direct-img.link/orange%2Bcat/i2-wikimedia` (either param order) |
|
||||
|
||||
A promoted file there serves it; otherwise the Worker gets the request, unfolds the path and handles it as before. Requests with any params (`?i=`, `?src=`), a literal `+` (`%2B`) or a trailing slash aren't rewritten and always go to the Worker. Analytics has no query string, so `?i=` views count toward the query's `?i=1`, and only `?i=1` is promoted.
|
||||
A promoted file there serves it; otherwise the Worker gets the request, restores the path and handles it as before. Requests with other params, a literal `+` (`%2B`) or a trailing slash aren't rewritten and always go to the Worker. `?i=1` and a bare query are separate files (`i1-` and `i-`), each promoted on its own views.
|
||||
|
||||
A rule's condition sees the original path, and the free plan has no `regex_replace` and allows one `wildcard_replace` per rule, so there's one rule per number of `+`s (0–7, 8 of the plan's 10). Queries of 9+ words are never promoted.
|
||||
The static file server redirects a name with `+`, `=` or `&` to its encoded form, which would change the visitor's URL, so the rules encode `+` as `%2B` and build the name from the param values. A rule's condition sees the original path, and the free plan has no `regex_replace` and allows one `wildcard_replace` per rule, so there's one rule per number of `+`s (0–7, 8 of the plan's 10). Queries of 9+ words are never promoted.
|
||||
|
||||
### KV: `DIRECT_IMG_CACHE`
|
||||
|
||||
|
||||
+5
-3
@@ -31,10 +31,12 @@ if (r.workers) {
|
||||
}
|
||||
if (r.workersError) lines.push(`**Worker requests:** couldn't read (${esc(r.workersError)})`);
|
||||
|
||||
// free.direct-img.link/1/orange+cat → [orange cat](https://free.direct-img.link/orange+cat) (free), so each opens on its site
|
||||
// free.direct-img.link/orange+cat/i2-wikimedia → [orange cat](https://free.direct-img.link/orange+cat?i=2&src=wikimedia)
|
||||
// i=2 src=wikimedia (free), so each opens on its site
|
||||
const link = id => {
|
||||
const [host, , name] = id.split("/"), site = host.split(".").length > 2 && host.split(".")[0];
|
||||
return `[${esc(name.replaceAll("+", " "))}](https://${host}/${name})${site ? ` (${site})` : ""}`;
|
||||
const [host, name, file] = id.split("/"), [, i, src] = file.match(/^i([^-]*)-(.*)$/), site = host.split(".").length > 2 && host.split(".")[0];
|
||||
const qs = new URLSearchParams(Object.entries({ i, src }).filter(([, v]) => v)).toString();
|
||||
return `[${esc(name.replaceAll("+", " "))}](https://${host}/${name}${qs && `?${qs}`})${qs && ` ${esc(qs.replace("&", " "))}`}${site ? ` (${site})` : ""}`;
|
||||
};
|
||||
|
||||
const failed = env.STATUS !== "success";
|
||||
|
||||
+44
-24
@@ -1,10 +1,11 @@
|
||||
// Copies the most viewed images from R2 into public/, so the next deploy serves them as static files: free, unlimited
|
||||
// and without running the worker. Views come from Cloudflare's analytics, which count static files too, so a promoted
|
||||
// image that stops being viewed drops out. Run before every deploy (.github/workflows/deploy.yml); writes
|
||||
// promoted/report.json for scripts/notify.js. promoted/ is kept between runs (Actions cache), so only new images download.
|
||||
// Copies the most viewed images (any ?i= and ?src=) from R2 into public/, so the next deploy serves them as static
|
||||
// files: free, unlimited and without running the worker. Views come from Cloudflare's analytics and the WAF rule's logs,
|
||||
// which count static files too, so a promoted image that stops being viewed drops out. Run before every deploy
|
||||
// (.github/workflows/deploy.yml); writes promoted/report.json for scripts/notify.js. promoted/ is kept between runs
|
||||
// (Actions cache), so only new images download.
|
||||
import { mkdir, readFile, writeFile, link, rm, readdir } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import { TTL_SECONDS, MAX_PLUSES, siteOf, normalizeQuery, cacheKey, r2Key } from "../src/_utils/keys.js";
|
||||
import { TTL_SECONDS, MAX_INDEX, MAX_PLUSES, parseIndex, siteOf, normalizeQuery, cacheKey, r2Key } from "../src/_utils/keys.js";
|
||||
import { parseFreeSource } from "../src/_utils/free.js";
|
||||
import { parseAiSource } from "../src/_utils/ai.js";
|
||||
|
||||
@@ -21,6 +22,8 @@ const MAX_BYTES = +(env.MAX_BYTES || 4e9);
|
||||
const TYPES = ["image/jpeg", "image/jpg", "image/pjpeg", "image/png", "image/gif", "image/webp"];
|
||||
// Queries whose file name the rules or the asset lookup would change, so a promoted file would never match
|
||||
const BAD_NAME = /[+.%\/\\?#\x00-\x1f\x7f]/;
|
||||
// Logged requests read per hour, enough to split a busy path's views across its ?i= and ?src=
|
||||
const PER_HOUR = 1000;
|
||||
const CACHE = "promoted", OUT = "public";
|
||||
const sleep = ms => new Promise(r => setTimeout(r, ms));
|
||||
|
||||
@@ -39,31 +42,48 @@ async function gql(query) {
|
||||
return r.data.viewer;
|
||||
}
|
||||
|
||||
// The image a request for a bare query (/orange+cat) was for. Analytics has no query string, so a ?i= view counts toward
|
||||
// its query's ?i=1 too, and only ?i=1 is promoted (other ?i= always run the worker). null for anything else (the
|
||||
// homepage, assets, probes like /wp-admin/x.php).
|
||||
async function parse(host, path) {
|
||||
const site = siteOf(host), m = path.match(/^\/([^/.]+)$/), query = m && normalizeQuery(m[1]);
|
||||
if (!query || query.length > 200) return null;
|
||||
const name = query.replaceAll(" ", "+");
|
||||
if (BAD_NAME.test(query) || Buffer.byteLength(name) > 200 || name.split("+").length > MAX_PLUSES + 1) return null;
|
||||
const src = site === "free" ? parseFreeSource(new URLSearchParams()) : site === "ai" ? parseAiSource(new URLSearchParams()) : null;
|
||||
return { id: `${host}/1/${name}`, r2: await r2Key(site, cacheKey(site, src, 1, query)) };
|
||||
// The image a request was for, from its path (/orange+cat) and query string (?i=2&src=wikimedia), checked like the
|
||||
// worker checks them. Its id is its static file's path, which the rules build the same way
|
||||
// (free.direct-img.link/orange+cat/i2-wikimedia). null for anything else (the homepage, assets, probes like
|
||||
// /wp-admin/x.php, params the worker rejects).
|
||||
async function image(host, path, qs) {
|
||||
const site = siteOf(host), m = path.match(/^\/([^/.]+)$/), query = m && normalizeQuery(m[1]), p = new URLSearchParams(qs);
|
||||
if (!query || query.length > 200 || [...p.keys()].some(k => k !== "i" && !(site && k === "src"))) return null;
|
||||
const i = parseIndex(p), src = site === "free" ? parseFreeSource(p) : site === "ai" ? parseAiSource(p) : null;
|
||||
if (!i || i > MAX_INDEX[site] || (site && !src)) return null;
|
||||
const name = query.replaceAll(" ", "+"), file = `i${p.get("i") ?? ""}-${p.get("src") ?? ""}`;
|
||||
if (BAD_NAME.test(query) || /[^\w-]/.test(file) || Buffer.byteLength(name) > 200 || name.split("+").length > MAX_PLUSES + 1) return null;
|
||||
return { id: `${host}/${name}/${file}`, r2: await r2Key(site, cacheKey(site, src, i, query)) };
|
||||
}
|
||||
|
||||
// Views per image over the last DAYS: one query per host and day, since a query returns at most 10,000 rows. Only 200s
|
||||
// count, so redirects (e.g. /favicon.ico) aren't.
|
||||
// Views per image over the last DAYS. Analytics counts requests per path (one query per host and day, at most 10,000
|
||||
// rows each; only 200s, so redirects like /favicon.ico aren't) but has no query string, so each path's count is split
|
||||
// across its ?i= and ?src= by that day's logged requests (the WAF rule from scripts/rules.js), read an hour at a time
|
||||
// so a busy day is sampled evenly. A path with none logged that day counts as the bare query.
|
||||
async function views(zoneTag) {
|
||||
const totals = new Map(), now = Date.now();
|
||||
const now = Date.now(), at = h => new Date(now - h * 36e5).toISOString();
|
||||
const logged = Array.from({ length: DAYS }, () => new Map());
|
||||
for (let h = 0; h < DAYS * 24; h++) {
|
||||
const { zones } = await gql(`zones(filter: {zoneTag: "${zoneTag}"}) { firewallEventsAdaptive(limit: ${PER_HOUR}, filter: {datetime_geq: "${at(h + 1)}", datetime_lt: "${at(h)}", action: "skip", source: "firewallCustom"}) { clientRequestHTTPHost clientRequestPath clientRequestQuery sampleInterval } }`);
|
||||
for (const e of zones[0].firewallEventsAdaptive) {
|
||||
const day = logged[Math.floor(h / 24)], key = e.clientRequestHTTPHost + e.clientRequestPath;
|
||||
const qs = day.get(key) || day.set(key, new Map()).get(key);
|
||||
qs.set(e.clientRequestQuery, (qs.get(e.clientRequestQuery) || 0) + e.sampleInterval);
|
||||
}
|
||||
}
|
||||
const totals = new Map();
|
||||
let total = 0;
|
||||
for (const host of HOSTS) for (let d = 0; d < DAYS; d++) {
|
||||
const range = `datetime_geq: "${new Date(now - (d + 1) * 864e5).toISOString()}", datetime_lt: "${new Date(now - d * 864e5).toISOString()}"`;
|
||||
const range = `datetime_geq: "${at((d + 1) * 24)}", datetime_lt: "${at(d * 24)}"`;
|
||||
const { zones } = await gql(`zones(filter: {zoneTag: "${zoneTag}"}) { httpRequestsAdaptiveGroups(limit: 10000, orderBy: [count_DESC], filter: {${range}, clientRequestHTTPHost: "${host}", edgeResponseStatus: 200}) { count dimensions { clientRequestPath } } }`);
|
||||
for (const { count, dimensions } of zones[0].httpRequestsAdaptiveGroups) {
|
||||
const img = await parse(host, dimensions.clientRequestPath);
|
||||
if (!img) continue;
|
||||
total += count;
|
||||
totals.set(img.id, { ...img, views: (totals.get(img.id)?.views || 0) + count });
|
||||
for (const { count, dimensions: { clientRequestPath: path } } of zones[0].httpRequestsAdaptiveGroups) {
|
||||
const qs = logged[d].get(host + path) || new Map([["", 1]]), weight = [...qs.values()].reduce((a, b) => a + b, 0);
|
||||
for (const [q, w] of qs) {
|
||||
const img = await image(host, path, q);
|
||||
if (!img) continue;
|
||||
total += (count * w) / weight;
|
||||
totals.set(img.id, { ...img, views: (totals.get(img.id)?.views || 0) + (count * w) / weight });
|
||||
}
|
||||
}
|
||||
}
|
||||
return { list: [...totals.values()], total };
|
||||
|
||||
+40
-23
@@ -1,32 +1,49 @@
|
||||
// Sets the zone's URL Rewrite Rules that turn a bare query into its static file's path, under its hostname
|
||||
// (free.direct-img.link/Orange+Cat → /free.direct-img.link/1/orange%2Bcat), so a promoted image is served without
|
||||
// running the worker and the visitor's URL doesn't change. +s become %2B (the static file's own encoding), since a +
|
||||
// would get a redirect to it. Replaces only the rules it made earlier.
|
||||
// Sets the zone's rules for static images, replacing only the ones it made earlier:
|
||||
// - URL Rewrite Rules give each image request a static file path under its hostname, named after its ?i= and ?src=
|
||||
// (free.direct-img.link/Orange+Cat?i=2 → /free.direct-img.link/orange%2Bcat/i2-), so a promoted image is served
|
||||
// without running the worker and the visitor's URL doesn't change. +s become %2B and the params skip = and &, since
|
||||
// the static file server redirects to those encodings otherwise.
|
||||
// - A WAF custom rule logs every image request with its query string (analytics has no query string), which is how
|
||||
// scripts/promote.js tells ?i= and ?src= apart. It's a skip that skips nothing: it's kept last in its ruleset.
|
||||
// node scripts/rules.js direct-img.link free.direct-img.link ai.direct-img.link
|
||||
// With no hostnames it removes them. Needs CLOUDFLARE_API_TOKEN with Zone > Transform Rules edit.
|
||||
// With no hostnames it removes them. Needs CLOUDFLARE_API_TOKEN with Zone > Transform Rules and Zone WAF edit.
|
||||
import { MAX_PLUSES } from "../src/_utils/keys.js";
|
||||
|
||||
const env = process.env, hosts = process.argv.slice(2), ZONE = env.ZONE || "direct-img.link", TAG = "direct-img fold:";
|
||||
const PHASE = "http_request_transform", PATH = "http.request.uri.path";
|
||||
const PATH = "http.request.uri.path", ARG = n => `http.request.uri.args["${n}"][0]`;
|
||||
const api = (path, init) => fetch(`https://api.cloudflare.com/client/v4${path}`, { ...init, headers: { Authorization: `Bearer ${env.CLOUDFLARE_API_TOKEN}`, "Content-Type": "application/json" } }).then(r => r.json());
|
||||
// A bare query with no params: not the homepage, assets/, robots.txt or an already folded path. A literal + (%2B) is
|
||||
// left to the worker, since a static file's %2B stands for a space.
|
||||
const BARE = `http.host in {${hosts.map(h => JSON.stringify(h)).join(" ")}} and http.request.uri.query eq "" and ${PATH} ne "/" and not ${PATH} contains "." and not ${PATH} wildcard "/*/*" and not lower(${PATH}) contains "%2b"`;
|
||||
// An image request: not the homepage, assets/, robots.txt, probes like /wp-admin/x.php or an already rewritten path
|
||||
const IMAGE = `http.host in {${hosts.map(h => JSON.stringify(h)).join(" ")}} and ${PATH} ne "/" and not ${PATH} contains "." and not ${PATH} wildcard "/*/*"`;
|
||||
// Folded only with no params but i and src (others get bad.webp from the worker), and without a literal + (%2B), since
|
||||
// a static file's %2B stands for a space
|
||||
const FOLDABLE = `${IMAGE} and (http.request.uri.query eq "" or all(http.request.uri.args.names[*] in {"i" "src"})) and not lower(${PATH}) contains "%2b"`;
|
||||
const pluses = n => `"${"*+".repeat(n)}*"`;
|
||||
|
||||
// A rule's condition sees the original path, and the free plan has no regex_replace and allows one wildcard_replace per
|
||||
// rule, so each rule handles an exact number of +s; a query with more always runs the worker
|
||||
const mine = hosts.length ? Array.from({ length: MAX_PLUSES + 1 }, (_, n) => ({
|
||||
description: `${TAG} bare query with ${n} +`,
|
||||
expression: `${BARE} and ${n ? `${PATH} wildcard ${pluses(n)}` : `not ${PATH} contains "+"`} and not ${PATH} wildcard ${pluses(n + 1)}`,
|
||||
action: "rewrite",
|
||||
action_parameters: { uri: { path: { expression: `concat("/", http.host, "/1", ${n ? `wildcard_replace(lower(${PATH}), ${pluses(n)}, "${Array.from({ length: n }, (_, k) => `\${${k + 1}}%2B`).join("")}\${${n + 1}}")` : `lower(${PATH})`})` } } },
|
||||
})) : [];
|
||||
const rules = {
|
||||
// A rule's condition sees the original path, and the free plan has no regex_replace and allows one wildcard_replace
|
||||
// per rule, so each rule handles an exact number of +s; a query with more always runs the worker. A missing param is
|
||||
// empty, so a bare query is i-.
|
||||
http_request_transform: hosts.length ? Array.from({ length: MAX_PLUSES + 1 }, (_, n) => ({
|
||||
description: `${TAG} query with ${n} +`,
|
||||
expression: `${FOLDABLE} and ${n ? `${PATH} wildcard ${pluses(n)}` : `not ${PATH} contains "+"`} and not ${PATH} wildcard ${pluses(n + 1)}`,
|
||||
action: "rewrite",
|
||||
action_parameters: { uri: { path: { expression: `concat("/", http.host, ${n ? `wildcard_replace(lower(${PATH}), ${pluses(n)}, "${Array.from({ length: n }, (_, k) => `\${${k + 1}}%2B`).join("")}\${${n + 1}}")` : `lower(${PATH})`}, "/i", ${ARG("i")}, "-", ${ARG("src")})` } } },
|
||||
})) : [],
|
||||
http_request_firewall_custom: hosts.length ? [{
|
||||
description: `${TAG} log image requests`,
|
||||
expression: IMAGE,
|
||||
action: "skip",
|
||||
action_parameters: { ruleset: "current" },
|
||||
logging: { enabled: true },
|
||||
}] : [],
|
||||
};
|
||||
|
||||
const zone = (await api(`/zones?name=${ZONE}`)).result[0].id;
|
||||
const others = ((await api(`/zones/${zone}/rulesets/phases/${PHASE}/entrypoint`)).result?.rules || [])
|
||||
.filter(r => !r.description?.startsWith(TAG))
|
||||
.map(({ id, version, last_updated, ref, ...r }) => r);
|
||||
const res = await api(`/zones/${zone}/rulesets/phases/${PHASE}/entrypoint`, { method: "PUT", body: JSON.stringify({ rules: [...others, ...mine] }) });
|
||||
console.log(res.success ? `${mine.length} set, ${others.length} others kept` : JSON.stringify(res.errors));
|
||||
if (!res.success) process.exitCode = 1;
|
||||
for (const [phase, mine] of Object.entries(rules)) {
|
||||
const others = ((await api(`/zones/${zone}/rulesets/phases/${phase}/entrypoint`)).result?.rules || [])
|
||||
.filter(r => !r.description?.startsWith(TAG))
|
||||
.map(({ id, version, last_updated, ref, ...r }) => r);
|
||||
const res = await api(`/zones/${zone}/rulesets/phases/${phase}/entrypoint`, { method: "PUT", body: JSON.stringify({ rules: [...others, ...mine] }) });
|
||||
console.log(phase, res.success ? `${mine.length} set, ${others.length} others kept` : JSON.stringify(res.errors));
|
||||
if (!res.success) process.exitCode = 1;
|
||||
}
|
||||
|
||||
@@ -10,6 +10,12 @@ export const TTL_SECONDS = 90 * 24 * 60 * 60;
|
||||
// Higher values serve max-i.webp, which lists these, so redraw it from design/max-i.html when they change.
|
||||
export const MAX_INDEX = { "": 40, free: 20, ai: 1000 };
|
||||
|
||||
// ?i=1 or higher (default 1). Returns null if invalid.
|
||||
export function parseIndex(searchParams) {
|
||||
const i = searchParams.get("i") || "1";
|
||||
return /^\d+$/.test(i) && +i >= 1 ? +i : null;
|
||||
}
|
||||
|
||||
// "free", "ai" or "" (main site), from the hostname's first label
|
||||
export const siteOf = hostname => ["free", "ai"].find(s => hostname.startsWith(`${s}.`)) || "";
|
||||
|
||||
|
||||
+10
-18
@@ -4,7 +4,7 @@ import { freeImageUrls, parseFreeSource, FREE_UA } from "./_utils/free.js";
|
||||
import { isFlickrPlaceholder } from "./_utils/flickr.js";
|
||||
import { aiImage, parseAiSource, DEFAULT_AI_SOURCE } from "./_utils/ai.js";
|
||||
import { surreal } from "./_utils/surreal.js";
|
||||
import { TTL_SECONDS, MAX_INDEX, siteOf, normalizeQuery, cacheKey as keyFor, r2Key as r2KeyFor, sha256 } from "./_utils/keys.js";
|
||||
import { TTL_SECONDS, MAX_INDEX, parseIndex, siteOf, normalizeQuery, cacheKey as keyFor, r2Key as r2KeyFor, sha256 } from "./_utils/keys.js";
|
||||
import * as LIMITS from "../public/assets/rate-limit.js";
|
||||
|
||||
// How long a failed search keeps serving bad.webp before the query is searched again
|
||||
@@ -23,8 +23,7 @@ const FIRECRAWL_SHARE = 500 / 1500;
|
||||
export default { fetch: handle };
|
||||
|
||||
async function handle(request, env, context) {
|
||||
const site = siteOf(new URL(request.url).hostname);
|
||||
const url = unfold(new URL(request.url), site);
|
||||
const url = unfold(new URL(request.url)), site = siteOf(url.hostname);
|
||||
const path = url.pathname.slice(1);
|
||||
const asset = f => env.ASSETS.fetch(new Request(new URL(`/assets/${f}`, url.origin)));
|
||||
// Every alert ends with the full request URL (site, query, ?i= and ?src=), so it's clear which request triggered it
|
||||
@@ -289,11 +288,6 @@ COMMIT TRANSACTION;`;
|
||||
return job;
|
||||
}
|
||||
|
||||
// ?i=1 or higher (default 1). Returns null if invalid.
|
||||
function parseIndex(searchParams) {
|
||||
const i = searchParams.get("i") || "1";
|
||||
return /^\d+$/.test(i) && +i >= 1 ? +i : null;
|
||||
}
|
||||
|
||||
// The query's provider's results (see FIRECRAWL_SHARE), or the other's if it has none or refuses (e.g. once its
|
||||
// credits run out). [] if either searched and found nothing, null if both failed, which means we're out of searches
|
||||
@@ -365,17 +359,15 @@ async function countHit(env, request, query, site, params) {
|
||||
} catch {}
|
||||
}
|
||||
|
||||
// The zone's URL Rewrite Rules (scripts/rules.js) turn a bare query into its static file's path, under its hostname
|
||||
// (free.direct-img.link/orange+cat → /free.direct-img.link/1/orange%2Bcat), so a promoted image is served without
|
||||
// running the worker. Unfolded back here, so the rest of the worker sees the original request: the rules' %2B were +s,
|
||||
// since they leave a query's own %2B alone. Links from when ?i= was folded too (/free/2/orange+cat) still work. A folded
|
||||
// path for another site is rejected like any slash.
|
||||
function unfold(url, site) {
|
||||
const m = url.pathname.match(/^\/([^/]+)\/([^/]*)\/([^/]+)$/);
|
||||
if (!m || ![url.hostname, site || "web"].includes(m[1])) return url;
|
||||
// The zone's URL Rewrite Rules (scripts/rules.js) give each image request a static file path under its hostname, named
|
||||
// after its ?i= and ?src= (free.direct-img.link/Orange+Cat?i=2 → /free.direct-img.link/orange%2Bcat/i2-), so a promoted
|
||||
// image is served without running the worker. Only the path is rewritten (the query string stays), so restoring it is
|
||||
// enough: the rules' %2B were +s, since they leave a query's own %2B alone.
|
||||
function unfold(url) {
|
||||
const m = url.pathname.match(/^\/([^/]+)\/([^/]+)\/i[^/]*$/);
|
||||
if (!m || m[1] !== url.hostname) return url;
|
||||
const u = new URL(url);
|
||||
u.pathname = `/${m[3].replace(/%2B/gi, "+")}`;
|
||||
if (m[2] && m[2] !== "1") u.searchParams.set("i", m[2]);
|
||||
u.pathname = `/${m[2].replace(/%2B/gi, "+")}`;
|
||||
return u;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user