mirror of
https://github.com/multipleof4/gitea-github-cron.git
synced 2026-10-08 21:35:21 +00:00
Sync descriptions and issues, run workflows nightly
mirror.js now keeps repo descriptions in step with GitHub. A new sync-issues workflow copies issues, PR conversations and comments onto the mirrors with GitHub's numbering. Shared API helpers move to gitea-mirrors.js. Every workflow now runs once a night, 1.5 hours apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -8,7 +8,7 @@ on:
|
|||||||
type: boolean
|
type: boolean
|
||||||
default: false
|
default: false
|
||||||
schedule:
|
schedule:
|
||||||
- cron: '0 */4 * * *' # Run every 4 hours
|
- cron: '0 5 * * *' # 12 AM CDT
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
mirror:
|
mirror:
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
name: Sync issues to Gitea
|
||||||
|
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
schedule:
|
||||||
|
- cron: '30 9 * * *' # 4:30 AM CDT
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: sync-issues
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
issues:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: 24
|
||||||
|
- name: Copy GitHub issues and PRs to Gitea mirrors
|
||||||
|
id: sync
|
||||||
|
run: node sync-issues.js
|
||||||
|
env:
|
||||||
|
GH_PAT: ${{ secrets.GH_PAT }}
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
GITEA_URL: ${{ secrets.GITEA_URL }}
|
||||||
|
|
||||||
|
- name: Notify via ntfy
|
||||||
|
if: ${{ always() }}
|
||||||
|
run: node notify.js
|
||||||
|
env:
|
||||||
|
NTFY_TOPIC_URL: ${{ secrets.NTFY_TOPIC_URL }}
|
||||||
|
WORKFLOW_KIND: issues
|
||||||
|
STEP_OUTCOME: ${{ steps.sync.outcome }}
|
||||||
|
RUN_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||||
@@ -3,7 +3,7 @@ name: Sync releases to Gitea
|
|||||||
on:
|
on:
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: '30 */4 * * *' # Every 4 hours, 30 min after the mirror run
|
- cron: '30 6 * * *' # 1:30 AM CDT
|
||||||
|
|
||||||
concurrency:
|
concurrency:
|
||||||
group: sync-releases
|
group: sync-releases
|
||||||
|
|||||||
@@ -8,11 +8,12 @@ GitHub Actions that back up every GitHub repo I own (personal and org) to a self
|
|||||||
|
|
||||||
| Workflow | Schedule (UTC) | Script | Does |
|
| Workflow | Schedule (UTC) | Script | Does |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| Mirror to Gitea | every 4h at :00 | `mirror.js` | Creates missing orgs and pull mirrors, syncs repo/org visibility and org avatars |
|
| Mirror to Gitea | 05:00 (12 AM CDT) | `mirror.js` | Creates missing orgs and pull mirrors, syncs repo/org visibility, descriptions and org avatars |
|
||||||
| Sync releases to Gitea | every 4h at :30 | `sync-releases.js` | Copies GitHub releases (title, notes, assets) onto the mirrors |
|
| Sync releases to Gitea | 06:30 (1:30 AM CDT) | `sync-releases.js` | Copies GitHub releases (title, notes, assets) onto the mirrors |
|
||||||
| Rename deleted GitHub mirrors | daily 08:00 | `rename-gone-mirrors.js` | Renames mirrors whose GitHub repo was deleted or moved to `<name>-goneN` and freezes them |
|
| Rename deleted GitHub mirrors | 08:00 (3 AM CDT) | `rename-gone-mirrors.js` | Renames mirrors whose GitHub repo was deleted or moved to `<name>-goneN` and freezes them |
|
||||||
|
| Sync issues to Gitea | 09:30 (4:30 AM CDT) | `sync-issues.js` | Copies GitHub issues, PR conversations and their comments onto the mirrors |
|
||||||
|
|
||||||
All three can also be run by hand from the Actions tab. Each run posts a summary to ntfy through `notify.js`.
|
Each runs once a day, 1.5 hours apart, so each one starts with a fresh hourly GitHub API budget. All can also be run by hand from the Actions tab. Each run posts a summary to ntfy through `notify.js`. `gitea-mirrors.js` holds the API helpers the two sync scripts share.
|
||||||
|
|
||||||
## Releases on mirrors
|
## Releases on mirrors
|
||||||
|
|
||||||
@@ -27,6 +28,18 @@ Gitea's migrate API drops `releases: true` for pull mirrors, so a mirror only ge
|
|||||||
|
|
||||||
This relies on Gitea ≥ 1.21.5, where a mirror sync only touches tag-only entries and keeps real releases ([go-gitea/gitea#28817](https://github.com/go-gitea/gitea/pull/28817)).
|
This relies on Gitea ≥ 1.21.5, where a mirror sync only touches tag-only entries and keeps real releases ([go-gitea/gitea#28817](https://github.com/go-gitea/gitea/pull/28817)).
|
||||||
|
|
||||||
|
## Issues on mirrors
|
||||||
|
|
||||||
|
Gitea drops issues for pull mirrors too, but the API still lets you create them. `sync-issues.js` copies them over through the API:
|
||||||
|
|
||||||
|
- Gitea issue numbers always match GitHub's, so `#N` in commits and comments points at the right thing:
|
||||||
|
- PRs become issues labelled `pull request`, closed or open like on GitHub. Only their conversation comes along, not code review comments or diffs.
|
||||||
|
- Numbers that aren't issues on GitHub (deleted, transferred, or used by a discussion) stay gaps on Gitea too. Gitea never reuses a deleted issue's number, so an issue that lands on a gap is deleted and made again until it gets GitHub's number.
|
||||||
|
- If Gitea's counter is already past a GitHub number, or `#N` on Gitea wasn't made by the sync, that repo stops with a failure instead of guessing.
|
||||||
|
- Everything is posted as the Gitea token user. Each issue and comment opens with a quoted line naming the GitHub author, the date, and a link back. The link is how comments are matched on later runs, so don't edit it away.
|
||||||
|
- Title, body, open/closed state, labels and comment edits follow GitHub. Nothing is deleted on Gitea.
|
||||||
|
- Images uploaded to GitHub stay as links to GitHub.
|
||||||
|
|
||||||
## Secrets
|
## Secrets
|
||||||
|
|
||||||
| Secret | Used for |
|
| Secret | Used for |
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
const { GH_PAT, GITEA_TOKEN, GITEA_URL } = process.env;
|
||||||
|
if (![GH_PAT, GITEA_TOKEN, GITEA_URL].every(Boolean)) {
|
||||||
|
throw new Error('GH_PAT, GITEA_TOKEN, and GITEA_URL are required');
|
||||||
|
}
|
||||||
|
|
||||||
|
const gitea = `https://${GITEA_URL.replace(/^https?:\/\//, '').replace(/\/$/, '')}/api/v1`;
|
||||||
|
const github = 'https://api.github.com';
|
||||||
|
const ghHeaders = { Authorization: `token ${GH_PAT}`, Accept: 'application/vnd.github+json' };
|
||||||
|
const gtHeaders = { Authorization: `token ${GITEA_TOKEN}`, 'Content-Type': 'application/json' };
|
||||||
|
const key = value => value.toLowerCase();
|
||||||
|
const enc = (...parts) => parts.map(encodeURIComponent).join('/');
|
||||||
|
|
||||||
|
const request = async (url, headers, method = 'GET', body, missingOK = false) => {
|
||||||
|
const res = await fetch(url, {
|
||||||
|
method, headers, body: body && JSON.stringify(body), signal: AbortSignal.timeout(30_000)
|
||||||
|
});
|
||||||
|
if (res.status === 404 && missingOK) return null;
|
||||||
|
if (!res.ok) throw new Error(`${method} ${url}: ${res.status} ${(await res.text()).slice(0, 300)}`);
|
||||||
|
return res.status === 204 ? null : res.json();
|
||||||
|
};
|
||||||
|
|
||||||
|
const pages = async (url, headers, sizeKey, missingOK = false) => {
|
||||||
|
const all = [];
|
||||||
|
for (let page = 1; page <= 1000; page++) {
|
||||||
|
const next = new URL(url);
|
||||||
|
next.searchParams.set(sizeKey, '100');
|
||||||
|
next.searchParams.set('page', String(page));
|
||||||
|
const batch = await request(next, headers, 'GET', null, missingOK);
|
||||||
|
if (batch === null) return null;
|
||||||
|
if (!Array.isArray(batch)) throw new Error(`Expected an array from ${next}`);
|
||||||
|
if (!batch.length) return all;
|
||||||
|
all.push(...batch);
|
||||||
|
}
|
||||||
|
throw new Error(`Pagination limit reached for ${url}`);
|
||||||
|
};
|
||||||
|
|
||||||
|
const source = repo => {
|
||||||
|
try {
|
||||||
|
const url = new URL(repo.original_url.replace(/^git@github\.com:/i, 'https://github.com/'));
|
||||||
|
const parts = url.pathname.replace(/\.git\/?$/i, '').split('/').filter(Boolean);
|
||||||
|
return url.hostname === 'github.com' && parts.length === 2 ? { owner: parts[0], name: parts[1] } : null;
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Pull mirrors of GitHub repos owned by the Gitea user or its orgs.
|
||||||
|
// Frozen mirrors (sync interval 0, e.g. renamed -goneN ones) are left out.
|
||||||
|
const listMirrors = async () => {
|
||||||
|
const [gtUser, gtOrgs] = await Promise.all([
|
||||||
|
request(`${gitea}/user`, gtHeaders),
|
||||||
|
pages(`${gitea}/user/orgs`, gtHeaders, 'limit')
|
||||||
|
]);
|
||||||
|
const owners = [gtUser.login, ...gtOrgs.map(org => org.username || org.name)];
|
||||||
|
const ownerKeys = new Set(owners.map(key));
|
||||||
|
const repos = (await Promise.all(owners.map((owner, i) =>
|
||||||
|
pages(i ? `${gitea}/orgs/${enc(owner)}/repos` : `${gitea}/user/repos`, gtHeaders, 'limit')
|
||||||
|
))).flat();
|
||||||
|
return [...new Map(repos.map(repo => [repo.id, repo])).values()]
|
||||||
|
.filter(repo => repo.mirror && repo.original_url && !repo.archived && repo.mirror_interval !== '0s')
|
||||||
|
.filter(repo => ownerKeys.has(key(repo.owner.login)))
|
||||||
|
.map(repo => ({
|
||||||
|
repo, gh: source(repo), name: `${repo.owner.login}/${repo.name}`,
|
||||||
|
gtRepo: `${gitea}/repos/${enc(repo.owner.login, repo.name)}`
|
||||||
|
}))
|
||||||
|
.filter(mirror => mirror.gh);
|
||||||
|
};
|
||||||
|
|
||||||
|
module.exports = { github, ghHeaders, gtHeaders, enc, request, pages, listMirrors };
|
||||||
@@ -3,7 +3,7 @@ const { writeFileSync } = require('node:fs');
|
|||||||
const { GH_PAT, GITEA_TOKEN, GITEA_URL } = process.env;
|
const { GH_PAT, GITEA_TOKEN, GITEA_URL } = process.env;
|
||||||
const facts = {
|
const facts = {
|
||||||
orgsChecked: 0, orgsCreated: 0, orgVisibilityChanged: 0, avatarsUpdated: 0,
|
orgsChecked: 0, orgsCreated: 0, orgVisibilityChanged: 0, avatarsUpdated: 0,
|
||||||
reposFound: 0, mirrorsCreated: 0, repoVisibilityChanged: 0, failures: 0, changes: []
|
reposFound: 0, mirrorsCreated: 0, repoVisibilityChanged: 0, descriptionsUpdated: 0, failures: 0, changes: []
|
||||||
};
|
};
|
||||||
const G_API = `https://${GITEA_URL.replace(/^https?:\/\//, '')}/api/v1`;
|
const G_API = `https://${GITEA_URL.replace(/^https?:\/\//, '')}/api/v1`;
|
||||||
const headers = {
|
const headers = {
|
||||||
@@ -111,11 +111,21 @@ const getPages = async (url) => {
|
|||||||
await req(`${G_API}/repos/migrate`, headers.GT, 'POST', payload);
|
await req(`${G_API}/repos/migrate`, headers.GT, 'POST', payload);
|
||||||
facts.mirrorsCreated++;
|
facts.mirrorsCreated++;
|
||||||
facts.changes.push(`Mirrored ${owner}/${r.name}`);
|
facts.changes.push(`Mirrored ${owner}/${r.name}`);
|
||||||
} else if (exists.private !== r.private) {
|
} else {
|
||||||
console.log(`Updating visibility: ${owner}/${r.name} → ${r.private ? 'private' : 'public'}`);
|
const patch = {};
|
||||||
await req(`${G_API}/repos/${owner}/${r.name}`, headers.GT, 'PATCH', { private: r.private });
|
if (exists.private !== r.private) {
|
||||||
facts.repoVisibilityChanged++;
|
console.log(`Updating visibility: ${owner}/${r.name} → ${r.private ? 'private' : 'public'}`);
|
||||||
facts.changes.push(`${owner}/${r.name} is now ${r.private ? 'private' : 'public'}`);
|
patch.private = r.private;
|
||||||
|
facts.repoVisibilityChanged++;
|
||||||
|
facts.changes.push(`${owner}/${r.name} is now ${r.private ? 'private' : 'public'}`);
|
||||||
|
}
|
||||||
|
if (exists.description !== (r.description || '')) {
|
||||||
|
console.log(`Updating description: ${owner}/${r.name}`);
|
||||||
|
patch.description = r.description || '';
|
||||||
|
facts.descriptionsUpdated++;
|
||||||
|
facts.changes.push(`Updated description of ${owner}/${r.name}`);
|
||||||
|
}
|
||||||
|
if (Object.keys(patch).length) await req(`${G_API}/repos/${owner}/${r.name}`, headers.GT, 'PATCH', patch);
|
||||||
}
|
}
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
repoFailures++;
|
repoFailures++;
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ const labels = {
|
|||||||
['org visibility change', 'orgVisibilityChanged'],
|
['org visibility change', 'orgVisibilityChanged'],
|
||||||
['new mirror', 'mirrorsCreated'],
|
['new mirror', 'mirrorsCreated'],
|
||||||
['repo visibility change', 'repoVisibilityChanged'],
|
['repo visibility change', 'repoVisibilityChanged'],
|
||||||
|
['description update', 'descriptionsUpdated'],
|
||||||
['failure', 'failures']
|
['failure', 'failures']
|
||||||
]],
|
]],
|
||||||
rename: ['Missing-mirror check', 'rename-facts.json', [
|
rename: ['Missing-mirror check', 'rename-facts.json', [
|
||||||
@@ -27,6 +28,15 @@ const labels = {
|
|||||||
['uploaded asset', 'assetsUploaded'],
|
['uploaded asset', 'assetsUploaded'],
|
||||||
['unsynced tag', 'unsyncedTags'],
|
['unsynced tag', 'unsyncedTags'],
|
||||||
['failure', 'failures']
|
['failure', 'failures']
|
||||||
|
]],
|
||||||
|
issues: ['Issue sync', 'issue-facts.json', [
|
||||||
|
['mirror', 'mirrorsChecked'],
|
||||||
|
['GitHub issue', 'issuesFound'],
|
||||||
|
['created issue', 'created'],
|
||||||
|
['updated issue', 'updated'],
|
||||||
|
['created comment', 'commentsCreated'],
|
||||||
|
['updated comment', 'commentsUpdated'],
|
||||||
|
['failure', 'failures']
|
||||||
]]
|
]]
|
||||||
};
|
};
|
||||||
const md = text => String(text).replace(/[\\`*_[\]#]/g, '\\$&');
|
const md = text => String(text).replace(/[\\`*_[\]#]/g, '\\$&');
|
||||||
|
|||||||
+133
@@ -0,0 +1,133 @@
|
|||||||
|
const { writeFileSync } = require('node:fs');
|
||||||
|
const { github, ghHeaders, gtHeaders, enc, request, pages, listMirrors } = require('./gitea-mirrors');
|
||||||
|
const facts = {
|
||||||
|
mirrorsChecked: 0, issuesFound: 0, created: 0, updated: 0,
|
||||||
|
commentsCreated: 0, commentsUpdated: 0, failures: 0, changes: []
|
||||||
|
};
|
||||||
|
const prLabel = { name: 'pull request', color: '8250df', description: 'Pull request on GitHub' };
|
||||||
|
const who = user => user ? `[@${user.login}](${user.html_url})` : '@ghost';
|
||||||
|
const quote = (header, body) => body ? `> ${header}\n\n${body}` : `> ${header}`;
|
||||||
|
const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`;
|
||||||
|
const names = labels => labels.map(label => label.name).sort().join('\n');
|
||||||
|
|
||||||
|
// Gitea numbers must match GitHub's so #N references in commits and comments stay correct,
|
||||||
|
// which is also why PRs come along as issues.
|
||||||
|
const syncIssues = async ({ gh, gtRepo }, ghIssues, count) => {
|
||||||
|
const [gtIssues, gtLabels] = await Promise.all([
|
||||||
|
pages(`${gtRepo}/issues?state=all&type=issues`, gtHeaders, 'limit'),
|
||||||
|
pages(`${gtRepo}/labels`, gtHeaders, 'limit')
|
||||||
|
]);
|
||||||
|
const byNumber = new Map(gtIssues.map(issue => [issue.number, issue]));
|
||||||
|
const labelIds = new Map(gtLabels.map(label => [label.name, label.id]));
|
||||||
|
const ids = async labels => {
|
||||||
|
for (const { name, color, description } of labels.filter(label => !labelIds.has(label.name))) {
|
||||||
|
const made = await request(`${gtRepo}/labels`, gtHeaders, 'POST', { name, color: `#${color}`, description: description || '' });
|
||||||
|
labelIds.set(name, made.id);
|
||||||
|
}
|
||||||
|
return labels.map(label => labelIds.get(label.name));
|
||||||
|
};
|
||||||
|
// Gitea never reuses a deleted issue's number, so an issue that lands on a GitHub gap
|
||||||
|
// (deleted, transferred, or a discussion) is deleted and made again, leaving the same gap
|
||||||
|
const create = async (issue, n) => {
|
||||||
|
for (;;) {
|
||||||
|
const { number } = await request(`${gtRepo}/issues`, gtHeaders, 'POST', issue);
|
||||||
|
if (number === n) return;
|
||||||
|
await request(`${gtRepo}/issues/${number}`, gtHeaders, 'DELETE');
|
||||||
|
if (number > n) throw new Error(`Gitea is already past #${n}, so numbers can't line up`);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const link = n => `https://github.com/${gh.owner}/${gh.name}/issues/${n}`;
|
||||||
|
let top = Math.max(0, ...byNumber.keys());
|
||||||
|
|
||||||
|
for (const issue of ghIssues.sort((a, b) => a.number - b.number)) {
|
||||||
|
const n = issue.number, kind = issue.pull_request ? 'pull request' : 'issue';
|
||||||
|
const header = `${who(issue.user)} opened ${kind} [GitHub #${n}](${link(n)}) on ${issue.created_at.slice(0, 10)}`;
|
||||||
|
const want = { title: issue.title, body: quote(header, issue.body), state: issue.state };
|
||||||
|
const labels = issue.pull_request ? [...issue.labels, prLabel] : issue.labels;
|
||||||
|
const gt = byNumber.get(n);
|
||||||
|
|
||||||
|
if (!gt) {
|
||||||
|
if (n < top) throw new Error(`Gitea has no #${n} and can't reuse the number`);
|
||||||
|
await create({ title: want.title, body: want.body, closed: want.state === 'closed', labels: await ids(labels) }, n);
|
||||||
|
top = n;
|
||||||
|
count.created++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!gt.body.includes(`(${link(n)})`)) throw new Error(`Gitea #${n} wasn't made by this sync`);
|
||||||
|
let changed = false;
|
||||||
|
if (Object.entries(want).some(([k, v]) => gt[k] !== v)) {
|
||||||
|
await request(`${gtRepo}/issues/${n}`, gtHeaders, 'PATCH', want);
|
||||||
|
changed = true;
|
||||||
|
}
|
||||||
|
if (names(gt.labels) !== names(labels)) {
|
||||||
|
await request(`${gtRepo}/issues/${n}/labels`, gtHeaders, 'PUT', { labels: await ids(labels) });
|
||||||
|
changed = true;
|
||||||
|
}
|
||||||
|
if (changed) count.updated++;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Conversation comments on issues and PRs; the GitHub comment link in each header maps them back
|
||||||
|
const syncComments = async ({ gh, gtRepo }, count) => {
|
||||||
|
const [ghComments, gtComments] = await Promise.all([
|
||||||
|
pages(`${github}/repos/${enc(gh.owner, gh.name)}/issues/comments?sort=created&direction=asc`, ghHeaders, 'per_page'),
|
||||||
|
pages(`${gtRepo}/issues/comments`, gtHeaders, 'limit')
|
||||||
|
]);
|
||||||
|
const mirrored = new Map(gtComments.map(comment =>
|
||||||
|
[comment.body.split('\n', 1)[0].match(/#issuecomment-(\d+)\)/)?.[1], comment]
|
||||||
|
));
|
||||||
|
for (const comment of ghComments) {
|
||||||
|
const body = quote(`${who(comment.user)} commented on [${comment.created_at.slice(0, 10)}](${comment.html_url})`, comment.body);
|
||||||
|
const gt = mirrored.get(String(comment.id));
|
||||||
|
if (!gt) {
|
||||||
|
await request(`${gtRepo}/issues/${comment.issue_url.split('/').pop()}/comments`, gtHeaders, 'POST', { body });
|
||||||
|
count.commentsCreated++;
|
||||||
|
} else if (gt.body !== body) {
|
||||||
|
await request(`${gtRepo}/issues/comments/${gt.id}`, gtHeaders, 'PATCH', { body });
|
||||||
|
count.commentsUpdated++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
(async () => {
|
||||||
|
const mirrors = await listMirrors();
|
||||||
|
facts.mirrorsChecked = mirrors.length;
|
||||||
|
console.log(`Checking issues for ${mirrors.length} GitHub mirror(s)...`);
|
||||||
|
|
||||||
|
for (const mirror of mirrors) {
|
||||||
|
const { gh, name } = mirror;
|
||||||
|
const count = { created: 0, updated: 0, commentsCreated: 0, commentsUpdated: 0 };
|
||||||
|
try {
|
||||||
|
const ghIssues = await pages(`${github}/repos/${enc(gh.owner, gh.name)}/issues?state=all`, ghHeaders, 'per_page', true);
|
||||||
|
if (!ghIssues?.length) continue;
|
||||||
|
facts.issuesFound += ghIssues.length;
|
||||||
|
await syncIssues(mirror, ghIssues, count);
|
||||||
|
if (ghIssues.some(issue => issue.comments)) await syncComments(mirror, count);
|
||||||
|
} catch (error) {
|
||||||
|
facts.failures++;
|
||||||
|
console.error(`Failed to sync issues for ${name}: ${error.message}`);
|
||||||
|
} finally {
|
||||||
|
for (const [k, v] of Object.entries(count)) facts[k] += v;
|
||||||
|
const parts = [
|
||||||
|
count.created && `+${plural(count.created, 'issue')}`,
|
||||||
|
count.updated && `${count.updated} updated`,
|
||||||
|
count.commentsCreated && `+${plural(count.commentsCreated, 'comment')}`,
|
||||||
|
count.commentsUpdated && `${plural(count.commentsUpdated, 'comment')} updated`
|
||||||
|
].filter(Boolean);
|
||||||
|
if (parts.length) {
|
||||||
|
facts.changes.push(`${name}: ${parts.join(', ')}`);
|
||||||
|
console.log(`${name}: ${parts.join(', ')}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`Created ${facts.created} issue(s), ${facts.commentsCreated} comment(s); updated ${facts.updated} issue(s), ${facts.commentsUpdated} comment(s); ${facts.failures} failure(s).`);
|
||||||
|
if (facts.failures) process.exitCode = 1;
|
||||||
|
})().catch(error => {
|
||||||
|
console.error(`Fatal: ${error.message}`);
|
||||||
|
facts.fatal = true;
|
||||||
|
process.exitCode = 1;
|
||||||
|
}).finally(() => {
|
||||||
|
if (process.env.GITHUB_ACTIONS) writeFileSync('issue-facts.json', JSON.stringify(facts));
|
||||||
|
});
|
||||||
+8
-68
@@ -1,55 +1,11 @@
|
|||||||
const { writeFileSync } = require('node:fs');
|
const { writeFileSync } = require('node:fs');
|
||||||
const { GH_PAT, GITEA_TOKEN, GITEA_URL } = process.env;
|
const { github, ghHeaders, gtHeaders, enc, request, pages, listMirrors } = require('./gitea-mirrors');
|
||||||
if (![GH_PAT, GITEA_TOKEN, GITEA_URL].every(Boolean)) {
|
|
||||||
throw new Error('GH_PAT, GITEA_TOKEN, and GITEA_URL are required');
|
|
||||||
}
|
|
||||||
|
|
||||||
const gitea = `https://${GITEA_URL.replace(/^https?:\/\//, '').replace(/\/$/, '')}/api/v1`;
|
|
||||||
const github = 'https://api.github.com';
|
|
||||||
const ghHeaders = { Authorization: `token ${GH_PAT}`, Accept: 'application/vnd.github+json' };
|
|
||||||
const gtHeaders = { Authorization: `token ${GITEA_TOKEN}`, 'Content-Type': 'application/json' };
|
|
||||||
const facts = {
|
const facts = {
|
||||||
mirrorsChecked: 0, releasesFound: 0, created: 0, updated: 0, assetsUploaded: 0,
|
mirrorsChecked: 0, releasesFound: 0, created: 0, updated: 0, assetsUploaded: 0,
|
||||||
unsyncedTags: 0, failures: 0, changes: []
|
unsyncedTags: 0, failures: 0, changes: []
|
||||||
};
|
};
|
||||||
const key = value => value.toLowerCase();
|
|
||||||
const enc = (...parts) => parts.map(encodeURIComponent).join('/');
|
|
||||||
const transferTimeout = () => AbortSignal.timeout(30 * 60_000);
|
const transferTimeout = () => AbortSignal.timeout(30 * 60_000);
|
||||||
|
|
||||||
const request = async (url, headers, method = 'GET', body, missingOK = false) => {
|
|
||||||
const res = await fetch(url, {
|
|
||||||
method, headers, body: body && JSON.stringify(body), signal: AbortSignal.timeout(30_000)
|
|
||||||
});
|
|
||||||
if (res.status === 404 && missingOK) return null;
|
|
||||||
if (!res.ok) throw new Error(`${method} ${url}: ${res.status} ${(await res.text()).slice(0, 300)}`);
|
|
||||||
return res.json();
|
|
||||||
};
|
|
||||||
|
|
||||||
const pages = async (url, headers, sizeKey, missingOK = false) => {
|
|
||||||
const all = [];
|
|
||||||
for (let page = 1; page <= 1000; page++) {
|
|
||||||
const next = new URL(url);
|
|
||||||
next.searchParams.set(sizeKey, '100');
|
|
||||||
next.searchParams.set('page', String(page));
|
|
||||||
const batch = await request(next, headers, 'GET', null, missingOK);
|
|
||||||
if (batch === null) return null;
|
|
||||||
if (!Array.isArray(batch)) throw new Error(`Expected an array from ${next}`);
|
|
||||||
if (!batch.length) return all;
|
|
||||||
all.push(...batch);
|
|
||||||
}
|
|
||||||
throw new Error(`Pagination limit reached for ${url}`);
|
|
||||||
};
|
|
||||||
|
|
||||||
const source = repo => {
|
|
||||||
try {
|
|
||||||
const url = new URL(repo.original_url.replace(/^git@github\.com:/i, 'https://github.com/'));
|
|
||||||
const parts = url.pathname.replace(/\.git\/?$/i, '').split('/').filter(Boolean);
|
|
||||||
return url.hostname === 'github.com' && parts.length === 2 ? { owner: parts[0], name: parts[1] } : null;
|
|
||||||
} catch {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Pipe the GitHub download straight into Gitea's raw upload, so assets never touch disk or memory
|
// Pipe the GitHub download straight into Gitea's raw upload, so assets never touch disk or memory
|
||||||
const copyAsset = async (gh, gtRepo, releaseId, asset) => {
|
const copyAsset = async (gh, gtRepo, releaseId, asset) => {
|
||||||
const download = await fetch(`${github}/repos/${enc(gh.owner, gh.name)}/releases/assets/${asset.id}`, {
|
const download = await fetch(`${github}/repos/${enc(gh.owner, gh.name)}/releases/assets/${asset.id}`, {
|
||||||
@@ -64,9 +20,8 @@ const copyAsset = async (gh, gtRepo, releaseId, asset) => {
|
|||||||
if (!upload.ok) throw new Error(`upload ${asset.name}: ${upload.status} ${(await upload.text()).slice(0, 300)}`);
|
if (!upload.ok) throw new Error(`upload ${asset.name}: ${upload.status} ${(await upload.text()).slice(0, 300)}`);
|
||||||
};
|
};
|
||||||
|
|
||||||
const syncRelease = async ({ repo, gh }, rel, existing) => {
|
const syncRelease = async ({ gh, gtRepo, name }, rel, existing) => {
|
||||||
const gtRepo = `${gitea}/repos/${enc(repo.owner.login, repo.name)}`;
|
const where = `${name}@${rel.tag_name}`;
|
||||||
const where = `${repo.owner.login}/${repo.name}@${rel.tag_name}`;
|
|
||||||
const want = { name: rel.name || rel.tag_name, body: rel.body || '', prerelease: rel.prerelease };
|
const want = { name: rel.name || rel.tag_name, body: rel.body || '', prerelease: rel.prerelease };
|
||||||
let gt = existing.get(rel.tag_name), action;
|
let gt = existing.get(rel.tag_name), action;
|
||||||
|
|
||||||
@@ -104,37 +59,22 @@ const syncRelease = async ({ repo, gh }, rel, existing) => {
|
|||||||
};
|
};
|
||||||
|
|
||||||
(async () => {
|
(async () => {
|
||||||
const [gtUser, gtOrgs] = await Promise.all([
|
const mirrors = await listMirrors();
|
||||||
request(`${gitea}/user`, gtHeaders),
|
|
||||||
pages(`${gitea}/user/orgs`, gtHeaders, 'limit')
|
|
||||||
]);
|
|
||||||
const owners = [gtUser.login, ...gtOrgs.map(org => org.username || org.name)];
|
|
||||||
const repos = (await Promise.all(owners.map((owner, i) =>
|
|
||||||
pages(i ? `${gitea}/orgs/${enc(owner)}/repos` : `${gitea}/user/repos`, gtHeaders, 'limit')
|
|
||||||
))).flat();
|
|
||||||
const ownerKeys = new Set(owners.map(key));
|
|
||||||
|
|
||||||
// Frozen mirrors (sync interval 0, e.g. renamed -goneN ones) never get new tags
|
|
||||||
const mirrors = [...new Map(repos.map(repo => [repo.id, repo])).values()]
|
|
||||||
.filter(repo => repo.mirror && repo.original_url && !repo.archived && repo.mirror_interval !== '0s')
|
|
||||||
.filter(repo => ownerKeys.has(key(repo.owner.login)))
|
|
||||||
.map(repo => ({ repo, gh: source(repo) }))
|
|
||||||
.filter(mirror => mirror.gh);
|
|
||||||
facts.mirrorsChecked = mirrors.length;
|
facts.mirrorsChecked = mirrors.length;
|
||||||
console.log(`Checking releases for ${mirrors.length} GitHub mirror(s)...`);
|
console.log(`Checking releases for ${mirrors.length} GitHub mirror(s)...`);
|
||||||
|
|
||||||
for (const mirror of mirrors) {
|
for (const mirror of mirrors) {
|
||||||
const { repo, gh } = mirror;
|
const { gh, gtRepo, name } = mirror;
|
||||||
let releases, existing;
|
let releases, existing;
|
||||||
try {
|
try {
|
||||||
releases = await pages(`${github}/repos/${enc(gh.owner, gh.name)}/releases`, ghHeaders, 'per_page', true);
|
releases = await pages(`${github}/repos/${enc(gh.owner, gh.name)}/releases`, ghHeaders, 'per_page', true);
|
||||||
releases = (releases || []).filter(rel => !rel.draft).reverse();
|
releases = (releases || []).filter(rel => !rel.draft).reverse();
|
||||||
if (!releases.length) continue;
|
if (!releases.length) continue;
|
||||||
const gtReleases = await pages(`${gitea}/repos/${enc(repo.owner.login, repo.name)}/releases`, gtHeaders, 'limit');
|
const gtReleases = await pages(`${gtRepo}/releases`, gtHeaders, 'limit');
|
||||||
existing = new Map(gtReleases.map(rel => [rel.tag_name, rel]));
|
existing = new Map(gtReleases.map(rel => [rel.tag_name, rel]));
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
facts.failures++;
|
facts.failures++;
|
||||||
console.error(`Failed to list releases for ${repo.owner.login}/${repo.name}: ${error.message}`);
|
console.error(`Failed to list releases for ${name}: ${error.message}`);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -144,7 +84,7 @@ const syncRelease = async ({ repo, gh }, rel, existing) => {
|
|||||||
await syncRelease(mirror, rel, existing);
|
await syncRelease(mirror, rel, existing);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
facts.failures++;
|
facts.failures++;
|
||||||
console.error(`Failed to sync ${repo.owner.login}/${repo.name}@${rel.tag_name}: ${error.message}`);
|
console.error(`Failed to sync ${name}@${rel.tag_name}: ${error.message}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user