rodado/linkme
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: dedupe job-room.ch and jobs.ch links stored without a protocol

Browse Source 
The link column was matched by an http(s):// pattern, but job-room.ch and
jobs.ch links are stored without a protocol (e.g. "www.jobs.ch/de/..."),
so the pattern silently skipped every one of them and re-added the same
posting on every run. Read the link by CSV column position instead.
This commit is contained in:
rodado
2026-09-10 10:39:51 +02:00
parent e3368ce161
commit 2e21d55807
1 changed files with 9 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
index.ts
+9 -3
View File
@@ -40,13 +40,19 @@ if (csvExists) {
}
}
// Load existing links for deduplication
// Load existing links for deduplication. The link column (6th field) is
// pulled by position, not by an http(s):// match — job-room.ch/jobs.ch links
// are stored without a protocol (e.g. "www.jobs.ch/de/..."), so a URL-pattern
// match silently skipped every one of them and re-added the same jobs.ch
// posting on every run (found 2026-09-08: 4 duplicate rows for one link,
// added on both 09-07 and 09-08).
const existingLinks = new Set<string>();
try {
const text = await Bun.file(CSV).text();
for (const line of text.split("\n").slice(1)) {
const url = line.match(/"(https?:\/\/[^"]+)"/)?.[1];
if (url) existingLinks.add(url);
if (!line.trim()) continue;
const link = line.replace(/^"|"$/g, "").split('","')[5];
if (link) existingLinks.add(link);
}
} catch {
// first run