rodado/linkme
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: cover the two new sources and the concurrent fetch pipeline

Browse Source
This commit is contained in:
rodado
2026-08-30 13:07:57 +02:00
parent ba3039150f
commit b61a75eca0
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4 -3
View File
@@ -17,7 +17,7 @@ bun daily # fetch, then Telegram the new jobs that match the profile
bun recheck # show what the filters are currently dropping + keyword suggestions bun recheck # show what the filters are currently dropping + keyword suggestions
``` ```
`bun fetch` needs no setup — reruns are safe, already-seen jobs are skipped by ID. Telegram needs the two-line setup below. `bun fetch` needs no setup — reruns are safe, already-seen jobs are skipped by ID, and every search/source runs concurrently (a full run is well under 2 minutes). Telegram needs the two-line setup below.
## Telegram setup ## Telegram setup
@@ -39,7 +39,8 @@ Then `bun daily --dry-run` prints the digest instead of sending it, and `bun dai
config.ts # keyword families, search areas, the sweep() generator config.ts # keyword families, search areas, the sweep() generator
fetch.ts # LinkedIn scraper, HTML parser, all title/content filters fetch.ts # LinkedIn scraper, HTML parser, all title/content filters
fetch-other.ts # extra sources: itjobs.ch, job-room.ch, stellen-basel.ch, fetch-other.ts # extra sources: itjobs.ch, job-room.ch, stellen-basel.ch,
# datacareer.ch (403, currently dead), Basel intl orgs # jobbasel.ch, englishjobsearch.ch, Basel intl orgs,
# datacareer.ch (403, currently dead)
index.ts # orchestrator: run searches → dedupe → content-check → CSV index.ts # orchestrator: run searches → dedupe → content-check → CSV
recheck.ts # raw-mode fetch: lists what the filters dropped recheck.ts # raw-mode fetch: lists what the filters dropped
daily.ts # digest: fetch, match against the profile, send to Telegram daily.ts # digest: fetch, match against the profile, send to Telegram
@@ -65,7 +66,7 @@ Two stages. Cheap title filters first, then one page fetch per survivor.
6. `SOFTWARE_TITLE_RE` — requires a real tech/role term. Skipped in `raw` mode, which is what `bun recheck` uses. 6. `SOFTWARE_TITLE_RE` — requires a real tech/role term. Skipped in `raw` mode, which is what `bun recheck` uses.
7. `GERMAN_TITLE_RE` — "(German-speaking)", "deutschsprachig". 7. `GERMAN_TITLE_RE` — "(German-speaking)", "deutschsprachig".
**Content check** (`index.ts` only, one fetch + 350 ms per job): `isViableContent` counts German/French indicator words and drops at ≥ 6, then `requiresGerman()` drops English postings demanding German above B1. **Content check**: `isViableContent`/`isViableUrl` (LinkedIn, one page fetch + 350 ms per job in `index.ts`) and `isViableDescription` (`fetch-other.ts` sources that already carry the description inline, no extra fetch) all run the same two checks — DE/FR indicator words dropped at ≥ 6, then `requiresGerman()` drops English postings demanding German above B1.
That last one is deliberately more than a keyword match. For each mention of German it takes the *nearest* qualifying marker, and ignores a marker when another language name sits between it and the mention — both rules came from real false results, like *"Fluent in English; German and/or French advantageous"*, where "Fluent" sits closer to "German" than "advantageous" does but qualifies English. That last one is deliberately more than a keyword match. For each mention of German it takes the *nearest* qualifying marker, and ignores a marker when another language name sits between it and the mention — both rules came from real false results, like *"Fluent in English; German and/or French advantageous"*, where "Fluent" sits closer to "German" than "advantageous" does but qualifies English.