Every LinkedIn search and every fetch-other.ts source hits a different
host, so there was no real reason for the old for-loop to wait on each
one in turn - that's what made a full run take 30-40+ minutes. Runs
them concurrently now (bounded: 3 at a time within each group, both
groups started together), which brought a real run down to under 90
seconds.
existingLinks.add() now happens synchronously right after the .has()
check and before any await, which is what makes concurrent workers
safe against double-claiming the same link.
Concurrency surfaced a real regression: LinkedIn started returning
occasional 429s under parallel load, and fetchJobs treated that the
same as end-of-results, silently truncating the rest of that search.
Added retry-with-backoff (3 attempts, 1s/2s/4s) so a rate limit no
longer costs real listings.
jobbasel.ch's search route (/job/suche-{slug}) is server-rendered with
a window.__PRELOADED_STATE__ JSON blob, no headless browser needed.
englishjobsearch.ch works the same way via its ?format=markdown param.
Both swept with the same fixed keyword list job-room.ch already uses.
basel-jobs.ch was evaluated too but left out: it's DataDome-protected,
which blocks plain curl outright, and this project's fetchers are
intentionally browser-free.
While wiring these in, found that isEnglishDescription (now
isViableDescription) only ever ran the DE/FR word-count check against
inline descriptions, never requiresGerman() - so an English-written
posting demanding fluent German (e.g. PRODYNA's "you can communicate
fluently in both German and English") sailed straight through. That
gap existed in itjobs.ch and job-room.ch too, not just the two new
sources, so the fix applies to all four.
fetch.ts already parses each job card's own location, but index.ts
discarded it and stored only the search-area label in the CSV, so
daily.ts's telegram grouping used the query that found a job instead
of where it actually is (e.g. a Bern search surfacing a Lausanne job).
Now the location is written to a new CSV column and used for grouping,
with a one-time migration for existing rows.
Rewrite the README around the current pipeline: the twice-daily Telegram
digest, the sciCORE watcher, Telegram setup, and the two-stage filter
pipeline.
CLAUDE.md gains the bun daily commands, the daily/profile/telegram
architecture notes, and a scheduled-automations section covering both
launchd agents. Records that fetch.ts gates what reaches the CSV while
profile.ts gates what reaches the phone.
Broaden the search beyond engineer/developer titles and stop three
classes of bad result from ever reaching the CSV.
config.ts
- PLATFORM_KEYWORDS (architect, SRE, infra, security, MLOps),
RESEARCH_KEYWORDS (research/scientific software, HPC, data platform)
and FIELD_KEYWORDS (forward-deployed, customer/sales, founding).
Separate OR-strings, not folded into KEYWORDS: LinkedIn ranks a
focused query far better than one 50-term disjunction.
- sweep() generates one search per area per family. 7 searches -> 17.
fetch.ts
- BIO_DOMAIN_RE drops life-science roles (bioinformatics, biostats,
GMP/MSAT, genomics, pathology, drug discovery). Filters on the role,
never the employer: software jobs at Roche/Novartis/sciCORE stay.
- requiresGerman() drops English postings demanding German above B1.
Two rules, both from real false results: the nearest marker wins
("Infrastructure skills are a plus" sat 95 chars before "Good
knowledge of German"), and an intervening language name breaks the
binding ("Fluent in English; German and/or French advantageous").
- GERMAN_TITLE_RE catches "(German-speaking)" without a page fetch.
- isViableContent/isViableUrl run the DE/FR and German-requirement
checks on a single fetch; isEnglishContent/isEnglishUrl kept for
recheck.ts and temp scripts.
- SOFTWARE_TITLE_RE widened so the new families survive it.
index.ts
- both content-check paths use the combined check and report which
filter dropped a job.
- move temp scripts to scripts/, cv files to cv/, jobs.csv to results/
- update index.ts and recheck.ts CSV path to results/jobs.csv
- fix imports in scripts/ to use ../config.ts and ../fetch.ts
- update CLAUDE.md and README.md to reflect new structure
- add plans/ with LinkedIn post drafts
- update .gitignore: ignore results/, jobs/, cv binaries, .DS_Store
- isEnglishContent() fetches each job page and drops descriptions with ≥6 German or French indicator words; called for every new job in index.ts before writing to CSV
- New `bun recheck` command (recheck.ts) runs searches in raw mode, lists filtered-out jobs, and prints word-frequency suggestions for SOFTWARE_TITLE_RE
- Expanded KEYWORDS and SOFTWARE_TITLE_RE to cover ML/AI, system/applications engineer, director, informatics, and more title variants
- Added `raw` flag to fetchJobs() to bypass the software-role filter
- Add CLAUDE.md with project workflow docs; add CV files and city temp scripts
rodado