datacareer.ch had been returning 403 on every page for months. The cause
was our own User-Agent: its nginx inverts the usual bot rule and blocks
browser-like UAs while serving honest ones (no UA, curl, Googlebot).
Give it PLAIN_HEADERS instead of the shared spoofed-Chrome HEADERS.
Basel has 8 jobs on a single page with no pagination, so page 2 returning
0 items is correct rather than truncation.
basel-jobs.ch was documented as un-integrable because DataDome blocks
plain curl. Only half of that was true. Measured: plain fetch 403s on any
UA, headless Chrome over CDP 403s, Playwright's own launch 403s even
headful — but a real Chrome launched normally with --remote-debugging-port
and attached over CDP afterwards gets 200. Clearance then lives entirely
in a datadome cookie that replays fine over plain fetch with a matching
UA, so a browser is touched only to mint, never to scrape. The earlier
"cookie can't be extracted" note was true of the browser extension, not
of CDP.
Minting needs a visible window, so launchd runs can't do it; a stale
cookie makes the source log and yield nothing instead of failing the run.
Refresh by hand with `bun run mint-datadome`, or set LINKME_NO_BROWSER=1.
Three passes, all needed — they cover different sets. The informatik
category is a curated taxonomy, not the whole board, and the two search
modes query different indexes: websearch=1 finds role words and postings
filed outside IT (+147 over the category), while the bare search is the
only mode matching stack terms — react, frontend and backend return zero
under websearch=1. Language checks run against div.plain-content only;
against the whole page the German site chrome alone clears the DE/FR
threshold and drops every job, English ones included.
Also fix a pre-existing double-wrapped Promise in the extraSources type.
telegram.ts
- Bot API wrapper: HTML escaping, 4096-char chunking, rate-limit
spacing, and --setup to discover the chat ID from the bot's inbox.
Reads TELEGRAM_BOT (or TELEGRAM_TOKEN) and TELEGRAM_CHAT_ID.
profile.ts
- matchesProfile() scores a title against cv/polo.md. Hard rejects for
Microsoft stack, Java/C++/SAP, aggregators (Jobgether, ALLPS),
non-software functions and life-science domains; then weighted
signals for the core stack, platform/DevOps, data, AI/ML, cloud and
seniority. Threshold is 4 so short bullseye titles ("Data Engineer",
"DevOps Engineer") pass, and forward-deployed is weighted to clear it
on its own.
daily.ts
- Runs the fetch, diffs jobs.csv, keeps only profile matches, groups by
area and pushes to Telegram. results/sent.txt records every link
actually sent, written only after a successful send, so a failure
retries instead of losing jobs and an ad is never sent twice.
- --dry-run previews, --resend --date= re-sends a past day.
- A failed fetch reports itself rather than going quiet.
Scheduled at 08:00 and 18:00 by a launchd agent
(com.rafapolo.linkme-daily). launchd runs missed jobs on wake, and the
digest is diff-based, so downtime delays delivery without losing jobs.
- move temp scripts to scripts/, cv files to cv/, jobs.csv to results/
- update index.ts and recheck.ts CSV path to results/jobs.csv
- fix imports in scripts/ to use ../config.ts and ../fetch.ts
- update CLAUDE.md and README.md to reflect new structure
- add plans/ with LinkedIn post drafts
- update .gitignore: ignore results/, jobs/, cv binaries, .DS_Store
- isEnglishContent() fetches each job page and drops descriptions with ≥6 German or French indicator words; called for every new job in index.ts before writing to CSV
- New `bun recheck` command (recheck.ts) runs searches in raw mode, lists filtered-out jobs, and prints word-frequency suggestions for SOFTWARE_TITLE_RE
- Expanded KEYWORDS and SOFTWARE_TITLE_RE to cover ML/AI, system/applications engineer, director, informatics, and more title variants
- Added `raw` flag to fetchJobs() to bypass the software-role filter
- Add CLAUDE.md with project workflow docs; add CV files and city temp scripts
- Paginated HTML scraper (up to 6 pages, 400ms delay between requests)
- Card-boundary-safe parsing to prevent field bleed across cards
- Language filter: drops German (m/w/d, umlauts) and French (H/F, accents) titles
- Software-role filter: requires title to contain a relevant tech term
- Global deduplication by job ID across runs
- 30-day recency filter via LinkedIn f_TPR parameter
- CSV output: date_found, posted_at, search, title, company, link
rodado