linkme/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

command	what it does
bun fetch	run all searches in config.ts, append new jobs to results/jobs.csv
bun recheck	fetch in raw mode, list filtered-out jobs + word-frequency keyword suggestions
bun daily	fetch, then push new profile-matching jobs to Telegram
bun daily --dry-run	same, but print the message instead of sending
bun daily --resend	skip fetching, re-send today's jobs from the CSV (--date=YYYY-MM-DD to pick a day)
bun run telegram.ts --setup	print the chat ID for .env after messaging the bot
bun run mint-datadome	open a real Chrome window to refresh the basel-jobs.ch DataDome cookie

Temp scripts: bun run scripts/<name>-temp.ts — never edit config.ts for one-off searches.

Structure

config.ts          # search keywords and areas
fetch.ts           # scraper, HTML parser, all filters
fetch-other.ts     # extra sources (datacareer.ch, stellen-basel.ch, itjobs.ch, job-room.ch, Basel intl orgs, jobbasel.ch, englishjobsearch.ch, basel-jobs.ch)
index.ts           # orchestrator: runs searches, writes CSV
recheck.ts         # raw-mode fetch, lists filtered-out jobs
daily.ts           # digest: fetch → match against profile.ts → send to Telegram
profile.ts         # title-level scoring; decides what's worth a notification
telegram.ts        # Telegram delivery, chunking, --setup chat-ID discovery
scripts/           # one-off search scripts; import from ../config.ts and ../fetch.ts
scripts/scicore-watch.sh  # daily launchd watch on sciCORE's open-positions page
results/sent.txt   # links already pushed to Telegram — never sent twice
cv/polo.md         # Rafael's profile — use for ranking jobs
cv/                # CV files (html, pdf, odt)
results/jobs.csv   # output — one row per job
jobs/              # notes on specific job leads
plans/             # LinkedIn post drafts

Architecture

• config.ts — KEYWORDS string (joined with OR) and searches: SearchConfig[]. Edit here to add/remove search areas or keywords.
  • Also exports PLATFORM_KEYWORDS (architect/SRE/infra/security/MLOps), RESEARCH_KEYWORDS (research & scientific software, HPC, data platform) and FIELD_KEYWORDS (forward-deployed, customer/sales/integration, founding/product). Kept as separate OR-strings rather than folded into KEYWORDS — LinkedIn ranks a focused query far better than one 50-term disjunction. The sweep() helper generates one search per area per family; Research skips Bern, Platform and Field add a Switzerland-wide remote pass.
  • RESEARCH_KEYWORDS deliberately contains no bioinformatics/computational-biology terms — see BIO_DOMAIN_RE.
• fetch.ts — HTTP scraping, HTML parsing, and all filters. Exports fetchJobs(config, { raw? }) and isEnglishContent(id).
• fetch-other.ts — scrapes extra sources (datacareer.ch, stellen-basel.ch, itjobs.ch, job-room.ch, Basel international orgs, jobbasel.ch, englishjobsearch.ch, basel-jobs.ch). Imported by index.ts.

  • datacareer.ch inverts the usual bot rule: its nginx 403s browser-like User-Agents and serves honest ones (no UA, curl/8.0, Googlebot) fine. The spoofed Chrome UA in the shared HEADERS was the entire cause of the long-standing 403 — it uses PLAIN_HEADERS instead. Small board: Basel returns 8 jobs on one page with no pagination, so ?page=2 legitimately yields 0 items.

  • job-room.ch carries its language requirement as a structured field (languageSkills[], e.g. {languageIsoCode: "de", spokenLevel: "PROFICIENT"}), entirely separate from the free-text jobDescriptions[].description — the UI renders it as "Deutsch (Fliessend)". requiresGerman()/isViableDescription() only ever scan the prose, so this field slipped through unchecked until 2026-09-04: measured against 1843 ads, 69 required PROFICIENT-level German and the prose check caught only 3 of them incidentally. jobRoomRequiresGerman() now checks the structured field directly (spokenLevel/writtenLevel === "PROFICIENT") before the description-based check runs. Levels seen: NONE, BASIC, INTERMEDIATE, PROFICIENT — only PROFICIENT (fluent/C1+) is dropped, matching the B1 cutoff used everywhere else.

  • jobbasel.ch looks like a JS-only React/Vite SPA (no jobs in the homepage's raw HTML), but its search route is server-rendered: GET /job/suche-{slug} returns full HTML with a window.__PRELOADED_STATE__ blob containing the matched vacancies as JSON — no headless browser needed. Pagination is a URL suffix (-seite-{n}); a bare ?page= query param is silently ignored. Swept with the same fixed keyword list as job-room.ch (no keyword-union endpoint exists) and deduped by vacancy id. Each item carries its full German description, so the DE/FR check runs in-process.

  • englishjobsearch.ch also looks JS-only at first glance, but any page accepts ?format=markdown and returns clean server-rendered markdown with full job cards — title, company, city, date, description. Hit the canton pages directly (/in/canton-basel-stadt, /in/canton-basel-landschaft) rather than the /in/basel/ landing page, which only lists canton counts. Every job link is a /clickout/<hash> tracking redirect rather than the real posting URL — there's no way to recover the underlying employer link without following the redirect, so the clickout link is stored as-is.

  • basel-jobs.ch is behind DataDome and is now integrated (fetchBaselJobs), contrary to the earlier note that it couldn't be. What the block actually permits, tested:

    • plain curl/fetch → 403 on every UA, honest or spoofed;
    • Chrome --headless=new over CDP → 403 (headless is detected);
    • Playwright's own chromium.launch(), even headful → 403 (launch-path artefacts);
    • a real Chrome launched normally with --remote-debugging-port, attached over CDP afterwards → 200.

    Clearance lives entirely in a datadome cookie (Max-Age 1 year) that replays fine over plain fetch with a matching User-Agent — the old claim that it "can't be extracted" was true of the browser-extension tooling, not of CDP. So mintDataDomeSession() opens a real window once, caches {cookie, ua} to results/.datadome.json (gitignored), and every run after that is plain HTTP. A rotated Set-Cookie is folded back into the cache; a 403 triggers one re-mint.

    Minting needs a visible window, so launchd runs can't do it. With a stale cookie an unattended run logs unavailable, returns [], and the rest of the fetch continues — it never fails the run. Refresh by hand with bun run mint-datadome, or set LINKME_NO_BROWSER=1 to suppress minting entirely.

    Two passes, both needed. Pass 1 walks the informatik-jobs category (~300 postings over 3 pages). Pass 2 sweeps BASELJOBS_TERMS through /search?terms={kw}&page=1&websearch=1. The category is a curated taxonomy, not the whole board — it holds only what the board filed under IT, so tech roles parked under Industrie/Pharma/Bau and roles whose relevance lives in the description never show up in pass 1. Measured 2026-09-01: pass 2 added 147 postings the category missed (incl. "Senior AI Infrastructure & Platform Engineer (Basel)", "Senior DevOps / Cloud Engineer (Kubernetes, CI/CD, Linux)"), while the category held 257 the search missed. 296 → 442 unique, 48 → 68 viable.

    Both search modes are swept — they query different indexes and each is blind to the other. Do not drop one as redundant:

    term	websearch=1	bare ?terms=
    engineer	90	260 found
    frontend	0	2
    backend	0	4
    react	0	2

    websearch=1 matches role/domain words and reaches postings filed outside the IT category. The bare search is the only mode that matches stack terms — react, frontend, backend, javascript return literally zero under websearch=1. Neither paginates: both ignore page and return the whole result set in one response (bare caps around 100 cards). Measured 2026-09-01 against the category: websearch added 147, bare a further 139 raw (23 past the title filters). 48 → 68 → 88 viable.

    German stems (entwickler, informatik) are in the term list deliberately: search matches description text, so they surface English-titled postings the category never filed under IT; the language filters drop the German-language ones afterwards. The bare search on architect/security drags in a lot of German noise (Architekt = building architect, Sicherheitsassistent, Gerüstbauer) — that's expected and NON_ENGLISH_RE/SOFTWARE_TITLE_RE handle it, 139 raw → 23.

    Pagination/dedup: page 4+ of the category still serves 2 sponsored slots, so "no new ids" is the end condition, not an empty list. Sponsored cards repeat within a page, across pages, and across both passes — dedupe on data-gtm-jid. Cards carry no date, so postedAt is the discovery date. Canton comes from data-gtm-jp (sometimes "BL,BS").

    Language-check the div.plain-content block only, never the whole page. The site chrome is German and on its own clears the ≥ 6 DE/FR word threshold — checking the full page dropped every job, English ones included. That block is the mobile rendering of the description; the desktop view iframes the same text, so it's the only copy in the server-rendered HTML.

  • Basel international orgs (fetchBaselIntlOrgs) sweeps three boards under one label, each only a handful of postings:

    • BIS — the career site is JS-rendered with no usable API, so we read the UNjobs mirror at unjobs.org/offices/bis_182320855689338880. Titles arrive as Role, City, Country and get split back apart. BIS also cycles Senior Software / Data / Data & Analytics Engineer roles through this board.
    • Swiss TPH — Umantis at jobs.swisstph.ch/Jobs/All. Site chrome is German even on English postings, so isEnglishUrl would reject everything; contentChecked: true and the title filters decide instead.
    • Basel Institute on Governance — baselgovernance.org/about/careers. Each vacancy is a linked PDF, so there's no HTML description to language-check. The rounded-3xl card class is what separates vacancy tiles from the annual-report PDF link.
    • These boards use UN-flavoured titles ("Information Management Officer", "ICT Specialist") that SOFTWARE_TITLE_RE alone drops, so INTL_TECH_RE widens the net for these sources only. Boards this small can afford the noise; missing a rare opening can't.

  • itjobs.ch is swept Switzerland-wide (no location param) — the board is small, ~210 jobs over 7 pages, and locking it to Basel returned only ~9. It 404s one page past the end instead of serving an empty list. Each item embeds its full description, so the DE/FR check runs in-process via isEnglishDescription rather than costing a page fetch per job. Use job_location.name for location — it's the English form; the bare location field is German.

• index.ts — orchestrator: loads existing IDs from results/jobs.csv, calls fetchJobs, calls isEnglishContent for each new job, appends survivors to CSV. Writes both search (the query/source that found the job) and location (the posting's own location, as parsed off its card by fetch.ts/fetch-other.ts — no extra page fetch) to the CSV. Migrates old CSVs on startup: adds source if missing, then location (backfilled "" for rows written before the column existed).
  • Every LinkedIn search and every fetch-other.ts source hits a different host, so they run concurrently (bounded: 3 at a time within each group, both groups started together) instead of one after another — cut a full bun fetch from ~16-20 min to well under 2 min. existingLinks.add() happens synchronously right after the .has() check and before any await, which is what makes concurrent workers safe against double-claiming the same link. Concurrency surfaced a real gap: LinkedIn started returning occasional 429s under parallel load, which fetchJobs used to treat as end-of-results (silently truncating that search) — fetchJobs now retries a 429 up to 3× with backoff (1s/2s/4s) before giving up.
• recheck.ts — calls fetchJobs(..., { raw: true }) to bypass SOFTWARE_TITLE_RE, then lists every new job that would be filtered out and prints word-frequency suggestions.
• daily.ts — the notification layer on top of index.ts. Snapshots the CSV's links, shells out to bun fetch, then diffs to find what's new. Survivors are filtered through matchesProfile and grouped by area (Basel / Remote / Zürich / Bern) so the message scans on a phone. results/sent.txt is written only after a successful send, so a Telegram failure retries next run instead of silently losing jobs. A non-zero exit from bun fetch sends a failure notice rather than failing quietly.
  • The area grouping matches against the CSV's location column (the posting's real location) first, falling back to search (the query/source label) only when location is empty — i.e. for rows written before the location column existed. This fixed two real mislabels: a LinkedIn "Bern" sweep can surface a job actually based in Lausanne (search label ≠ real location), and fetch-other.ts sources (job-room.ch, itjobs.ch, stellen-basel.ch, datacareer.ch) write the source name into search instead of a city, which used to dump their jobs into "⚪️ Other" regardless of where they actually were.
• profile.ts — matchesProfile(title, company, minScore = 4). Hard rejects run first (Microsoft stack, Java/C++/SAP, life-science, non-software functions, blocked companies Jobgether and ALLPS), then SIGNALS scores stack and seniority terms from cv/polo.md. This is a second, stricter gate than the fetch filters: fetch.ts decides what lands in the CSV, profile.ts decides what's worth a phone buzz. Widen SOFTWARE_TITLE_RE to catch more in the CSV; tune SIGNALS/minScore to change what gets sent.
• telegram.ts — reads .env directly (no dotenv dep). Accepts TELEGRAM_BOT or TELEGRAM_TOKEN for the token. Splits at 3900 chars on line boundaries (Telegram hard-caps at 4096) and sleeps 400 ms between parts to stay under the rate limit. Messages are HTML — run titles/companies through escapeHtml.

Filter pipeline (applied in order)

1. Switzerland location guard (card HTML) — extracts job-search-card__location from each card; drops jobs where location doesn't contain "Switzerland". Prevents LinkedIn from returning global results when geoId is ignored on unauthenticated requests.
2. NON_ENGLISH_RE (title) — drops German gender markers, umlauts, /in suffix, French accents/markers. Runs inside fetchJobs.
3. JUNIOR_TITLE_RE (title) — drops junior/intern/trainee/werkstudent. Runs inside fetchJobs.
4. JAVA_TITLE_RE (title) — drops Java-titled roles. Runs inside fetchJobs.
5. BIO_DOMAIN_RE (title) — drops roles whose domain is biology/chemistry/the lab bench (bioinformatics, biostatistics, GMP/MSAT, genomics, pathology, drug discovery, bench Scientist titles). Filters on the role, not the employer — software/platform/data jobs at Roche, Novartis, Swiss TPH and sciCORE stay in scope. Runs inside fetchJobs.
6. SOFTWARE_TITLE_RE (title) — requires a relevant tech/role term. Skipped when raw: true. Runs inside fetchJobs. 6.5. GERMAN_TITLE_RE (title) — drops "(German-speaking)" / "deutschsprachig" titles without a description fetch. Runs inside fetchJobs.
7. isViableContent / isViableUrl (description page) — one fetch, two checks: (a) counts DE/FR indicator words, drops if ≥ 6; (b) requiresGerman() drops English postings that demand German above B1. Runs in index.ts only (not recheck.ts). Adds 350 ms delay per job. isEnglishContent / isEnglishUrl remain exported for recheck.ts and temp scripts.

requiresGerman(text) is deliberately more than a keyword match — Rafael is German B1 (TELC), so B1/"basic"/"a plus" is fine and B2+ is not. For each mention of German it takes the nearest qualifying marker, and a marker is ignored when another language name sits between it and the mention. Both rules come from real false results:

• "Infrastructure skills are a plus … Good knowledge of German" (Helvetia) — presence-based matching let an unrelated "a plus" 95 chars away cancel a real requirement.
• "Fluent in English; German and/or French advantageous" (albedis) — "Fluent" is nearer to "German" than "advantageous" is, but qualifies English.

Three further rules, added 2026-09-01 after an audit found the filter catching 0 of 24 basel-jobs.ch postings that mention German:

• Coordination beats the intervening-language rule. "Excellent communication skills in both English and German" (Karakun) and "Fluency in English and German" (J. Safra Sarasin) are hard requirements, but "English" sitting between the marker and the mention handed the marker to English and waved them through. A marker now still binds when the other language is coordinated with German (and/und/sowie/&/as well as). Deliberately excludes and/or and a bare or — "Very good English and/or German" (BBF) is satisfied by English alone and must stay optional.
• German-language markers actually match now. sehr\s+gute? could never match "Sehr gutes Deutsch" — the trailing \b fails against the "s". It's sehr\s+gut\w* now, plus fliessend\w*, Deutschkenntnisse and erforderlich. This is why every German-written Swisscom posting ("Sehr gutes Deutsch in Wort und Schrift") passed the filter.
• Spec-sheet language lines. "Languages: German & English" (contractor postings) has no adjective to bind to, so the Languages:/Sprachen: label itself counts as a required marker. It still goes through the nearest-marker comparison, so "Language: English and German (bonus)" correctly loses to the closer "bonus".

Regression cases for all of the above live in the git history of this change; re-check them before touching requiresGerman — the optional/required boundary is where every past bug has been.

Deduplication: job IDs are extracted from results/jobs.csv on startup; any ID already seen is skipped before content-checking.

SearchConfig

url overrides all filter params. Otherwise build from: keywords, geoId/location, workType, jobType, timePosted, seniority.

Known Swiss geoIds:

• Basel city: 103383283
• Basel Metropolitan Area: 90009885
• Basel-Country: 107117530
• Zürich city: 102436504
• Zürich Metropolitan Area: 107814425
• Zürich Area: 90009888

Use location: "City, Switzerland" (no geoId) when the geoId is unknown — LinkedIn resolves by name.

Auth warning: Do NOT send the li_at cookie in job search requests (/jobs/search/). With the cookie, LinkedIn serves the React SPA instead of server-rendered HTML, breaking the card parser entirely. The cookie is only useful for individual profile pages.

Searching any city / mode temporarily

Write a throwaway script in scripts/ (e.g. scripts/bern-temp.ts) — never touch config.ts unless the user explicitly asks to save the change. Delete temp scripts after the session unless the user asks to keep them.

Common patterns:

// City search (onsite + hybrid)
import { KEYWORDS } from "../config.ts";
import { fetchJobs } from "../fetch.ts";
import type { SearchConfig } from "../config.ts";

const config: SearchConfig = {
  label: "Bern", keywords: KEYWORDS, location: "Bern, Switzerland",
  workType: ["onsite", "hybrid"], timePosted: "month",
};
const jobs = await fetchJobs(config);
for (const j of jobs) console.log(j.title, j.company, j.link);

// Remote-only across all Switzerland
const config: SearchConfig = {
  label: "Switzerland Remote", keywords: KEYWORDS, location: "Switzerland",
  workType: ["remote"], timePosted: "month",
};

// Description-level keyword check (e.g. Ruby/Rails)
const RUBY_RE = /\bruby\b|\brails\b/i;
// fetch each job page, test RUBY_RE against the description text

Extra filters for temp scripts (import from ../fetch.ts):

• JAVA_TITLE_RE — drop Java roles
• JUNIOR_TITLE_RE — drop junior/intern
• isEnglishContent(id) — drop DE/FR description content
• Add your own inline RE for EMEA, .NET, etc.

Ranking jobs against the CV

Read cv/polo.md for the full profile. Key signals:

• Seniority: Senior / Staff / Principal / Lead / Architect — weight titles accordingly
• Core stack: Python · TypeScript · Ruby on Rails · React · Node.js
• Domain strengths: platform engineering · data pipelines · AI/LLM integrations · DevOps (Docker, Kubernetes, CI/CD, AWS)
• Background edges: fintech/CBDC (Moedax), civic tech (Tactical Tech / GFOSS), independent consulting (ExtraPolo, 40+ projects)
• Languages: English (full) · German B1 TELC certified · Spanish (full) — drop roles requiring German above B1; requiresGerman() does this automatically at fetch time
• Hard no: biological domains — drop bioinformatics, computational biology, biostatistics, lab/bench Scientist, GMP/MSAT, clinical/assay and drug-discovery-science roles; he has no life-science background. Software roles at pharma companies are still strong targets
• Hard no: Microsoft stack — drop .NET/C#, Azure-first, Windows/SCCM/Intune, SharePoint, Dynamics, Power Platform roles entirely; don't present them even as weak fits
• Location: based in Basel, open to Basel / Bern / Zürich; EU citizen, no permit needed

Output format

Always present jobs as a markdown table — never as a bullet list or prose: #, Title, Company, Canton, Why it fits, Link.

Canton is mandatory on every table, including single-city searches and the list of dropped/rejected jobs (there Why dropped replaces Why it fits). Derive it from the actual location on the posting, not from the search label — recruiters and multi-site companies often post under a different area than the job sits in. Use the two-letter code: BS, BL, ZH, BE, ZG, LU, VD, GE, TI, AG, SG, TG. For remote roles write Remote (add scope if known, e.g. Remote (EU)).

Links use just the job ID as anchor text: [4192954199](https://www.linkedin.com/jobs/view/4192954199/)

Some employers (Optiml, LocalBini seen repeatedly) repost the same role under a new job ID every few weeks rather than keeping one listing open. When several rows share the same title + company, show only the most recent job ID and drop the older duplicates from the table — they're the same opening, not new leads.

Workflow: "find jobs in X city / remote / custom search"

1. Write a temp script in scripts/ with the right SearchConfig (city, workType, custom keywords).
2. Apply extra inline filters as needed: EMEA, .NET, description-level keyword check.
3. Run isEnglishContent per job to drop DE/FR content.
4. Rank survivors against cv/polo.md, present as a table.
5. Delete temp scripts after the session unless asked to keep them.

Scheduled automations

Both run as launchd agents — this machine has no cron at all, so don't reach for crontab.

plist (~/Library/LaunchAgents/)	when	runs
com.rafapolo.linkme-daily	every 6h (00:00 / 06:00 / 12:00 / 18:00)	bun run daily.ts → results/daily.log
ch.polo.scicore-watch	08:47	scripts/scicore-watch.sh → results/.scicore-watch/watch.log

If the Mac is asleep at the scheduled time, launchd fires the job on wake rather than skipping the day. launchctl list | grep -E 'linkme|scicore' to check they're loaded; launchctl unload <plist> to stop one.

scripts/scicore-watch.sh watches https://scicore.unibas.ch/about-scicore/open-positions/ for a software engineering role Rafael is waiting on. It extracts the "Join the team!" block (keeping each posting's <a href> so the alert is tappable), diffs it against results/.scicore-watch/snapshot.txt, and notifies only when a newly added line matches $WANTED ("software"). Every other change still updates the snapshot silently — that's what keeps the eventual real alert clean instead of tangled with months of sysadmin churn. Three consecutive fetch failures alert, so a broken watcher can't look like "no news".

Test the alert path without waiting for the page to change:

SCICORE_URL="file://$PWD/fixture.html" bash scripts/scicore-watch.sh

The whole extractor is a python3 -c program inside shell single quotes, so it must not contain a literal apostrophe — that's why the <a> quote class is built with chr(39) instead of written inline.

LinkedIn session token

.env holds LI_AT (LinkedIn session cookie), plus TELEGRAM_BOT and TELEGRAM_CHAT_ID for the digest. It is gitignored.

LI_AT is the li_at cookie value. Use it to fetch authenticated LinkedIn pages via curl:

curl -s "https://www.linkedin.com/in/<slug>" \
  -H "Cookie: li_at=$(grep LI_AT .env | cut -d= -f2-)" \
  -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36" \
  --compressed -o /tmp/profile.html

What works: Full HTML pages for profiles that don't 999-block (varies by profile). Extract data with Python regex on the HTML.

What doesn't: LinkedIn Voyager API returns 401/403 (CSRF issues). ch.linkedin.com slugs sometimes work when www.linkedin.com returns HTTP 999. Public profiles without login redirect to a 1530-byte JS stub — no useful data.

Tip: Profile data is embedded as escaped JSON in the HTML. Use html.unescape() + regex for "headline", "locationName", defaultLocalizedName fields.

Workflow: "check if new keywords should be added"

1. Run bun recheck.
2. Identify false negatives — real software roles that got dropped.
3. Ignore generic frequency suggestions (engineer, development, senior).
4. Propose specific additions to SOFTWARE_TITLE_RE in fetch.ts or words to NON_ENGLISH_RE.