# linkme

A personal job-hunting pipeline for the Swiss market. It scrapes LinkedIn (plus a handful of Swiss boards), filters out everything that doesn't fit the profile in `cv/polo.md`, appends what survives to a CSV, and pushes the good ones to Telegram throughout the day.

One thing runs on its own once installed:

| automation | when | what it does |
|---|---|---|
| **daily digest** | 09:00 / 12:00 / 15:00 / 17:00 / 20:00 | fetch every search + source, push new profile-matching jobs to Telegram |

sciCORE used to have its own separate 08:47 watcher; it's now just another source swept inside `bun fetch`/`bun daily` (see Sources below).

## Quick start

```bash
bun fetch      # scrape all searches, append new jobs to results/jobs.csv
bun daily      # fetch, then Telegram the new jobs that match the profile
bun recheck    # show what the filters are currently dropping + keyword suggestions

bun run mint-datadome   # refresh the basel-jobs.ch cookie (opens a Chrome window)
```

`bun fetch` needs no setup — reruns are safe, already-seen jobs are skipped by ID, and every search/source runs concurrently (a full run is well under 2 minutes). Telegram needs the two-line setup below.

## Telegram setup

```bash
# 1. Message @BotFather → /newbot → copy the token into .env
echo 'TELEGRAM_BOT=<token>' >> .env

# 2. Send your bot any message ("hi"), then discover your chat ID
bun run telegram.ts --setup      # prints TELEGRAM_CHAT_ID=... — paste it into .env
```

Then `bun daily --dry-run` prints the digest instead of sending it, and `bun daily --resend` re-sends today's jobs from the CSV without re-fetching.

`.env` also holds `LI_AT`, a LinkedIn session cookie used only for fetching individual *profile* pages. Never send it with a job search — with the cookie LinkedIn serves a React SPA instead of server-rendered HTML and the card parser gets nothing. `.env` is gitignored.

## Structure

```
config.ts          # keyword families, search areas, the sweep() generator
fetch.ts           # LinkedIn scraper, HTML parser, all title/content filters
fetch-other.ts     # the ten non-LinkedIn sources (see Sources below)
index.ts           # orchestrator: run searches → dedupe → content-check → CSV
recheck.ts         # raw-mode fetch: lists what the filters dropped
daily.ts           # digest: fetch, match against the profile, send to Telegram
profile.ts         # title-level scoring — decides what's worth a notification
telegram.ts        # Telegram delivery + --setup chat-ID discovery
scripts/           # one-off searches (*-temp.ts); scicore-watch.sh kept for history, retired
cv/polo.md         # the profile everything is ranked against
results/jobs.csv   # output, one row per job
results/sent.txt   # links already pushed — never sent twice
```

## Sources

LinkedIn is swept by `config.ts`'s search families. Everything else lives in `fetch-other.ts` and runs concurrently with it — a full `bun fetch` stays under two minutes.

| source | scope | notes |
|---|---|---|
| **LinkedIn** | 23 searches | keyword families × areas, generated by `sweep()` |
| **job-room.ch** | Switzerland | the federal employment service board (RAV); by far the largest feed |
| **basel-jobs.ch** | Basel region | behind DataDome — needs a cookie, see below |
| **jobbasel.ch** | Basel region | React SPA, but `/job/suche-{slug}` is server-rendered with a `window.__PRELOADED_STATE__` JSON blob |
| **englishjobsearch.ch** | BS + BL cantons | any page accepts `?format=markdown` and returns clean server-rendered markdown |
| **itjobs.ch** | Switzerland | small (~210 jobs); locking it to Basel returned only ~9, so it's swept nationally |
| **datacareer.ch** | Basel | 8 jobs on one page, no pagination |
| **stellen-basel.ch** | Basel region | WordPress REST API |
| **Basel intl orgs** | Basel | BIS (via the UNjobs mirror), Swiss TPH, Basel Institute on Governance |
| **scicore.unibas.ch** | Basel | sciCORE's open-positions announcement list; used to be its own separate 08:47 watcher, now just another source |
| **swisscom.com** | Switzerland | Swisscom's own Workday career site, pulled via its public CXS JSON API; small board and mostly German-language, so few survive the filters |

Two of these were dead until recently and are worth knowing about:

**datacareer.ch** spent months returning 403 on every request. The cause was our own User-Agent — its nginx inverts the usual bot rule and blocks browser-like UAs while serving honest ones (no UA, `curl`, Googlebot). It gets `PLAIN_HEADERS` instead of the shared spoofed-Chrome ones.

**basel-jobs.ch** sits behind DataDome. Plain `fetch` gets 403 on any UA, headless Chrome over CDP gets 403, and Playwright's own `launch()` gets 403 even headful — but a real Chrome launched normally with `--remote-debugging-port` and attached to afterwards gets through. Clearance then lives entirely in a `datadome` cookie that replays fine over plain `fetch`, so a browser is only ever needed to *mint*, never to scrape:

```bash
bun run mint-datadome    # opens a real Chrome window for ~10s, caches the cookie
```

The cache is `results/.datadome.json` (gitignored). Because minting needs a visible window, a launchd run can't do it — if the cookie goes stale the source logs `unavailable` and yields nothing while every other source continues, so the digest never fails over it. Set `LINKME_NO_BROWSER=1` to suppress minting entirely.

It's scraped in three passes, all of which cover different sets: the `informatik-jobs` category is a curated taxonomy rather than the whole board, and the two search modes query different indexes — `?terms=X&websearch=1` finds role words and postings filed outside IT, while the bare `?terms=X` is the only mode that matches stack terms (`react`, `frontend` and `backend` return literally zero under `websearch=1`).

## How a job gets through

Two stages. Cheap title filters first, then one page fetch per survivor.

**Title filters** (inside `fetchJobs`, no network cost):

1. **Switzerland guard** — the card's location must say Switzerland. LinkedIn ignores `geoId` on unauthenticated requests and will happily return global results.
2. `NON_ENGLISH_RE` — German gender markers (`m/w/d`, `/in`), umlauts, French `H/F` and accents.
3. `JUNIOR_TITLE_RE` — junior, intern, trainee, werkstudent, entry-level.
4. `JAVA_TITLE_RE` — Java-titled roles.
5. `BIO_DOMAIN_RE` — bioinformatics, biostatistics, genomics, GMP/MSAT, bench-scientist titles. Filters on the **role, not the employer**: software and data jobs at Roche, Novartis or Swiss TPH stay in.
6. `SOFTWARE_TITLE_RE` — requires a real tech/role term. Skipped in `raw` mode, which is what `bun recheck` uses.
7. `GERMAN_TITLE_RE` — "(German-speaking)", "deutschsprachig".

**Content check**: `isViableContent`/`isViableUrl` (LinkedIn, one page fetch + 350 ms per job in `index.ts`) and `isViableDescription` (`fetch-other.ts` sources that already carry the description inline, no extra fetch) all run the same two checks — DE/FR indicator words dropped at ≥ 6, then `requiresGerman()` drops English postings demanding German above B1.

That last one is deliberately more than a keyword match, because the optional/required boundary is where every bug in it has been. Rafael is German B1, so *"a plus"* / *"advantageous"* / *"B1"* must survive and *"fluent"* / *"C1"* / *"sehr gutes Deutsch"* must not.

For each mention of German it takes the *nearest* qualifying marker, and ignores a marker when another language name sits between it and the mention — *"Fluent in English; German and/or French advantageous"* qualifies English, not German. But that rule has an exception: when the other language is **coordinated** with German the marker governs both, so *"Excellent communication skills in both English and German"* is correctly a hard requirement. Coordination means `and`/`und`/`sowie`/`&` — explicitly not `and/or`, since *"Very good English and/or German"* is satisfiable by English alone.

Two more cases it handles: German-language phrasing (*"Sehr gutes Deutsch in Wort und Schrift"*, which for a long time matched nothing at all), and spec-sheet lines like *"Languages: German & English"* that have no adjective to bind to — there the `Languages:` label itself counts, though still only if no optional marker sits closer.

**Profile match** (`profile.ts`, digest only) scores the title against the CV's stack and seniority signals, needing 4+ points to notify. Hard rejects encode standing decisions: Microsoft stack, Java/C++/SAP, life-science domains, non-software functions, and the blocked companies Jobgether (an aggregator, not an employer) and ALLPS (postings require German despite passing the English filter).

Jobs are deduped globally by ID against `results/jobs.csv` on startup, so nothing is ever fetched or sent twice.

## Output

`results/jobs.csv` — `date_found`, `posted_at`, `search`, `title`, `company`, `link`, `source`, `location`. `search` is the query or source that found the job; `location` is the posting's own location, which is what the digest groups on (a "Bern" sweep can surface a job actually based in Lausanne). Open it in any spreadsheet and delete rows as you apply or dismiss.

## Configuring searches

`config.ts` holds four keyword families as separate OR-strings — `KEYWORDS`, `PLATFORM_KEYWORDS`, `RESEARCH_KEYWORDS`, `FIELD_KEYWORDS`. They're kept apart on purpose: LinkedIn ranks a focused query far better than one 50-term disjunction. `sweep()` generates one search per area per family (Research skips Bern; Platform and Field add a Switzerland-wide remote pass).

```ts
export const searches: SearchConfig[] = [
  {
    label: "Basel (city)",
    keywords: KEYWORDS,
    geoId: "103383283",
    workType: ["onsite", "hybrid"],
    timePosted: "month",
  },
];
```

`SearchConfig` also takes `jobType`, `seniority`, and `location` (use `"City, Switzerland"` when the geoId is unknown — LinkedIn resolves by name). A `url:` field overrides all filter params, so you can paste a LinkedIn search URL directly.

Known Swiss geoIds:

| area | geoId |
|---|---|
| Basel (city) | `103383283` |
| Basel Metropolitan Area | `90009885` |
| Basel-Country | `107117530` |
| Zürich (city) | `102436504` |
| Zürich Metropolitan Area | `107814425` |
| Zürich Area | `90009888` |

## One-off searches

Don't edit `config.ts` for a single search — write a throwaway in `scripts/`:

```ts
import { KEYWORDS } from "../config.ts";
import { fetchJobs } from "../fetch.ts";

const jobs = await fetchJobs({
  label: "Luzern", keywords: KEYWORDS,
  location: "Luzern, Switzerland",
  workType: ["onsite", "hybrid"], timePosted: "month",
});
for (const j of jobs) console.log(j.title, j.company, j.link);
```

```bash
bun run scripts/luzern-temp.ts
```

`*-temp.ts` scripts are disposable — delete them when the search is done.

## Scheduling

The digest runs as a launchd agent, not cron — this machine has no cron at all.

```bash
launchctl list | grep linkme                                             # is it loaded?
launchctl unload ~/Library/LaunchAgents/com.rafapolo.linkme-daily.plist   # stop it
```

If the Mac is asleep at the scheduled time, launchd runs the job once it wakes rather than skipping the day.

Logs: `results/daily.log`.

## recheck

`bun recheck` re-runs every search with `SOFTWARE_TITLE_RE` bypassed and lists each job the filters would drop, plus a word-frequency table of those titles. Use it to spot false negatives; ignore the generic suggestions (`engineer`, `development`, `senior`) and look for real role terms worth adding to the regex.