rodado
7276aded92
Rewrite the README around the current pipeline: the twice-daily Telegram digest, the sciCORE watcher, Telegram setup, and the two-stage filter pipeline. CLAUDE.md gains the bun daily commands, the daily/profile/telegram architecture notes, and a scheduled-automations section covering both launchd agents. Records that fetch.ts gates what reaches the CSV while profile.ts gates what reaches the phone.
152 lines
7.5 KiB
Markdown
152 lines
7.5 KiB
Markdown
# linkme
|
|
|
|
A personal job-hunting pipeline for the Swiss market. It scrapes LinkedIn (plus a handful of Swiss boards), filters out everything that doesn't fit the profile in `cv/polo.md`, appends what survives to a CSV, and pushes the good ones to Telegram twice a day.
|
|
|
|
Two things run on their own once installed:
|
|
|
|
| automation | when | what it does |
|
|
|---|---|---|
|
|
| **daily digest** | 08:00 + 18:00 | fetch every search, push new profile-matching jobs to Telegram |
|
|
| **sciCORE watch** | 08:47 | alert if sciCORE's open-positions page lists a *software* role |
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
bun fetch # scrape all searches, append new jobs to results/jobs.csv
|
|
bun daily # fetch, then Telegram the new jobs that match the profile
|
|
bun recheck # show what the filters are currently dropping + keyword suggestions
|
|
```
|
|
|
|
`bun fetch` needs no setup — reruns are safe, already-seen jobs are skipped by ID. Telegram needs the two-line setup below.
|
|
|
|
## Telegram setup
|
|
|
|
```bash
|
|
# 1. Message @BotFather → /newbot → copy the token into .env
|
|
echo 'TELEGRAM_BOT=<token>' >> .env
|
|
|
|
# 2. Send your bot any message ("hi"), then discover your chat ID
|
|
bun run telegram.ts --setup # prints TELEGRAM_CHAT_ID=... — paste it into .env
|
|
```
|
|
|
|
Then `bun daily --dry-run` prints the digest instead of sending it, and `bun daily --resend` re-sends today's jobs from the CSV without re-fetching.
|
|
|
|
`.env` also holds `LI_AT`, a LinkedIn session cookie used only for fetching individual *profile* pages. Never send it with a job search — with the cookie LinkedIn serves a React SPA instead of server-rendered HTML and the card parser gets nothing. `.env` is gitignored.
|
|
|
|
## Structure
|
|
|
|
```
|
|
config.ts # keyword families, search areas, the sweep() generator
|
|
fetch.ts # LinkedIn scraper, HTML parser, all title/content filters
|
|
fetch-other.ts # extra sources: itjobs.ch, job-room.ch, stellen-basel.ch,
|
|
# datacareer.ch (403, currently dead), Basel intl orgs
|
|
index.ts # orchestrator: run searches → dedupe → content-check → CSV
|
|
recheck.ts # raw-mode fetch: lists what the filters dropped
|
|
daily.ts # digest: fetch, match against the profile, send to Telegram
|
|
profile.ts # title-level scoring — decides what's worth a notification
|
|
telegram.ts # Telegram delivery + --setup chat-ID discovery
|
|
scripts/ # one-off searches (*-temp.ts) and scicore-watch.sh
|
|
cv/polo.md # the profile everything is ranked against
|
|
results/jobs.csv # output, one row per job
|
|
results/sent.txt # links already pushed — never sent twice
|
|
```
|
|
|
|
## How a job gets through
|
|
|
|
Two stages. Cheap title filters first, then one page fetch per survivor.
|
|
|
|
**Title filters** (inside `fetchJobs`, no network cost):
|
|
|
|
1. **Switzerland guard** — the card's location must say Switzerland. LinkedIn ignores `geoId` on unauthenticated requests and will happily return global results.
|
|
2. `NON_ENGLISH_RE` — German gender markers (`m/w/d`, `/in`), umlauts, French `H/F` and accents.
|
|
3. `JUNIOR_TITLE_RE` — junior, intern, trainee, werkstudent, entry-level.
|
|
4. `JAVA_TITLE_RE` — Java-titled roles.
|
|
5. `BIO_DOMAIN_RE` — bioinformatics, biostatistics, genomics, GMP/MSAT, bench-scientist titles. Filters on the **role, not the employer**: software and data jobs at Roche, Novartis or Swiss TPH stay in.
|
|
6. `SOFTWARE_TITLE_RE` — requires a real tech/role term. Skipped in `raw` mode, which is what `bun recheck` uses.
|
|
7. `GERMAN_TITLE_RE` — "(German-speaking)", "deutschsprachig".
|
|
|
|
**Content check** (`index.ts` only, one fetch + 350 ms per job): `isViableContent` counts German/French indicator words and drops at ≥ 6, then `requiresGerman()` drops English postings demanding German above B1.
|
|
|
|
That last one is deliberately more than a keyword match. For each mention of German it takes the *nearest* qualifying marker, and ignores a marker when another language name sits between it and the mention — both rules came from real false results, like *"Fluent in English; German and/or French advantageous"*, where "Fluent" sits closer to "German" than "advantageous" does but qualifies English.
|
|
|
|
**Profile match** (`profile.ts`, digest only) scores the title against the CV's stack and seniority signals, needing 4+ points to notify. Hard rejects encode standing decisions: Microsoft stack, Java/C++/SAP, life-science domains, non-software functions, and the blocked companies Jobgether (an aggregator, not an employer) and ALLPS (postings require German despite passing the English filter).
|
|
|
|
Jobs are deduped globally by ID against `results/jobs.csv` on startup, so nothing is ever fetched or sent twice.
|
|
|
|
## Output
|
|
|
|
`results/jobs.csv` — `date_found`, `posted_at`, `search`, `title`, `company`, `link`, `source`. Open it in any spreadsheet and delete rows as you apply or dismiss.
|
|
|
|
## Configuring searches
|
|
|
|
`config.ts` holds four keyword families as separate OR-strings — `KEYWORDS`, `PLATFORM_KEYWORDS`, `RESEARCH_KEYWORDS`, `FIELD_KEYWORDS`. They're kept apart on purpose: LinkedIn ranks a focused query far better than one 50-term disjunction. `sweep()` generates one search per area per family (Research skips Bern; Platform and Field add a Switzerland-wide remote pass).
|
|
|
|
```ts
|
|
export const searches: SearchConfig[] = [
|
|
{
|
|
label: "Basel (city)",
|
|
keywords: KEYWORDS,
|
|
geoId: "103383283",
|
|
workType: ["onsite", "hybrid"],
|
|
timePosted: "month",
|
|
},
|
|
];
|
|
```
|
|
|
|
`SearchConfig` also takes `jobType`, `seniority`, and `location` (use `"City, Switzerland"` when the geoId is unknown — LinkedIn resolves by name). A `url:` field overrides all filter params, so you can paste a LinkedIn search URL directly.
|
|
|
|
Known Swiss geoIds:
|
|
|
|
| area | geoId |
|
|
|---|---|
|
|
| Basel (city) | `103383283` |
|
|
| Basel Metropolitan Area | `90009885` |
|
|
| Basel-Country | `107117530` |
|
|
| Zürich (city) | `102436504` |
|
|
| Zürich Metropolitan Area | `107814425` |
|
|
| Zürich Area | `90009888` |
|
|
|
|
## One-off searches
|
|
|
|
Don't edit `config.ts` for a single search — write a throwaway in `scripts/`:
|
|
|
|
```ts
|
|
import { KEYWORDS } from "../config.ts";
|
|
import { fetchJobs } from "../fetch.ts";
|
|
|
|
const jobs = await fetchJobs({
|
|
label: "Luzern", keywords: KEYWORDS,
|
|
location: "Luzern, Switzerland",
|
|
workType: ["onsite", "hybrid"], timePosted: "month",
|
|
});
|
|
for (const j of jobs) console.log(j.title, j.company, j.link);
|
|
```
|
|
|
|
```bash
|
|
bun run scripts/luzern-temp.ts
|
|
```
|
|
|
|
`*-temp.ts` scripts are disposable — delete them when the search is done.
|
|
|
|
## Scheduling
|
|
|
|
Both automations are launchd agents, not cron — this machine has no cron at all.
|
|
|
|
```bash
|
|
launchctl list | grep -E 'linkme|scicore' # are they loaded?
|
|
launchctl unload ~/Library/LaunchAgents/com.rafapolo.linkme-daily.plist # stop the digest
|
|
launchctl unload ~/Library/LaunchAgents/ch.polo.scicore-watch.plist # stop the watcher
|
|
```
|
|
|
|
If the Mac is asleep at the scheduled time, launchd runs the job once it wakes rather than skipping the day.
|
|
|
|
Logs: `results/daily.log` for the digest, `results/.scicore-watch/watch.log` for the watcher. The watcher logs every run — `no change`, `change ignored`, or the diff it sent — so silence on your phone can always be told apart from a broken script. Test its alert path without waiting for the real page to change:
|
|
|
|
```bash
|
|
SCICORE_URL="file://$PWD/fixture.html" bash scripts/scicore-watch.sh
|
|
```
|
|
|
|
## recheck
|
|
|
|
`bun recheck` re-runs every search with `SOFTWARE_TITLE_RE` bypassed and lists each job the filters would drop, plus a word-frequency table of those titles. Use it to spot false negatives; ignore the generic suggestions (`engineer`, `development`, `senior`) and look for real role terms worth adding to the regex.
|