fix: catch job-room.ch's structured German-fluency requirement

languageSkills[] (e.g. {languageIsoCode: "de", spokenLevel: "PROFICIENT"})
sits outside jobDescriptions[].description, so requiresGerman()/
isViableDescription() never saw it — the UI renders this as
"Deutsch (Fliessend)", which is exactly what kept reaching Telegram.
Measured: 69 of 1843 ads required PROFICIENT German, prose check caught
only 3.
This commit is contained in:
2026-09-04 11:18:15 +02:00
parent fccecb2254
commit b2e5c7865a
2 changed files with 24 additions and 0 deletions
+1
View File
@@ -45,6 +45,7 @@ plans/ # LinkedIn post drafts
- **`fetch.ts`** — HTTP scraping, HTML parsing, and all filters. Exports `fetchJobs(config, { raw? })` and `isEnglishContent(id)`.
- **`fetch-other.ts`** — scrapes extra sources (datacareer.ch, stellen-basel.ch, itjobs.ch, job-room.ch, Basel international orgs, jobbasel.ch, englishjobsearch.ch, basel-jobs.ch). Imported by `index.ts`.
- **datacareer.ch** inverts the usual bot rule: its nginx **403s browser-like User-Agents** and serves honest ones (no UA, `curl/8.0`, Googlebot) fine. The spoofed Chrome UA in the shared `HEADERS` was the entire cause of the long-standing 403 — it uses `PLAIN_HEADERS` instead. Small board: Basel returns **8 jobs on one page with no pagination**, so `?page=2` legitimately yields 0 items.
- **job-room.ch** carries its language requirement as a **structured field** (`languageSkills[]`, e.g. `{languageIsoCode: "de", spokenLevel: "PROFICIENT"}`), entirely separate from the free-text `jobDescriptions[].description` — the UI renders it as "Deutsch (Fliessend)". `requiresGerman()`/`isViableDescription()` only ever scan the prose, so this field slipped through unchecked until 2026-09-04: measured against 1843 ads, 69 required PROFICIENT-level German and the prose check caught only 3 of them incidentally. `jobRoomRequiresGerman()` now checks the structured field directly (`spokenLevel`/`writtenLevel === "PROFICIENT"`) before the description-based check runs. Levels seen: `NONE`, `BASIC`, `INTERMEDIATE`, `PROFICIENT` — only `PROFICIENT` (fluent/C1+) is dropped, matching the B1 cutoff used everywhere else.
- **jobbasel.ch** looks like a JS-only React/Vite SPA (no jobs in the homepage's raw HTML), but its search route is server-rendered: `GET /job/suche-{slug}` returns full HTML with a `window.__PRELOADED_STATE__` blob containing the matched vacancies as JSON — no headless browser needed. Pagination is a URL suffix (`-seite-{n}`); a bare `?page=` query param is silently ignored. Swept with the same fixed keyword list as job-room.ch (no keyword-union endpoint exists) and deduped by vacancy id. Each item carries its full German description, so the DE/FR check runs in-process.
- **englishjobsearch.ch** also looks JS-only at first glance, but any page accepts `?format=markdown` and returns clean server-rendered markdown with full job cards — title, company, city, date, description. Hit the canton pages directly (`/in/canton-basel-stadt`, `/in/canton-basel-landschaft`) rather than the `/in/basel/` landing page, which only lists canton counts. Every job link is a `/clickout/<hash>` tracking redirect rather than the real posting URL — there's no way to recover the underlying employer link without following the redirect, so the clickout link is stored as-is.
- **basel-jobs.ch** is behind DataDome and **is now integrated** (`fetchBaselJobs`), contrary to the earlier note that it couldn't be. What the block actually permits, tested:
+23
View File
@@ -236,9 +236,28 @@ interface JobRoomAd {
jobDescriptions?: Array<{ title: string; description?: string }>;
company?: { name?: string; city?: string };
location?: { city?: string; cantonCode?: string };
languageSkills?: JobRoomLanguageSkill[];
};
}
interface JobRoomLanguageSkill {
languageIsoCode?: string;
spokenLevel?: string;
writtenLevel?: string;
}
// job-room's language requirement is a structured field, not prose — it never
// touches jobDescriptions[].description, so isViableDescription()/requiresGerman()
// can't see it. The UI renders it as e.g. "Deutsch (Fliessend)"; PROFICIENT is
// that "Fliessend" (fluent/C1+) tier, well above Rafael's German B1. BASIC and
// INTERMEDIATE stay in scope.
function jobRoomRequiresGerman(skills?: JobRoomLanguageSkill[]): boolean {
if (!skills) return false;
return skills.some(
(s) => s.languageIsoCode === "de" && (s.spokenLevel === "PROFICIENT" || s.writtenLevel === "PROFICIENT")
);
}
// jobs.ch links arrive with a tracking query string that would break dedup
// across runs if it ever changes; the path alone identifies the posting.
function stripTracking(url: string): string {
@@ -299,6 +318,10 @@ export async function fetchJobRoom(): Promise<Job[]> {
if (MS_STACK_RE.test(title)) continue;
if (PHARMA_ROLE_RE.test(title)) continue;
// Structured language requirement — separate from the free-text description
// entirely, so it must be checked even when the description reads clean.
if (jobRoomRequiresGerman(content?.languageSkills)) { langDropped++; continue; }
// A handful of ads ship without a description. isViableDescription passes
// those through by design, so leave contentChecked unset and let index.ts
// fall back to fetching the page.