fix: catch German requirements requiresGerman was missing
An audit of 89 basel-jobs.ch postings against their live descriptions
found 24 that mention German and 0 of them flagged. Three separate bugs,
each verified against real postings:
Coordination broke the intervening-language rule. In "Excellent
communication skills in both English and German" (Karakun) and "Fluency
in English and German" (J. Safra Sarasin), "English" sits between the
marker and the mention, so the marker was handed to English and a hard
requirement waved through. A marker now still binds when the other
language is coordinated with German (and/und/sowie/&/as well as), but
not through "and/or" or a bare "or" — "Very good English and/or German"
is satisfiable by English alone and must stay optional.
German-language markers never matched at all: `sehr\s+gute?` cannot match
"Sehr gutes Deutsch" because the trailing \b fails on the "s". Widened to
`sehr\s+gut\w*` plus fliessend\w*, Deutschkenntnisse and erforderlich.
This is why German-written postings passed the filter untouched.
Spec-sheet language lines ("Languages: German & English") have no
adjective to bind to, so the Languages:/Sprachen: label now counts as a
required marker. It still goes through the nearest-marker comparison, so
"Language: English and German (bonus)" correctly stays optional.
28 regression cases pass, covering the 15 "a plus" phrasings that must
not be dropped. On basel-jobs.ch this moves 88 viable to 85, removing
exactly the three hard requirements and keeping every optional one.
The filter is shared by every source, so rows already in jobs.csv were
written under the broken logic and are not retroactively corrected.
This commit is contained in:
@@ -104,6 +104,13 @@ plans/ # LinkedIn post drafts
|
||||
- *"Infrastructure skills are **a plus** … Good knowledge of **German**"* (Helvetia) — presence-based matching let an unrelated "a plus" 95 chars away cancel a real requirement.
|
||||
- *"Fluent in **English**; **German** and/or French advantageous"* (albedis) — "Fluent" is nearer to "German" than "advantageous" is, but qualifies English.
|
||||
|
||||
Three further rules, added 2026-09-01 after an audit found the filter catching **0 of 24** basel-jobs.ch postings that mention German:
|
||||
- **Coordination beats the intervening-language rule.** *"Excellent communication skills in both **English** and **German**"* (Karakun) and *"Fluency in **English** and **German**"* (J. Safra Sarasin) are hard requirements, but "English" sitting between the marker and the mention handed the marker to English and waved them through. A marker now still binds when the other language is *coordinated* with German (`and`/`und`/`sowie`/`&`/`as well as`). Deliberately **excludes `and/or` and a bare `or`** — *"Very good English and/or German"* (BBF) is satisfied by English alone and must stay optional.
|
||||
- **German-language markers actually match now.** `sehr\s+gute?` could never match *"Sehr gutes Deutsch"* — the trailing `\b` fails against the "s". It's `sehr\s+gut\w*` now, plus `fliessend\w*`, `Deutschkenntnisse` and `erforderlich`. This is why every German-written Swisscom posting ("Sehr gutes Deutsch in Wort und Schrift") passed the filter.
|
||||
- **Spec-sheet language lines.** *"Languages: German & English"* (contractor postings) has no adjective to bind to, so the `Languages:`/`Sprachen:` label itself counts as a required marker. It still goes through the nearest-marker comparison, so *"Language: English and German (bonus)"* correctly loses to the closer "bonus".
|
||||
|
||||
Regression cases for all of the above live in the git history of this change; re-check them before touching `requiresGerman` — the optional/required boundary is where every past bug has been.
|
||||
|
||||
Deduplication: job IDs are extracted from `results/jobs.csv` on startup; any ID already seen is skipped before content-checking.
|
||||
|
||||
### SearchConfig
|
||||
|
||||
@@ -158,7 +158,16 @@ const GERMAN_OPTIONAL_RE =
|
||||
|
||||
/** Within a mention's window: signals it is genuinely required at B2+. */
|
||||
const GERMAN_REQUIRED_RE =
|
||||
/\b(?:required|require[sd]?|must|mandatory|essential|necessary|fluen\w*|proficien\w*|excellent|strong|very\s+good|good\s+(?:knowledge|command|level)|native|mother\s*tongue|business\s+level|negotiat\w*|B2|C1|C2|verhandlungssicher\w*|fliessend|fließend|muttersprach\w*|sehr\s+gute?|gute?\s+Deutschkenntnisse|voraussetzung|zwingend)\b/i;
|
||||
/\b(?:required|require[sd]?|must|mandatory|essential|necessary|fluen\w*|proficien\w*|excellent|strong|very\s+good|good\s+(?:knowledge|command|level)|native|mother\s*tongue|business\s+level|negotiat\w*|B2|C1|C2|verhandlungssicher\w*|fliessend\w*|fließend\w*|muttersprach\w*|sehr\s+gut\w*|gut\w*\s+Deutschkenntnisse|Deutschkenntnisse|voraussetzung|zwingend|erforderlich)\b/i;
|
||||
|
||||
/**
|
||||
* Spec-sheet style language line ("Languages: German & English"), common on
|
||||
* contractor postings. There is no adjective for the nearest-marker logic to
|
||||
* bind to, so the label itself counts as a requirement — but it goes through
|
||||
* the same nearest-marker comparison, so "Language: English and German (bonus)"
|
||||
* still loses to the "bonus" sitting closer to the mention.
|
||||
*/
|
||||
const GERMAN_SPEC_LABEL_RE = /\b(?:languages?|sprachen|sprache)\s*[::]/i;
|
||||
|
||||
/** Title-level marker — no description fetch needed. */
|
||||
export const GERMAN_TITLE_RE =
|
||||
@@ -175,19 +184,35 @@ export const GERMAN_TITLE_RE =
|
||||
export function requiresGerman(text: string): boolean {
|
||||
const flat = text.replace(/\s+/g, " ");
|
||||
const optional = new RegExp(GERMAN_OPTIONAL_RE.source, "gi");
|
||||
const required = new RegExp(GERMAN_REQUIRED_RE.source, "gi");
|
||||
const required = new RegExp(
|
||||
`${GERMAN_REQUIRED_RE.source}|${GERMAN_SPEC_LABEL_RE.source}`,
|
||||
"gi"
|
||||
);
|
||||
|
||||
// A marker binds to the German mention only if no OTHER language name sits
|
||||
// between them: in "Fluent in English; German ... advantageous", "Fluent"
|
||||
// qualifies English, not German.
|
||||
const OTHER_LANG = /\b(english|french|italian|spanish|portuguese|anglais|français|englisch|franz|italienisch)\b/i;
|
||||
|
||||
// …except when that other language is *coordinated* with German, where the
|
||||
// marker governs both: "Excellent communication skills in both English and
|
||||
// German" (Karakun) and "Fluency in English and German" (J. Safra Sarasin)
|
||||
// are hard requirements that the bare rule above waved through — the entire
|
||||
// reason 24 basel-jobs.ch postings mentioning German were caught 0 times.
|
||||
// Deliberately excludes "and/or" and a bare "or": "Very good English and/or
|
||||
// German" (BBF) is satisfied by English alone, so it must stay optional.
|
||||
const CONJOINED = /\b(?:and|und|sowie|as\s+well\s+as|&)\s*$/i;
|
||||
const ANDOR = /\b(?:and\s*\/\s*or|or)\s*$/i;
|
||||
|
||||
const nearest = (re: RegExp, window: string, at: number, bind: boolean): number => {
|
||||
let best = Infinity;
|
||||
for (const m of window.matchAll(re)) {
|
||||
const i = m.index ?? 0;
|
||||
const between = i < at ? window.slice(i + m[0].length, at) : window.slice(at, i);
|
||||
if (bind && OTHER_LANG.test(between)) continue; // belongs to a different language
|
||||
// "…English and German" keeps the marker; "…English; German" or
|
||||
// "…English and/or German" hands it to the other language.
|
||||
const conjoined = CONJOINED.test(between) && !ANDOR.test(between);
|
||||
if (bind && OTHER_LANG.test(between) && !conjoined) continue;
|
||||
const d = Math.abs(i - at);
|
||||
if (d < best) best = d;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user