fix: systematic sweep of NON_ENGLISH_RE gaps in gender/compound notation

Audited all 2949 titles in jobs.csv against broader German-signal
heuristics than the regex itself uses, to find what it still misses:

- gender suffix generalized beyond slash/colon to also catch */_ and
  the adjectival -r form (Engineer*in, Mitarbeiter_in, Verantwortliche:r)
- 'und'/'oder' were missing entirely — basic conjunctions, zero
  English overlap
- occupational nouns (Mitarbeiter, Ingenieur, Berater, Leiter,
  Informatiker, Verantwortliche) now also match as compound suffixes
  (Aussendienstmitarbeiter, Vertriebsingenieur) instead of only
  standalone, since German compounds have no separator
- Berater, Leiter(in), Informatiker, Verantwortliche, unbefristet/
  befristet added — words that weren't on the list at all

Verified: 72 titles in jobs.csv history now correctly flag as
non-English that didn't before, all genuinely German on inspection;
re-ran the broad-heuristic audit after the fix and every remaining
candidate was a false positive from the heuristic itself (ambiguous
'Team', an English/German loanword).
This commit is contained in:
2026-09-04 13:35:27 +02:00
parent db396d0a45
commit afddbe496d
+31 -14
View File
@@ -75,21 +75,38 @@ export const NON_ENGLISH_RE = new RegExp(
String.raw`\s\(a\)(\s*[\d%]|\s*$)`, String.raw`\s\(a\)(\s*[\d%]|\s*$)`,
// German/Swiss umlauts and ß // German/Swiss umlauts and ß
String.raw`[äöüÄÖÜß]`, String.raw`[äöüÄÖÜß]`,
// /in word-ending (e.g. Mitarbeiter/in) — excludes m/f, n/a, etc. Also // Gender-neutral suffix, in all four notations Swiss postings use
// catches the gender-colon variant (Entwickler:in) — ZHAW's "Senior Full // (Mitarbeiter/in, Entwickler:in, Engineer*in, Mitarbeiter_in) — excludes
// Stack Entwickler:in 80-100 %" (via job-room.ch) used ":in" and slipped // m/f, n/a, etc. Also covers the adjectival-noun form, which appends "r"
// past both this rule (slash-only) and the word list ("Entwickler" wasn't // instead of "in" (Verantwortliche:r, Angestellte:r, Studierende:r) since
// on it either). // the base word already ends in "-e". ZHAW's "Senior Full Stack
String.raw`[a-zäöü][\/:]in\b`, // Entwickler:in 80-100 %" used ":in" and slipped past both this rule
// (slash-only, at the time) and the word list ("Entwickler" wasn't on it
// either); the audit below found the *in/_in/:r variants were equally
// unhandled (OT-System Engineer*in, Applikationsverantwortliche:r).
String.raw`[a-zäöü][\/:*_](in|r)\b`,
// German-only words (case-insensitive handled by flag) // German-only words (case-insensitive handled by flag)
String.raw`\b(Deine|Mitarbeiter|Pflegehelfer|Köchin|Koch EFZ|Kellner|Sozialpädagog|Lehrperson|Produktionsmitarbeiter|Produktionsplaner|Geschäftsführer|Spontanbewerbung|Arztsekretär|Physiotherapeut|Empfangsmitarbeiter|Automobilmechatroniker|Verkäufer|Projektleiter|Crewmitarbeiter|Crewmember|Systemgastronomie|Studienteilnehmende|Teilnehmende|Bauingenieur|Tiefbau|Hochbau|Elektroingenieur|Maschinenbau|gesucht|EFZ|Schweiz|Filiale|Ingenieur|Spezialist|Perspektive|Stellvertretung|Leitung|Beratung|Entwicklung|Betreuung|Verwaltung)\b`, String.raw`\b(Deine|Mitarbeiter|Pflegehelfer|Köchin|Koch EFZ|Kellner|Sozialpädagog|Lehrperson|Produktionsmitarbeiter|Produktionsplaner|Geschäftsführer|Spontanbewerbung|Arztsekretär|Physiotherapeut|Empfangsmitarbeiter|Automobilmechatroniker|Verkäufer|Projektleiter|Crewmitarbeiter|Crewmember|Systemgastronomie|Studienteilnehmende|Teilnehmende|Bauingenieur|Tiefbau|Hochbau|Elektroingenieur|Maschinenbau|gesucht|EFZ|Schweiz|Filiale|Ingenieur|Spezialist|Perspektive|Stellvertretung|Leitung|Beratung|Entwicklung|Betreuung|Verwaltung|unbefristet|befristet)\b`,
// German prepositions/articles in title context (must appear as whole words) — // German occupational suffixes — matched with NO leading \b, deliberately,
// none of these exist as English words, so they're safe unambiguous markers. // because German compounds glue words together with no separator
// Added after "Software Engineer auf dem Weg zum Data / AI Engineer" (ti&m, // (Aussendienstmitarbeiter, Vertriebsingenieur, Applikationsverantwortliche)
// via job-room.ch) sailed through: none of the German-word list above // and a leading \b would only ever match the bare word. Found by auditing
// matched, and the job-room description was too short (~1 hit) to clear the // jobs.csv for titles containing these strings that the rest of the regex
// separate ≥6 DE/FR word-count threshold either. // still let through: "Software Berater Fokus Lohn/HR in Bern",
String.raw`\bmit\b|\bauf\b|\bzum\b|\bzur\b|\bvom\b|\bfür\b|\bbeim\b`, // "Leiter Business Applikationen, CRM & Analytics", "Leiterin Trading
// Data, Integration & Reporting", "ICT System Engineer / Informatiker
// Systemtechnik". All are German-only strings with no English overlap.
String.raw`(mitarbeiter|ingenieur|berater|leiterin|leiter|informatiker|verantwortliche)\b`,
// German prepositions/articles/conjunctions in title context (must appear
// as whole words) — none of these exist as English words, so they're safe
// unambiguous markers. Added after "Software Engineer auf dem Weg zum
// Data / AI Engineer" (ti&m, via job-room.ch) sailed through: none of the
// German-word list above matched, and the job-room description was too
// short (~1 hit) to clear the separate ≥6 DE/FR word-count threshold
// either. "und"/"oder" added after the same audit found "System Engineer
// Microsoft und OnPrem Infrastruktur" slipping through — even the most
// basic German conjunctions weren't covered.
String.raw`\bmit\b|\bauf\b|\bzum\b|\bzur\b|\bvom\b|\bfür\b|\bbeim\b|\bund\b|\boder\b`,
// French gender markers // French gender markers
String.raw`\(H\/F\)|\(F\/H\)|\(h\/f\)|\(f\/h\)`, String.raw`\(H\/F\)|\(F\/H\)|\(h\/f\)|\(f\/h\)`,
// French-only accented characters // French-only accented characters