A Hybrid Ranking System for Job Feeds

2026-05-02
, , , , , , ,

Introduction

LinkedIn’s boolean queries surface more signal than free text, but recall isn’t deterministic. Two constraints shaped jobpipe’s design: filtering written as code is frozen at compile time, and relevance is not static. The system needed a query layer changeable without recompiling and a scoring layer that learned from context.

jobpipe evolved from a simple crawler into a hybrid ranking system: a custom DSL with an AST evaluator, NLP-based feature extraction aligned to a skill ontology, and a calibrated learning loop.

The key architectural shift was making Rust own the deterministic feature authority layer, while Python owns probabilistic ranking. The boundary is explicit.

crawl -> parse -> tokenize -> concepts -> weak labels -> filter -> score -> DSL -> output

The DSL: Parser, AST, Evaluator

The DSL is a prefix, fully parenthesised predicate language restricted to a fixed operator set (and, or, not, contains, remote, score).

Parsing and evaluation are separate passes. The parser produces an untyped Expr AST. The evaluator interprets it against a typed Job (eval : Expr -> Job -> Bool). Invalid field references collapse to false at evaluation – keeping the DSL permissive at the query surface while the data model remains strict.

(and
  (contains text "rust")
  (not (contains text "wordpress"))
  (score > 50))

Rust Abstractions: Feature Authority

Early versions had hidden mutation, clone-heavy synchronisation, and feature drift between scoring and training. Ownership-oriented stage separation clarified which phases read, mutate, and export:

This removed implicit shared state between the DSL scoring layer and the feature extractor. Train/inference feature consistency became a structural guarantee. The ontology layer maps symbolic concepts (rust, kubernetes) into stable feature IDs consumed by both the DSL evaluator and train.py, ensuring the sparse vector produced at crawl time matches the one produced at train time.

Feature Extraction: Text to Concept Vectors

The feature layer converts unstructured job descriptions into structured signals by aligning tokens against profile.json – a hand-maintained ontology of skills and synonyms.

A job “Rust Kubernetes Engineer” becomes \(x = [1, 1, 1, 0, \ldots]\). Ownership of feature semantics lives in Rust; the DSL and ML layer share a single ground truth.

The naive implementation had hidden \(O(n^2)\) patterns (nested scans for synonyms). Fixing this with a HashMap<Token, Concepts> index dropped the total pipeline from \(O(n^2)\) to \(O(n)\).

The Learning Loop and Embedding Space

Concept vectors and labelled examples (want = 1 or 0 in SQLite) feed a logistic regression model. MultiLabelBinarizer converts concept lists into binary presence vectors.

Sentence-transformer embeddings project jobs into geometric vector space (\(x_i \in \mathbb{R}^{d}\)). The ranking model learns \(P(y=1 \mid x) = \sigma(w^T x + b)\), inferring preference structure from feedback history rather than manual rules. The loop is:

crawl -> extract -> score -> rank -> label uncertain results -> retrain

The Manifold Hypothesis in Practice

High-dimensional real-world data tends to lie on lower-dimensional structured surfaces. Although embeddings exist in \(\mathbb{R}^{768}\), meaningful data occupies a smaller manifold \(M\).

After the Rust abstraction cleanup stabilised ontology extraction, observed prediction statistics converged strongly (mean prediction 0.88 for positives, 0.11 for negatives). The positives formed strong semantic neighbourhoods around titles like “Platform Engineer” and “Deployment Infrastructure” – clustering on a locally coherent semantic surface despite differing surface forms. Stable features produce stable embeddings; stable embeddings produce a stable manifold.

Operational Semantics Observability: Validating Refactors

Maintaining strict feature consistency across a Rust DSL evaluator and a Python ML pipeline during refactors is difficult. Unit tests only verify final outputs; if a refactor causes a silent logic divergence in intermediate AST evaluation or feature vector assembly, tests might pass, but the model’s inputs are corrupted.

To solve this, jobpipe acts as the primary testbed for trace-align, a research-level operational semantics observability tool.

Instead of diffing source code, trace-align diffs execution. By instrumenting jobpipe’s Rust pipeline to emit JSONL execution traces (snapshots of AST nodes, feature vectors, and loop states), trace-align aligns the operational histories of the old and new code. It uses type-driven similarity scoring to pinpoint the exact iteration where a refactor broke the logic.

This elevates the CI/CD gate from “Did the test pass?” to “Did the refactor preserve the exact operational semantics?” It ensures that shifting from a recursive AST evaluator to a compiled bytecode loop – or migrating feature extraction logic – does not silently alter the model’s inputs.

Concurrency and the Rust Memory Model

Feed-level parallelism spawns an async task per source, while request-level throttling uses a semaphore to bound simultaneous HTTP connections. Arc provides shared ownership across tasks without deep copying. Performance is controlled IO throughput, avoiding retry storms and smoothing network load against rate-limiting servers.

What This System Actually Is

It is a feedback-driven ranking system where the DSL handles known constraints, and the learned model handles discovered preferences. The architecture combines symbolic reasoning (DSL, ontology, deterministic scoring) with vector semantics (embeddings, probabilistic ranking).

The shift is from

string -> features -> classifier

to

symbolic ontology -> structured feature graph -> vector space projection -> calibrated ranker

Bias is explicit in the Rust scoring layer, and overfitting risk is reduced by separation of concerns, verified by operational semantics diffing.

Webmentions

Leave a comment

Comments are verified via IndieAuth. You will be redirected to authenticate before your comment is published.