Ranking New Papers By Meaning Instead Of Keywords, In One PHP App

Leader ●4 ●53 ●178
calendar_today • schedule2 min read

Most of us have built some version of a scheduled job that watches a feed and mails out a summary, and it almost never fails on the plumbing. It fails on relevance, and this is a writeup of a small PHP and SQLite project that puts a model scoring step where the keyword filter usually goes.

Why Keyword Alerts Keep Missing The Right Paper

Every major index offers alerts, and they all match on words. That is the whole problem. A paper that applies your exact method in a neighboring field will share none of your keywords, so it never reaches you. A paper that happens to use your words while solving an unrelated problem arrives every single week.

Word overlap is a proxy for relevance, and it is a bad one the moment your work sits between two subfields. Any watcher you build on top of a keyword query inherits that ceiling no matter how good the rest of the code is.

The Shape Of The Pipeline

Literature Radar splits the job into four steps, and the split is the interesting part.

First, an interest profile. Not a keyword list, a short paragraph written the way you would describe your work to a colleague. Separately you define topic queries that tell the poller what to pull, so retrieval stays broad while ranking stays narrow.

Second, a scheduled poll against arXiv, PubMed, Crossref, bioRxiv and medRxiv, writing into one local SQLite file. Anything already seen is skipped, so the table doubles as the dedupe layer.

Third, the ranking pass. Your model reads each new title and abstract and scores it 0 to 10 against the profile, attaching a one line reason. This is where a paper about your method in another field can outscore a paper that matched your keywords exactly.

Fourth, the digest. One email on your schedule, best first, score and reason attached, nothing repeated across sends.

Choosing Where The Ranking Runs

The ranking backend is resolved in order: the Claude command line tool, the Codex command line tool, an Anthropic API key, then any OpenAI compatible endpoint, which covers OpenAI, OpenRouter, and local servers like Ollama or LM Studio.

That order matters for cost. If you already pay for a Claude or ChatGPT subscription, the CLI path means the ranking step adds no per token cost at all, which is what makes a daily job over a few hundred abstracts practical for one person. Falling back to a local model through an OpenAI compatible endpoint keeps abstracts on your own machine entirely.

The whole thing is one PHP app plus one SQLite file, driven by a single cron entry that decides internally whether a poll, a rank, or a digest is due:

0 */2 * * * php /path/to/Literature-Radar/cli/cron.php

Each step also runs standalone, which is what you want while tuning the profile.

What Transfers To Your Own Watcher

Three things generalize past papers. Keep retrieval broad and put the judgment in a separate ranking pass, because a narrow query loses items you can never recover later. Store the reason next to the score, since a ranked list with no explanation gives you nothing to tune against. And make the scoring backend swappable from day one, because the model you start on will not be the one you finish on.

It is MIT licensed with no account and no service in the middle, so the abstracts go to whichever model you picked and nowhere else.

🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

A Textile Mill's Dyehouse Ramp Curve: Get One Step Wrong and the Sheet Belongs to the Master

aiomniu - Oct 18

Dozens of email accounts: learning AI coding tools as a student

rahuldk1105 - Oct 27

One PHP App and One SQLite File: A Self Hosted Workflow Engine

AIAppsAPI - Aug 14

Cloudflare Blocked My Server, So I Made My Visitors' Browsers Do the Fetching

bryanhamiltondevverified - Nov 15

A Message From Our Founder: On the Winding Down of LaaSy™

Ken W. Algerverified - Oct 29
chevron_left
5.3k Points • 235 Badges
United States • t.co/5LlztlB5C5
118Posts
23Comments
46Connections
Our AI Apps are a self expanding AI SaaS ecosystem used to create the custom web application of your... Show more

Related Jobs

Commenters (This Week)

1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!