Scheduled keyword checks that separate ranking changes from failed measurements.
This SEO reporting tool case study covers a tracker that records where a domain shows up for the keywords an operator picks. It keeps a daily history of each one. The country, the device and an optional location stay with every keyword. A position is always tied to the search conditions used to measure it.
Users can group those keywords, open the page that each one matched and also pull in search performance data. Clicks and impressions then sit next to the ranking history.
The challenge
A missing result does not always mean a ranking has dropped. A provider request can fail, return partial data or miss the page range where the domain sits. Recording each one as a lost ranking would mislead the history.
Searching more result pages can cost more requests and still return nothing useful. We needed page-selection options and a retry path. Together they keep a failed check apart from a real ranking result.
What we built
The tracker groups keywords by domain. It supports tags, pinned items and detail views, and a CSV export is available too. Users open the latest result list and its matched page.
Each calendar day gets one stored position per tracked keyword. A later successful check on the same day replaces that entry, which keeps the product focused on day-to-day comparison.
Scheduled checks run through configured search-data providers or a proxy adapter. Domain settings can fix a page range or follow a strategy built around the last known position. Email digests and an API cover reporting outside the main interface. An optional search performance integration adds clicks and impressions. Its matching uses the keyword, the country and the device together as one key. It does not treat every variant of a term as the same observation.
Position-aware page selection
A basic check asks for the first page, while a custom check covers a set number of pages. The adaptive strategy is the interesting one. It starts near the keyword's previous position, looks at the neighbouring pages and can widen the search when the domain does not turn up.
The adapter recalculates each position from the page offset. That keeps a stored position consistent even when a provider returns page-relative numbering.
Failures stay separate from ranking history
A scraping result returns an error state as well as whatever position it managed to find. When the domain is absent and too many attempted pages fail, the refresh counts as an unsuccessful measurement rather than a result.
Failed keywords move to a retry list for a later run. A successful refresh updates the keyword's dated position map. A provider failure therefore stays a problem with the measurement, and the history does not log it as a ranking loss.
A compact scheduled runtime
A scheduler calls the application's API, and the web process runs the checks. SQLite stores the domains and keywords. Settings and retry data sit in local files. This avoids a separate queue service, though it does tie running work to the application staying available.
Provider adapters define the request details and the result extraction. Some adapters can send parallel requests. Others run one request at a time with an optional delay between them. Scheduling and batch size then have to match the chosen provider and the environment it runs in.
Project success
A daily comparison keeps the keyword's country, device and location context. Users can open the matching result, review the performance data and export or receive summaries of the recorded positions.
Failed measurements stay identifiable and can be retried. Page-selection settings let the operator choose how far each check searches, including whether to start from the previous position. The application serves this internal tracking workflow through a single configured login.
Technology stack
Application
TypeScript, Next.js, React
Persistence
SQLite, Sequelize, local settings files
Scheduling
Croner
Search and reporting
Provider adapters, Search Console API
Parsing and delivery
Cheerio, Nodemailer
Other engagements
Recognize the pattern?
If one of these looks like your situation, the discovery call is where we find out how close.
