Feed Methodology & Deduplication

Harvesting Cadence

Every active feed is polled by an edge cron worker on a 15-minute interval. Requests supply conditional HTTP headers (If-None-Match with ETag, If-Modified-Since) to minimize bandwidth.

Sanitization & Security

All HTML tags inside RSS titles and descriptions are stripped using strict regex filters before rendering. No untrusted third-party scripts or iframes are ever rendered.

Deduplication & Storage

Items are hashed by canonical URL. Identical links across cycles are ignored, ensuring fresh headlines without duplicate noise.