loading page…
A Next.js web app for managing website scraping jobs and RSS feed generation. Authenticated users view configured sites, trigger scrapes, preview feed items, and copy generated RSS URLs. The UI is a Backend-for-Frontend (BFF): browser calls same-origin API routes; Next.js forwards to an Express API with JWT from HTTP-only cookies — never localStorage.
Built for PEI Obituaries — Prince Edward Island news and obituary sources — with an architecture that supports any sites configured in the backend.
System architecture
BFF keeps JWT off the client and out of localStorage
React dashboard
→POST /api/auth/login
←Set-Cookie: auth-token
BFF + middleware
→GET /api/sites · POST scrape
←{ sites } · { rssUrl }
Scrape + RSS
React dashboard
→POST /api/auth/login
←Set-Cookie: auth-token
BFF + middleware
→GET /api/sites · POST scrape
←{ sites } · { rssUrl }
Scrape + RSS
Session
auth-token · 7d · httpOnly
BFF surface
/api/auth/* · /api/sites/*
Output
RSS feeds · XML on disk
Email/password login; session stored in an httpOnly auth-token cookie (7 days). Client state via AuthContext and /api/auth/verify.
Middleware gates every page except /login. Unauthenticated users redirect before any site data loads.
List backend-configured scraping targets with status, metadata, and last-scraped timestamps.
Trigger per-site jobs from the UI and receive updated RSS feed URLs when the backend finishes.
Expandable detail panels load recent items — title, link, date, description — without leaving the dashboard.
robots.txt disallows crawlers; global X-Robots-Tag: noindex, nofollow; middleware blocks common bot user-agents.
App Router pages for auth and operations; API route handlers proxy to the backend with Authorization: Bearer <token> read from the cookie.
| Route | Access |
|---|---|
| / | ProtectedLanding dashboard — PEI Obituaries branding, links to Sites |
| /login | PublicEmail/password sign-in |
| /sites | ProtectedSite grid — scrape, RSS URL, feed item preview |
| /change-password | ProtectedUpdate password with confirmation |
Sample RSS output
Feeds are generated by the backend after a successful scrape; the dashboard surfaces URLs and in-app previews.
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>PEI Obituaries — Example source</title>
<link>https://example.com/obituaries</link>
<description>Recent items from a configured scrape target</description>
<item>
<title>Community remembers local educator</title>
<link>https://example.com/obituaries/educator-may-2025</link>
<pubDate>Tue, 27 May 2025 10:00:00 GMT</pubDate>
<description>Summary text from the scraped listing page…</description>
</item>
<item>
<title>Island family announces memorial service</title>
<link>https://example.com/obituaries/memorial-may-2025</link>
<pubDate>Mon, 26 May 2025 14:30:00 GMT</pubDate>
</item>
</channel>
</rss>This is a private production project — no public repository. Happy to walk through the BFF pattern, auth flow, and dashboard UX on a call.
Discuss this project