loading page…
A Next.js web app for managing website scraping jobs and RSS feed generation. Authenticated users view configured sites, trigger scrapes, preview feed items, and copy generated RSS URLs. The UI is a Backend-for-Frontend (BFF): browser calls same-origin API routes; Next.js forwards to an Express API with JWT from HTTP-only cookies — never localStorage.
Built for PEI Obituaries — Prince Edward Island news and obituary sources — with an architecture that supports any sites configured in the backend.
“Ali is exceptional to work with. Prompt, very knowledgeable and engaged. Clear communications. Works until an issue is resolved. Could not ask for a better partner.”
System architecture
BFF keeps JWT off the client and out of localStorage
React dashboard
→POST /api/auth/login
←Set-Cookie: auth-token
BFF + middleware
→GET /api/sites · POST scrape
←{ sites } · { rssUrl }
Scrape + RSS
React dashboard
→POST /api/auth/login
←Set-Cookie: auth-token
BFF + middleware
→GET /api/sites · POST scrape
←{ sites } · { rssUrl }
Scrape + RSS
Session
auth-token · 7d · httpOnly
BFF surface
/api/auth/* · /api/sites/*
Output
RSS feeds · XML on disk
Email/password login; session stored in an httpOnly auth-token cookie (7 days). Client state via AuthContext and /api/auth/verify.
Middleware gates every page except /login. Unauthenticated users redirect before any site data loads.
List backend-configured scraping targets with status, metadata, and last-scraped timestamps.
Trigger per-site jobs from the UI and receive updated RSS feed URLs when the backend finishes.
Expandable detail panels load recent items — title, link, date, description — without leaving the dashboard.
robots.txt disallows crawlers; global X-Robots-Tag: noindex, nofollow; middleware blocks common bot user-agents.
App Router pages for auth and operations; API route handlers proxy to the backend with Authorization: Bearer <token> read from the cookie.
| Route | Access |
|---|---|
| / | ProtectedLanding dashboard — PEI Obituaries branding, links to Sites |
| /login | PublicEmail/password sign-in |
| /sites | ProtectedSite grid — scrape, RSS URL, feed item preview |
| /change-password | ProtectedUpdate password with confirmation |
Sample RSS output
Feeds are generated by the backend after a successful scrape; the dashboard surfaces URLs and in-app previews.
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>PEI Obituaries — Example source</title>
<link>https://example.com/obituaries</link>
<description>Recent items from a configured scrape target</description>
<item>
<title>Community remembers local educator</title>
<link>https://example.com/obituaries/educator-may-2025</link>
<pubDate>Tue, 27 May 2025 10:00:00 GMT</pubDate>
<description>Summary text from the scraped listing page…</description>
</item>
<item>
<title>Island family announces memorial service</title>
<link>https://example.com/obituaries/memorial-may-2025</link>
<pubDate>Mon, 26 May 2025 14:30:00 GMT</pubDate>
</item>
</channel>
</rss>This is a private production project — no public repository. Happy to walk through the BFF pattern, auth flow, and dashboard UX on a call.
Discuss this project