feat: add historical backfill with --init CLI and episode numbering
Adds a --init mode that seeds the database with past shows from a given anchor episode/date forward, batch-fetching likes from SoundCloud and partitioning them into weekly buckets. Episode numbers are tracked in the shows table and auto-incremented by the poller for new shows. Includes full API documentation (docs/api.md) and updated README. Made-with: Cursor
This commit is contained in:
39
chat-summaries/2026-03-12_00-00-summary.md
Normal file
39
chat-summaries/2026-03-12_00-00-summary.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# NtR SoundCloud Fetcher — Full Implementation
|
||||
|
||||
## Task Description
|
||||
|
||||
Designed and implemented a Python service that polls NicktheRat's SoundCloud likes, builds weekly playlists aligned to the Wednesday 22:00 ET show schedule, and serves them via a JSON API for an IRC bot.
|
||||
|
||||
## Changes Made
|
||||
|
||||
### Design Phase
|
||||
- Brainstormed requirements through 6 clarifying questions
|
||||
- Evaluated 3 architectural approaches, selected single-process daemon
|
||||
- Produced design doc covering architecture, data model, API, poller logic
|
||||
- Produced 13-task TDD implementation plan
|
||||
|
||||
### Implementation (42 tests, all passing, lint clean)
|
||||
|
||||
| Module | File | Purpose |
|
||||
|--------|------|---------|
|
||||
| Config | `src/ntr_fetcher/config.py` | Pydantic settings with `NTR_` env prefix |
|
||||
| Week | `src/ntr_fetcher/week.py` | DST-aware Wednesday 22:00 ET boundary computation |
|
||||
| Models | `src/ntr_fetcher/models.py` | Track, Show, ShowTrack dataclasses |
|
||||
| Database | `src/ntr_fetcher/db.py` | SQLite schema, CRUD, track sync with unlike removal |
|
||||
| SoundCloud | `src/ntr_fetcher/soundcloud.py` | client_id extraction, user resolution, likes fetching |
|
||||
| Poller | `src/ntr_fetcher/poller.py` | Hourly polling with supervised restart |
|
||||
| API | `src/ntr_fetcher/api.py` | FastAPI routes for playlist, shows, admin, health |
|
||||
| Main | `src/ntr_fetcher/main.py` | Entry point wiring everything together |
|
||||
|
||||
### Key Design Decisions
|
||||
- Tracks removed when Nick unlikes them (positions re-compact)
|
||||
- Cursor-seeking for efficient SoundCloud API pagination
|
||||
- Automatic client_id rotation on 401
|
||||
- Supervisor restarts poller on failure without affecting API
|
||||
|
||||
## Follow-up Items
|
||||
|
||||
- **Incremental fetching**: Currently fetches full week every poll; could optimize to stop at known tracks
|
||||
- **Retry/backoff for non-401 errors**: 429, 5xx, timeouts not yet handled with retries
|
||||
- **`full` parameter**: Accepted but currently equivalent to normal poll (no incremental to differentiate from)
|
||||
- **`soundcloud_url` in admin add track**: Removed from API; only `track_id` supported
|
||||
Reference in New Issue
Block a user