Commit Graph
17 Commits
Author SHA1 Message Date
bot-ci dcf37c4a1d test(scheduler): nail H7 progress keep + H8 deferred task:state
Address Michał NEED-CHANGES on !7 tip f408da93: fail/defer omit-progress
preserves progress/stat_date; seed + quiet-window defer assert
status=deferred and future due_at on task:state. Soft: seeded due≠wall;
cleanup failure reads task:state.
2026-11-03 01:59:18 +08:00
bot-ci f408da93c2 feat(scheduler): persist task state/progress/due_at across restart (#8)
Replace process-start next_run=0 guesses with kv task:state records
(status, due_at, stat_date, progress). Startup recovers stale running,
honors future due_at for delayed retries, and resumes the same
stat_date. Seed only enqueues evening seal; seal force keeps non-seed
ok=1 as progress (keep_scraped). Keeps !7 fail-logging and cleanup retry.
2026-11-03 01:50:00 +08:00
bot-ci 51ffa44577 fix(scheduler): do not force full snapshot on post-seed restart
Schedule snapshot_seed/snapshot/fans to the next wall-clock seed_at /
snapshot_at on process start (optionally from last task record), instead
of next_run=0. Force-seal of seed shells only after today's snapshot_at
(or CLI --force), so a daytime restart cannot scrape the whole site.
2026-11-03 01:09:50 +08:00
bot-ci 1697573d95 fix(scheduler): address !7 REQUEST_CHANGES
- seed: check completed_key before no_source so same-day last_completed
  is already_completed (not 10m retry)
- scheduler: resolve TaskControl ok/error before logging; ok=False logs
  task_fail, not task_ok
- candidate_cleanup: catch failures, record task, retry ~5m like discover
2026-11-03 01:08:34 +08:00
bot-ci bb18d232f6 feat(scheduler): add snapshot-seed CLI and 10m seed deferral
Add `python app.py snapshot-seed [--date]` (offline clone, Debian /
docker compose exec friendly). When seed fires while full-site snapshot
is running, or last_completed is missing (no_source), record skip as
non-success and retry in 10 minutes until a real seed succeeds; fans
and quiet-window rules unchanged.
2026-11-03 00:22:11 +08:00
kairee c5b7185675 feat(growth): same-day 1-day baselines for new videos
Let 1-day view growth use a zero baseline for videos published today, and the first successful sample for older videos ingested today, without mixing those into 7/30-day rankings.
2026-11-02 20:44:17 +08:00
kairee cdd11823b9 feat(stats): add migration v7 visit table and client-ip helpers 2026-11-01 15:29:43 +08:00
kairee 2ff3e7952b feat(web): up avatars, follower count, and dedupe views/avg secondary 2026-11-01 14:34:23 +08:00
kairee 6c4287b43a refactor(sync): drop favorite-folder tracking and deal 2026-11-01 13:59:10 +08:00
kairee 519fc39fca feat(up): capture follower snapshots at ingest and backfill history 2026-11-01 11:18:06 +08:00
kairee 257fbe009b fix(web): align trend ranges, next runs and preview task records 2026-10-31 23:46:49 +08:00
kairee 19572171b8 feat(web): add read-only dashboard server and query layer 2026-10-31 23:13:54 +08:00
kairee d971e718e9 feat(p3): add discovery, telegram notifications, and one-click favorites
Implement WBI video search with legacy fallback, classification scoring, candidate pool, permanent seen-aid dedupe, blacklist filtering, and batched Telegram notifications. Add write client for fav deal/sort, target-folder selection, insertion position calculation, read-back verification, fav_action locking, retry on failure, ignore and blacklist actions. Wire discovery, candidate cleanup, Telegram long polling, and new discover/candidates CLI commands into the scheduler. Add migration v3 for candidate duration and index stats.
2026-10-31 21:23:12 +08:00
kairee 6a21b5d905 fix: close data integrity and hot-reload review findings
Backfill same-day snapshots for newly added or restored videos, preserve successful fan snapshots on resume, persist incomplete state before forced re-runs, and split collection completion from backup completion so failed backups can retry without API calls. Reject missing data payloads instead of treating them as empty folders, guard against mass-unfav on suspicious empty remote folders, rebuild Bilibili clients when credentials change, recompute primary folder ownership from active relations with per-folder fav_time, preserve original titles for dropped invalid videos, poll config every 60 seconds, and fix favsort verify interval and mixed resource types.
2026-10-31 20:58:36 +08:00
kairee 178ab64b18 feat(auth): separate read and write bilibili accounts 2026-10-31 20:01:21 +08:00
kairee 5c2ab7b187 feat(p1): add daily snapshots and fan tracking
Implement P1: seven-metric view snapshots, relation/stat fan snapshots, failure placeholders, resume by daily completion markers, Cookie login warnings, daily SQLite backups, Shanghai daily scheduling, and CLI snapshot/auth-check commands. Real read-only run captured 795/795 videos successfully over 1124s with a verified daily backup.
2026-10-31 19:19:24 +08:00
kairee e2cdf997af feat(p0): add favorite folder sync and sqlite data foundation
Implement P0 of the reaction dashboard: config loading with hot reload, SQLite migrations with backups, single-writer queue, read-only Bilibili fav client, folder import/diff sync, invalid-resource handling, blacklist seed, scheduler, and JSON redacted logging. Includes offline unit and transport-level contract tests; all existing and new tests pass.
2026-10-31 18:08:33 +08:00