fix: harden download lifecycle, subscriptions, and UI robustness

Addresses a full-project review. Backend correctness and availability:

- ytdl: cancel() only SIGKILLs the child's process group when the child
  actually became its own group leader, so a race (or failed setpgrp)
  can no longer kill the whole server; kill the group on cancel and on
  shutdown to avoid orphaned ffmpeg children
- ytdl: dedicated ThreadPoolExecutor for download supervision so active
  downloads can't starve extract_info / live probes on the default pool
- ytdl/main/subscriptions: route fire-and-forget tasks through a
  bg_tasks helper that keeps a strong ref and logs failures
- subscriptions: run flat-playlist extraction in an executor and check
  feeds with bounded concurrency so one slow feed can't block the loop;
  set last_checked on failure so broken feeds aren't retried every 60s
- main: validate ids on /start & /delete and numeric env vars at startup;
  return 400 (not 500) on bad subscriptions/update input; serve /history
  from memory; move get_custom_dirs off the event loop; restrict t=
  stripping to YouTube hosts; drop double percent-decode in state guard
- dl_formats/ytdl: enforce requested caption format via
  FFmpegSubtitlesConvertor and strip VTT header metadata only in the
  pre-cue region so real dialogue is preserved
- ytdl: throttle progress events, dedup adds against pending, clear
  filename/size on error and reject out-of-dir trashcan deletes, pin
  fork start-method on Linux only

Frontend:

- retry deletes the done record only after a successful re-add
- surface HTTP errors for delete/start and reset the deleting flag
- ignore late 'updated' events for rows no longer in the queue
- track table rows by map key; FileSizePipe uses base-1024

Also: HTTPS-aware Docker healthcheck, dead-code removal, and shared
helpers for path-containment and yt-dlp option merging. Adds/updates
unit tests throughout (250 backend tests passing).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Alex Shnitman
2026-07-12 08:08:14 +03:00
parent e2c777842e
commit 3ea4732c5d
21 changed files with 1392 additions and 166 deletions
+44 -1
View File
@@ -10,6 +10,7 @@ from app.dl_formats import (
_normalize_subtitle_language,
get_format,
get_opts,
merge_ytdl_option_layers,
)
@@ -118,7 +119,31 @@ class DlFormatsTests(unittest.TestCase):
def test_get_opts_captions_txt_maps_to_srt_format(self):
opts = get_opts("captions", "auto", "txt", "best", {})
self.assertEqual(opts["subtitlesformat"], "srt")
self.assertEqual(opts["subtitlesformat"], "srt/best")
keys = [p["key"] for p in opts["postprocessors"]]
self.assertIn("FFmpegSubtitlesConvertor", keys)
convertor = next(p for p in opts["postprocessors"] if p["key"] == "FFmpegSubtitlesConvertor")
self.assertEqual(convertor["format"], "srt")
def test_get_opts_captions_srt_guarantees_convertor(self):
opts = get_opts("captions", "auto", "srt", "best", {})
self.assertEqual(opts["subtitlesformat"], "srt/best")
keys = [p["key"] for p in opts["postprocessors"]]
self.assertIn("FFmpegSubtitlesConvertor", keys)
def test_get_opts_captions_vtt_guarantees_convertor(self):
opts = get_opts("captions", "auto", "vtt", "best", {})
self.assertEqual(opts["subtitlesformat"], "vtt/best")
keys = [p["key"] for p in opts["postprocessors"]]
self.assertIn("FFmpegSubtitlesConvertor", keys)
convertor = next(p for p in opts["postprocessors"] if p["key"] == "FFmpegSubtitlesConvertor")
self.assertEqual(convertor["format"], "vtt")
def test_get_opts_captions_ttml_has_no_convertor(self):
opts = get_opts("captions", "auto", "ttml", "best", {})
self.assertEqual(opts["subtitlesformat"], "ttml/best")
keys = [p["key"] for p in opts["postprocessors"]]
self.assertNotIn("FFmpegSubtitlesConvertor", keys)
def test_get_opts_merges_existing_postprocessors(self):
opts = get_opts("audio", "auto", "opus", "best", {"postprocessors": [{"key": "SponsorBlock"}]})
@@ -135,5 +160,23 @@ class DlFormatsTests(unittest.TestCase):
self.assertEqual(_normalize_subtitle_language(" "), "en")
class MergeYtdlOptionLayersTests(unittest.TestCase):
def test_presets_applied_in_order_then_overrides(self):
presets_config = {
"a": {"x": 1, "y": 1},
"b": {"y": 2, "z": 2},
}
merged = merge_ytdl_option_layers(["a", "b"], {"z": 3, "w": 4}, presets_config)
# b overrides a's y; explicit overrides win over presets.
self.assertEqual(merged, {"x": 1, "y": 2, "z": 3, "w": 4})
def test_no_base_options_included(self):
# The helper only produces the preset/override layer, never base opts.
self.assertEqual(merge_ytdl_option_layers(None, None, {}), {})
def test_unknown_preset_names_ignored(self):
self.assertEqual(merge_ytdl_option_layers(["missing"], {"a": 1}, {}), {"a": 1})
if __name__ == "__main__":
unittest.main()