Sites that put a description in the title produce output names past what
the filesystem accepts, and the download fails outright with
"[Errno 36] File name too long". The name is now shortened to fit inside
prepare_filename, which every output path already passes through, so the
main file, its chapter files, thumbnails and subtitles stay consistent.
The limit is read from the filesystem (PC_NAME_MAX, falling back to 255)
and counted in bytes, not characters: a title of accented or CJK
characters reaches it in a third of the characters. The extension is
kept, a cut landing inside a multi-byte character does not leave a broken
sequence, and a reserve is held back for the suffixes yt-dlp appends
afterwards -- '.part', '.ytdl', '.f<format_id>', '-Frag<n>' -- since it
is those that pushed the reported name over the limit.
Names that already fit are untouched, so nothing that downloads today
changes name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds __record_add_failure so a URL that fails before a real download starts
(unsupported/unextractable URL, SSRF-rejected, extraction error) appears in the
Completed list as a red-cross entry with retry and error-detail, instead of only
a transient toast and a server log line. Keyed by info.url like any errored
download. Includes _short_title_for_failed_url for a readable hostname title.
The frontend hunk removing the 'Click for details' hint from every error row was
dropped from this merge; that affordance (added in fd3aaea, #143) is kept.
Co-authored-by: streamer1122 <streamer1122@users.noreply.github.com>
Adds MusicMetadataPreProcessor (app/music_metadata.py), a pre_process
postprocessor that enriches the info dict using only extractor-owned fields:
track-number/total resolution, album-title fallback, and square-thumbnail
preference. Track numbers and album flow into files through yt-dlp's existing
FFmpegMetadata embedding, so no per-format tag-writing code and no new
dependency. The earlier mutagen writer and its download-failing
PostProcessingError path were dropped per review.
Co-authored-by: jahruz67 <jahruz67@users.noreply.github.com>
The per-download chapter_template was validated only against literal ".."
in the template string, but yt-dlp expands %(section_title)s (and every
other field) from attacker-controlled metadata at download time. On POSIX
hosts yt-dlp does not neutralise a ".." path component, so a chapter titled
".." turns a guard-passing template like
"%(section_title)s/%(section_title)s/x.%(ext)s" into "../../x.mp4" and writes
outside DOWNLOAD_DIR. The same class of escape applies to any multi-segment
output template (default/playlist/channel) whose fields resolve to "..".
The template string can never see the "..": it only exists after expansion.
So move the check to the one point every output path flows through —
YoutubeDL.prepare_filename — via a _ConfinedYoutubeDL subclass that refuses
any resolved path outside the download/temp roots (fail closed). This covers
the main file, split-chapter files, thumbnails and subtitles in one place.
With the chokepoint authoritative, the scattered ingress string checks
(chapter_template, custom_name_prefix) and the weaker _output_dir_escapes
literal-prefix check are removed. Tests move from the ingress layer to the
chokepoint, exercising the real metadata-resolution vector.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sanitize path separators and .. segments in playlist/channel titles before they are baked into yt-dlp output templates, and refuse downloads whose resolved output directory escapes DOWNLOAD_DIR.
Addresses a full-project review. Backend correctness and availability:
- ytdl: cancel() only SIGKILLs the child's process group when the child
actually became its own group leader, so a race (or failed setpgrp)
can no longer kill the whole server; kill the group on cancel and on
shutdown to avoid orphaned ffmpeg children
- ytdl: dedicated ThreadPoolExecutor for download supervision so active
downloads can't starve extract_info / live probes on the default pool
- ytdl/main/subscriptions: route fire-and-forget tasks through a
bg_tasks helper that keeps a strong ref and logs failures
- subscriptions: run flat-playlist extraction in an executor and check
feeds with bounded concurrency so one slow feed can't block the loop;
set last_checked on failure so broken feeds aren't retried every 60s
- main: validate ids on /start & /delete and numeric env vars at startup;
return 400 (not 500) on bad subscriptions/update input; serve /history
from memory; move get_custom_dirs off the event loop; restrict t=
stripping to YouTube hosts; drop double percent-decode in state guard
- dl_formats/ytdl: enforce requested caption format via
FFmpegSubtitlesConvertor and strip VTT header metadata only in the
pre-cue region so real dialogue is preserved
- ytdl: throttle progress events, dedup adds against pending, clear
filename/size on error and reject out-of-dir trashcan deletes, pin
fork start-method on Linux only
Frontend:
- retry deletes the done record only after a successful re-add
- surface HTTP errors for delete/start and reset the deleting flag
- ignore late 'updated' events for rows no longer in the queue
- track table rows by map key; FileSizePipe uses base-1024
Also: HTTPS-aware Docker healthcheck, dead-code removal, and shared
helpers for path-containment and yt-dlp option merging. Adds/updates
unit tests throughout (250 backend tests passing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The playlist/channel processing loop now sets playlist_count,
playlist_autonumber, n_entries, and __last_playlist_index on each
video entry so that templates like %(playlist_autonumber)s,
%(playlist_count)s, and %(playlist_index&{} - |)s resolve correctly
instead of showing NA.
Also updates _compact_persisted_entry to preserve n_entries and
__last_playlist_index across restarts.
Fixes#692
Agent-Logs-Url: https://github.com/alexta69/metube/sessions/b5aeb55a-3197-4a14-b8b4-96c9a67796e8
Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
Replace the hand-rolled _outtmpl_substitute_field() / _compile_outtmpl_pattern()
with a new _resolve_outtmpl_fields() that delegates to yt-dlp's
YoutubeDL.evaluate_outtmpl(). This gives playlist/channel output templates
access to yt-dlp's full template syntax: defaults (%(field|fallback)s),
conditional formatting (%(field&prefix {})s), math (%(field+N)d),
datetime formatting (%(field>%Y-%m-%d)s), and more.
Only field references whose root name matches the targeted prefix (e.g.
"playlist" or "channel") are resolved; all other references remain as
template placeholders for yt-dlp to fill during the actual download.
Agent-Logs-Url: https://github.com/alexta69/metube/sessions/0ae5ff34-540f-4fc8-a81c-358fb92b7c15
Co-authored-by: alexta69 <7450369+alexta69@users.noreply.github.com>
tjelite1986