Compare commits

...

29 Commits

Author SHA1 Message Date
Alex Shnitman 0445f5858b docs: put the closing keyword in the commit subject
GitHub matches a closing keyword anywhere in the message, so the previous
"on its own line in the body" rule was invented. Use the parenthesised subject
form instead: "fix: ... (closes #1040)".
2026-07-27 23:13:07 +03:00
Alex Shnitman 6551f7ad58 docs: require Closes #N in commits that resolve an issue
Master is the default branch and is released on every push, so a commit body
that says "Closes #N." makes the issue close exactly when the fix ships and
leaves a permanent link from the issue to the commit — instead of a separate
manual close afterwards. Notes that a bare (#123) in the subject is only a
reference, and that auto-closing still needs an explanatory comment on the
issue.
2026-07-27 23:09:58 +03:00
Alex Shnitman 06c63ec6e5 feat: write playlist/channel metadata files where their items go (#660)
yt-dlp emits the feed-level .info.json, description and thumbnail from
__process_playlist_result without consulting `download`, so they fell out of
MeTube's classification pass with nothing steering them: they landed in
DOWNLOAD_DIR under yt-dlp's own pl_* names, ignoring OUTPUT_TEMPLATE, the
download's folder and AUDIO_DOWNLOAD_DIR. #1040 reported the names; #660 asked
for control over them. Same accident from both sides.

The classification pass now writes nothing at all, and the files are produced
once the feed has been accepted and its type is known, reusing the template its
items use — OUTPUT_TEMPLATE_CHANNEL for a channel, OUTPUT_TEMPLATE_PLAYLIST for
a playlist, falling back to OUTPUT_TEMPLATE — evaluated against the feed dict.
With the defaults that puts them in the same folder as the videos, named after
the feed, which is the Jellyfin layout #660 asked for. No new environment
variable: knowing the feed type is what makes reusing the item template
possible, and doing this after extraction is what makes the type known.

The write re-runs yt-dlp over a copy of the feed with its entries removed,
which reaches the playlist-file writing without re-extracting anything and
without touching yt-dlp's private write helpers. It runs in an executor and
never fails the add.

Nothing new appears for anyone who hasn't enabled writeinfojson /
writethumbnail, an explicit allow_playlist_files=false still turns it off, and
a failed or cancelled add no longer leaves metadata behind.

Subscription scans keep allow_playlist_files=False: a scan is a timer-driven
poll, and items it finds are queued through the download queue, which does the
writing. Previously every check interval rewrote these files.

Test doubles for __extract_info now take *args/**kwargs.
2026-07-27 22:47:15 +03:00
Alex Shnitman d66b04ccf5 feat: let subscriptions be renamed from the list (#1044)
A subscription's name is captured once at subscribe time from the feed's own
title, so playlists — particularly the UULF-prefixed channel-uploads playlists
— all come back named "Videos" and stay that way. Adds an inline editor on the
Name cell, mirroring the existing title-filter edit next to it.

The update route already whitelisted `name` and update_subscription already
applied it, so this is mostly the missing UI. The backend side is validation:
the old `str(changes["name"])` accepted any type, any length and any
whitespace, for a value that is persisted and broadcast to every connected
client. validate_subscription_name now requires a string, collapses interior
whitespace to keep the label single-line, and caps it at 200 characters.

The name is display-only — it is used for the subscription list and log lines,
never for download paths — so renaming cannot move where files land.
2026-07-27 22:25:51 +03:00
Alex Shnitman 2744f36b44 Merge PR #1043: bump actions/setup-python from 6 to 7
Dependabot github-actions group update. Single-line bump in update-yt-dlp.yml,
the last v6 action left; consistent with checkout@v7 and setup-node@v7.

Co-authored-by: dependabot[bot] <support@github.com>
2026-07-27 21:22:03 +03:00
Alex Shnitman ff1b73a576 refactor: make POST /retry take a singular id
The endpoint accepted {ids: [x]} and then rejected anything but exactly one
id, so the schema advertised a batch it never supported. Retry is genuinely
singular: unlike the /delete, /start and /cancel batches, which act on local
state and can't meaningfully fail for one id and not another, each retry
re-extracts the URL and the caller removes that item's done record only once
it is confirmed re-queued. A real batch form would need per-id results in the
response for the caller to know which records to remove; that only becomes
worth designing alongside moving done-record deletion server-side.

/retry has not shipped yet, so there is no compatibility cost.
2026-07-27 21:19:01 +03:00
Alex Shnitman 1a09dbd686 Merge PR #1041: preserve playlist folder when retrying failed downloads
Replaces the client-side retry (which rebuilt an /add payload from the public
download dict) with a server-side POST /retry that re-adds from the stored
DownloadInfo. The public dict deliberately excludes `entry`, so the UI could
never carry playlist context across a retry: retried playlist items lost
playlist_index and landed in the root directory instead of their playlist
folder. The completed queue now persists the compacted entry for status=error
records only, so the context also survives a restart; successful records still
drop it. Also adds track_number to _COMPACT_ENTRY_EXTRA_KEYS so the #1031 music
metadata survives a failure/retry cycle.

Merged with two review fixes (08dccd9): retry_entry is now carried through the
url/url_transparent recursion in __add_entry, and retry() re-applies the
ALLOW_YTDL_OPTIONS_OVERRIDES and configured-preset gates that /add enforces via
parse_download_options.

Co-authored-by: jahruz67 <jahruz67@users.noreply.github.com>
2026-07-27 21:14:21 +03:00
Alex Shnitman 08dccd98fb fix: carry retry context through url indirection and re-gate retry options
Two review fixes on top of the retry endpoint:

- __add_entry dropped retry_entry when extraction returned an unprocessed
  url/url_transparent result and it recursed back into add(). Since
  __extract_info runs with extract_flat=True, that path is live, and a retried
  playlist item taking it fell back to OUTPUT_TEMPLATE and landed in the root
  directory instead of its playlist folder. The playlist child loop keeps
  passing retry_entry=None on purpose: those entries get fresh playlist context
  stamped on them from the current extraction.

- retry() called dqueue.add() directly, so it bypassed the
  parse_download_options gates that /add applies. Stored ytdl_options_overrides
  were re-applied even after ALLOW_YTDL_OPTIONS_OVERRIDES was turned off, and
  preset names removed from the configuration were still passed through. Both
  are re-checked against the current configuration at retry time.
2026-07-27 21:13:59 +03:00
dependabot[bot] 1f20aaee94 build(deps): bump actions/setup-python in the github-actions group
Bumps the github-actions group with 1 update: [actions/setup-python](https://github.com/actions/setup-python).


Updates `actions/setup-python` from 6 to 7
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-26 16:12:34 +00:00
jahruz67 8a29f3a084 fix: add track_number to compact entry extra keys
Added track_number to the set of keys preserved when compacting persisted playlist entries, ensuring this metadata is retained for accurate track ordering and display.
2026-07-24 19:49:48 -07:00
jahruz67 1839e5484d feat: add retry functionality for failed downloads 2026-07-24 10:21:06 -07:00
Alex Shnitman fceac97033 docs: cache-bust screenshot embed so the refreshed GIF shows
The README image URL was unchanged, so GitHub's Camo image proxy kept
serving the old cached GIF. Add ?v=2 to change the cache key and force a
re-fetch of the new screenshot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 14:02:12 +03:00
Alex Shnitman a13762aa61 docs: refresh screenshot.gif for the current UI [skip ci]
Regenerated the README demo GIF against today's UI: paste a playlist, watch
it queue and download, with a brief Advanced Options peek. Replaces the 2021
recording.

[skip ci] keeps this from cutting a build+release, since a screenshot swap
doesn't change the image (like a **.md-only change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 13:52:37 +03:00
Alex Shnitman 50250f8374 Merge PR #1037: bump actions/setup-node from 6 to 7
Dependabot github-actions group update. Single-line bump in main.yml,
consistent with the existing actions/checkout@v7.

Co-authored-by: dependabot[bot] <support@github.com>
2026-07-24 12:09:53 +03:00
Alex Shnitman 926d392926 Merge PR #1038: surface add-time failures as failed done entries
Adds __record_add_failure so a URL that fails before a real download starts
(unsupported/unextractable URL, SSRF-rejected, extraction error) appears in the
Completed list as a red-cross entry with retry and error-detail, instead of only
a transient toast and a server log line. Keyed by info.url like any errored
download. Includes _short_title_for_failed_url for a readable hostname title.

The frontend hunk removing the 'Click for details' hint from every error row was
dropped from this merge; that affordance (added in fd3aaea, #143) is kept.

Co-authored-by: streamer1122 <streamer1122@users.noreply.github.com>
2026-07-24 12:04:08 +03:00
Alex Shnitman 7f13784445 Merge PR #1031: conservative music metadata enrichment for audio downloads
Adds MusicMetadataPreProcessor (app/music_metadata.py), a pre_process
postprocessor that enriches the info dict using only extractor-owned fields:
track-number/total resolution, album-title fallback, and square-thumbnail
preference. Track numbers and album flow into files through yt-dlp's existing
FFmpegMetadata embedding, so no per-format tag-writing code and no new
dependency. The earlier mutagen writer and its download-failing
PostProcessingError path were dropped per review.

Co-authored-by: jahruz67 <jahruz67@users.noreply.github.com>
2026-07-24 12:00:38 +03:00
Alex Shnitman 4aa20890d4 Merge PR #1035: release per-download status_queue proxy on close to fix fd leak (#485)
Wraps Download.close() in try/finally and nulls self.status_queue so the
per-download manager.Queue() proxy is released once the completed Download is
retained in the done list. Previously every finished download permanently
pinned one Manager-process connection, accumulating file descriptors until
the instance hit 'too many open files' and self-terminated (#485, #980).

Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
2026-07-24 11:44:47 +03:00
Alex Shnitman e061a8a5ba feat: ALLOW_PRIVATE_ADDRESSES to opt out of the SSRF checks (closes #1036)
The SSRF guard rejects any address that isn't globally routable, which breaks
proxy/VPN setups that resolve hosts into private or special-use ranges. The
reported case is Fake-IP clients (sing-box, Clash, Mihomo) that map YouTube to
the RFC 2544 benchmarking range 198.18.0.0/15 to tunnel the traffic; MeTube
rejected it with "Refusing to fetch internal address" even though yt-dlp itself
handles it fine.

Add a boolean ALLOW_PRIVATE_ADDRESSES (default false) that, when set, skips both
layers: validate_url returns after scheme validation without the internal-host
checks, and the connect-time socket guard is not installed. Threaded to the
download subprocess via Download so the guard sees it too. Scheme validation
(http/https only) still applies. Documented in the README as a trust-your-network
opt-out that disables SSRF protection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 07:44:38 +03:00
Alex Shnitman 13cb65d931 docs: document the SSRF guard's connect-time coverage limitations
The connect-time getaddrinfo guard added for redirect/rebinding SSRF covers
only the download subprocess: metadata extraction runs in the main process
(where a process-wide guard would reject the server's own HOST=0.0.0.0 bind),
and native curl_cffi/libcurl resolution used by --impersonate bypasses Python's
socket module. Record both in url_guard's docstring and at the extraction site
so the boundary is explicit; network isolation remains the backstop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 07:41:01 +03:00
Alex Shnitman 1b02a99510 fix: re-validate outbound connections at fetch time against internal hosts
validate_url only inspects the submitted URL string. yt-dlp then follows HTTP
redirects and resolves media URLs from remote metadata without re-checking, so
an allowed URL that 302s to http://169.254.169.254/ (cloud metadata) or an
RFC1918 host is still fetched — the guard's own docstring scoped this out.

Install a getaddrinfo guard in the download subprocess that re-validates every
resolved address at actual connect time, covering redirects and DNS rebinding
for any backend resolving through Python's socket module (urllib, requests).
Loopback is permitted so locally-configured proxies keep working; link-local,
RFC1918 and unique-local are blocked. Native resolvers (curl_cffi/libcurl via
--impersonate) bypass this and rely on network isolation as the backstop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 06:55:29 +03:00
James Tew 4e27600329 Added handling for unsupported URL 2026-07-20 22:20:18 +01:00
Alex Shnitman ebcfe577bc fix: fail closed when an SSRF-guarded host cannot be resolved
validate_url() previously returned None (allow) on socket.gaierror, so a
host that failed to resolve at check time was passed straight to yt-dlp. A
host we cannot resolve is a host we cannot verify as non-internal, and it may
resolve differently when yt-dlp fetches it. Reject it instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 23:45:08 +03:00
Alex Shnitman 3bd2c3e366 fix: enforce download-dir containment at the resolved-path chokepoint
The per-download chapter_template was validated only against literal ".."
in the template string, but yt-dlp expands %(section_title)s (and every
other field) from attacker-controlled metadata at download time. On POSIX
hosts yt-dlp does not neutralise a ".." path component, so a chapter titled
".." turns a guard-passing template like
"%(section_title)s/%(section_title)s/x.%(ext)s" into "../../x.mp4" and writes
outside DOWNLOAD_DIR. The same class of escape applies to any multi-segment
output template (default/playlist/channel) whose fields resolve to "..".

The template string can never see the "..": it only exists after expansion.
So move the check to the one point every output path flows through —
YoutubeDL.prepare_filename — via a _ConfinedYoutubeDL subclass that refuses
any resolved path outside the download/temp roots (fail closed). This covers
the main file, split-chapter files, thumbnails and subtitles in one place.

With the chokepoint authoritative, the scattered ingress string checks
(chapter_template, custom_name_prefix) and the weaker _output_dir_escapes
literal-prefix check are removed. Tests move from the ingress layer to the
chokepoint, exercising the real metadata-resolution vector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 23:44:43 +03:00
dependabot[bot] 1c7261ab59 build(deps): bump actions/setup-node in the github-actions group
Bumps the github-actions group with 1 update: [actions/setup-node](https://github.com/actions/setup-node).


Updates `actions/setup-node` from 6 to 7
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-19 16:12:37 +00:00
Matt Van Horn 4cf2b1b0d0 fix: release per-download status_queue proxy on close to stop FD leak 2026-07-19 01:13:16 -07:00
Alex Shnitman 707f700609 ci: update releases in place instead of delete-and-recreate
The same-day rerun path deleted and recreated the release tag within
seconds, which corrupted GitHub's release index (release 2026.07.05 was
hidden from the public list and floated to the top for maintainers).
Replace it with gh release create/edit: force-move the tag in place on
reruns so it matches the rebuilt Docker image, exclude today's tag when
collecting release notes so reruns cover the whole day, and pin the
created tag to the triggering commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:44:52 +03:00
Alex Shnitman c519f45908 chore: rework issue and discussion templates around scope policy
Issue forms: trim prerequisite checkboxes to the two that matter, make the
yt-dlp test field explicitly conditional (UI bugs write "UI bug"), lead the
feature form with the scope line and the already-decided list, and retire
the question template in favor of Discussions Q&A.

Discussion forms: rename q-and-a.yml to q-a.yml so it matches the actual
category slug (it never applied), add ideas.yml for the category where
feature requests actually land, drop configuration-help.yml (no matching
category) and the inert config.yml (blank_discussions_enabled is not a
GitHub feature), and remove checkboxes from low-stakes discussion forms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:50:34 +03:00
Your GitHub Name f3d670e288 refactor: simplify music metadata processing by removing unused code and improving album signal detection 2026-07-17 10:43:39 -07:00
Your GitHub Name edf101faa0 feat: add music metadata processing and writing functionality 2026-07-16 19:31:57 -07:00
32 changed files with 1741 additions and 420 deletions
-1
View File
@@ -1 +0,0 @@
blank_discussions_enabled: false
@@ -1,65 +0,0 @@
name: ⚙️ Configuration Help
description: Get help with MeTube configuration and setup
title: "[Config]: "
labels: ["configuration", "help"]
assignees: []
body:
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before asking for configuration help
options:
- label: I have searched existing discussions and issues for similar configuration problems
required: true
- label: I have read the [configuration section](https://github.com/alexta69/metube#%EF%B8%8F-configuration-via-environment-variables) in the README
required: true
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
required: true
- type: markdown
attributes:
value: |
## Configuration Resources
Before asking for help, please check these resources:
- **[Configuration Guide](https://github.com/alexta69/metube#%EF%B8%8F-configuration-via-environment-variables)** - All available environment variables
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
- type: textarea
id: configuration-goal
attributes:
label: What are you trying to configure?
description: Describe what you want to achieve with your MeTube configuration
placeholder: |
What specific behavior are you trying to achieve?
What's not working as expected?
What have you tried so far?
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Your configuration, environment details, errors, and any other helpful information
placeholder: |
Current Configuration:
```yaml
# Your docker-compose.yml or environment variables
```
Environment Details:
- MeTube version: [e.g., latest, specific version]
- Docker image: [e.g., ghcr.io/alexta69/metube:latest]
- Operating System: [e.g., Ubuntu 20.04, Windows 10, macOS 12]
Error Messages or Issues:
[Paste any error messages, logs, or unexpected behavior here]
Other relevant information:
[Screenshots, examples, etc.]
validations:
required: true
+5 -31
View File
@@ -1,30 +1,13 @@
name: 💬 General Discussion
description: Start a general discussion about MeTube
title: "[Discussion]: "
labels: ["discussion"]
assignees: []
body:
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before starting a discussion
options:
- label: I have searched existing discussions to ensure this topic hasn't been discussed before
required: true
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
required: true
- type: markdown
attributes:
value: |
## Discussion Guidelines
This is for general discussions about MeTube. For specific issues, please use:
- **Bug reports** → Use the Bug Report issue template
- **Feature requests** → Use the Feature Request issue template
- **Questions** → Use the Question issue template
This is for general discussions about MeTube. For specific topics, better homes exist:
- **Bug reports** → [open an issue](https://github.com/alexta69/metube/issues/new?template=bug_report.yml)
- **Feature requests** → [open an issue](https://github.com/alexta69/metube/issues/new?template=feature_request.yml) or post in [Ideas](https://github.com/alexta69/metube/discussions/categories/ideas)
- **Questions** → post in [Q&A](https://github.com/alexta69/metube/discussions/categories/q-a)
- type: textarea
id: discussion-topic
@@ -34,12 +17,3 @@ body:
placeholder: Please provide a clear topic for discussion
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Any other information that might be helpful for the discussion
placeholder: Links, examples, or other relevant information
validations:
required: false
+31
View File
@@ -0,0 +1,31 @@
body:
- type: markdown
attributes:
value: |
## Read this first — MeTube's scope
MeTube's scope is deliberately narrow: **it downloads well and stops once the file is written.**
Ideas that improve the download itself are welcome — usually as community PRs, since the
maintainer is unlikely to implement requests ([details](https://github.com/alexta69/metube#-submitting-feature-requests)).
Post-download file management is out of scope regardless of implementation quality.
**Already decided — please don't re-request:**
- 🔒 Built-in authentication / password → use a reverse proxy ([#931](https://github.com/alexta69/metube/issues/931), [wiki](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations))
- 🔔 Notifications on download completion → available today via the `Exec` postprocessor ([recipe](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook))
- ✏️ Renaming, converting, tagging, or organizing files after download → out of scope ([#495](https://github.com/alexta69/metube/issues/495), [#1027](https://github.com/alexta69/metube/issues/1027))
- 🎚️ Pre-download format/quality/audio-track picker → tracked in [#1032](https://github.com/alexta69/metube/issues/1032)
- 📋 Pre-download playlist item browser → tracked in [#1030](https://github.com/alexta69/metube/issues/1030)
- 🎛️ Config switches to disable individual features → declined ([#976](https://github.com/alexta69/metube/issues/976))
- ⏰ Download scheduler → declined; add items with auto-start off and start them when you like ([#838](https://github.com/alexta69/metube/issues/838))
- type: textarea
id: idea
attributes:
label: Your Idea
description: Describe the idea and what problem it solves
placeholder: |
What would you like to see?
What problem does it solve?
Would you be willing to implement it as a PR?
validations:
required: true
+23
View File
@@ -0,0 +1,23 @@
body:
- type: markdown
attributes:
value: |
## Quick Resources
Your question may already be answered here:
- **[README](https://github.com/alexta69/metube#readme)** — complete setup and configuration guide
- **[Troubleshooting FAQ](https://github.com/alexta69/metube/wiki/Troubleshooting-FAQ)** — common problems and their fixes
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** — common yt-dlp configurations (notifications, metadata, audio extraction, ...)
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** — filename template examples
- type: textarea
id: question
attributes:
label: Your Question
description: What would you like to know about MeTube?
placeholder: |
What are you trying to achieve?
What's your current setup (docker-compose / environment variables)?
What have you tried so far, and what happened (errors, logs)?
validations:
required: true
-60
View File
@@ -1,60 +0,0 @@
name: ❓ Q&A
description: Ask a question and get answers from the community
title: "[Q&A]: "
labels: ["q-and-a"]
assignees: []
body:
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before asking your question
options:
- label: I have searched existing discussions and issues to see if my question has been answered before
required: true
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
required: true
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
required: true
- type: markdown
attributes:
value: |
## Quick Resources
Before asking your question, please check these resources:
- **[README](https://github.com/alexta69/metube#readme)** - Complete setup and configuration guide
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
- type: textarea
id: question
attributes:
label: Your Question
description: What would you like to know about MeTube?
placeholder: |
What are you trying to achieve?
What's your current setup?
What have you tried so far?
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Any other information that might be helpful (configuration, errors, screenshots, etc.)
placeholder: |
Configuration:
```yaml
# Your docker-compose.yml or environment variables
```
Error messages or logs:
[Paste any error messages or logs here]
Other relevant information:
[Screenshots, examples, etc.]
validations:
required: false
+12 -13
View File
@@ -9,24 +9,20 @@ body:
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before submitting your bug report
description: Please confirm before submitting
options:
- label: I have searched existing issues and discussions to ensure this bug hasn't been reported before
required: true
- label: I have read the [troubleshooting section](https://github.com/alexta69/metube#-troubleshooting-and-submitting-issues) in the README
required: true
- label: I have tested this issue with yt-dlp directly (not just through MeTube UI) as described in the README
required: true
- label: I have checked that this is not a yt-dlp issue (if it is, please report it to [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues) instead)
- label: If the download itself fails, I have tested the same URL and options with yt-dlp directly (see the note below) — or this bug is not download-related
required: true
- type: markdown
attributes:
value: |
## Important Notes
- **MeTube is only a UI for yt-dlp** - issues with authentication, postprocessing, permissions, or other yt-dlp functionality should be reported to the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
- Before reporting, please test with yt-dlp directly using: `docker exec -ti metube sh` then `cd /downloads` and run yt-dlp commands
- If yt-dlp works directly but MeTube doesn't, then it's a MeTube issue
- **MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp)** issues with authentication, postprocessing, site support, or other download functionality should be reported to the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
- To test with yt-dlp directly: `docker exec -ti metube sh`, then `cd /downloads` and run `yt-dlp` with your URL and options
- If yt-dlp works directly but MeTube doesn't, then it's a MeTube issue — report it here
- type: textarea
id: bug-description
@@ -47,10 +43,13 @@ body:
id: ytdl-test-results
attributes:
label: yt-dlp Direct Test Results
description: Results of testing the same URL/configuration directly with yt-dlp (required)
description: >-
If the download itself fails: paste the exact yt-dlp command you ran AND its output —
download-failure reports without this will be closed as needs-info.
If this is a UI or app bug that doesn't involve a failing download, just write "UI bug".
placeholder: |
Command used: yt-dlp [your-command-here]
Result: [success/error and output]
Result: [paste the output here]
validations:
required: true
@@ -64,12 +63,12 @@ body:
- MeTube version: [e.g., latest, specific version]
- Docker image: [e.g., ghcr.io/alexta69/metube:latest]
- Operating System: [e.g., Ubuntu 20.04, Windows 10, macOS 12]
Configuration:
```yaml
# Your docker-compose.yml or environment variables
```
Logs:
```bash
docker logs metube
+7 -4
View File
@@ -1,8 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: MeTube Community Discussions
url: https://github.com/alexta69/metube/discussions
about: Ask questions and discuss MeTube with the community
- name: yt-dlp Issues
- name: ❓ Questions & Support
url: https://github.com/alexta69/metube/discussions/categories/q-a
about: Ask usage and configuration questions in Discussions Q&A — issues are for bugs and feature requests
- name: 📖 Troubleshooting FAQ
url: https://github.com/alexta69/metube/wiki/Troubleshooting-FAQ
about: Common problems and their fixes
- name: ⬇️ yt-dlp Issues
url: https://github.com/yt-dlp/yt-dlp/issues
about: Report issues related to video downloading, authentication, or site support
+22 -15
View File
@@ -5,28 +5,35 @@ labels: ["enhancement"]
assignees: []
body:
- type: markdown
attributes:
value: |
## Read this first — MeTube's scope
MeTube's scope is deliberately narrow: **it downloads well and stops once the file is written.**
Features that improve the download itself are welcome — usually as community PRs, since the
maintainer is unlikely to implement requests ([details](https://github.com/alexta69/metube#-submitting-feature-requests)).
Post-download file management is out of scope regardless of implementation quality.
**Already decided — please don't re-request:**
- 🔒 Built-in authentication / password → use a reverse proxy ([#931](https://github.com/alexta69/metube/issues/931), [wiki](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations))
- 🔔 Notifications on download completion → available today via the `Exec` postprocessor ([recipe](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook))
- ✏️ Renaming, converting, tagging, or organizing files after download → out of scope ([#495](https://github.com/alexta69/metube/issues/495), [#1027](https://github.com/alexta69/metube/issues/1027))
- 🎚️ Pre-download format/quality/audio-track picker → tracked in [#1032](https://github.com/alexta69/metube/issues/1032)
- 📋 Pre-download playlist item browser → tracked in [#1030](https://github.com/alexta69/metube/issues/1030)
- 🎛️ Config switches to disable individual features → declined ([#976](https://github.com/alexta69/metube/issues/976))
- ⏰ Download scheduler → declined; add items with auto-start off and start them when you like ([#838](https://github.com/alexta69/metube/issues/838))
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before submitting your feature request
description: Please confirm before submitting
options:
- label: I have searched existing issues and discussions to ensure this feature hasn't been requested before
- label: I have searched existing issues and checked the "Already decided" list above
required: true
- label: I have read the [feature request guidelines](https://github.com/alexta69/metube#-submitting-feature-requests) in the README
- label: I have considered if this feature belongs in yt-dlp instead (downloading, processing, site support) — if so, please request it in the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
required: true
- label: I understand that MeTube development relies on community contributions and the maintainer is not likely to implement this feature
required: true
- label: I have considered if this feature should be implemented in yt-dlp instead of MeTube (if so, please report to [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues))
required: true
- type: markdown
attributes:
value: |
## Important Notes
- **MeTube development relies on code contributions by the community** - the project is feature-complete for the maintainer's use cases
- **Consider if this belongs in yt-dlp** - if it's related to video downloading, processing, or site support, it might belong in the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues) instead
- **Some features may not be accepted** - in an effort to reduce bloat, some PRs may not be accepted
- type: textarea
id: feature-description
-62
View File
@@ -1,62 +0,0 @@
name: ❓ Question
description: Ask a question about MeTube usage, configuration, or general help
title: "[Question]: "
labels: ["question"]
assignees: []
body:
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm you have completed these steps before asking your question
options:
- label: I have searched existing issues and discussions to see if my question has been answered before
required: true
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
required: true
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
required: true
- label: I have read the [troubleshooting section](https://github.com/alexta69/metube#-troubleshooting-and-submitting-issues) if this is a technical issue
required: true
- type: markdown
attributes:
value: |
## Quick Resources
Before asking your question, please check these resources:
- **[README](https://github.com/alexta69/metube#readme)** - Complete setup and configuration guide
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
- type: textarea
id: question
attributes:
label: Your Question
description: What would you like to know about MeTube?
placeholder: |
What are you trying to achieve?
What's your current setup?
What have you tried so far?
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Any other information that might be helpful (configuration, errors, screenshots, etc.)
placeholder: |
Configuration:
```yaml
# Your docker-compose.yml or environment variables
```
Error messages or logs:
[Paste any error messages or logs here]
Other relevant information:
[Screenshots, examples, etc.]
validations:
required: false
+33 -47
View File
@@ -14,7 +14,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v7
- name: Set up Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: lts/*
- name: Enable pnpm
@@ -117,25 +117,27 @@ jobs:
- name: Get current date
id: date
run: echo "date=$(date +'%Y.%m.%d')" >> $GITHUB_OUTPUT
- name: Checkout
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Get commits since last release
id: commits
env:
DATE: ${{ steps.date.outputs.date }}
run: |
# Fetch all tags
git fetch --tags
# Get the last tag (sorted by version, using date format YYYY.MM.DD)
LAST_TAG=$(git tag -l --sort=-version:refname | grep -E '^[0-9]{4}\.[0-9]{2}\.[0-9]{2}$' | head -n 1)
# Exclude today's tag: on a same-day rerun the notes must cover the
# whole day, not just the commits since the morning release.
LAST_TAG=$(git tag -l --sort=-version:refname | grep -E '^[0-9]{4}\.[0-9]{2}\.[0-9]{2}$' | grep -v "^${DATE}$" | head -n 1)
if [ -z "$LAST_TAG" ]; then
# No previous release, skip commits for first release
COMMITS=""
echo "has_commits=false" >> $GITHUB_OUTPUT
else
# Get commits since last tag
COMMITS=$(git log ${LAST_TAG}..HEAD --pretty=format:"- %s (%h)" --no-merges)
if [ -z "$COMMITS" ]; then
echo "has_commits=false" >> $GITHUB_OUTPUT
@@ -143,18 +145,13 @@ jobs:
echo "has_commits=true" >> $GITHUB_OUTPUT
fi
fi
# Escape for use in YAML/multiline output
{
echo 'commits<<EOF'
echo "$COMMITS"
echo EOF
} >> $GITHUB_OUTPUT
# Also output for debugging
echo "Last tag: ${LAST_TAG:-none}"
echo "Commits since last release:"
echo "$COMMITS"
- name: Generate release body
id: release_body
env:
@@ -176,7 +173,7 @@ jobs:
echo '**GitHub Container Registry:**'
echo "- \`${GHCR_REPO}:latest\`"
echo "- \`${GHCR_REPO}:${DATE}\`"
if [ "$HAS_COMMITS" = "true" ] && [ -n "$COMMITS" ]; then
echo ''
echo '## Changes'
@@ -184,39 +181,28 @@ jobs:
echo "$COMMITS"
fi
} > release_body.txt
{
echo 'body<<EOF'
cat release_body.txt
echo EOF
} >> $GITHUB_OUTPUT
- name: Delete existing release if present
- name: Create or update GitHub Release (mark as latest)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG_NAME: ${{ steps.date.outputs.date }}
run: |
# Check if release exists and delete it
if gh release view "$TAG_NAME" &>/dev/null; then
echo "Release $TAG_NAME already exists, deleting it..."
gh release delete "$TAG_NAME" --yes || true
if gh release view "$TAG_NAME" >/dev/null 2>&1; then
echo "Release $TAG_NAME exists; updating."
# Force-move the tag in place so it matches the rebuilt Docker
# image. Never delete+recreate the tag: that corrupts GitHub's
# release index (broke release 2026.07.05).
gh api -X PATCH "repos/${GITHUB_REPOSITORY}/git/refs/tags/${TAG_NAME}" \
-f sha="$GITHUB_SHA" -F force=true
gh release edit "$TAG_NAME" \
--title "Release $TAG_NAME" \
--notes-file release_body.txt \
--latest
else
echo "Release $TAG_NAME does not exist; creating."
gh release create "$TAG_NAME" \
--target "$GITHUB_SHA" \
--title "Release $TAG_NAME" \
--notes-file release_body.txt \
--latest
fi
# Fetch tags to check remote
git fetch --tags
# Check if tag exists (locally or remotely) and delete it
if git rev-parse "$TAG_NAME" &>/dev/null 2>&1 || git ls-remote --tags origin "$TAG_NAME" | grep -q "$TAG_NAME"; then
echo "Tag $TAG_NAME already exists, deleting it..."
git tag -d "$TAG_NAME" 2>/dev/null || true
git push origin ":refs/tags/$TAG_NAME" || true
fi
- name: Create GitHub Release
uses: softprops/action-gh-release@v3
with:
tag_name: ${{ steps.date.outputs.date }}
name: Release ${{ steps.date.outputs.date }}
body_path: release_body.txt
draft: false
prerelease: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
token: ${{ secrets.AUTOUPDATE_PAT }}
-
name: Set up Python
uses: actions/setup-python@v6
uses: actions/setup-python@v7
with:
python-version: '3.13'
-
+17 -1
View File
@@ -96,7 +96,23 @@ release the same day. **Master is continuously released** — a PR must be
release-ready exactly as merged; there is no stabilization window for follow-up
fixes.
## Code style
## Commit messages
A commit that resolves an issue must close it, with a GitHub closing keyword in
parentheses at the end of the subject line:
```
fix: stop metadata probes from writing playlist sidecar files (closes #1040)
```
Because master is the default branch and is released on every push, the issue
closes at the moment the fix ships, and keeps a permanent link to the commit that
fixed it. A bare `(#1040)` is only a reference — and reads as a pull-request
number — so it does not count; the keyword is what closes the issue.
Auto-closing leaves only a commit stub on the issue, which is not an answer to
whoever reported it. Post an explanatory comment as well: what the cause was, what
changed, and anything the reporter needs to do differently.
Follow `.editorconfig`:
- Python: 4-space indent
+4 -1
View File
@@ -10,7 +10,7 @@ Key capabilities:
* Download playlists and channels, with configurable output and download options.
* [Subscribe](https://github.com/alexta69/metube/wiki/Subscriptions) to channels and playlists, periodically check for new items, and queue new uploads automatically.
![screenshot1](https://github.com/alexta69/metube/raw/master/screenshot.gif)
![screenshot1](https://github.com/alexta69/metube/raw/master/screenshot.gif?v=2)
## 🐳 Run using Docker
@@ -80,8 +80,11 @@ Certain values can be set via environment variables, using the `-e` parameter on
* __YTDL_OPTIONS_PRESETS__: Named bundles of yt-dlp options, selectable per download in the UI. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options) for format and examples.
* __YTDL_OPTIONS_PRESETS_FILE__: Path to a JSON file containing presets. Monitored and reloaded automatically on changes. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options).
* __ALLOW_YTDL_OPTIONS_OVERRIDES__: Whether to show a free-text field in the UI for per-download yt-dlp option overrides. Defaults to `false`. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options) for details and security considerations.
* __ALLOW_PRIVATE_ADDRESSES__: Whether to allow downloads from private, loopback, link-local and other non-global addresses. Defaults to `false`, which protects against SSRF by refusing URLs that resolve to internal hosts. Set to `true` only in trusted environments — for example when routing traffic through a proxy/VPN client in Fake-IP mode (sing-box, Clash, Mihomo), which resolves hosts to the `198.18.0.0/15` range. Enabling this disables the SSRF protection entirely, so only use it when you control the network.
* __YTDL_NIGHTLY_UPDATE_TIME__: If set, will cause MeTube to use [nightly yt-dlp builds](https://github.com/yt-dlp/yt-dlp-nightly-builds) instead of the stable releases. Set to the time (`HH:MM`, 24-hour) when you want the daily upgrades and MeTube restart to happen. Defaults to empty (disabled).
Enabling `writeinfojson` or `writethumbnail` in `YTDL_OPTIONS` also writes a feed-level `.info.json` and thumbnail when you add a playlist or channel. These reuse the template of the items they belong to — `OUTPUT_TEMPLATE_CHANNEL` or `OUTPUT_TEMPLATE_PLAYLIST` — evaluated against the feed itself, so with the defaults they land in the same folder as the videos, named after the feed. Set `allow_playlist_files` to `false` in `YTDL_OPTIONS` to skip them.
### 🌐 Web Server & URLs
* __HOST__: The host address the web server will bind to. Defaults to `0.0.0.0` (all interfaces).
+21 -5
View File
@@ -81,6 +81,7 @@ class Config:
'YTDL_OPTIONS_PRESETS': '{}',
'YTDL_OPTIONS_PRESETS_FILE': '',
'ALLOW_YTDL_OPTIONS_OVERRIDES': 'false',
'ALLOW_PRIVATE_ADDRESSES': 'false',
'CORS_ALLOWED_ORIGINS': '',
'ROBOTS_TXT': '',
'HOST': '0.0.0.0',
@@ -96,7 +97,7 @@ class Config:
'YTDL_NIGHTLY_UPDATE_TIME': '',
}
_BOOLEAN = ('DOWNLOAD_DIRS_INDEXABLE', 'CUSTOM_DIRS', 'CREATE_CUSTOM_DIRS', 'DELETE_FILE_ON_TRASHCAN', 'HTTPS', 'ENABLE_ACCESSLOG', 'ALLOW_YTDL_OPTIONS_OVERRIDES')
_BOOLEAN = ('DOWNLOAD_DIRS_INDEXABLE', 'CUSTOM_DIRS', 'CREATE_CUSTOM_DIRS', 'DELETE_FILE_ON_TRASHCAN', 'HTTPS', 'ENABLE_ACCESSLOG', 'ALLOW_YTDL_OPTIONS_OVERRIDES', 'ALLOW_PRIVATE_ADDRESSES')
def __init__(self):
for k, v in self._DEFAULTS.items():
@@ -717,8 +718,6 @@ def parse_download_options(post: dict) -> dict:
if custom_name_prefix is None:
custom_name_prefix = ''
if custom_name_prefix and ('..' in custom_name_prefix or custom_name_prefix.startswith('/') or custom_name_prefix.startswith('\\')):
raise web.HTTPBadRequest(reason='custom_name_prefix must not contain ".." or start with a path separator')
if auto_start is None:
auto_start = True
if playlist_item_limit is None:
@@ -743,8 +742,6 @@ def parse_download_options(post: dict) -> dict:
enabled=config.ALLOW_YTDL_OPTIONS_OVERRIDES,
)
if chapter_template and ('..' in chapter_template or chapter_template.startswith('/') or chapter_template.startswith('\\')):
raise web.HTTPBadRequest(reason='chapter_template must not contain ".." or start with a path separator')
if not SUBTITLE_LANGUAGE_RE.fullmatch(subtitle_language):
raise web.HTTPBadRequest(reason='subtitle_language must match pattern [A-Za-z0-9-] and be at most 35 characters')
if subtitle_mode not in VALID_SUBTITLE_MODES:
@@ -896,6 +893,17 @@ async def cancel_add(request):
return web.Response(text=serializer.encode({'status': 'ok'}), content_type='application/json')
@routes.post(config.URL_PREFIX + 'retry')
async def retry(request):
# Singular by design, unlike the 'ids' batch endpoints: a retry re-extracts
# the URL, so it can fail per item, and the caller removes that item's done
# record only once it is confirmed re-queued. A batch form would have to
# report per-id results for the caller to know which ones to remove.
post = await _read_json_request(request)
status = await dqueue.retry(_require_id(post))
return web.Response(text=serializer.encode(status), content_type='application/json')
@routes.post(config.URL_PREFIX + 'subscribe')
async def subscribe(request):
post = await _read_json_request(request)
@@ -988,6 +996,13 @@ async def subscriptions_check(request):
result = await submgr.check_now([str(i) for i in ids] if ids else None)
return web.Response(text=serializer.encode(result))
def _require_id(post: dict) -> str:
id = post.get('id')
if not isinstance(id, str) or not id:
raise web.HTTPBadRequest(reason="'id' must be a non-empty string")
return id
def _require_id_list(post: dict) -> list:
ids = post.get('ids')
if not isinstance(ids, list) or not ids or not all(isinstance(i, str) for i in ids):
@@ -1230,6 +1245,7 @@ async def add_cors(request):
app.router.add_route('OPTIONS', config.URL_PREFIX + 'add', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'cancel-add', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'retry', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscribe', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions', add_cors)
app.router.add_route('OPTIONS', config.URL_PREFIX + 'subscriptions/update', add_cors)
+124
View File
@@ -0,0 +1,124 @@
"""Conservative music metadata enrichment for audio downloads.
This module only consumes fields already supplied by yt-dlp or retained on
MeTube's queued playlist entry. It intentionally performs no external lookup
or site-specific album detection.
"""
from __future__ import annotations
from typing import Any, Optional
from yt_dlp.postprocessor.common import PostProcessor
def _has_value(value: Any) -> bool:
if isinstance(value, str):
return bool(value.strip())
if isinstance(value, (list, tuple)):
return any(_has_value(item) for item in value)
return value is not None
def _positive_int(value: Any) -> Optional[int]:
if isinstance(value, bool):
return None
try:
number = int(value)
except (TypeError, ValueError):
return None
return number if number > 0 else None
def _track_position(value: Any) -> tuple[Optional[int], Optional[int]]:
"""Return a track number and optional total from a scalar or ``n/total``."""
if isinstance(value, str) and '/' in value:
number, total = value.split('/', 1)
return _positive_int(number.strip()), _positive_int(total.strip())
return _positive_int(value), None
def _first_positive_int(*values: Any) -> Optional[int]:
return next((number for value in values if (number := _positive_int(value))), None)
def _has_album_signal(info: dict[str, Any], source_entry: dict[str, Any]) -> bool:
"""Use only extractor-owned fields to identify album-level metadata."""
return any(
_has_value(entry.get(key))
for entry in (info, source_entry)
for key in ('album', 'track_number')
)
def _is_music_audio(info: dict[str, Any], source_entry: dict[str, Any]) -> bool:
return _has_album_signal(info, source_entry) or any(
_has_value(entry.get(key))
for entry in (info, source_entry)
for key in ('track', 'artists')
)
def prefer_square_thumbnail(info: dict[str, Any]) -> None:
"""Move the largest known square thumbnail to yt-dlp's preferred slot."""
thumbnails = info.get('thumbnails')
if not isinstance(thumbnails, list) or len(thumbnails) < 2:
return
candidates: list[tuple[int, int]] = []
for index, thumbnail in enumerate(thumbnails):
if not isinstance(thumbnail, dict):
continue
width = _positive_int(thumbnail.get('width'))
height = _positive_int(thumbnail.get('height'))
if width is not None and width == height:
candidates.append((width * height, index))
if not candidates:
return
_, selected_index = max(candidates)
selected = thumbnails.pop(selected_index)
thumbnails.append(selected)
if selected.get('url'):
info['thumbnail'] = selected['url']
class MusicMetadataPreProcessor(PostProcessor):
"""Enrich extracted audio metadata using extractor-owned album signals."""
def __init__(self, downloader=None, *, source_entry=None):
super().__init__(downloader)
self._source_entry = source_entry if isinstance(source_entry, dict) else {}
def run(self, info):
if _has_album_signal(info, self._source_entry):
number, inline_total = _track_position(info.get('track_number'))
if number is None:
number, source_inline_total = _track_position(
self._source_entry.get('track_number')
)
inline_total = inline_total or source_inline_total
if number is None:
number = _positive_int(self._source_entry.get('playlist_index'))
total = inline_total or _first_positive_int(
info.get('track_count'),
info.get('track_total'),
self._source_entry.get('track_count'),
self._source_entry.get('track_total'),
self._source_entry.get('playlist_count'),
self._source_entry.get('n_entries'),
)
if number is not None:
info['track_number'] = f'{number}/{total}' if total is not None else number
if not _has_value(info.get('album')):
album = self._source_entry.get('album') or self._source_entry.get(
'playlist_title'
)
if isinstance(album, str) and album.strip():
info['album'] = album.strip()
if _is_music_audio(info, self._source_entry):
prefer_square_thumbnail(info)
return [], info
+36 -4
View File
@@ -64,6 +64,12 @@ def _build_ydl_params(
"paths": {"home": config.DOWNLOAD_DIR, "temp": config.TEMP_DIR},
**config.YTDL_OPTIONS,
**(extra_opts or {}),
# A scan is a poll, not an add: it runs on a timer and queues items
# through the download queue, which writes the feed metadata itself.
# yt-dlp emits the playlist-level infojson/description/thumbnail
# regardless of `download`, so without this a writeinfojson user would
# get those files rewritten on every check interval. See issue #1040.
"allow_playlist_files": False,
}
params = _impersonate_opt(params)
if playlistend is not None and playlistend > 0:
@@ -115,7 +121,7 @@ def extract_flat_playlist(
if not nested_url:
continue
# nested_url comes from remote playlist content; guard it too.
if validate_url(nested_url) is not None:
if validate_url(nested_url, allow_private=getattr(config, "ALLOW_PRIVATE_ADDRESSES", False)) is not None:
continue
nested_info, nested_entries = extract_flat_playlist(
config,
@@ -287,6 +293,24 @@ def validate_title_regex(value: Any) -> str:
return s
# The name is a display label the user picks; it is persisted and broadcast to
# every connected client, so keep it a bounded single-line string.
SUBSCRIPTION_NAME_MAX_LENGTH = 200
def validate_subscription_name(value: Any) -> str:
"""Return a stored subscription name, or raise ValueError if unusable."""
if not isinstance(value, str):
raise ValueError("name must be a string")
# Collapse newlines/tabs so a pasted title can't break the table layout.
name = " ".join(value.split())
if not name:
raise ValueError("name must not be empty")
if len(name) > SUBSCRIPTION_NAME_MAX_LENGTH:
raise ValueError(f"name must be at most {SUBSCRIPTION_NAME_MAX_LENGTH} characters")
return name
def _coerce_bool(value: Any) -> bool:
"""Accept JSON booleans and common string forms used by API clients."""
if isinstance(value, bool):
@@ -548,7 +572,8 @@ class SubscriptionManager:
return {"status": "error", "msg": "Missing URL"}
# SSRF guard: block non-http(s) schemes and internal/metadata hosts
# before yt-dlp fetches the feed. May do a DNS lookup, so run off-loop.
url_error = await asyncio.get_running_loop().run_in_executor(None, validate_url, url)
url_error = await asyncio.get_running_loop().run_in_executor(
None, partial(validate_url, url, allow_private=getattr(self.config, "ALLOW_PRIVATE_ADDRESSES", False)))
if url_error is not None:
log.warning('Rejected subscription URL "%s": %s', url, url_error)
return {"status": "error", "msg": url_error}
@@ -673,6 +698,13 @@ class SubscriptionManager:
return {"status": "ok"}
async def update_subscription(self, sub_id: str, changes: dict) -> dict:
validated_name: Optional[str] = None
if "name" in changes:
try:
validated_name = validate_subscription_name(changes["name"])
except ValueError as exc:
return {"status": "error", "msg": str(exc)}
validated_tr: Optional[str] = None
if "title_regex" in changes:
try:
@@ -721,8 +753,8 @@ class SubscriptionManager:
sub.enabled = validated_enabled
if interval_set:
sub.check_interval_minutes = validated_interval
if "name" in changes and changes["name"]:
sub.name = str(changes["name"])
if validated_name is not None:
sub.name = validated_name
if validated_tr is not None:
sub.title_regex = validated_tr
if skip_so_set:
+17 -19
View File
@@ -20,6 +20,7 @@ def mock_dqueue(monkeypatch):
d = MagicMock()
d.initialize = AsyncMock(return_value=None)
d.add = AsyncMock(return_value={"status": "ok"})
d.retry = AsyncMock(return_value={"status": "ok"})
d.cancel = AsyncMock(return_value={"status": "ok"})
d.clear = AsyncMock(return_value={"status": "ok"})
d.start_pending = AsyncMock(return_value={"status": "ok"})
@@ -69,6 +70,22 @@ async def test_add_ok(mock_dqueue):
mock_dqueue.add.assert_awaited_once()
@pytest.mark.asyncio
async def test_retry_passes_failed_download_id(mock_dqueue):
req = _json_request({"id": "https://example.com/watch?v=1"})
resp = await main.retry(req)
assert resp.status == 200
mock_dqueue.retry.assert_awaited_once_with("https://example.com/watch?v=1")
@pytest.mark.asyncio
@pytest.mark.parametrize("body", [{}, {"id": ""}, {"id": ["a"]}, {"ids": ["a"]}])
async def test_retry_rejects_missing_or_non_string_id(mock_dqueue, body):
with pytest.raises(web.HTTPBadRequest):
await main.retry(_json_request(body))
mock_dqueue.retry.assert_not_awaited()
@pytest.mark.asyncio
async def test_add_passes_preset_and_overrides(mock_dqueue, monkeypatch):
monkeypatch.setattr(main.config, "YTDL_OPTIONS_PRESETS", {"Preset A": {"writesubtitles": True}})
@@ -138,25 +155,6 @@ async def test_add_invalid_subtitle_language(mock_dqueue):
await main.add(req)
@pytest.mark.asyncio
async def test_add_custom_name_prefix_path_traversal(mock_dqueue):
req = _json_request(_valid_video_add_body(custom_name_prefix="../evil"))
with pytest.raises(web.HTTPBadRequest):
await main.add(req)
@pytest.mark.asyncio
async def test_add_chapter_template_path_traversal(mock_dqueue):
req = _json_request(
_valid_video_add_body(
split_by_chapters=True,
chapter_template="/etc/passwd%(title)s",
)
)
with pytest.raises(web.HTTPBadRequest):
await main.add(req)
@pytest.mark.asyncio
async def test_add_invalid_json_body(mock_dqueue):
req = MagicMock(spec=web.Request)
+444 -8
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import copy
import os
import re
import tempfile
@@ -89,7 +90,7 @@ def test_get_returns_tuple_of_lists(dq_env):
async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
@@ -115,11 +116,59 @@ async def test_add_single_video_goes_to_pending_when_auto_start_false(dq_env):
assert dq.pending.exists("https://example.com/watch?v=1")
@pytest.mark.asyncio
async def test_add_unsupported_url_recorded_as_failed_entry(dq_env):
"""An unsupported/unextractable URL must show up as a red-cross entry in the
done list, not just a transient toast and a server log line."""
import ytdl
notifier = AsyncMock()
url = "https://example.com/not-a-video"
def boom(self, url, *_args, **_kwargs):
raise ytdl.yt_dlp.utils.YoutubeDLError(f'Unsupported URL: {url}')
dq = DownloadQueue(dq_env, notifier)
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", boom):
result = await dq.add(
url, "video", "auto", "any", "best", "", "", 0, auto_start=True,
)
assert result["status"] == "error"
assert dq.done.exists(url)
failed = dq.done.get(url)
assert failed.info.status == "error"
assert failed.info.error == result["msg"]
assert failed.info.url == url
# The full URL stays in .url/.error for the detail panel; the display
# title is shortened to the hostname so the Completed row stays readable.
assert failed.info.title == "example.com"
notifier.completed.assert_awaited()
@pytest.mark.asyncio
async def test_add_ssrf_rejected_url_recorded_as_failed_entry(dq_env):
"""A URL rejected by the SSRF guard (before yt-dlp ever runs) must also
surface as a failed entry, not just an error status returned to the caller."""
notifier = AsyncMock()
url = "file:///etc/passwd"
dq = DownloadQueue(dq_env, notifier)
result = await dq.add(
url, "video", "auto", "any", "best", "", "", 0, auto_start=True,
)
assert result["status"] == "error"
assert dq.done.exists(url)
failed = dq.done.get(url)
assert failed.info.status == "error"
assert failed.info.error == result["msg"]
notifier.completed.assert_awaited()
@pytest.mark.asyncio
async def test_cancel_removes_from_pending(dq_env):
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
@@ -156,7 +205,7 @@ async def test_cancel_before_start_marks_download_canceled(dq_env):
cancelling, because its ``download.canceled`` guard was never flipped."""
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
@@ -194,7 +243,7 @@ async def test_cancel_before_start_marks_download_canceled(dq_env):
async def test_start_pending_moves_to_queue(dq_env):
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
@@ -254,6 +303,179 @@ async def test_add_entry_queues_single_video_without_reextracting(dq_env):
assert dq.pending.exists("https://example.com/watch?v=1")
@pytest.mark.asyncio
async def test_retry_restores_playlist_output_context(dq_env):
notifier = AsyncMock()
dq_env.OUTPUT_TEMPLATE_PLAYLIST = "%(playlist_title)s/%(title)s.%(ext)s"
dq = DownloadQueue(dq_env, notifier)
url = "https://example.com/watch?v=1"
failed_info = DownloadInfo(
id="vid1",
title="Test Video",
url=url,
quality="best",
download_type="video",
codec="auto",
format="any",
folder="",
custom_name_prefix="",
error="temporary failure",
entry={
"playlist_index": "01",
"playlist_title": "My Playlist",
"playlist_count": 10,
},
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
)
failed_info.status = "error"
dq.done.put(Download(None, None, None, None, "best", "any", {}, failed_info))
def fake_extract(self, extracted_url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": extracted_url,
"webpage_url": extracted_url,
}
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
result = await dq.retry(url)
assert result["status"] == "ok"
queued = dq.queue.get(url)
assert queued.output_template == "My Playlist/%(title)s.%(ext)s"
assert queued.info.entry["playlist_index"] == "01"
assert queued.info.entry["playlist_title"] == "My Playlist"
def _failed_playlist_item(url, **overrides):
"""A done-list entry for a playlist item that failed mid-download."""
info = DownloadInfo(
id="vid1",
title="Test Video",
url=url,
quality="best",
download_type="video",
codec="auto",
format="any",
folder="",
custom_name_prefix="",
error="temporary failure",
entry={
"playlist_index": "01",
"playlist_title": "My Playlist",
"playlist_count": 10,
},
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
**overrides,
)
info.status = "error"
return info
@pytest.mark.asyncio
async def test_retry_keeps_playlist_context_through_url_indirection(dq_env):
# extract_flat=True makes yt-dlp hand back url/url_transparent results
# unprocessed, so __add_entry recurses into add() a second time. The retry
# context has to survive that hop or the item lands in the root directory.
notifier = AsyncMock()
dq_env.OUTPUT_TEMPLATE_PLAYLIST = "%(playlist_title)s/%(title)s.%(ext)s"
dq = DownloadQueue(dq_env, notifier)
url = "https://example.com/watch?v=1"
resolved = "https://example.com/resolved?v=1"
dq.done.put(Download(None, None, None, None, "best", "any", {}, _failed_playlist_item(url)))
def fake_extract(self, extracted_url, *_args, **_kwargs):
if extracted_url == url:
return {"_type": "url", "url": resolved, "id": "vid1"}
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": extracted_url,
"webpage_url": extracted_url,
}
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
result = await dq.retry(url)
assert result["status"] == "ok"
queued = dq.queue.get(resolved)
assert queued.output_template == "My Playlist/%(title)s.%(ext)s"
assert queued.info.entry["playlist_title"] == "My Playlist"
@pytest.mark.asyncio
async def test_retry_reapplies_current_options_gates(dq_env):
# The stored options passed parse_download_options when first submitted, but
# the configuration can have changed since; retry must not resurrect
# overrides or presets the current configuration no longer allows.
notifier = AsyncMock()
dq_env.ALLOW_YTDL_OPTIONS_OVERRIDES = False
dq_env.YTDL_OPTIONS_PRESETS = {"Still There": {"writesubtitles": True}}
dq = DownloadQueue(dq_env, notifier)
url = "https://example.com/watch?v=1"
info = _failed_playlist_item(
url,
ytdl_options_presets=["Still There", "Removed Preset"],
ytdl_options_overrides={"paths": {"home": "/etc"}},
)
dq.done.put(Download(None, None, None, None, "best", "any", {}, info))
def fake_extract(self, extracted_url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": extracted_url,
"webpage_url": extracted_url,
}
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
result = await dq.retry(url)
assert result["status"] == "ok"
queued = dq.queue.get(url)
assert queued.info.ytdl_options_overrides == {}
assert queued.info.ytdl_options_presets == ["Still There"]
assert queued.ytdl_opts.get("paths", {}).get("home") != "/etc"
@pytest.mark.asyncio
async def test_retry_keeps_overrides_while_still_allowed(dq_env):
notifier = AsyncMock()
dq_env.ALLOW_YTDL_OPTIONS_OVERRIDES = True
dq_env.YTDL_OPTIONS_PRESETS = {}
dq = DownloadQueue(dq_env, notifier)
url = "https://example.com/watch?v=1"
info = _failed_playlist_item(url, ytdl_options_overrides={"writesubtitles": True})
dq.done.put(Download(None, None, None, None, "best", "any", {}, info))
def fake_extract(self, extracted_url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
"title": "Test Video",
"url": extracted_url,
"webpage_url": extracted_url,
}
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", fake_extract), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
result = await dq.retry(url)
assert result["status"] == "ok"
assert dq.queue.get(url).info.ytdl_options_overrides == {"writesubtitles": True}
@pytest.mark.asyncio
async def test_add_entry_duplicate_while_pending_is_skipped_not_clobbered(dq_env):
notifier = AsyncMock()
@@ -316,7 +538,7 @@ async def test_channel_download_uses_output_template_when_channel_template_empty
channel_id = "UCabcd123"
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "playlist",
"id": channel_id,
@@ -365,7 +587,7 @@ async def test_playlist_download_not_treated_as_channel(dq_env):
dq_env.OUTPUT_TEMPLATE_CHANNEL = ""
dq_env.OUTPUT_TEMPLATE_PLAYLIST = "%(playlist_title)s/%(title)s.%(ext)s"
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "playlist",
"id": "PLxyz789",
@@ -412,7 +634,7 @@ async def test_add_merges_global_preset_and_override_options(dq_env):
"Preset B": {"writesubtitles": False, "ratelimit": 1000},
}
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid2",
@@ -535,11 +757,191 @@ async def test_extract_info_metube_extract_keys_win_over_preset(dq_env):
assert captured_params[0]["noplaylist"] is True
def _feed_extract(feed):
"""Patch for __extract_info that returns a playlist/channel feed dict."""
def fake_extract(self, url, *_args, **_kwargs):
return copy.deepcopy(feed)
return fake_extract
_CHANNEL_FEED = {
"_type": "playlist",
"id": "UC123",
"title": "Vanessa - Videos",
"channel": "Vanessa",
"channel_id": "UC123",
"uploader": "Vanessa",
"extractor": "youtube:tab",
"extractor_key": "YoutubeTab",
"webpage_url": "https://example.com/@vanessa/videos",
"entries": [
{"id": "v1", "title": "One", "url": "https://example.com/v1",
"webpage_url": "https://example.com/v1", "_type": "url"},
],
}
_PLAYLIST_FEED = {
"_type": "playlist",
"id": "PL123",
"title": "My Playlist",
"extractor": "generic",
"extractor_key": "Generic",
"webpage_url": "https://example.com/playlist?list=PL123",
"entries": [
{"id": "v1", "title": "One", "url": "https://example.com/v1",
"webpage_url": "https://example.com/v1", "_type": "url"},
],
}
def _written_files(root):
found = []
for dirpath, _dirs, files in os.walk(root):
for f in files:
found.append(os.path.relpath(os.path.join(dirpath, f), root))
return sorted(found)
@pytest.mark.asyncio
async def test_channel_feed_metadata_lands_beside_its_items(dq_env):
"""Issues #660/#1040: the feed-level .info.json follows the same template
the items use, so it sits in the channel's own folder rather than in
DOWNLOAD_DIR under yt-dlp's pl_* default name."""
dq_env.YTDL_OPTIONS = {"writeinfojson": True}
dq_env.OUTPUT_TEMPLATE_CHANNEL = "%(channel)s/%(title)s.%(ext)s"
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_CHANNEL_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
result = await dq.add(
"https://example.com/@vanessa/videos", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert result["status"] == "ok"
assert _written_files(dq_env.DOWNLOAD_DIR) == [
os.path.join("Vanessa", "Vanessa - Videos.info.json")
]
@pytest.mark.asyncio
async def test_playlist_feed_metadata_uses_the_playlist_template(dq_env):
dq_env.YTDL_OPTIONS = {"writeinfojson": True}
dq_env.OUTPUT_TEMPLATE_PLAYLIST = "%(playlist_title)s/%(title)s.%(ext)s"
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_PLAYLIST_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
await dq.add(
"https://example.com/playlist?list=PL123", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert _written_files(dq_env.DOWNLOAD_DIR) == [
os.path.join("My Playlist", "My Playlist.info.json")
]
@pytest.mark.asyncio
async def test_feed_metadata_honours_custom_folder(dq_env):
dq_env.YTDL_OPTIONS = {"writeinfojson": True}
dq_env.OUTPUT_TEMPLATE_PLAYLIST = "%(playlist_title)s/%(title)s.%(ext)s"
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_PLAYLIST_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
await dq.add(
"https://example.com/playlist?list=PL123", "video", "auto", "any", "best",
"Music", "", 0, auto_start=False,
)
assert _written_files(dq_env.DOWNLOAD_DIR) == [
os.path.join("Music", "My Playlist", "My Playlist.info.json")
]
@pytest.mark.asyncio
async def test_no_feed_metadata_without_writeinfojson(dq_env):
"""Nothing new appears for users who never asked for these files."""
dq_env.YTDL_OPTIONS = {}
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_PLAYLIST_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
await dq.add(
"https://example.com/playlist?list=PL123", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert _written_files(dq_env.DOWNLOAD_DIR) == []
@pytest.mark.asyncio
async def test_feed_metadata_can_be_turned_off_by_the_user(dq_env):
dq_env.YTDL_OPTIONS = {"writeinfojson": True, "allow_playlist_files": False}
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_PLAYLIST_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()):
await dq.add(
"https://example.com/playlist?list=PL123", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert _written_files(dq_env.DOWNLOAD_DIR) == []
@pytest.mark.asyncio
async def test_feed_metadata_failure_does_not_fail_the_add(dq_env):
dq_env.YTDL_OPTIONS = {"writeinfojson": True}
dq = DownloadQueue(dq_env, AsyncMock())
with patch.object(DownloadQueue, "_DownloadQueue__extract_info", _feed_extract(_PLAYLIST_FEED)), \
patch.object(DownloadQueue, "_DownloadQueue__start_download", new=AsyncMock()), \
patch.object(
DownloadQueue, "_DownloadQueue__write_feed_metadata_sync",
side_effect=OSError("read-only filesystem"),
):
result = await dq.add(
"https://example.com/playlist?list=PL123", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert result["status"] == "ok"
assert dq.pending.exists("https://example.com/v1")
@pytest.mark.asyncio
async def test_extraction_pass_never_writes_feed_metadata(dq_env):
"""The classification pass must not produce files: it runs before the add is
known to succeed, and yt-dlp writes playlist files regardless of `download`."""
dq_env.YTDL_OPTIONS = {"writeinfojson": True, "allow_playlist_files": True}
captured: list = []
class FakeYoutubeDL:
def __init__(self, params=None):
captured.append(params)
def extract_info(self, url, download=False):
return {"_type": "video", "id": "v", "title": "V", "url": url, "webpage_url": url}
dq = DownloadQueue(dq_env, AsyncMock())
with patch("ytdl.yt_dlp.YoutubeDL", FakeYoutubeDL):
await dq.add(
"https://example.com/watch?v=1", "video", "auto", "any", "best",
"", "", 0, auto_start=False,
)
assert captured[0]["allow_playlist_files"] is False
@pytest.mark.asyncio
async def test_add_sets_clip_bounds_on_download_info(dq_env):
notifier = AsyncMock()
def fake_extract(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
def fake_extract(self, url, *_args, **_kwargs):
return {
"_type": "video",
"id": "vid1",
@@ -898,6 +1300,40 @@ def _make_download(dq_env, *, download_type="video", status="downloading", filen
)
def test_download_close_releases_status_queue(dq_env):
download = _make_download(dq_env)
status_queue = MagicMock()
proc = MagicMock()
download.status_queue = status_queue
download.proc = proc
download.close()
proc.close.assert_called_once()
assert download.status_queue is None
def test_download_close_releases_status_queue_without_process(dq_env):
download = _make_download(dq_env)
download.status_queue = MagicMock()
download.close()
assert download.status_queue is None
def test_download_close_releases_status_queue_when_process_close_fails(dq_env):
download = _make_download(dq_env)
download.status_queue = MagicMock()
download.proc = MagicMock()
download.proc.close.side_effect = RuntimeError('close failed')
with pytest.raises(RuntimeError, match='close failed'):
download.close()
assert download.status_queue is None
@pytest.mark.asyncio
async def test_post_download_cleanup_clears_filename_on_error(dq_env):
notifier = AsyncMock()
+119
View File
@@ -0,0 +1,119 @@
"""Tests for conservative audio metadata enrichment."""
from __future__ import annotations
from music_metadata import MusicMetadataPreProcessor
def _preprocess(source_entry, info):
processor = MusicMetadataPreProcessor(source_entry=source_entry)
_, result = processor.run(info)
return result
def test_album_uses_existing_order_and_total_when_track_number_is_missing():
result = _preprocess(
{
'playlist_index': '03',
'playlist_count': 12,
'playlist_title': 'Example Album',
},
{'title': 'Track', 'album': 'Example Album'},
)
assert result['track_number'] == '3/12'
assert result['album'] == 'Example Album'
def test_official_track_number_wins_over_album_order():
result = _preprocess(
{'playlist_index': 3, 'playlist_count': 12},
{'track_number': 7, 'album': 'Official Album'},
)
assert result['track_number'] == '7/12'
assert result['album'] == 'Official Album'
def test_inline_official_track_total_is_preserved():
result = _preprocess(
{'playlist_count': 12},
{'track_number': '4/10', 'album': 'Official Album'},
)
assert result['track_number'] == '4/10'
def test_source_track_number_and_total_are_retained_from_flat_extraction():
result = _preprocess(
{'track_number': 2, 'track_count': 9, 'playlist_index': 4},
{'title': 'Track'},
)
assert result['track_number'] == '2/9'
def test_album_title_falls_back_to_source_playlist_title():
result = _preprocess(
{'playlist_title': 'Example Album'},
{'track_number': 4},
)
assert result['album'] == 'Example Album'
assert result['track_number'] == 4
def test_playlist_without_extractor_album_signals_is_not_changed():
result = _preprocess(
{
'playlist_index': 3,
'playlist_count': 12,
'playlist_title': 'Example Playlist',
},
{'title': 'Track'},
)
assert 'album' not in result
assert 'track_number' not in result
def test_regular_video_artwork_is_not_changed():
thumbnails = [
{'url': 'square.jpg', 'width': 500, 'height': 500},
{'url': 'landscape.jpg', 'width': 1280, 'height': 720},
]
result = _preprocess({}, {'title': 'Regular Video', 'thumbnails': thumbnails.copy()})
assert result['thumbnails'] == thumbnails
assert 'thumbnail' not in result
def test_music_audio_prefers_largest_existing_square_thumbnail():
result = _preprocess(
{},
{
'track': 'Track',
'thumbnails': [
{'url': 'small-square.jpg', 'width': 200, 'height': 200},
{'url': 'large-square.jpg', 'width': 1000, 'height': 1000},
{'url': 'landscape.jpg', 'width': 1280, 'height': 720},
],
},
)
assert result['thumbnails'][-1]['url'] == 'large-square.jpg'
assert result['thumbnail'] == 'large-square.jpg'
def test_landscape_only_music_artwork_keeps_existing_order():
thumbnails = [
{'url': 'small.jpg', 'width': 640, 'height': 360},
{'url': 'large.jpg', 'width': 1280, 'height': 720},
]
result = _preprocess(
{},
{'track': 'Track', 'thumbnails': thumbnails.copy()},
)
assert result['thumbnails'] == thumbnails
assert 'thumbnail' not in result
+15 -3
View File
@@ -146,12 +146,12 @@ class PersistentQueueTests(unittest.TestCase):
self.assertNotIn("formats", record["entry"])
self.assertNotIn("description", record["entry"])
def test_completed_queue_does_not_persist_entry_or_transient_progress(self):
def test_completed_queue_persists_only_failed_retry_context(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "completed")
pq = PersistentQueue("completed", path)
info = _make_info("http://done.example")
info.status = "finished"
info.status = "error"
info.percent = 88
info.speed = 123
info.eta = 9
@@ -167,12 +167,24 @@ class PersistentQueueTests(unittest.TestCase):
payload = json.load(f)
record = payload["items"][0]["info"]
self.assertNotIn("entry", record)
self.assertEqual(
record["entry"],
{
"playlist_index": "01",
"playlist_title": "Playlist",
},
)
self.assertNotIn("percent", record)
self.assertNotIn("speed", record)
self.assertNotIn("eta", record)
self.assertEqual(record["filename"], "done.mp4")
info.status = "finished"
pq.put(_FakeDownload(info))
with open(path + ".json", encoding="utf-8") as f:
payload = json.load(f)
self.assertNotIn("entry", payload["items"][0]["info"])
def test_invalid_json_is_quarantined_and_legacy_is_imported(self):
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "queue")
+108
View File
@@ -821,6 +821,76 @@ class SubscriptionPersistenceTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(upd["subscription"]["title_regex"], "foo|bar")
self.assertEqual(mgr.list_all()[0].title_regex, "foo|bar")
async def _add_one_subscription(self, mgr):
with patch(
"subscriptions.extract_flat_playlist",
return_value=(
{"_type": "channel", "title": "Videos"},
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
),
):
result = await mgr.add_subscription(
"https://example.com/playlist?list=UULFabc",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
)
return result["subscription"]["id"]
async def test_update_subscription_renames(self):
"""Issue #1044: UULF-style uploads playlists all come back named 'Videos',
so the user needs to be able to relabel them."""
with tempfile.TemporaryDirectory() as tmp:
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
sub_id = await self._add_one_subscription(mgr)
self.assertEqual(mgr.list_all()[0].name, "Videos")
upd = await mgr.update_subscription(sub_id, {"name": " Jane's uploads \n"})
self.assertEqual(upd["status"], "ok")
# Surrounding and interior whitespace is collapsed to keep the name
# a single-line label.
self.assertEqual(upd["subscription"]["name"], "Jane's uploads")
self.assertEqual(mgr.list_all()[0].name, "Jane's uploads")
async def test_update_subscription_rename_survives_reload(self):
with tempfile.TemporaryDirectory() as tmp:
cfg = _Config(tmp)
mgr = SubscriptionManager(cfg, _Queue(), _Notifier())
sub_id = await self._add_one_subscription(mgr)
await mgr.update_subscription(sub_id, {"name": "Renamed"})
reloaded = SubscriptionManager(cfg, _Queue(), _Notifier())
self.assertEqual(reloaded.get(sub_id).name, "Renamed")
async def test_update_subscription_rejects_unusable_name(self):
with tempfile.TemporaryDirectory() as tmp:
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
sub_id = await self._add_one_subscription(mgr)
for bad in ("", " ", "\n\t", 42, None, ["a"], "x" * 201):
upd = await mgr.update_subscription(sub_id, {"name": bad})
self.assertEqual(upd["status"], "error", f"expected {bad!r} to be rejected")
self.assertEqual(mgr.list_all()[0].name, "Videos")
async def test_update_subscription_accepts_name_at_length_limit(self):
with tempfile.TemporaryDirectory() as tmp:
mgr = SubscriptionManager(_Config(tmp), _Queue(), _Notifier())
sub_id = await self._add_one_subscription(mgr)
upd = await mgr.update_subscription(sub_id, {"name": "x" * 200})
self.assertEqual(upd["status"], "ok")
self.assertEqual(mgr.list_all()[0].name, "x" * 200)
async def test_update_subscription_skip_subscriber_only(self):
with tempfile.TemporaryDirectory() as tmp:
queue = _Queue()
@@ -1101,6 +1171,44 @@ class SubscriptionScanExtraOptsTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(captured_params[0].get("cookiefile"), "preset.txt")
self.assertEqual(captured_params[0].get("extra"), "override")
async def test_scan_never_writes_playlist_sidecar_files(self):
"""A subscription scan is a metadata probe. yt-dlp writes the
playlist-level infojson/description/thumbnail regardless of ``download``,
so without this a writeinfojson/writethumbnail user would get stray files
in DOWNLOAD_DIR on every check interval. Issue #1040."""
captured_params: list = []
fake_ydl = _make_scan_capturing_fake_ydl(
captured_params,
[{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}],
)
with tempfile.TemporaryDirectory() as tmp:
cfg = _Config(tmp)
cfg.YTDL_OPTIONS = {"writeinfojson": True, "writethumbnail": True}
mgr = SubscriptionManager(cfg, _Queue(), _Notifier())
with patch("subscriptions.yt_dlp.YoutubeDL", fake_ydl, create=True):
await mgr.add_subscription(
"https://example.com/channel",
check_interval_minutes=60,
download_type="video",
codec="auto",
format="any",
quality="best",
folder="",
custom_name_prefix="",
auto_start=True,
playlist_item_limit=0,
split_by_chapters=False,
chapter_template="",
subtitle_language="en",
subtitle_mode="prefer_manual",
ytdl_options_overrides={"allow_playlist_files": True},
)
self.assertTrue(captured_params)
self.assertIs(captured_params[0].get("allow_playlist_files"), False)
async def test_check_now_scan_applies_stored_subscription_presets(self):
entries = [{"id": "v1", "title": "One", "webpage_url": "https://example.com/v1"}]
+89 -3
View File
@@ -6,7 +6,13 @@ import socket
import unittest
from unittest import mock
from url_guard import validate_url
import url_guard
from url_guard import (
validate_url,
_address_allowed_at_connect,
_guarded_getaddrinfo,
install_socket_guard,
)
def _addrinfo(*addrs, family=socket.AF_INET):
@@ -93,9 +99,89 @@ class AddressResolutionTests(unittest.TestCase):
# If any resolved address is internal, reject the whole URL.
self.assertIsNotNone(self._validate_with_addrs("http://mixed/x", "142.250.1.1", "127.0.0.1"))
def test_resolution_failure_defers_to_ytdlp(self):
def test_resolution_failure_is_rejected(self):
# Fail closed: an unresolvable host cannot be verified as non-internal.
with mock.patch("url_guard.socket.getaddrinfo", side_effect=socket.gaierror):
self.assertIsNone(validate_url("http://does-not-resolve.example/x"))
self.assertIsNotNone(validate_url("http://does-not-resolve.example/x"))
class ConnectAddressPolicyTests(unittest.TestCase):
"""Connect-time policy: allow global + loopback, block everything else."""
def test_global_allowed(self):
self.assertTrue(_address_allowed_at_connect("142.250.1.1"))
def test_loopback_allowed(self):
# Loopback stays reachable so locally-configured proxies keep working.
self.assertTrue(_address_allowed_at_connect("127.0.0.1"))
self.assertTrue(_address_allowed_at_connect("::1"))
def test_link_local_metadata_blocked(self):
self.assertFalse(_address_allowed_at_connect("169.254.169.254"))
def test_private_blocked(self):
self.assertFalse(_address_allowed_at_connect("10.0.0.5"))
self.assertFalse(_address_allowed_at_connect("192.168.1.10"))
def test_ipv4_mapped_metadata_blocked(self):
self.assertFalse(_address_allowed_at_connect("::ffff:169.254.169.254"))
class GuardedGetaddrinfoTests(unittest.TestCase):
def test_internal_only_raises(self):
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("169.254.169.254")):
with self.assertRaises(socket.gaierror):
_guarded_getaddrinfo("metadata", 80)
def test_filters_internal_keeps_global(self):
# Split-horizon rebinding: keep the public address, drop the internal one.
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("142.250.1.1", "10.0.0.1")):
results = _guarded_getaddrinfo("mixed", 80)
self.assertEqual([r[4][0] for r in results], ["142.250.1.1"])
def test_loopback_passes(self):
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("127.0.0.1")):
results = _guarded_getaddrinfo("localproxy", 9050)
self.assertEqual([r[4][0] for r in results], ["127.0.0.1"])
class AllowPrivateBypassTests(unittest.TestCase):
"""ALLOW_PRIVATE_ADDRESSES: trusted proxy/VPN environments opt out of the
SSRF address checks (e.g. Fake-IP clients that resolve to 198.18.0.0/15)."""
def test_internal_address_allowed_when_bypassed(self):
# Fake-IP benchmarking range that is_global rejects by default.
self.assertIsNone(validate_url("http://www.youtube.com/x", allow_private=True))
def test_private_host_allowed_when_bypassed(self):
# No DNS lookup needed: the bypass returns before resolution.
with mock.patch("url_guard.socket.getaddrinfo") as gai:
self.assertIsNone(validate_url("http://192.168.1.1/x", allow_private=True))
gai.assert_not_called()
def test_scheme_still_enforced_when_bypassed(self):
self.assertIsNotNone(validate_url("file:///etc/passwd", allow_private=True))
def test_socket_guard_not_installed_when_bypassed(self):
original = socket.getaddrinfo
try:
install_socket_guard(allow_private=True)
self.assertIs(socket.getaddrinfo, original)
finally:
socket.getaddrinfo = original
class InstallSocketGuardTests(unittest.TestCase):
def test_install_replaces_and_is_idempotent(self):
original = socket.getaddrinfo
try:
install_socket_guard()
self.assertIs(socket.getaddrinfo, url_guard._guarded_getaddrinfo)
# Re-installing must not wrap the wrapper (real fn captured at import).
install_socket_guard()
self.assertIs(socket.getaddrinfo, url_guard._guarded_getaddrinfo)
finally:
socket.getaddrinfo = original
if __name__ == "__main__":
+95 -15
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import os
import pickle
import signal
import sys
@@ -31,6 +32,25 @@ class _PostProcessor:
self._downloader = downloader
class _YoutubeDL:
"""Minimal stand-in so ``_ConfinedYoutubeDL`` can subclass it under the shim.
``prepare_filename`` is patched per-test; the real containment logic lives in
the ``_ConfinedYoutubeDL`` override, which is what the tests exercise.
"""
def __init__(self, params=None, **kwargs):
self.params = params or {}
def prepare_filename(self, *args, **kwargs):
return ""
def add_post_processor(self, *args, **kwargs):
pass
fake_utils.DownloadError = type("DownloadError", (Exception,), {})
fake_yt_dlp.YoutubeDL = _YoutubeDL
fake_impersonate.ImpersonateTarget = _ImpersonateTarget
fake_networking.impersonate = fake_impersonate
fake_postprocessor_common.PostProcessor = _PostProcessor
@@ -53,18 +73,22 @@ import ytdl
from ytdl import (
Download,
DownloadInfo,
MusicMetadataPreProcessor,
_compact_persisted_entry,
_convert_srt_to_txt_file,
_AlbumArtistPostProcessor,
_output_dir_escapes,
_resolve_outtmpl_fields,
_sanitize_entry_for_pickle,
_sanitize_path_component,
_short_title_for_failed_url,
)
# Detect whether the real yt-dlp is loaded (as opposed to the minimal fake
# shim above). _resolve_outtmpl_fields needs YoutubeDL at runtime.
_has_real_ytdlp = hasattr(sys.modules.get("yt_dlp"), "YoutubeDL")
# shim above). _resolve_outtmpl_fields needs YoutubeDL.evaluate_outtmpl at
# runtime, which the shim's YoutubeDL stand-in deliberately does not provide.
_has_real_ytdlp = hasattr(
getattr(sys.modules.get("yt_dlp"), "YoutubeDL", None), "evaluate_outtmpl"
)
class AlbumArtistPostProcessorTests(unittest.TestCase):
@@ -175,19 +199,25 @@ class AlbumArtistRegistrationTests(unittest.TestCase):
download.info.download_type = 'audio'
fake_ydl = MagicMock()
with patch('ytdl.yt_dlp.YoutubeDL', return_value=fake_ydl):
with patch('ytdl._ConfinedYoutubeDL', return_value=fake_ydl):
result = download._make_youtube_dl({'quiet': True})
self.assertIs(result, fake_ydl)
postprocessor, = fake_ydl.add_post_processor.call_args.args
album_artist_call = fake_ydl.add_post_processor.call_args_list[0]
postprocessor, = album_artist_call.args
self.assertIsInstance(postprocessor, _AlbumArtistPostProcessor)
self.assertEqual(fake_ydl.add_post_processor.call_args.kwargs, {'when': 'pre_process'})
self.assertEqual(album_artist_call.kwargs, {'when': 'pre_process'})
metadata_pre_call = fake_ydl.add_post_processor.call_args_list[1]
metadata_preprocessor, = metadata_pre_call.args
self.assertIsInstance(metadata_preprocessor, MusicMetadataPreProcessor)
self.assertEqual(metadata_pre_call.kwargs, {'when': 'pre_process'})
self.assertEqual(fake_ydl.add_post_processor.call_count, 2)
def test_video_download_does_not_register_postprocessor(self):
download = _make_test_download()
fake_ydl = MagicMock()
with patch('ytdl.yt_dlp.YoutubeDL', return_value=fake_ydl):
with patch('ytdl._ConfinedYoutubeDL', return_value=fake_ydl):
download._make_youtube_dl({'quiet': True})
fake_ydl.add_post_processor.assert_not_called()
@@ -301,18 +331,50 @@ class ResolveOuttmplFieldsTests(unittest.TestCase):
self.assertFalse(literal_prefix.startswith('\\'))
class OutputDirEscapesTests(unittest.TestCase):
class ConfinedYoutubeDLTests(unittest.TestCase):
"""The chokepoint: ``_ConfinedYoutubeDL.prepare_filename`` validates the
*resolved* output path (after yt-dlp expands the template) and refuses any
write outside the allowed roots. This is the single guard for the download-
directory invariant across the main file, split-chapter files, thumbnails,
subtitles, etc. the ``..`` only exists post-expansion, so it is caught here
rather than by any ingress string check.
"""
def setUp(self):
self.base_dir = tempfile.mkdtemp()
self.base = os.path.realpath(tempfile.mkdtemp())
def test_relative_traversal_escapes(self):
self.assertTrue(_output_dir_escapes(self.base_dir, '../../tmp/x/%(title)s.%(ext)s'))
def _prepared_path(self, resolved, roots=None):
ydl = ytdl._ConfinedYoutubeDL.__new__(ytdl._ConfinedYoutubeDL)
ydl._allowed_roots = [self.base] if roots is None else roots
with patch.object(
ytdl.yt_dlp.YoutubeDL, "prepare_filename", return_value=resolved
):
return ydl.prepare_filename({})
def test_absolute_path_escapes(self):
self.assertTrue(_output_dir_escapes(self.base_dir, '/tmp/x/%(title)s.%(ext)s'))
def test_chapter_traversal_via_metadata_is_blocked(self):
# e.g. chapter_template '%(section_title)s/%(section_title)s/pwned.%(ext)s'
# with a chapter titled '..' expands to '../../pwned.mp4'.
escaping = os.path.join(self.base, "..", "..", "pwned.mp4")
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
self._prepared_path(escaping)
def test_normal_playlist_dir_stays_inside(self):
self.assertFalse(_output_dir_escapes(self.base_dir, 'Playlist/%(title)s.%(ext)s'))
def test_absolute_output_path_is_blocked(self):
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
self._prepared_path("/etc/cron.d/evil")
def test_path_inside_download_dir_is_allowed(self):
ok = os.path.join(self.base, "Playlist", "video.mp4")
self.assertEqual(self._prepared_path(ok), ok)
def test_sibling_prefix_directory_is_blocked(self):
# base '/x/downloads' must not be escapable to '/x/downloads-secret'.
sibling = self.base + "-secret"
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
self._prepared_path(os.path.join(sibling, "video.mp4"))
def test_empty_and_stdout_targets_pass_through(self):
self.assertEqual(self._prepared_path(""), "")
self.assertEqual(self._prepared_path("-"), "-")
class SanitizeEntryForPickleTests(unittest.TestCase):
@@ -747,5 +809,23 @@ class CompactPersistedEntryTests(unittest.TestCase):
self.assertIsNone(_compact_persisted_entry({"id": "x", "title": "y"}))
class ShortTitleForFailedUrlTests(unittest.TestCase):
def test_uses_hostname_for_a_normal_url(self):
self.assertEqual(
_short_title_for_failed_url("https://example.com/watch?v=1"),
"example.com",
)
def test_falls_back_to_raw_value_when_there_is_no_hostname(self):
# file:// URIs and bare search terms/video IDs have no netloc to extract.
self.assertEqual(_short_title_for_failed_url("file:///etc/passwd"), "file:///etc/passwd")
self.assertEqual(_short_title_for_failed_url("ytsearch:some query"), "ytsearch:some query")
def test_falls_back_to_raw_value_on_unparseable_input(self):
# A malformed IPv6-looking host raises ValueError in urlsplit().hostname.
malformed = "https://[::1/watch"
self.assertEqual(_short_title_for_failed_url(malformed), malformed)
if __name__ == "__main__":
unittest.main()
+89 -12
View File
@@ -5,10 +5,26 @@ any ``http(s)`` URL. Without a guard, an attacker can make the server fetch
internal endpoints (cloud metadata services, loopback, RFC1918 hosts, etc.) and
have the response saved to the download directory and served back.
This module provides a single cheap validator applied at every URL ingress. It
intentionally does NOT attempt DNS-rebinding pinning, redirect-chain
re-validation, or validation of every media URL yt-dlp derives from remote
metadata network isolation (e.g. Docker) remains the backstop for those.
This module provides two layers:
* ``validate_url`` a cheap validator applied at every URL ingress.
* ``install_socket_guard`` a connect-time ``getaddrinfo`` guard installed in
the download subprocess, which re-validates every resolved address and so
covers redirects, DNS rebinding, and media URLs yt-dlp derives from remote
metadata for any backend that resolves through Python's socket module.
Known limitations network isolation (e.g. Docker) remains the backstop for
all of these:
* The socket guard is installed only in the download subprocess. Metadata
extraction (``ytdl.DownloadQueue.__extract_info``) runs in the main process,
where installing a process-wide guard would reject the server's own bind on
``HOST=0.0.0.0``. So extraction which also follows redirects is covered
only by ``validate_url`` at ingress, not at connect time; a redirect from an
allowed host to an internal one during extraction is not blocked (a lower-
impact, blind SSRF, since the extraction response is not written to disk).
* Native resolvers (curl_cffi/libcurl via ``--impersonate``) resolve outside
Python's socket module and bypass the connect-time guard entirely.
"""
import ipaddress
@@ -34,24 +50,79 @@ def _hostname_is_blocked(hostname: str) -> bool:
return False
def _address_is_global(addr: str) -> bool:
def _normalise_ip(addr: str):
"""Parse *addr*, unwrapping IPv4-mapped IPv6 (e.g. ``::ffff:169.254.169.254``)
so the embedded IPv4 address is judged on its own merits. Returns ``None``
when *addr* is not a valid IP literal."""
try:
ip = ipaddress.ip_address(addr)
except ValueError:
return False
# Unwrap IPv4-mapped/compatible IPv6 (e.g. ::ffff:169.254.169.254) so the
# embedded IPv4 address is judged on its own merits.
return None
if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
ip = ip.ipv4_mapped
return ip.is_global
return ip
def validate_url(url: str) -> str | None:
def _address_is_global(addr: str) -> bool:
ip = _normalise_ip(addr)
return ip is not None and ip.is_global
def _address_allowed_at_connect(addr: str) -> bool:
"""True if *addr* may be connected to at download time.
Permits global addresses and loopback loopback so that locally-configured
proxies (e.g. ``proxy: http://127.0.0.1:9050``) keep working. Blocks the SSRF
targets that matter: link-local (cloud metadata at 169.254.169.254), private
(RFC1918), unique-local and every other non-global, non-loopback range.
"""
ip = _normalise_ip(addr)
return ip is not None and (ip.is_global or ip.is_loopback)
# Captured at import so re-installing the guard never wraps the wrapper.
_real_getaddrinfo = socket.getaddrinfo
def _guarded_getaddrinfo(host, *args, **kwargs):
results = _real_getaddrinfo(host, *args, **kwargs)
allowed = [r for r in results if _address_allowed_at_connect(r[4][0])]
if not allowed:
raise socket.gaierror(f'Refusing to connect to non-global address for host {host!r}')
return allowed
def install_socket_guard(allow_private: bool = False) -> None:
"""Enforce the no-internal-hosts policy at actual connection time.
``validate_url`` only checks the *submitted* URL string; yt-dlp then follows
HTTP redirects and resolves media URLs from remote metadata without
re-validating them. Installing this in the download subprocess re-checks
every resolved address at connect time, covering redirects and DNS rebinding
for any networking backend that resolves through Python's socket module
(urllib, requests). Native resolvers notably curl_cffi/libcurl used by
``--impersonate`` bypass this and rely on network isolation as the backstop.
When *allow_private* is set (``ALLOW_PRIVATE_ADDRESSES``), the guard is not
installed at all, so proxy/VPN setups that route through private or Fake-IP
ranges keep working.
"""
if allow_private:
return
socket.getaddrinfo = _guarded_getaddrinfo
def validate_url(url: str, allow_private: bool = False) -> str | None:
"""Return an error message if the URL is disallowed, else ``None``.
Inputs without a ``://`` scheme separator (bare video IDs, ``ytsearch:``
and other yt-dlp search/extractor prefixes) are allowed unchanged so that
non-URL entries keep working.
When *allow_private* is set (``ALLOW_PRIVATE_ADDRESSES``), the internal-host
and internal-address checks are skipped so that trusted proxy/VPN setups
e.g. Fake-IP clients that resolve YouTube to ``198.18.0.0/15`` can be used.
Scheme validation (http/https only) still applies.
"""
if not isinstance(url, str):
return 'Invalid URL'
@@ -70,14 +141,20 @@ def validate_url(url: str) -> str | None:
if not hostname:
return 'URL is missing a host'
if allow_private:
# Environment is explicitly trusted: skip the SSRF address checks.
return None
if _hostname_is_blocked(hostname):
return f'Refusing to fetch internal host "{hostname}"'
try:
addrinfo = socket.getaddrinfo(hostname, parts.port, proto=socket.IPPROTO_TCP)
except socket.gaierror:
# Let yt-dlp surface a normal resolution error rather than masking it.
return None
# Fail closed: a host we cannot resolve is a host we cannot verify as
# non-internal, so refuse it rather than letting the download proceed
# to a target that may resolve differently at fetch time.
return f'Could not resolve host "{hostname}"'
except (UnicodeError, ValueError):
return f'Invalid host "{hostname}"'
+286 -26
View File
@@ -23,10 +23,12 @@ from yt_dlp.postprocessor.common import PostProcessor
from yt_dlp.utils import STR_FORMAT_RE_TMPL, STR_FORMAT_TYPES
import bg_tasks
from dl_formats import get_format, get_opts, AUDIO_FORMATS, merge_ytdl_option_layers
from music_metadata import MusicMetadataPreProcessor
from datetime import datetime
from state_store import AtomicJsonStore, from_json_compatible, read_legacy_shelf, to_json_compatible
from subscriptions import _entry_id
from url_guard import validate_url
from url_guard import validate_url, install_socket_guard
from urllib.parse import urlsplit
log = logging.getLogger('ytdl')
@@ -145,16 +147,36 @@ def _sanitize_path_component(value: Any) -> Any:
return value.lstrip('.').strip() or '_'
def _output_dir_escapes(base_dir: str, output_template: str) -> bool:
"""True when the literal directory prefix of *output_template* resolves outside *base_dir*."""
marker = output_template.find('%(')
literal = output_template if marker == -1 else output_template[:marker]
dir_prefix = os.path.dirname(literal)
if not dir_prefix:
return False
real_base = os.path.realpath(base_dir)
real_target = os.path.realpath(os.path.join(base_dir, dir_prefix))
return not _is_within_directory(real_base, real_target)
class _ConfinedYoutubeDL(yt_dlp.YoutubeDL):
"""A ``YoutubeDL`` that refuses to emit any output path outside the allowed roots.
This is the single authoritative enforcement of MeTube's download-directory
containment invariant. yt-dlp expands output templates at download time using
metadata that is fully attacker-controlled (``%(title)s``, ``%(uploader)s``,
``%(section_title)s`` from chapter titles, ) and, on POSIX hosts, does *not*
neutralise a ``..`` path component so any template segment resolving to
``..`` next to a literal separator (or an absolute template) can traverse out
of the download directory. Every output path main file, split-chapter files,
thumbnails, subtitles, infojson is produced by ``prepare_filename``, so
validating its result here covers them all, regardless of which template or
metadata field carries the traversal. Checking the resolved path (rather than
the template string) is what makes this robust: the ``..`` only exists after
expansion, so no ingress string check can see it.
"""
def __init__(self, params=None, *, allowed_roots=(), **kwargs):
self._allowed_roots = [os.path.realpath(r) for r in allowed_roots if r]
super().__init__(params=params, **kwargs)
def prepare_filename(self, *args, **kwargs):
filename = super().prepare_filename(*args, **kwargs)
if filename and filename != '-' and self._allowed_roots:
resolved = os.path.realpath(filename)
if not any(_is_within_directory(root, resolved) for root in self._allowed_roots):
raise yt_dlp.utils.DownloadError(
f'Refusing to write outside the download directory: {filename}'
)
return filename
# Regex matching yt-dlp output-template field references, e.g. ``%(title)s``
@@ -477,7 +499,18 @@ _PERSISTED_DOWNLOAD_FIELDS = (
)
_COMPACT_ENTRY_EXTRA_KEYS = frozenset(("n_entries", "__last_playlist_index"))
def _short_title_for_failed_url(url: str) -> str:
"""A concise display title for a URL that failed before yt-dlp could extract a
real title (unsupported URL, SSRF-rejected, extraction error). The full URL
remains available in DownloadInfo.url and the error-detail panel."""
try:
hostname = urlsplit(url).hostname
except ValueError:
hostname = None
return hostname or url
_COMPACT_ENTRY_EXTRA_KEYS = frozenset(("n_entries", "__last_playlist_index", "track_number"))
def _compact_persisted_entry(entry: Any) -> Optional[dict[str, Any]]:
@@ -537,11 +570,12 @@ class Download:
cls.manager.shutdown()
cls.manager = None
def __init__(self, download_dir, temp_dir, output_template, output_template_chapter, quality, format, ytdl_opts, info):
def __init__(self, download_dir, temp_dir, output_template, output_template_chapter, quality, format, ytdl_opts, info, allow_private=False):
self.download_dir = download_dir
self.temp_dir = temp_dir
self.output_template = output_template
self.output_template_chapter = output_template_chapter
self.allow_private = allow_private
self.info = info
self.format = get_format(
getattr(info, 'download_type', 'video'),
@@ -598,9 +632,19 @@ class Download:
return put_status
def _make_youtube_dl(self, params):
ydl = yt_dlp.YoutubeDL(params=params)
ydl = _ConfinedYoutubeDL(
params=params,
allowed_roots=(self.download_dir, self.temp_dir),
)
if getattr(self.info, 'download_type', '') == 'audio':
ydl.add_post_processor(_AlbumArtistPostProcessor(ydl), when='pre_process')
ydl.add_post_processor(
MusicMetadataPreProcessor(
ydl,
source_entry=getattr(self.info, 'entry', None),
),
when='pre_process',
)
return ydl
def _download(self):
@@ -612,6 +656,11 @@ class Download:
os.setpgrp()
except OSError:
pass
# Re-validate every outbound connection at fetch time. validate_url only
# saw the submitted URL string; this catches redirects and DNS rebinding
# to internal hosts (cloud metadata, RFC1918) that it cannot. Skipped when
# ALLOW_PRIVATE_ADDRESSES trusts the environment (e.g. Fake-IP proxies).
install_socket_guard(self.allow_private)
log.info(f"Starting download for: {self.info.title} ({self.info.url})")
try:
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
@@ -758,8 +807,11 @@ class Download:
def close(self):
log.info(f"Closing download process for: {self.info.title}")
if self.started():
self.proc.close()
try:
if self.started():
self.proc.close()
finally:
self.status_queue = None
def running(self):
try:
@@ -892,8 +944,12 @@ class PersistentQueue:
]
return sorted(items, key=lambda item: item[1].timestamp)
def _should_persist_entry(self) -> bool:
return self.identifier != "completed"
def _should_persist_entry(self, info: DownloadInfo | dict[str, Any]) -> bool:
# Failed downloads need their compact playlist/channel context so a
# retry after a server restart still resolves the original outtmpl.
# Successful completed entries continue to omit extractor metadata.
status = info.get("status") if isinstance(info, dict) else info.status
return self.identifier != "completed" or status == "error"
def _serialize_items(self):
return [
@@ -901,7 +957,7 @@ class PersistentQueue:
"key": key,
"info": _download_info_to_record(
download.info,
include_entry=self._should_persist_entry(),
include_entry=self._should_persist_entry(download.info),
),
}
for key, download in self.dict.items()
@@ -920,7 +976,7 @@ class PersistentQueue:
"key": item["key"],
"info": _download_info_to_record(
_download_info_from_record(item["info"]),
include_entry=self._should_persist_entry(),
include_entry=self._should_persist_entry(item["info"]),
),
}
for item in items
@@ -941,7 +997,7 @@ class PersistentQueue:
"key": key,
"info": _download_info_to_record(
value,
include_entry=self._should_persist_entry(),
include_entry=self._should_persist_entry(value),
),
}
for key, value in sorted(legacy_items, key=lambda item: item[1].timestamp)
@@ -1242,6 +1298,11 @@ class DownloadQueue:
return opts
def __extract_info(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
# NOTE: extraction runs in the main process, so the connect-time socket
# guard (installed only in the download subprocess) does not apply here.
# The ingress validate_url check guards the submitted URL, but redirects
# followed during extraction are not re-validated. See url_guard's module
# docstring for why the guard can't be installed process-wide.
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
user_opts = self._build_ytdl_options(ytdl_options_presets, ytdl_options_overrides)
params = {
@@ -1253,6 +1314,14 @@ class DownloadQueue:
'ignore_no_formats_error': True,
'noplaylist': True,
'paths': {"home": self.config.DOWNLOAD_DIR, "temp": self.config.TEMP_DIR},
# This is a classification pass, not a download. yt-dlp emits the
# feed-level infojson/description/thumbnail from
# __process_playlist_result without consulting `download`, so
# without this a writeinfojson user gets stray files here — in
# DOWNLOAD_DIR, under yt-dlp's pl_* names, even for an add that goes
# on to fail. __write_feed_metadata writes them properly once the
# feed is accepted. See issues #1040 and #660.
'allow_playlist_files': False,
}
imp = user_opts.get('impersonate')
if imp is not None:
@@ -1301,9 +1370,7 @@ class DownloadQueue:
if playlist_item_limit > 0:
log.info(f'playlist limit is set. Processing only first {playlist_item_limit} entries')
ytdl_options['playlistend'] = playlist_item_limit
if _output_dir_escapes(dldirectory, output):
return {'status': 'error', 'msg': 'Refusing download: resolved output path escapes the download directory'}
download = Download(dldirectory, self.config.TEMP_DIR, output, output_chapter, dl.quality, dl.format, ytdl_options, dl)
download = Download(dldirectory, self.config.TEMP_DIR, output, output_chapter, dl.quality, dl.format, ytdl_options, dl, allow_private=self.config.ALLOW_PRIVATE_ADDRESSES)
is_upcoming = (
getattr(dl, 'live_status', None) == 'is_upcoming'
or getattr(dl, 'status', None) == 'scheduled'
@@ -1318,6 +1385,81 @@ class DownloadQueue:
self.pending.put(download)
await self.notifier.added(dl)
def __write_feed_metadata_sync(self, entry, etype, download_type, folder,
ytdl_options_presets, ytdl_options_overrides):
"""Write the feed-level .info.json/description/thumbnail for a playlist
or channel add, using the same output template its items will use.
yt-dlp produces these from __process_playlist_result, which ignores
``download`` so they used to fall out of the classification pass with
yt-dlp's own pl_* names, in DOWNLOAD_DIR, ignoring the download's folder
(issue #1040) and with no way to steer them (issue #660). Doing it here
instead means the feed type is already known, so the file lands beside
the items rather than in a differently-named sibling directory.
Re-runs yt-dlp on a copy of the feed with no entries: that reaches the
playlist-file writing without re-extracting anything or touching
yt-dlp's private write helpers.
"""
user_opts = self._build_ytdl_options(ytdl_options_presets, ytdl_options_overrides)
wants = ('writeinfojson', 'writedescription', 'writethumbnail', 'write_all_thumbnails')
if not any(user_opts.get(key) for key in wants):
return
# An explicit allow_playlist_files=false is the user asking for exactly
# this to not happen.
if user_opts.get('allow_playlist_files') is False:
return
dldirectory, error_message = self.__calc_download_path(download_type, folder)
if error_message is not None:
return
template = (
self.config.OUTPUT_TEMPLATE_CHANNEL if etype == 'channel'
else self.config.OUTPUT_TEMPLATE_PLAYLIST
) or self.config.OUTPUT_TEMPLATE
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
params = {
**user_opts,
'quiet': not debug_logging,
'verbose': debug_logging,
'no_color': True,
'skip_download': True,
'extract_flat': True,
'allow_playlist_files': True,
'paths': {"home": dldirectory, "temp": self.config.TEMP_DIR},
# Feed-level keys only; per-item names are resolved by __add_download.
'outtmpl': {
'pl_infojson': template,
'pl_thumbnail': template,
'pl_description': template,
},
}
imp = user_opts.get('impersonate')
if imp is not None:
params['impersonate'] = yt_dlp.networking.impersonate.ImpersonateTarget.from_str(imp)
# A copy: process_ie_result mutates entries/requested_entries, and the
# caller still needs the real feed dict to queue the items.
feed = {k: v for k, v in entry.items() if k != 'entries'}
feed['entries'] = []
yt_dlp.YoutubeDL(params=params).process_ie_result(feed, download=False)
async def __write_feed_metadata(self, entry, etype, download_type, folder,
ytdl_options_presets, ytdl_options_overrides):
try:
await asyncio.get_running_loop().run_in_executor(
None,
partial(
self.__write_feed_metadata_sync, entry, etype, download_type, folder,
ytdl_options_presets, ytdl_options_overrides,
),
)
except Exception as exc:
# Supplemental output must never fail the add.
log.warning(f'Could not write {etype} metadata files: {exc}')
async def __add_entry(
self,
entry,
@@ -1339,6 +1481,7 @@ class DownloadQueue:
clip_end,
already,
_add_gen=None,
retry_entry=None,
):
if not entry:
return {'status': 'error', 'msg': "Invalid/empty data was given."}
@@ -1357,6 +1500,10 @@ class DownloadQueue:
if etype.startswith('url'):
log.debug('Processing as a url')
# retry_entry must ride along: extraction can hand back an
# unprocessed url/url_transparent result, and dropping the retry
# context here would send the retried item back to the root
# directory instead of its original playlist folder.
return await self.add(
entry['url'],
download_type,
@@ -1377,6 +1524,7 @@ class DownloadQueue:
clip_end,
already,
_add_gen,
retry_entry,
)
elif etype == 'playlist' or etype == 'channel':
if etype == 'playlist' and self.__is_channel_extraction(entry):
@@ -1388,6 +1536,10 @@ class DownloadQueue:
entries = list(entries)
total_entries = len(entries)
log.info(f'{etype} detected with {total_entries} entries')
await self.__write_feed_metadata(
entry, etype, download_type, folder,
ytdl_options_presets, ytdl_options_overrides,
)
index_digits = len(str(total_entries))
results = []
if playlist_item_limit > 0:
@@ -1485,6 +1637,59 @@ class DownloadQueue:
return {'status': 'ok'}
return {'status': 'error', 'msg': f'Unsupported resource "{etype}"'}
async def __record_add_failure(
self,
url,
msg,
download_type,
codec,
format,
quality,
folder,
custom_name_prefix,
playlist_item_limit,
split_by_chapters,
chapter_template,
subtitle_language,
subtitle_mode,
ytdl_options_presets,
ytdl_options_overrides,
clip_start,
clip_end,
entry=None,
):
"""Surface a URL that failed before a DownloadInfo could be created (unsupported
URL, SSRF-rejected, extraction error) as a failed entry in the done list, so the
frontend shows it with the same red-cross/retry/error-detail treatment as a
download that failed mid-stream, instead of only a toast and a server log line."""
info = DownloadInfo(
id=url,
title=_short_title_for_failed_url(url),
url=url,
quality=quality,
download_type=download_type,
codec=codec,
format=format,
folder=folder,
custom_name_prefix=custom_name_prefix,
error=msg,
entry=entry,
playlist_item_limit=playlist_item_limit,
split_by_chapters=split_by_chapters,
chapter_template=chapter_template,
subtitle_language=subtitle_language,
subtitle_mode=subtitle_mode,
ytdl_options_presets=ytdl_options_presets,
ytdl_options_overrides=ytdl_options_overrides,
clip_start=clip_start,
clip_end=clip_end,
)
info.status = 'error'
info.msg = msg
download = Download(None, None, None, None, quality, format, {}, info)
self.done.put(download)
await self.notifier.completed(info)
async def add(
self,
url,
@@ -1506,6 +1711,7 @@ class DownloadQueue:
clip_end=None,
already=None,
_add_gen=None,
retry_entry=None,
):
if ytdl_options_presets is None:
ytdl_options_presets = []
@@ -1526,9 +1732,16 @@ class DownloadQueue:
# SSRF guard: reject non-http(s) schemes and hosts resolving to
# internal/loopback/link-local/metadata addresses before yt-dlp fetches
# anything. run_in_executor because validate_url may perform a DNS lookup.
url_error = await asyncio.get_running_loop().run_in_executor(None, validate_url, url)
url_error = await asyncio.get_running_loop().run_in_executor(
None, partial(validate_url, url, allow_private=self.config.ALLOW_PRIVATE_ADDRESSES))
if url_error is not None:
log.warning('Rejected URL "%s": %s', url, url_error)
await self.__record_add_failure(
url, url_error, download_type, codec, format, quality, folder,
custom_name_prefix, playlist_item_limit, split_by_chapters, chapter_template,
subtitle_language, subtitle_mode, ytdl_options_presets, ytdl_options_overrides,
clip_start, clip_end, retry_entry,
)
return {'status': 'error', 'msg': url_error}
try:
entry = await asyncio.get_running_loop().run_in_executor(
@@ -1536,7 +1749,17 @@ class DownloadQueue:
partial(self.__extract_info, url, ytdl_options_presets, ytdl_options_overrides),
)
except yt_dlp.utils.YoutubeDLError as exc:
return {'status': 'error', 'msg': str(exc)}
msg = str(exc)
await self.__record_add_failure(
url, msg, download_type, codec, format, quality, folder,
custom_name_prefix, playlist_item_limit, split_by_chapters, chapter_template,
subtitle_language, subtitle_mode, ytdl_options_presets, ytdl_options_overrides,
clip_start, clip_end, retry_entry,
)
return {'status': 'error', 'msg': msg}
retry_context = _compact_persisted_entry(retry_entry)
if isinstance(entry, dict) and retry_context is not None:
entry = {**entry, **copy.deepcopy(retry_context)}
return await self.__add_entry(
entry,
download_type,
@@ -1557,6 +1780,43 @@ class DownloadQueue:
clip_end,
already,
_add_gen,
retry_entry,
)
async def retry(self, id):
if not self.done.exists(id):
return {'status': 'error', 'msg': 'Failed download no longer exists.'}
info = self.done.get(id).info
if info.status != 'error':
return {'status': 'error', 'msg': 'Only failed downloads can be retried.'}
# The stored options were validated by parse_download_options when the
# download was first submitted, but the configuration can have changed
# since. Re-apply the same gates here so a retry can't resurrect
# overrides or presets the current configuration no longer allows.
overrides = info.ytdl_options_overrides if self.config.ALLOW_YTDL_OPTIONS_OVERRIDES else {}
presets = [p for p in info.ytdl_options_presets if p in self.config.YTDL_OPTIONS_PRESETS]
return await self.add(
info.url,
info.download_type,
info.codec,
info.format,
info.quality,
info.folder,
info.custom_name_prefix,
info.playlist_item_limit,
True,
info.split_by_chapters,
info.chapter_template,
info.subtitle_language,
info.subtitle_mode,
presets,
overrides,
info.clip_start,
info.clip_end,
retry_entry=info.entry,
)
async def add_entry(
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 885 KiB

After

Width:  |  Height:  |  Size: 1.9 MiB

+27 -1
View File
@@ -958,7 +958,33 @@
[disabled]="downloads.loading"
[attr.aria-label]="'Select subscription ' + entry[1].name" />
</td>
<td>{{ entry[1].name }}</td>
<td>
@if (editingNameId === entry[0]) {
<div class="d-flex flex-wrap gap-1 align-items-center">
<input type="text"
class="form-control form-control-sm flex-grow-1"
[name]="'subName' + entry[0]"
[(ngModel)]="nameEditDraft"
[maxlength]="subscriptionNameMaxLength"
[disabled]="downloads.loading"
[attr.aria-label]="'Subscription name for ' + entry[1].name" />
<button type="button" class="btn btn-sm btn-outline-secondary"
(click)="saveName(entry[0])"
[disabled]="downloads.loading">Save</button>
<button type="button" class="btn btn-sm btn-outline-secondary"
(click)="cancelEditName()"
[disabled]="downloads.loading">Cancel</button>
</div>
} @else {
<div class="d-flex flex-wrap gap-1 align-items-center">
<span class="text-break">{{ entry[1].name }}</span>
<button type="button" class="btn btn-link btn-sm p-0"
(click)="beginEditName(entry[0], entry[1].name)"
[disabled]="downloads.loading"
ngbTooltip="Rename this subscription (display name only; does not affect the download folder)">Edit</button>
</div>
}
</td>
<td class="text-break"><a [href]="entry[1].url" target="_blank" rel="noopener">{{ entry[1].url }}</a></td>
<td>
@if (editingTitleRegexId === entry[0]) {
+70 -1
View File
@@ -19,6 +19,7 @@ class DownloadsServiceStub {
customDirsChanged = new Subject<Record<string, string[]>>();
ytdlOptionsChanged = new Subject<Record<string, unknown>>();
updated = new Subject<void>();
retryCalls: string[] = [];
getCookieStatus() {
return of({ status: 'ok', has_cookies: false });
@@ -32,6 +33,11 @@ class DownloadsServiceStub {
return of({ status: 'ok' as const });
}
retry(id: string) {
this.retryCalls.push(id);
return of({ status: 'ok' as const });
}
cancelAdd() {
return of({ status: 'ok' as const });
}
@@ -75,7 +81,10 @@ class SubscriptionsServiceStub {
return of({});
}
update() {
updateCalls: [string, unknown][] = [];
update(id: string, changes: unknown) {
this.updateCalls.push([id, changes]);
return of({ status: 'ok' as const });
}
@@ -269,6 +278,33 @@ describe('App', () => {
expect(payload.clipEnd).toBe('1:20');
});
it('retries a failed download by its server-side queue id', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const download = {
id: 'vid1',
title: 'Test Video',
url: 'https://example.com/v',
download_type: 'video',
quality: 'best',
format: 'any',
folder: '',
custom_name_prefix: '',
playlist_item_limit: 0,
status: 'error',
msg: 'temporary failure',
percent: 0,
speed: 0,
eta: 0,
filename: '',
checked: false,
};
app.retryDownload(download.url, download);
expect(downloads.retryCalls).toEqual([download.url]);
});
it('blocks subscribe with invalid title regex', () => {
const toasts = TestBed.inject(ToastService);
const errorSpy = vi.spyOn(toasts, 'error').mockImplementation(() => undefined);
@@ -282,4 +318,37 @@ describe('App', () => {
expect(errorSpy).toHaveBeenCalledWith('Invalid subscription title filter (regex)');
errorSpy.mockRestore();
});
it('renames a subscription and closes the inline editor', () => {
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.beginEditName('sub1', 'Videos');
expect(app.editingNameId).toBe('sub1');
expect(app.nameEditDraft).toBe('Videos');
app.nameEditDraft = ' Jane uploads ';
app.saveName('sub1');
expect(subs.updateCalls).toEqual([['sub1', { name: 'Jane uploads' }]]);
expect(app.editingNameId).toBeNull();
});
it('blocks renaming a subscription to an empty name', () => {
const toasts = TestBed.inject(ToastService);
const errorSpy = vi.spyOn(toasts, 'error').mockImplementation(() => undefined);
const fixture = TestBed.createComponent(App);
const app = fixture.componentInstance;
const subs = TestBed.inject(SubscriptionsService) as unknown as SubscriptionsServiceStub;
app.beginEditName('sub1', 'Videos');
app.nameEditDraft = ' ';
app.saveName('sub1');
expect(subs.updateCalls.length).toBe(0);
expect(app.editingNameId).toBe('sub1');
expect(errorSpy).toHaveBeenCalledWith('Subscription name must not be empty');
errorSpy.mockRestore();
});
});
+32 -22
View File
@@ -102,6 +102,9 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
skipSubscriberOnly = false;
editingTitleRegexId: string | null = null;
titleRegexEditDraft = '';
editingNameId: string | null = null;
nameEditDraft = '';
readonly subscriptionNameMaxLength = 200;
cachedSubs: [string, SubscriptionRow][] = [];
selectedSubscriptionIds = new Set<string>();
checkingSubscriptionIds = new Set<string>();
@@ -663,6 +666,34 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
});
}
beginEditName(id: string, current: string | undefined) {
this.editingNameId = id;
this.nameEditDraft = current ?? '';
this.cdr.markForCheck();
}
cancelEditName() {
this.editingNameId = null;
this.nameEditDraft = '';
this.cdr.markForCheck();
}
saveName(id: string) {
const name = (this.nameEditDraft || '').trim();
if (!name) {
this.toasts.error('Subscription name must not be empty');
return;
}
this.subscriptionsSvc.update(id, { name }).subscribe((res) => {
const error = this.getStatusError(res);
if (error) {
this.toasts.error(error || 'Update subscription failed');
return;
}
this.cancelEditName();
});
}
deleteSubscription(id: string) {
this.subscriptionsSvc.delete([id]).subscribe((res) => {
const error = this.getStatusError(res);
@@ -1146,30 +1177,9 @@ export class App implements AfterViewInit, OnInit, OnDestroy {
}
retryDownload(key: string, download: Download) {
const payload = this.buildAddPayload({
url: download.url,
downloadType: download.download_type,
codec: download.codec,
quality: download.quality,
format: download.format,
folder: download.folder,
customNamePrefix: download.custom_name_prefix,
playlistItemLimit: download.playlist_item_limit,
autoStart: true,
splitByChapters: download.split_by_chapters,
chapterTemplate: download.chapter_template,
subtitleLanguage: download.subtitle_language,
subtitleMode: download.subtitle_mode,
ytdlOptionsPresets: download.ytdl_options_presets?.length
? [...download.ytdl_options_presets]
: [],
ytdlOptionsOverrides: download.ytdl_options_overrides ? JSON.stringify(download.ytdl_options_overrides) : '',
clipStart: download.clip_start != null ? String(download.clip_start) : '',
clipEnd: download.clip_end != null ? String(download.clip_end) : '',
});
// Only remove the done-list record once the retry is confirmed queued —
// deleting it eagerly would silently lose history if the re-add fails.
this.downloads.add(payload)
this.downloads.retry(key)
.pipe(takeUntilDestroyed(this.destroyRef))
.subscribe((status: Status) => {
if (status.status === 'error') {
@@ -117,6 +117,14 @@ describe('DownloadsService', () => {
req.flush({ presets: ['Preset A'] });
});
it('retry() posts the failed download id', () => {
service.retry('https://example.com/v').subscribe();
const req = httpMock.expectOne('retry');
expect(req.request.method).toBe('POST');
expect(req.request.body).toEqual({ id: 'https://example.com/v' });
req.flush({ status: 'ok' });
});
it('cancelAdd posts to cancel-add', () => {
service.cancelAdd().subscribe();
const req = httpMock.expectOne('cancel-add');
+6
View File
@@ -169,6 +169,12 @@ export class DownloadsService {
);
}
public retry(id: string) {
return this.http.post<Status>('retry', { id: id }).pipe(
catchError(this.handleHTTPError)
);
}
public startById(ids: string[]) {
return this.http.post<Status>('start', {ids: ids}).pipe(
catchError(this.handleHTTPError)