ainur/metube
1
0
Fork 0
mirror of https://github.com/alexta69/metube.git synced 2026-09-21 13:35:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: stop resolving submitted URLs locally when a proxy will (closes #1079)

Browse Source 
validate_url resolved every submitted hostname in the server process before
yt-dlp saw it. Behind a proxy that does its own DNS -- an HTTP proxy, socks5h,
socks4a, or the plain socks5 yt-dlp rewrites to socks5h -- that lookup is both
wrong and harmful: it describes this host's network rather than the proxy's,
and it leaks the hostname of every queued URL to the local resolver, which is
the one thing a SOCKS/Tor setup exists to prevent. It also failed closed when
only the proxy could resolve the name, so a container pointed at the proxy's
DNS port refused every add with 'Could not resolve host'.

The address check is now skipped for hostnames that the carrying proxy will
resolve, and kept everywhere else: for direct fetches, for hosts excluded by
no_proxy, for socks4 (which resolves locally), and for hosts written as IP
literals, which need no lookup and leak nothing. Scheme validation, the
localhost/metadata blocklist and the connect-time socket guard are unchanged.

download_proxies mirrors YoutubeDL.proxies rather than importing it: that
property is only reachable from a constructed instance, and since it decides
whether a security check runs, a quiet upstream change should leave the check
in place rather than silently skip it.

Also makes ALLOW_PRIVATE_ADDRESSES explicit in the download-queue test config
-- unset on a MagicMock it is truthy, which had validate_url bypassing every
check those tests asked it to run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Alex Shnitman
2026-09-15 21:46:17 +03:00
parent 79388370e9
commit 5cac98a6d1
6 changed files with 271 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/subscriptions.py
+18 -4
View File
@@ -19,7 +19,7 @@ import yt_dlp.networking.impersonate
import bg_tasks
from dl_formats import merge_ytdl_option_layers
from state_store import AtomicJsonStore, read_legacy_shelf
from url_guard import validate_url
from url_guard import validate_url, download_proxies
log = logging.getLogger("subscriptions")
@@ -116,12 +116,18 @@ def extract_flat_playlist(
if media_entries:
return info, media_entries
if _depth < 1:
proxies = download_proxies({**config.YTDL_OPTIONS, **(extra_opts or {})})
for ent in entries[:5]:
nested_url = _entry_video_url(ent)
if not nested_url:
continue
# nested_url comes from remote playlist content; guard it too.
if validate_url(nested_url, allow_private=getattr(config, "ALLOW_PRIVATE_ADDRESSES", False)) is not None:
# nested_url comes from remote playlist content; guard it too,
# against the same proxy map this scan is using.
if validate_url(
nested_url,
allow_private=getattr(config, "ALLOW_PRIVATE_ADDRESSES", False),
proxies=proxies,
) is not None:
continue
nested_info, nested_entries = extract_flat_playlist(
config,
@@ -621,8 +627,16 @@ class SubscriptionManager:
return {"status": "error", "msg": "Missing URL"}
# SSRF guard: block non-http(s) schemes and internal/metadata hosts
# before yt-dlp fetches the feed. May do a DNS lookup, so run off-loop.
# The scan's own options pick the proxy, so a feed fetched through one
# is not resolved here — see validate_url.
proxies = download_proxies({
**self.config.YTDL_OPTIONS,
**self._scan_extra_opts(ytdl_options_presets, ytdl_options_overrides),
})
url_error = await asyncio.get_running_loop().run_in_executor(
None, partial(validate_url, url, allow_private=getattr(self.config, "ALLOW_PRIVATE_ADDRESSES", False)))
None, partial(validate_url, url,
allow_private=getattr(self.config, "ALLOW_PRIVATE_ADDRESSES", False),
proxies=proxies))
if url_error is not None:
log.warning('Rejected subscription URL "%s": %s', url, url_error)
return {"status": "error", "msg": url_error}