mirror of
https://github.com/alexta69/metube.git
synced 2026-07-23 13:22:48 +00:00
Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e061a8a5ba | |||
| 13cb65d931 | |||
| 1b02a99510 | |||
| ebcfe577bc | |||
| 3bd2c3e366 | |||
| 707f700609 | |||
| c519f45908 |
@@ -1 +0,0 @@
|
||||
blank_discussions_enabled: false
|
||||
@@ -1,65 +0,0 @@
|
||||
name: ⚙️ Configuration Help
|
||||
description: Get help with MeTube configuration and setup
|
||||
title: "[Config]: "
|
||||
labels: ["configuration", "help"]
|
||||
assignees: []
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before asking for configuration help
|
||||
options:
|
||||
- label: I have searched existing discussions and issues for similar configuration problems
|
||||
required: true
|
||||
- label: I have read the [configuration section](https://github.com/alexta69/metube#%EF%B8%8F-configuration-via-environment-variables) in the README
|
||||
required: true
|
||||
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Configuration Resources
|
||||
|
||||
Before asking for help, please check these resources:
|
||||
- **[Configuration Guide](https://github.com/alexta69/metube#%EF%B8%8F-configuration-via-environment-variables)** - All available environment variables
|
||||
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
|
||||
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
|
||||
|
||||
- type: textarea
|
||||
id: configuration-goal
|
||||
attributes:
|
||||
label: What are you trying to configure?
|
||||
description: Describe what you want to achieve with your MeTube configuration
|
||||
placeholder: |
|
||||
What specific behavior are you trying to achieve?
|
||||
What's not working as expected?
|
||||
What have you tried so far?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Your configuration, environment details, errors, and any other helpful information
|
||||
placeholder: |
|
||||
Current Configuration:
|
||||
```yaml
|
||||
# Your docker-compose.yml or environment variables
|
||||
```
|
||||
|
||||
Environment Details:
|
||||
- MeTube version: [e.g., latest, specific version]
|
||||
- Docker image: [e.g., ghcr.io/alexta69/metube:latest]
|
||||
- Operating System: [e.g., Ubuntu 20.04, Windows 10, macOS 12]
|
||||
|
||||
Error Messages or Issues:
|
||||
[Paste any error messages, logs, or unexpected behavior here]
|
||||
|
||||
Other relevant information:
|
||||
[Screenshots, examples, etc.]
|
||||
validations:
|
||||
required: true
|
||||
@@ -1,30 +1,13 @@
|
||||
name: 💬 General Discussion
|
||||
description: Start a general discussion about MeTube
|
||||
title: "[Discussion]: "
|
||||
labels: ["discussion"]
|
||||
assignees: []
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before starting a discussion
|
||||
options:
|
||||
- label: I have searched existing discussions to ensure this topic hasn't been discussed before
|
||||
required: true
|
||||
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Discussion Guidelines
|
||||
|
||||
This is for general discussions about MeTube. For specific issues, please use:
|
||||
- **Bug reports** → Use the Bug Report issue template
|
||||
- **Feature requests** → Use the Feature Request issue template
|
||||
- **Questions** → Use the Question issue template
|
||||
|
||||
This is for general discussions about MeTube. For specific topics, better homes exist:
|
||||
- **Bug reports** → [open an issue](https://github.com/alexta69/metube/issues/new?template=bug_report.yml)
|
||||
- **Feature requests** → [open an issue](https://github.com/alexta69/metube/issues/new?template=feature_request.yml) or post in [Ideas](https://github.com/alexta69/metube/discussions/categories/ideas)
|
||||
- **Questions** → post in [Q&A](https://github.com/alexta69/metube/discussions/categories/q-a)
|
||||
|
||||
- type: textarea
|
||||
id: discussion-topic
|
||||
@@ -34,12 +17,3 @@ body:
|
||||
placeholder: Please provide a clear topic for discussion
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other information that might be helpful for the discussion
|
||||
placeholder: Links, examples, or other relevant information
|
||||
validations:
|
||||
required: false
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Read this first — MeTube's scope
|
||||
|
||||
MeTube's scope is deliberately narrow: **it downloads well and stops once the file is written.**
|
||||
Ideas that improve the download itself are welcome — usually as community PRs, since the
|
||||
maintainer is unlikely to implement requests ([details](https://github.com/alexta69/metube#-submitting-feature-requests)).
|
||||
Post-download file management is out of scope regardless of implementation quality.
|
||||
|
||||
**Already decided — please don't re-request:**
|
||||
- 🔒 Built-in authentication / password → use a reverse proxy ([#931](https://github.com/alexta69/metube/issues/931), [wiki](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations))
|
||||
- 🔔 Notifications on download completion → available today via the `Exec` postprocessor ([recipe](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook))
|
||||
- ✏️ Renaming, converting, tagging, or organizing files after download → out of scope ([#495](https://github.com/alexta69/metube/issues/495), [#1027](https://github.com/alexta69/metube/issues/1027))
|
||||
- 🎚️ Pre-download format/quality/audio-track picker → tracked in [#1032](https://github.com/alexta69/metube/issues/1032)
|
||||
- 📋 Pre-download playlist item browser → tracked in [#1030](https://github.com/alexta69/metube/issues/1030)
|
||||
- 🎛️ Config switches to disable individual features → declined ([#976](https://github.com/alexta69/metube/issues/976))
|
||||
- ⏰ Download scheduler → declined; add items with auto-start off and start them when you like ([#838](https://github.com/alexta69/metube/issues/838))
|
||||
|
||||
- type: textarea
|
||||
id: idea
|
||||
attributes:
|
||||
label: Your Idea
|
||||
description: Describe the idea and what problem it solves
|
||||
placeholder: |
|
||||
What would you like to see?
|
||||
What problem does it solve?
|
||||
Would you be willing to implement it as a PR?
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,23 @@
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Quick Resources
|
||||
|
||||
Your question may already be answered here:
|
||||
- **[README](https://github.com/alexta69/metube#readme)** — complete setup and configuration guide
|
||||
- **[Troubleshooting FAQ](https://github.com/alexta69/metube/wiki/Troubleshooting-FAQ)** — common problems and their fixes
|
||||
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** — common yt-dlp configurations (notifications, metadata, audio extraction, ...)
|
||||
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** — filename template examples
|
||||
|
||||
- type: textarea
|
||||
id: question
|
||||
attributes:
|
||||
label: Your Question
|
||||
description: What would you like to know about MeTube?
|
||||
placeholder: |
|
||||
What are you trying to achieve?
|
||||
What's your current setup (docker-compose / environment variables)?
|
||||
What have you tried so far, and what happened (errors, logs)?
|
||||
validations:
|
||||
required: true
|
||||
@@ -1,60 +0,0 @@
|
||||
name: ❓ Q&A
|
||||
description: Ask a question and get answers from the community
|
||||
title: "[Q&A]: "
|
||||
labels: ["q-and-a"]
|
||||
assignees: []
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before asking your question
|
||||
options:
|
||||
- label: I have searched existing discussions and issues to see if my question has been answered before
|
||||
required: true
|
||||
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
|
||||
required: true
|
||||
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Quick Resources
|
||||
|
||||
Before asking your question, please check these resources:
|
||||
- **[README](https://github.com/alexta69/metube#readme)** - Complete setup and configuration guide
|
||||
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
|
||||
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
|
||||
|
||||
- type: textarea
|
||||
id: question
|
||||
attributes:
|
||||
label: Your Question
|
||||
description: What would you like to know about MeTube?
|
||||
placeholder: |
|
||||
What are you trying to achieve?
|
||||
What's your current setup?
|
||||
What have you tried so far?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other information that might be helpful (configuration, errors, screenshots, etc.)
|
||||
placeholder: |
|
||||
Configuration:
|
||||
```yaml
|
||||
# Your docker-compose.yml or environment variables
|
||||
```
|
||||
|
||||
Error messages or logs:
|
||||
[Paste any error messages or logs here]
|
||||
|
||||
Other relevant information:
|
||||
[Screenshots, examples, etc.]
|
||||
validations:
|
||||
required: false
|
||||
@@ -9,24 +9,20 @@ body:
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before submitting your bug report
|
||||
description: Please confirm before submitting
|
||||
options:
|
||||
- label: I have searched existing issues and discussions to ensure this bug hasn't been reported before
|
||||
required: true
|
||||
- label: I have read the [troubleshooting section](https://github.com/alexta69/metube#-troubleshooting-and-submitting-issues) in the README
|
||||
required: true
|
||||
- label: I have tested this issue with yt-dlp directly (not just through MeTube UI) as described in the README
|
||||
required: true
|
||||
- label: I have checked that this is not a yt-dlp issue (if it is, please report it to [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues) instead)
|
||||
- label: If the download itself fails, I have tested the same URL and options with yt-dlp directly (see the note below) — or this bug is not download-related
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Important Notes
|
||||
- **MeTube is only a UI for yt-dlp** - issues with authentication, postprocessing, permissions, or other yt-dlp functionality should be reported to the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
|
||||
- Before reporting, please test with yt-dlp directly using: `docker exec -ti metube sh` then `cd /downloads` and run yt-dlp commands
|
||||
- If yt-dlp works directly but MeTube doesn't, then it's a MeTube issue
|
||||
- **MeTube is only a UI for [yt-dlp](https://github.com/yt-dlp/yt-dlp)** — issues with authentication, postprocessing, site support, or other download functionality should be reported to the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
|
||||
- To test with yt-dlp directly: `docker exec -ti metube sh`, then `cd /downloads` and run `yt-dlp` with your URL and options
|
||||
- If yt-dlp works directly but MeTube doesn't, then it's a MeTube issue — report it here
|
||||
|
||||
- type: textarea
|
||||
id: bug-description
|
||||
@@ -47,10 +43,13 @@ body:
|
||||
id: ytdl-test-results
|
||||
attributes:
|
||||
label: yt-dlp Direct Test Results
|
||||
description: Results of testing the same URL/configuration directly with yt-dlp (required)
|
||||
description: >-
|
||||
If the download itself fails: paste the exact yt-dlp command you ran AND its output —
|
||||
download-failure reports without this will be closed as needs-info.
|
||||
If this is a UI or app bug that doesn't involve a failing download, just write "UI bug".
|
||||
placeholder: |
|
||||
Command used: yt-dlp [your-command-here]
|
||||
Result: [success/error and output]
|
||||
Result: [paste the output here]
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -64,12 +63,12 @@ body:
|
||||
- MeTube version: [e.g., latest, specific version]
|
||||
- Docker image: [e.g., ghcr.io/alexta69/metube:latest]
|
||||
- Operating System: [e.g., Ubuntu 20.04, Windows 10, macOS 12]
|
||||
|
||||
|
||||
Configuration:
|
||||
```yaml
|
||||
# Your docker-compose.yml or environment variables
|
||||
```
|
||||
|
||||
|
||||
Logs:
|
||||
```bash
|
||||
docker logs metube
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: MeTube Community Discussions
|
||||
url: https://github.com/alexta69/metube/discussions
|
||||
about: Ask questions and discuss MeTube with the community
|
||||
- name: yt-dlp Issues
|
||||
- name: ❓ Questions & Support
|
||||
url: https://github.com/alexta69/metube/discussions/categories/q-a
|
||||
about: Ask usage and configuration questions in Discussions Q&A — issues are for bugs and feature requests
|
||||
- name: 📖 Troubleshooting FAQ
|
||||
url: https://github.com/alexta69/metube/wiki/Troubleshooting-FAQ
|
||||
about: Common problems and their fixes
|
||||
- name: ⬇️ yt-dlp Issues
|
||||
url: https://github.com/yt-dlp/yt-dlp/issues
|
||||
about: Report issues related to video downloading, authentication, or site support
|
||||
|
||||
@@ -5,28 +5,35 @@ labels: ["enhancement"]
|
||||
assignees: []
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Read this first — MeTube's scope
|
||||
|
||||
MeTube's scope is deliberately narrow: **it downloads well and stops once the file is written.**
|
||||
Features that improve the download itself are welcome — usually as community PRs, since the
|
||||
maintainer is unlikely to implement requests ([details](https://github.com/alexta69/metube#-submitting-feature-requests)).
|
||||
Post-download file management is out of scope regardless of implementation quality.
|
||||
|
||||
**Already decided — please don't re-request:**
|
||||
- 🔒 Built-in authentication / password → use a reverse proxy ([#931](https://github.com/alexta69/metube/issues/931), [wiki](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations))
|
||||
- 🔔 Notifications on download completion → available today via the `Exec` postprocessor ([recipe](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook))
|
||||
- ✏️ Renaming, converting, tagging, or organizing files after download → out of scope ([#495](https://github.com/alexta69/metube/issues/495), [#1027](https://github.com/alexta69/metube/issues/1027))
|
||||
- 🎚️ Pre-download format/quality/audio-track picker → tracked in [#1032](https://github.com/alexta69/metube/issues/1032)
|
||||
- 📋 Pre-download playlist item browser → tracked in [#1030](https://github.com/alexta69/metube/issues/1030)
|
||||
- 🎛️ Config switches to disable individual features → declined ([#976](https://github.com/alexta69/metube/issues/976))
|
||||
- ⏰ Download scheduler → declined; add items with auto-start off and start them when you like ([#838](https://github.com/alexta69/metube/issues/838))
|
||||
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before submitting your feature request
|
||||
description: Please confirm before submitting
|
||||
options:
|
||||
- label: I have searched existing issues and discussions to ensure this feature hasn't been requested before
|
||||
- label: I have searched existing issues and checked the "Already decided" list above
|
||||
required: true
|
||||
- label: I have read the [feature request guidelines](https://github.com/alexta69/metube#-submitting-feature-requests) in the README
|
||||
- label: I have considered if this feature belongs in yt-dlp instead (downloading, processing, site support) — if so, please request it in the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues)
|
||||
required: true
|
||||
- label: I understand that MeTube development relies on community contributions and the maintainer is not likely to implement this feature
|
||||
required: true
|
||||
- label: I have considered if this feature should be implemented in yt-dlp instead of MeTube (if so, please report to [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues))
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Important Notes
|
||||
- **MeTube development relies on code contributions by the community** - the project is feature-complete for the maintainer's use cases
|
||||
- **Consider if this belongs in yt-dlp** - if it's related to video downloading, processing, or site support, it might belong in the [yt-dlp repository](https://github.com/yt-dlp/yt-dlp/issues) instead
|
||||
- **Some features may not be accepted** - in an effort to reduce bloat, some PRs may not be accepted
|
||||
|
||||
- type: textarea
|
||||
id: feature-description
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
name: ❓ Question
|
||||
description: Ask a question about MeTube usage, configuration, or general help
|
||||
title: "[Question]: "
|
||||
labels: ["question"]
|
||||
assignees: []
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
description: Please confirm you have completed these steps before asking your question
|
||||
options:
|
||||
- label: I have searched existing issues and discussions to see if my question has been answered before
|
||||
required: true
|
||||
- label: I have read the [README](https://github.com/alexta69/metube#readme) and relevant sections
|
||||
required: true
|
||||
- label: I have checked the [Wiki](https://github.com/alexta69/metube/wiki) for configuration examples
|
||||
required: true
|
||||
- label: I have read the [troubleshooting section](https://github.com/alexta69/metube#-troubleshooting-and-submitting-issues) if this is a technical issue
|
||||
required: true
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Quick Resources
|
||||
|
||||
Before asking your question, please check these resources:
|
||||
- **[README](https://github.com/alexta69/metube#readme)** - Complete setup and configuration guide
|
||||
- **[YTDL_OPTIONS Cookbook](https://github.com/alexta69/metube/wiki/YTDL_OPTIONS-Cookbook)** - Common yt-dlp configurations
|
||||
- **[OUTPUT_TEMPLATE Cookbook](https://github.com/alexta69/metube/wiki/OUTPUT_TEMPLATE-Cookbook)** - Filename template examples
|
||||
|
||||
- type: textarea
|
||||
id: question
|
||||
attributes:
|
||||
label: Your Question
|
||||
description: What would you like to know about MeTube?
|
||||
placeholder: |
|
||||
What are you trying to achieve?
|
||||
What's your current setup?
|
||||
What have you tried so far?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
description: Any other information that might be helpful (configuration, errors, screenshots, etc.)
|
||||
placeholder: |
|
||||
Configuration:
|
||||
```yaml
|
||||
# Your docker-compose.yml or environment variables
|
||||
```
|
||||
|
||||
Error messages or logs:
|
||||
[Paste any error messages or logs here]
|
||||
|
||||
Other relevant information:
|
||||
[Screenshots, examples, etc.]
|
||||
validations:
|
||||
required: false
|
||||
+32
-46
@@ -117,25 +117,27 @@ jobs:
|
||||
- name: Get current date
|
||||
id: date
|
||||
run: echo "date=$(date +'%Y.%m.%d')" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Get commits since last release
|
||||
id: commits
|
||||
env:
|
||||
DATE: ${{ steps.date.outputs.date }}
|
||||
run: |
|
||||
# Fetch all tags
|
||||
git fetch --tags
|
||||
|
||||
# Get the last tag (sorted by version, using date format YYYY.MM.DD)
|
||||
LAST_TAG=$(git tag -l --sort=-version:refname | grep -E '^[0-9]{4}\.[0-9]{2}\.[0-9]{2}$' | head -n 1)
|
||||
|
||||
|
||||
# Exclude today's tag: on a same-day rerun the notes must cover the
|
||||
# whole day, not just the commits since the morning release.
|
||||
LAST_TAG=$(git tag -l --sort=-version:refname | grep -E '^[0-9]{4}\.[0-9]{2}\.[0-9]{2}$' | grep -v "^${DATE}$" | head -n 1)
|
||||
|
||||
if [ -z "$LAST_TAG" ]; then
|
||||
# No previous release, skip commits for first release
|
||||
COMMITS=""
|
||||
echo "has_commits=false" >> $GITHUB_OUTPUT
|
||||
else
|
||||
# Get commits since last tag
|
||||
COMMITS=$(git log ${LAST_TAG}..HEAD --pretty=format:"- %s (%h)" --no-merges)
|
||||
if [ -z "$COMMITS" ]; then
|
||||
echo "has_commits=false" >> $GITHUB_OUTPUT
|
||||
@@ -143,18 +145,13 @@ jobs:
|
||||
echo "has_commits=true" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
fi
|
||||
|
||||
# Escape for use in YAML/multiline output
|
||||
|
||||
{
|
||||
echo 'commits<<EOF'
|
||||
echo "$COMMITS"
|
||||
echo EOF
|
||||
} >> $GITHUB_OUTPUT
|
||||
|
||||
# Also output for debugging
|
||||
echo "Last tag: ${LAST_TAG:-none}"
|
||||
echo "Commits since last release:"
|
||||
echo "$COMMITS"
|
||||
|
||||
- name: Generate release body
|
||||
id: release_body
|
||||
env:
|
||||
@@ -176,7 +173,7 @@ jobs:
|
||||
echo '**GitHub Container Registry:**'
|
||||
echo "- \`${GHCR_REPO}:latest\`"
|
||||
echo "- \`${GHCR_REPO}:${DATE}\`"
|
||||
|
||||
|
||||
if [ "$HAS_COMMITS" = "true" ] && [ -n "$COMMITS" ]; then
|
||||
echo ''
|
||||
echo '## Changes'
|
||||
@@ -184,39 +181,28 @@ jobs:
|
||||
echo "$COMMITS"
|
||||
fi
|
||||
} > release_body.txt
|
||||
|
||||
{
|
||||
echo 'body<<EOF'
|
||||
cat release_body.txt
|
||||
echo EOF
|
||||
} >> $GITHUB_OUTPUT
|
||||
- name: Delete existing release if present
|
||||
|
||||
- name: Create or update GitHub Release (mark as latest)
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
TAG_NAME: ${{ steps.date.outputs.date }}
|
||||
run: |
|
||||
# Check if release exists and delete it
|
||||
if gh release view "$TAG_NAME" &>/dev/null; then
|
||||
echo "Release $TAG_NAME already exists, deleting it..."
|
||||
gh release delete "$TAG_NAME" --yes || true
|
||||
if gh release view "$TAG_NAME" >/dev/null 2>&1; then
|
||||
echo "Release $TAG_NAME exists; updating."
|
||||
# Force-move the tag in place so it matches the rebuilt Docker
|
||||
# image. Never delete+recreate the tag: that corrupts GitHub's
|
||||
# release index (broke release 2026.07.05).
|
||||
gh api -X PATCH "repos/${GITHUB_REPOSITORY}/git/refs/tags/${TAG_NAME}" \
|
||||
-f sha="$GITHUB_SHA" -F force=true
|
||||
gh release edit "$TAG_NAME" \
|
||||
--title "Release $TAG_NAME" \
|
||||
--notes-file release_body.txt \
|
||||
--latest
|
||||
else
|
||||
echo "Release $TAG_NAME does not exist; creating."
|
||||
gh release create "$TAG_NAME" \
|
||||
--target "$GITHUB_SHA" \
|
||||
--title "Release $TAG_NAME" \
|
||||
--notes-file release_body.txt \
|
||||
--latest
|
||||
fi
|
||||
|
||||
# Fetch tags to check remote
|
||||
git fetch --tags
|
||||
|
||||
# Check if tag exists (locally or remotely) and delete it
|
||||
if git rev-parse "$TAG_NAME" &>/dev/null 2>&1 || git ls-remote --tags origin "$TAG_NAME" | grep -q "$TAG_NAME"; then
|
||||
echo "Tag $TAG_NAME already exists, deleting it..."
|
||||
git tag -d "$TAG_NAME" 2>/dev/null || true
|
||||
git push origin ":refs/tags/$TAG_NAME" || true
|
||||
fi
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v3
|
||||
with:
|
||||
tag_name: ${{ steps.date.outputs.date }}
|
||||
name: Release ${{ steps.date.outputs.date }}
|
||||
body_path: release_body.txt
|
||||
draft: false
|
||||
prerelease: false
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
@@ -80,6 +80,7 @@ Certain values can be set via environment variables, using the `-e` parameter on
|
||||
* __YTDL_OPTIONS_PRESETS__: Named bundles of yt-dlp options, selectable per download in the UI. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options) for format and examples.
|
||||
* __YTDL_OPTIONS_PRESETS_FILE__: Path to a JSON file containing presets. Monitored and reloaded automatically on changes. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options).
|
||||
* __ALLOW_YTDL_OPTIONS_OVERRIDES__: Whether to show a free-text field in the UI for per-download yt-dlp option overrides. Defaults to `false`. See [Configuring yt-dlp options](#%EF%B8%8F-configuring-yt-dlp-options) for details and security considerations.
|
||||
* __ALLOW_PRIVATE_ADDRESSES__: Whether to allow downloads from private, loopback, link-local and other non-global addresses. Defaults to `false`, which protects against SSRF by refusing URLs that resolve to internal hosts. Set to `true` only in trusted environments — for example when routing traffic through a proxy/VPN client in Fake-IP mode (sing-box, Clash, Mihomo), which resolves hosts to the `198.18.0.0/15` range. Enabling this disables the SSRF protection entirely, so only use it when you control the network.
|
||||
* __YTDL_NIGHTLY_UPDATE_TIME__: If set, will cause MeTube to use [nightly yt-dlp builds](https://github.com/yt-dlp/yt-dlp-nightly-builds) instead of the stable releases. Set to the time (`HH:MM`, 24-hour) when you want the daily upgrades and MeTube restart to happen. Defaults to empty (disabled).
|
||||
|
||||
### 🌐 Web Server & URLs
|
||||
|
||||
+2
-5
@@ -81,6 +81,7 @@ class Config:
|
||||
'YTDL_OPTIONS_PRESETS': '{}',
|
||||
'YTDL_OPTIONS_PRESETS_FILE': '',
|
||||
'ALLOW_YTDL_OPTIONS_OVERRIDES': 'false',
|
||||
'ALLOW_PRIVATE_ADDRESSES': 'false',
|
||||
'CORS_ALLOWED_ORIGINS': '',
|
||||
'ROBOTS_TXT': '',
|
||||
'HOST': '0.0.0.0',
|
||||
@@ -96,7 +97,7 @@ class Config:
|
||||
'YTDL_NIGHTLY_UPDATE_TIME': '',
|
||||
}
|
||||
|
||||
_BOOLEAN = ('DOWNLOAD_DIRS_INDEXABLE', 'CUSTOM_DIRS', 'CREATE_CUSTOM_DIRS', 'DELETE_FILE_ON_TRASHCAN', 'HTTPS', 'ENABLE_ACCESSLOG', 'ALLOW_YTDL_OPTIONS_OVERRIDES')
|
||||
_BOOLEAN = ('DOWNLOAD_DIRS_INDEXABLE', 'CUSTOM_DIRS', 'CREATE_CUSTOM_DIRS', 'DELETE_FILE_ON_TRASHCAN', 'HTTPS', 'ENABLE_ACCESSLOG', 'ALLOW_YTDL_OPTIONS_OVERRIDES', 'ALLOW_PRIVATE_ADDRESSES')
|
||||
|
||||
def __init__(self):
|
||||
for k, v in self._DEFAULTS.items():
|
||||
@@ -717,8 +718,6 @@ def parse_download_options(post: dict) -> dict:
|
||||
|
||||
if custom_name_prefix is None:
|
||||
custom_name_prefix = ''
|
||||
if custom_name_prefix and ('..' in custom_name_prefix or custom_name_prefix.startswith('/') or custom_name_prefix.startswith('\\')):
|
||||
raise web.HTTPBadRequest(reason='custom_name_prefix must not contain ".." or start with a path separator')
|
||||
if auto_start is None:
|
||||
auto_start = True
|
||||
if playlist_item_limit is None:
|
||||
@@ -743,8 +742,6 @@ def parse_download_options(post: dict) -> dict:
|
||||
enabled=config.ALLOW_YTDL_OPTIONS_OVERRIDES,
|
||||
)
|
||||
|
||||
if chapter_template and ('..' in chapter_template or chapter_template.startswith('/') or chapter_template.startswith('\\')):
|
||||
raise web.HTTPBadRequest(reason='chapter_template must not contain ".." or start with a path separator')
|
||||
if not SUBTITLE_LANGUAGE_RE.fullmatch(subtitle_language):
|
||||
raise web.HTTPBadRequest(reason='subtitle_language must match pattern [A-Za-z0-9-] and be at most 35 characters')
|
||||
if subtitle_mode not in VALID_SUBTITLE_MODES:
|
||||
|
||||
@@ -115,7 +115,7 @@ def extract_flat_playlist(
|
||||
if not nested_url:
|
||||
continue
|
||||
# nested_url comes from remote playlist content; guard it too.
|
||||
if validate_url(nested_url) is not None:
|
||||
if validate_url(nested_url, allow_private=getattr(config, "ALLOW_PRIVATE_ADDRESSES", False)) is not None:
|
||||
continue
|
||||
nested_info, nested_entries = extract_flat_playlist(
|
||||
config,
|
||||
@@ -548,7 +548,8 @@ class SubscriptionManager:
|
||||
return {"status": "error", "msg": "Missing URL"}
|
||||
# SSRF guard: block non-http(s) schemes and internal/metadata hosts
|
||||
# before yt-dlp fetches the feed. May do a DNS lookup, so run off-loop.
|
||||
url_error = await asyncio.get_running_loop().run_in_executor(None, validate_url, url)
|
||||
url_error = await asyncio.get_running_loop().run_in_executor(
|
||||
None, partial(validate_url, url, allow_private=getattr(self.config, "ALLOW_PRIVATE_ADDRESSES", False)))
|
||||
if url_error is not None:
|
||||
log.warning('Rejected subscription URL "%s": %s', url, url_error)
|
||||
return {"status": "error", "msg": url_error}
|
||||
|
||||
@@ -138,25 +138,6 @@ async def test_add_invalid_subtitle_language(mock_dqueue):
|
||||
await main.add(req)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_custom_name_prefix_path_traversal(mock_dqueue):
|
||||
req = _json_request(_valid_video_add_body(custom_name_prefix="../evil"))
|
||||
with pytest.raises(web.HTTPBadRequest):
|
||||
await main.add(req)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_chapter_template_path_traversal(mock_dqueue):
|
||||
req = _json_request(
|
||||
_valid_video_add_body(
|
||||
split_by_chapters=True,
|
||||
chapter_template="/etc/passwd%(title)s",
|
||||
)
|
||||
)
|
||||
with pytest.raises(web.HTTPBadRequest):
|
||||
await main.add(req)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_invalid_json_body(mock_dqueue):
|
||||
req = MagicMock(spec=web.Request)
|
||||
|
||||
@@ -6,7 +6,13 @@ import socket
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from url_guard import validate_url
|
||||
import url_guard
|
||||
from url_guard import (
|
||||
validate_url,
|
||||
_address_allowed_at_connect,
|
||||
_guarded_getaddrinfo,
|
||||
install_socket_guard,
|
||||
)
|
||||
|
||||
|
||||
def _addrinfo(*addrs, family=socket.AF_INET):
|
||||
@@ -93,9 +99,89 @@ class AddressResolutionTests(unittest.TestCase):
|
||||
# If any resolved address is internal, reject the whole URL.
|
||||
self.assertIsNotNone(self._validate_with_addrs("http://mixed/x", "142.250.1.1", "127.0.0.1"))
|
||||
|
||||
def test_resolution_failure_defers_to_ytdlp(self):
|
||||
def test_resolution_failure_is_rejected(self):
|
||||
# Fail closed: an unresolvable host cannot be verified as non-internal.
|
||||
with mock.patch("url_guard.socket.getaddrinfo", side_effect=socket.gaierror):
|
||||
self.assertIsNone(validate_url("http://does-not-resolve.example/x"))
|
||||
self.assertIsNotNone(validate_url("http://does-not-resolve.example/x"))
|
||||
|
||||
|
||||
class ConnectAddressPolicyTests(unittest.TestCase):
|
||||
"""Connect-time policy: allow global + loopback, block everything else."""
|
||||
|
||||
def test_global_allowed(self):
|
||||
self.assertTrue(_address_allowed_at_connect("142.250.1.1"))
|
||||
|
||||
def test_loopback_allowed(self):
|
||||
# Loopback stays reachable so locally-configured proxies keep working.
|
||||
self.assertTrue(_address_allowed_at_connect("127.0.0.1"))
|
||||
self.assertTrue(_address_allowed_at_connect("::1"))
|
||||
|
||||
def test_link_local_metadata_blocked(self):
|
||||
self.assertFalse(_address_allowed_at_connect("169.254.169.254"))
|
||||
|
||||
def test_private_blocked(self):
|
||||
self.assertFalse(_address_allowed_at_connect("10.0.0.5"))
|
||||
self.assertFalse(_address_allowed_at_connect("192.168.1.10"))
|
||||
|
||||
def test_ipv4_mapped_metadata_blocked(self):
|
||||
self.assertFalse(_address_allowed_at_connect("::ffff:169.254.169.254"))
|
||||
|
||||
|
||||
class GuardedGetaddrinfoTests(unittest.TestCase):
|
||||
def test_internal_only_raises(self):
|
||||
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("169.254.169.254")):
|
||||
with self.assertRaises(socket.gaierror):
|
||||
_guarded_getaddrinfo("metadata", 80)
|
||||
|
||||
def test_filters_internal_keeps_global(self):
|
||||
# Split-horizon rebinding: keep the public address, drop the internal one.
|
||||
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("142.250.1.1", "10.0.0.1")):
|
||||
results = _guarded_getaddrinfo("mixed", 80)
|
||||
self.assertEqual([r[4][0] for r in results], ["142.250.1.1"])
|
||||
|
||||
def test_loopback_passes(self):
|
||||
with mock.patch("url_guard._real_getaddrinfo", return_value=_addrinfo("127.0.0.1")):
|
||||
results = _guarded_getaddrinfo("localproxy", 9050)
|
||||
self.assertEqual([r[4][0] for r in results], ["127.0.0.1"])
|
||||
|
||||
|
||||
class AllowPrivateBypassTests(unittest.TestCase):
|
||||
"""ALLOW_PRIVATE_ADDRESSES: trusted proxy/VPN environments opt out of the
|
||||
SSRF address checks (e.g. Fake-IP clients that resolve to 198.18.0.0/15)."""
|
||||
|
||||
def test_internal_address_allowed_when_bypassed(self):
|
||||
# Fake-IP benchmarking range that is_global rejects by default.
|
||||
self.assertIsNone(validate_url("http://www.youtube.com/x", allow_private=True))
|
||||
|
||||
def test_private_host_allowed_when_bypassed(self):
|
||||
# No DNS lookup needed: the bypass returns before resolution.
|
||||
with mock.patch("url_guard.socket.getaddrinfo") as gai:
|
||||
self.assertIsNone(validate_url("http://192.168.1.1/x", allow_private=True))
|
||||
gai.assert_not_called()
|
||||
|
||||
def test_scheme_still_enforced_when_bypassed(self):
|
||||
self.assertIsNotNone(validate_url("file:///etc/passwd", allow_private=True))
|
||||
|
||||
def test_socket_guard_not_installed_when_bypassed(self):
|
||||
original = socket.getaddrinfo
|
||||
try:
|
||||
install_socket_guard(allow_private=True)
|
||||
self.assertIs(socket.getaddrinfo, original)
|
||||
finally:
|
||||
socket.getaddrinfo = original
|
||||
|
||||
|
||||
class InstallSocketGuardTests(unittest.TestCase):
|
||||
def test_install_replaces_and_is_idempotent(self):
|
||||
original = socket.getaddrinfo
|
||||
try:
|
||||
install_socket_guard()
|
||||
self.assertIs(socket.getaddrinfo, url_guard._guarded_getaddrinfo)
|
||||
# Re-installing must not wrap the wrapper (real fn captured at import).
|
||||
install_socket_guard()
|
||||
self.assertIs(socket.getaddrinfo, url_guard._guarded_getaddrinfo)
|
||||
finally:
|
||||
socket.getaddrinfo = original
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import pickle
|
||||
import signal
|
||||
import sys
|
||||
@@ -31,6 +32,25 @@ class _PostProcessor:
|
||||
self._downloader = downloader
|
||||
|
||||
|
||||
class _YoutubeDL:
|
||||
"""Minimal stand-in so ``_ConfinedYoutubeDL`` can subclass it under the shim.
|
||||
|
||||
``prepare_filename`` is patched per-test; the real containment logic lives in
|
||||
the ``_ConfinedYoutubeDL`` override, which is what the tests exercise.
|
||||
"""
|
||||
|
||||
def __init__(self, params=None, **kwargs):
|
||||
self.params = params or {}
|
||||
|
||||
def prepare_filename(self, *args, **kwargs):
|
||||
return ""
|
||||
|
||||
def add_post_processor(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
|
||||
fake_utils.DownloadError = type("DownloadError", (Exception,), {})
|
||||
fake_yt_dlp.YoutubeDL = _YoutubeDL
|
||||
fake_impersonate.ImpersonateTarget = _ImpersonateTarget
|
||||
fake_networking.impersonate = fake_impersonate
|
||||
fake_postprocessor_common.PostProcessor = _PostProcessor
|
||||
@@ -56,15 +76,17 @@ from ytdl import (
|
||||
_compact_persisted_entry,
|
||||
_convert_srt_to_txt_file,
|
||||
_AlbumArtistPostProcessor,
|
||||
_output_dir_escapes,
|
||||
_resolve_outtmpl_fields,
|
||||
_sanitize_entry_for_pickle,
|
||||
_sanitize_path_component,
|
||||
)
|
||||
|
||||
# Detect whether the real yt-dlp is loaded (as opposed to the minimal fake
|
||||
# shim above). _resolve_outtmpl_fields needs YoutubeDL at runtime.
|
||||
_has_real_ytdlp = hasattr(sys.modules.get("yt_dlp"), "YoutubeDL")
|
||||
# shim above). _resolve_outtmpl_fields needs YoutubeDL.evaluate_outtmpl at
|
||||
# runtime, which the shim's YoutubeDL stand-in deliberately does not provide.
|
||||
_has_real_ytdlp = hasattr(
|
||||
getattr(sys.modules.get("yt_dlp"), "YoutubeDL", None), "evaluate_outtmpl"
|
||||
)
|
||||
|
||||
|
||||
class AlbumArtistPostProcessorTests(unittest.TestCase):
|
||||
@@ -175,7 +197,7 @@ class AlbumArtistRegistrationTests(unittest.TestCase):
|
||||
download.info.download_type = 'audio'
|
||||
fake_ydl = MagicMock()
|
||||
|
||||
with patch('ytdl.yt_dlp.YoutubeDL', return_value=fake_ydl):
|
||||
with patch('ytdl._ConfinedYoutubeDL', return_value=fake_ydl):
|
||||
result = download._make_youtube_dl({'quiet': True})
|
||||
|
||||
self.assertIs(result, fake_ydl)
|
||||
@@ -187,7 +209,7 @@ class AlbumArtistRegistrationTests(unittest.TestCase):
|
||||
download = _make_test_download()
|
||||
fake_ydl = MagicMock()
|
||||
|
||||
with patch('ytdl.yt_dlp.YoutubeDL', return_value=fake_ydl):
|
||||
with patch('ytdl._ConfinedYoutubeDL', return_value=fake_ydl):
|
||||
download._make_youtube_dl({'quiet': True})
|
||||
|
||||
fake_ydl.add_post_processor.assert_not_called()
|
||||
@@ -301,18 +323,50 @@ class ResolveOuttmplFieldsTests(unittest.TestCase):
|
||||
self.assertFalse(literal_prefix.startswith('\\'))
|
||||
|
||||
|
||||
class OutputDirEscapesTests(unittest.TestCase):
|
||||
class ConfinedYoutubeDLTests(unittest.TestCase):
|
||||
"""The chokepoint: ``_ConfinedYoutubeDL.prepare_filename`` validates the
|
||||
*resolved* output path (after yt-dlp expands the template) and refuses any
|
||||
write outside the allowed roots. This is the single guard for the download-
|
||||
directory invariant across the main file, split-chapter files, thumbnails,
|
||||
subtitles, etc. — the ``..`` only exists post-expansion, so it is caught here
|
||||
rather than by any ingress string check.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.base_dir = tempfile.mkdtemp()
|
||||
self.base = os.path.realpath(tempfile.mkdtemp())
|
||||
|
||||
def test_relative_traversal_escapes(self):
|
||||
self.assertTrue(_output_dir_escapes(self.base_dir, '../../tmp/x/%(title)s.%(ext)s'))
|
||||
def _prepared_path(self, resolved, roots=None):
|
||||
ydl = ytdl._ConfinedYoutubeDL.__new__(ytdl._ConfinedYoutubeDL)
|
||||
ydl._allowed_roots = [self.base] if roots is None else roots
|
||||
with patch.object(
|
||||
ytdl.yt_dlp.YoutubeDL, "prepare_filename", return_value=resolved
|
||||
):
|
||||
return ydl.prepare_filename({})
|
||||
|
||||
def test_absolute_path_escapes(self):
|
||||
self.assertTrue(_output_dir_escapes(self.base_dir, '/tmp/x/%(title)s.%(ext)s'))
|
||||
def test_chapter_traversal_via_metadata_is_blocked(self):
|
||||
# e.g. chapter_template '%(section_title)s/%(section_title)s/pwned.%(ext)s'
|
||||
# with a chapter titled '..' expands to '../../pwned.mp4'.
|
||||
escaping = os.path.join(self.base, "..", "..", "pwned.mp4")
|
||||
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
|
||||
self._prepared_path(escaping)
|
||||
|
||||
def test_normal_playlist_dir_stays_inside(self):
|
||||
self.assertFalse(_output_dir_escapes(self.base_dir, 'Playlist/%(title)s.%(ext)s'))
|
||||
def test_absolute_output_path_is_blocked(self):
|
||||
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
|
||||
self._prepared_path("/etc/cron.d/evil")
|
||||
|
||||
def test_path_inside_download_dir_is_allowed(self):
|
||||
ok = os.path.join(self.base, "Playlist", "video.mp4")
|
||||
self.assertEqual(self._prepared_path(ok), ok)
|
||||
|
||||
def test_sibling_prefix_directory_is_blocked(self):
|
||||
# base '/x/downloads' must not be escapable to '/x/downloads-secret'.
|
||||
sibling = self.base + "-secret"
|
||||
with self.assertRaises(ytdl.yt_dlp.utils.DownloadError):
|
||||
self._prepared_path(os.path.join(sibling, "video.mp4"))
|
||||
|
||||
def test_empty_and_stdout_targets_pass_through(self):
|
||||
self.assertEqual(self._prepared_path(""), "")
|
||||
self.assertEqual(self._prepared_path("-"), "-")
|
||||
|
||||
|
||||
class SanitizeEntryForPickleTests(unittest.TestCase):
|
||||
|
||||
+89
-12
@@ -5,10 +5,26 @@ any ``http(s)`` URL. Without a guard, an attacker can make the server fetch
|
||||
internal endpoints (cloud metadata services, loopback, RFC1918 hosts, etc.) and
|
||||
have the response saved to the download directory and served back.
|
||||
|
||||
This module provides a single cheap validator applied at every URL ingress. It
|
||||
intentionally does NOT attempt DNS-rebinding pinning, redirect-chain
|
||||
re-validation, or validation of every media URL yt-dlp derives from remote
|
||||
metadata — network isolation (e.g. Docker) remains the backstop for those.
|
||||
This module provides two layers:
|
||||
|
||||
* ``validate_url`` — a cheap validator applied at every URL ingress.
|
||||
* ``install_socket_guard`` — a connect-time ``getaddrinfo`` guard installed in
|
||||
the download subprocess, which re-validates every resolved address and so
|
||||
covers redirects, DNS rebinding, and media URLs yt-dlp derives from remote
|
||||
metadata — for any backend that resolves through Python's socket module.
|
||||
|
||||
Known limitations — network isolation (e.g. Docker) remains the backstop for
|
||||
all of these:
|
||||
|
||||
* The socket guard is installed only in the download subprocess. Metadata
|
||||
extraction (``ytdl.DownloadQueue.__extract_info``) runs in the main process,
|
||||
where installing a process-wide guard would reject the server's own bind on
|
||||
``HOST=0.0.0.0``. So extraction — which also follows redirects — is covered
|
||||
only by ``validate_url`` at ingress, not at connect time; a redirect from an
|
||||
allowed host to an internal one during extraction is not blocked (a lower-
|
||||
impact, blind SSRF, since the extraction response is not written to disk).
|
||||
* Native resolvers (curl_cffi/libcurl via ``--impersonate``) resolve outside
|
||||
Python's socket module and bypass the connect-time guard entirely.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
@@ -34,24 +50,79 @@ def _hostname_is_blocked(hostname: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _address_is_global(addr: str) -> bool:
|
||||
def _normalise_ip(addr: str):
|
||||
"""Parse *addr*, unwrapping IPv4-mapped IPv6 (e.g. ``::ffff:169.254.169.254``)
|
||||
so the embedded IPv4 address is judged on its own merits. Returns ``None``
|
||||
when *addr* is not a valid IP literal."""
|
||||
try:
|
||||
ip = ipaddress.ip_address(addr)
|
||||
except ValueError:
|
||||
return False
|
||||
# Unwrap IPv4-mapped/compatible IPv6 (e.g. ::ffff:169.254.169.254) so the
|
||||
# embedded IPv4 address is judged on its own merits.
|
||||
return None
|
||||
if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
|
||||
ip = ip.ipv4_mapped
|
||||
return ip.is_global
|
||||
return ip
|
||||
|
||||
|
||||
def validate_url(url: str) -> str | None:
|
||||
def _address_is_global(addr: str) -> bool:
|
||||
ip = _normalise_ip(addr)
|
||||
return ip is not None and ip.is_global
|
||||
|
||||
|
||||
def _address_allowed_at_connect(addr: str) -> bool:
|
||||
"""True if *addr* may be connected to at download time.
|
||||
|
||||
Permits global addresses and loopback — loopback so that locally-configured
|
||||
proxies (e.g. ``proxy: http://127.0.0.1:9050``) keep working. Blocks the SSRF
|
||||
targets that matter: link-local (cloud metadata at 169.254.169.254), private
|
||||
(RFC1918), unique-local and every other non-global, non-loopback range.
|
||||
"""
|
||||
ip = _normalise_ip(addr)
|
||||
return ip is not None and (ip.is_global or ip.is_loopback)
|
||||
|
||||
|
||||
# Captured at import so re-installing the guard never wraps the wrapper.
|
||||
_real_getaddrinfo = socket.getaddrinfo
|
||||
|
||||
|
||||
def _guarded_getaddrinfo(host, *args, **kwargs):
|
||||
results = _real_getaddrinfo(host, *args, **kwargs)
|
||||
allowed = [r for r in results if _address_allowed_at_connect(r[4][0])]
|
||||
if not allowed:
|
||||
raise socket.gaierror(f'Refusing to connect to non-global address for host {host!r}')
|
||||
return allowed
|
||||
|
||||
|
||||
def install_socket_guard(allow_private: bool = False) -> None:
|
||||
"""Enforce the no-internal-hosts policy at actual connection time.
|
||||
|
||||
``validate_url`` only checks the *submitted* URL string; yt-dlp then follows
|
||||
HTTP redirects and resolves media URLs from remote metadata without
|
||||
re-validating them. Installing this in the download subprocess re-checks
|
||||
every resolved address at connect time, covering redirects and DNS rebinding
|
||||
for any networking backend that resolves through Python's socket module
|
||||
(urllib, requests). Native resolvers — notably curl_cffi/libcurl used by
|
||||
``--impersonate`` — bypass this and rely on network isolation as the backstop.
|
||||
|
||||
When *allow_private* is set (``ALLOW_PRIVATE_ADDRESSES``), the guard is not
|
||||
installed at all, so proxy/VPN setups that route through private or Fake-IP
|
||||
ranges keep working.
|
||||
"""
|
||||
if allow_private:
|
||||
return
|
||||
socket.getaddrinfo = _guarded_getaddrinfo
|
||||
|
||||
|
||||
def validate_url(url: str, allow_private: bool = False) -> str | None:
|
||||
"""Return an error message if the URL is disallowed, else ``None``.
|
||||
|
||||
Inputs without a ``://`` scheme separator (bare video IDs, ``ytsearch:``
|
||||
and other yt-dlp search/extractor prefixes) are allowed unchanged so that
|
||||
non-URL entries keep working.
|
||||
|
||||
When *allow_private* is set (``ALLOW_PRIVATE_ADDRESSES``), the internal-host
|
||||
and internal-address checks are skipped so that trusted proxy/VPN setups —
|
||||
e.g. Fake-IP clients that resolve YouTube to ``198.18.0.0/15`` — can be used.
|
||||
Scheme validation (http/https only) still applies.
|
||||
"""
|
||||
if not isinstance(url, str):
|
||||
return 'Invalid URL'
|
||||
@@ -70,14 +141,20 @@ def validate_url(url: str) -> str | None:
|
||||
if not hostname:
|
||||
return 'URL is missing a host'
|
||||
|
||||
if allow_private:
|
||||
# Environment is explicitly trusted: skip the SSRF address checks.
|
||||
return None
|
||||
|
||||
if _hostname_is_blocked(hostname):
|
||||
return f'Refusing to fetch internal host "{hostname}"'
|
||||
|
||||
try:
|
||||
addrinfo = socket.getaddrinfo(hostname, parts.port, proto=socket.IPPROTO_TCP)
|
||||
except socket.gaierror:
|
||||
# Let yt-dlp surface a normal resolution error rather than masking it.
|
||||
return None
|
||||
# Fail closed: a host we cannot resolve is a host we cannot verify as
|
||||
# non-internal, so refuse it rather than letting the download proceed
|
||||
# to a target that may resolve differently at fetch time.
|
||||
return f'Could not resolve host "{hostname}"'
|
||||
except (UnicodeError, ValueError):
|
||||
return f'Invalid host "{hostname}"'
|
||||
|
||||
|
||||
+50
-17
@@ -26,7 +26,7 @@ from dl_formats import get_format, get_opts, AUDIO_FORMATS, merge_ytdl_option_la
|
||||
from datetime import datetime
|
||||
from state_store import AtomicJsonStore, from_json_compatible, read_legacy_shelf, to_json_compatible
|
||||
from subscriptions import _entry_id
|
||||
from url_guard import validate_url
|
||||
from url_guard import validate_url, install_socket_guard
|
||||
|
||||
log = logging.getLogger('ytdl')
|
||||
|
||||
@@ -145,16 +145,36 @@ def _sanitize_path_component(value: Any) -> Any:
|
||||
return value.lstrip('.').strip() or '_'
|
||||
|
||||
|
||||
def _output_dir_escapes(base_dir: str, output_template: str) -> bool:
|
||||
"""True when the literal directory prefix of *output_template* resolves outside *base_dir*."""
|
||||
marker = output_template.find('%(')
|
||||
literal = output_template if marker == -1 else output_template[:marker]
|
||||
dir_prefix = os.path.dirname(literal)
|
||||
if not dir_prefix:
|
||||
return False
|
||||
real_base = os.path.realpath(base_dir)
|
||||
real_target = os.path.realpath(os.path.join(base_dir, dir_prefix))
|
||||
return not _is_within_directory(real_base, real_target)
|
||||
class _ConfinedYoutubeDL(yt_dlp.YoutubeDL):
|
||||
"""A ``YoutubeDL`` that refuses to emit any output path outside the allowed roots.
|
||||
|
||||
This is the single authoritative enforcement of MeTube's download-directory
|
||||
containment invariant. yt-dlp expands output templates at download time using
|
||||
metadata that is fully attacker-controlled (``%(title)s``, ``%(uploader)s``,
|
||||
``%(section_title)s`` from chapter titles, …) and, on POSIX hosts, does *not*
|
||||
neutralise a ``..`` path component — so any template segment resolving to
|
||||
``..`` next to a literal separator (or an absolute template) can traverse out
|
||||
of the download directory. Every output path — main file, split-chapter files,
|
||||
thumbnails, subtitles, infojson — is produced by ``prepare_filename``, so
|
||||
validating its result here covers them all, regardless of which template or
|
||||
metadata field carries the traversal. Checking the resolved path (rather than
|
||||
the template string) is what makes this robust: the ``..`` only exists after
|
||||
expansion, so no ingress string check can see it.
|
||||
"""
|
||||
|
||||
def __init__(self, params=None, *, allowed_roots=(), **kwargs):
|
||||
self._allowed_roots = [os.path.realpath(r) for r in allowed_roots if r]
|
||||
super().__init__(params=params, **kwargs)
|
||||
|
||||
def prepare_filename(self, *args, **kwargs):
|
||||
filename = super().prepare_filename(*args, **kwargs)
|
||||
if filename and filename != '-' and self._allowed_roots:
|
||||
resolved = os.path.realpath(filename)
|
||||
if not any(_is_within_directory(root, resolved) for root in self._allowed_roots):
|
||||
raise yt_dlp.utils.DownloadError(
|
||||
f'Refusing to write outside the download directory: {filename}'
|
||||
)
|
||||
return filename
|
||||
|
||||
|
||||
# Regex matching yt-dlp output-template field references, e.g. ``%(title)s``
|
||||
@@ -537,11 +557,12 @@ class Download:
|
||||
cls.manager.shutdown()
|
||||
cls.manager = None
|
||||
|
||||
def __init__(self, download_dir, temp_dir, output_template, output_template_chapter, quality, format, ytdl_opts, info):
|
||||
def __init__(self, download_dir, temp_dir, output_template, output_template_chapter, quality, format, ytdl_opts, info, allow_private=False):
|
||||
self.download_dir = download_dir
|
||||
self.temp_dir = temp_dir
|
||||
self.output_template = output_template
|
||||
self.output_template_chapter = output_template_chapter
|
||||
self.allow_private = allow_private
|
||||
self.info = info
|
||||
self.format = get_format(
|
||||
getattr(info, 'download_type', 'video'),
|
||||
@@ -598,7 +619,10 @@ class Download:
|
||||
return put_status
|
||||
|
||||
def _make_youtube_dl(self, params):
|
||||
ydl = yt_dlp.YoutubeDL(params=params)
|
||||
ydl = _ConfinedYoutubeDL(
|
||||
params=params,
|
||||
allowed_roots=(self.download_dir, self.temp_dir),
|
||||
)
|
||||
if getattr(self.info, 'download_type', '') == 'audio':
|
||||
ydl.add_post_processor(_AlbumArtistPostProcessor(ydl), when='pre_process')
|
||||
return ydl
|
||||
@@ -612,6 +636,11 @@ class Download:
|
||||
os.setpgrp()
|
||||
except OSError:
|
||||
pass
|
||||
# Re-validate every outbound connection at fetch time. validate_url only
|
||||
# saw the submitted URL string; this catches redirects and DNS rebinding
|
||||
# to internal hosts (cloud metadata, RFC1918) that it cannot. Skipped when
|
||||
# ALLOW_PRIVATE_ADDRESSES trusts the environment (e.g. Fake-IP proxies).
|
||||
install_socket_guard(self.allow_private)
|
||||
log.info(f"Starting download for: {self.info.title} ({self.info.url})")
|
||||
try:
|
||||
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
|
||||
@@ -1242,6 +1271,11 @@ class DownloadQueue:
|
||||
return opts
|
||||
|
||||
def __extract_info(self, url, ytdl_options_presets=None, ytdl_options_overrides=None):
|
||||
# NOTE: extraction runs in the main process, so the connect-time socket
|
||||
# guard (installed only in the download subprocess) does not apply here.
|
||||
# The ingress validate_url check guards the submitted URL, but redirects
|
||||
# followed during extraction are not re-validated. See url_guard's module
|
||||
# docstring for why the guard can't be installed process-wide.
|
||||
debug_logging = logging.getLogger().isEnabledFor(logging.DEBUG)
|
||||
user_opts = self._build_ytdl_options(ytdl_options_presets, ytdl_options_overrides)
|
||||
params = {
|
||||
@@ -1301,9 +1335,7 @@ class DownloadQueue:
|
||||
if playlist_item_limit > 0:
|
||||
log.info(f'playlist limit is set. Processing only first {playlist_item_limit} entries')
|
||||
ytdl_options['playlistend'] = playlist_item_limit
|
||||
if _output_dir_escapes(dldirectory, output):
|
||||
return {'status': 'error', 'msg': 'Refusing download: resolved output path escapes the download directory'}
|
||||
download = Download(dldirectory, self.config.TEMP_DIR, output, output_chapter, dl.quality, dl.format, ytdl_options, dl)
|
||||
download = Download(dldirectory, self.config.TEMP_DIR, output, output_chapter, dl.quality, dl.format, ytdl_options, dl, allow_private=self.config.ALLOW_PRIVATE_ADDRESSES)
|
||||
is_upcoming = (
|
||||
getattr(dl, 'live_status', None) == 'is_upcoming'
|
||||
or getattr(dl, 'status', None) == 'scheduled'
|
||||
@@ -1526,7 +1558,8 @@ class DownloadQueue:
|
||||
# SSRF guard: reject non-http(s) schemes and hosts resolving to
|
||||
# internal/loopback/link-local/metadata addresses before yt-dlp fetches
|
||||
# anything. run_in_executor because validate_url may perform a DNS lookup.
|
||||
url_error = await asyncio.get_running_loop().run_in_executor(None, validate_url, url)
|
||||
url_error = await asyncio.get_running_loop().run_in_executor(
|
||||
None, partial(validate_url, url, allow_private=self.config.ALLOW_PRIVATE_ADDRESSES))
|
||||
if url_error is not None:
|
||||
log.warning('Rejected URL "%s": %s', url, url_error)
|
||||
return {'status': 'error', 'msg': url_error}
|
||||
|
||||
Reference in New Issue
Block a user