mirror of
https://github.com/alexta69/metube.git
synced 2026-09-21 21:45:04 +00:00
Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Alex Shnitman
|
a23d1689e3 | ||
|
Alex Shnitman
|
4b05022b91 | ||
|
Alex Shnitman
|
0dc9b0b3d6 | ||
|
Alex Shnitman
|
96f5ffe34a | ||
|
Alex Shnitman
|
1eafecd1cc | ||
|
Alex Shnitman
|
8fe81dea18 | ||
|
Alex
|
f054d84108 | ||
|
tjelite1986
|
fa02717e12 |
@@ -0,0 +1,21 @@
|
||||
name: readme-size
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- 'README.md'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'README.md'
|
||||
|
||||
jobs:
|
||||
check-size:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v7
|
||||
- name: Check README stays under Docker Hub's 25k character limit
|
||||
run: |
|
||||
size=$(wc -c < README.md)
|
||||
echo "README.md is ${size} bytes (limit 25000)"
|
||||
test "$size" -lt 25000
|
||||
@@ -1,5 +1,53 @@
|
||||
# Agent Guidelines
|
||||
|
||||
## Project scope — read this before planning a feature
|
||||
|
||||
MeTube's contract is: give it a URL, it runs yt-dlp well, and correct files appear.
|
||||
The maintainer holds a deliberate line on what belongs inside that contract, and PRs
|
||||
on the wrong side of it are declined **regardless of code quality**. Check your plan
|
||||
against this line before writing any code.
|
||||
|
||||
**In scope — improving the write itself:**
|
||||
|
||||
- Features that make the file yt-dlp writes at download time come out more correct,
|
||||
using only data the extractor already provides (e.g. filling a missing album-artist
|
||||
tag from the extractor's own metadata).
|
||||
- Surfacing functionality yt-dlp itself owns and maintains as first-class UI options
|
||||
(e.g. a SponsorBlock toggle that just passes yt-dlp postprocessor params).
|
||||
- Download queue, subscriptions, output templates, and UI improvements to the
|
||||
download workflow.
|
||||
|
||||
**Out of scope — managing files after they exist:**
|
||||
|
||||
- Tag editors, metadata dialogs, or any workflow that rewrites files after the
|
||||
download has finished. This holds even for slimmed-down versions.
|
||||
- Lookups against external metadata services (iTunes, Deezer, MusicBrainz, etc.).
|
||||
More broadly: any new dependency on a third-party API, or new network egress from
|
||||
self-hosted instances, beyond what yt-dlp itself performs.
|
||||
- Library organization: moving/renaming existing files into Artist/Album layouts,
|
||||
watch-folder processing, and similar media-manager features. Dedicated tools
|
||||
(beets, MusicBrainz Picard, Lidarr) do this properly; the README points users
|
||||
to them.
|
||||
|
||||
**Corollaries that shape borderline PRs:**
|
||||
|
||||
- Site-specific intelligence (parsing playlist-ID prefixes, URL path conventions,
|
||||
and other platform internals) is extractor work and belongs upstream in yt-dlp,
|
||||
not re-implemented here — it silently breaks when the platform changes and
|
||||
MeTube would own the breakage.
|
||||
- Prefer enriching yt-dlp's info dict and letting its existing pipeline
|
||||
(FFmpegMetadata etc.) do the writing, over adding custom per-format tag-writing
|
||||
code to MeTube.
|
||||
- Supplemental processing must never fail a download that otherwise succeeded:
|
||||
warn and continue, don't raise.
|
||||
- Keep feature scope minimal on first submission. A hardcoded sensible default
|
||||
beats a configuration surface; follow-ups can add options when users actually
|
||||
ask. PRs that bundle several "reasonable next steps" invite rejection of the
|
||||
whole.
|
||||
|
||||
If a feature idea fails this test, the accepted alternative is usually a README
|
||||
section documenting how to pair MeTube with the right dedicated tool.
|
||||
|
||||
## README.md size constraint
|
||||
|
||||
The README.md is synced to Docker Hub, which has a **25,000 character limit**.
|
||||
@@ -30,6 +78,24 @@ uv run pytest app/tests/
|
||||
|
||||
All of these run in CI (`.github/workflows/main.yml`) on every push to master and must pass.
|
||||
|
||||
Gotchas:
|
||||
|
||||
- Backend tests must run **from the repo root**: `main.py` resolves the static-assets
|
||||
path relative to the cwd, and several test modules import `main`. Running from
|
||||
`app/` makes five test modules fail to import.
|
||||
- The frontend must be **built before** running backend tests (same reason — the
|
||||
assets at `ui/dist/metube/browser` must exist). The command order above is
|
||||
load-bearing.
|
||||
- `app/tests/test_ytdl_utils.py` stubs `yt_dlp` at import time. Run standalone,
|
||||
two tests fail with `AttributeError: <module 'yt_dlp'> does not have the
|
||||
attribute 'YoutubeDL'`; under the full suite the real module is imported first
|
||||
and they pass. This is a known quirk, not a bug to fix in the code under test.
|
||||
|
||||
Every non-markdown push to master builds multi-arch Docker images and cuts a dated
|
||||
release the same day. **Master is continuously released** — a PR must be
|
||||
release-ready exactly as merged; there is no stabilization window for follow-up
|
||||
fixes.
|
||||
|
||||
## Code style
|
||||
|
||||
Follow `.editorconfig`:
|
||||
@@ -56,5 +122,36 @@ ui/src/app/ — Angular standalone components (no NgModules)
|
||||
- Backend configuration lives in the `Config` class in `app/main.py` with env-var defaults in `_DEFAULTS`. New env vars go there.
|
||||
- Real-time communication uses Socket.IO events, not REST polling.
|
||||
- Frontend uses standalone Angular components with `inject()` for DI, RxJS Subjects for state, and `takeUntilDestroyed()` for cleanup.
|
||||
- Frontend components use OnPush change detection: subscribe callbacks must call `cdr.markForCheck()`.
|
||||
- State is persisted as JSON files via `AtomicJsonStore` in `app/state_store.py`.
|
||||
- Persisted state stays compact: the completed queue deliberately drops bulky entry data (see `_compact_persisted_entry` in `app/ytdl.py`). Don't expand what gets persisted without discussion.
|
||||
- Custom yt-dlp postprocessors added to `ytdl_params['postprocessors']` run in **list order** within a stage. When combining postprocessors, mirror the ordering the yt-dlp CLI would produce (e.g. sponsor-segment removal before chapter splitting).
|
||||
- No pre-commit hooks — linting and tests are enforced in CI only.
|
||||
|
||||
## Checklist: adding a per-download option
|
||||
|
||||
New options on the download form (the `split_by_chapters` pattern) need **all** of
|
||||
these pieces — the last three are the ones commonly missed:
|
||||
|
||||
1. `parse_download_options` in `app/main.py`.
|
||||
2. A field on `DownloadInfo` in `app/ytdl.py`.
|
||||
3. A `hasattr` backfill in `DownloadInfo.__setstate__` for old persisted records.
|
||||
4. The safe-deserialization field list in `app/ytdl.py`.
|
||||
5. UI form control + cookie persistence in `ui/src/app/app.ts` / `app.html`, and
|
||||
the payload in `downloads.service.ts` (plus its spec).
|
||||
6. The redownload path in `app.ts`, so retries carry the option.
|
||||
7. If the option makes sense for unattended downloads: threading through
|
||||
`app/subscriptions.py` (`SubscriptionInfo` field, serializer, add/update
|
||||
routes, the enqueue call) — or a note in the PR that it's deliberately
|
||||
direct-downloads-only.
|
||||
|
||||
## Security invariants
|
||||
|
||||
User input and extractor-provided metadata (titles, playlist names, URLs) are
|
||||
untrusted. Use the existing guards instead of hand-rolling:
|
||||
|
||||
- User-submitted URLs go through the SSRF guard (see `test_url_guard.py` for the
|
||||
expected behavior).
|
||||
- Anything that becomes a filesystem path goes through `_is_within_directory` and
|
||||
`_sanitize_path_component` in `app/ytdl.py` — including values that arrive via
|
||||
yt-dlp metadata, which sites can influence.
|
||||
|
||||
@@ -3,12 +3,12 @@
|
||||
|
||||
|
||||
|
||||
MeTube is a self-hosted web UI for `yt-dlp`, for downloading media from YouTube and [dozens of other sites](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md).
|
||||
MeTube is a self-hosted web UI for `yt-dlp`, for downloading media from YouTube and [dozens of other sites](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md). Docker images are multi-arch (amd64/arm64).
|
||||
|
||||
Key capabilities:
|
||||
* Download videos, audio, captions, and thumbnails from a browser UI.
|
||||
* Download playlists and channels, with configurable output and download options.
|
||||
* Subscribe to channels and playlists, periodically check for new items, and queue new uploads automatically.
|
||||
* [Subscribe](https://github.com/alexta69/metube/wiki/Subscriptions) to channels and playlists, periodically check for new items, and queue new uploads automatically.
|
||||
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Key capabilities:
|
||||
docker run -d -p 8081:8081 -v /path/to/downloads:/downloads ghcr.io/alexta69/metube
|
||||
```
|
||||
|
||||
## 🐳 Run using docker-compose
|
||||
## 🐳 Run using Docker Compose
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -34,11 +34,20 @@ services:
|
||||
|
||||
## ⚙️ Configuration via environment variables
|
||||
|
||||
Certain values can be set via environment variables, using the `-e` parameter on the docker command line, or the `environment:` section in docker-compose.
|
||||
Certain values can be set via environment variables, using the `-e` parameter on the docker command line, or the `environment:` section in Docker Compose.
|
||||
|
||||
### 🏠 Runtime & Permissions
|
||||
|
||||
* __PUID__: User under which MeTube will run. Defaults to `1000` (legacy `UID` also supported).
|
||||
* __PGID__: Group under which MeTube will run. Defaults to `1000` (legacy `GID` also supported).
|
||||
* __UMASK__: Umask value used by MeTube. Defaults to `022`.
|
||||
* __DEFAULT_THEME__: Default theme to use for the UI, can be set to `light`, `dark`, or `auto`. Defaults to `auto`.
|
||||
* __LOGLEVEL__: Log level, can be set to `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`, or `NONE`. Defaults to `INFO`.
|
||||
* __ENABLE_ACCESSLOG__: Whether to enable access log. Defaults to `false`.
|
||||
|
||||
### ⬇️ Download Behavior
|
||||
|
||||
* __MAX_CONCURRENT_DOWNLOADS__: Maximum number of simultaneous downloads allowed. For example, if set to `5`, then at most five downloads will run concurrently, and any additional downloads will wait until one of the active downloads completes. Defaults to `3`.
|
||||
* __MAX_CONCURRENT_DOWNLOADS__: Maximum number of simultaneous downloads allowed. For example, if set to `5`, then at most five downloads will run concurrently, and any additional downloads will wait until one of the active downloads completes. Defaults to `3`.
|
||||
* __DELETE_FILE_ON_TRASHCAN__: if `true`, downloaded files are deleted on the server, when they are trashed from the "Completed" section of the UI. Defaults to `false`.
|
||||
* __DEFAULT_OPTION_PLAYLIST_ITEM_LIMIT__: Maximum number of playlist items that can be downloaded. Defaults to `0` (no limit).
|
||||
* __SUBSCRIPTION_DEFAULT_CHECK_INTERVAL__: Default minutes between automatic checks for each subscription. Defaults to `60`.
|
||||
@@ -83,18 +92,9 @@ Certain values can be set via environment variables, using the `-e` parameter on
|
||||
* __HTTPS__: Use `https` instead of `http` (__CERTFILE__ and __KEYFILE__ required). Defaults to `false`.
|
||||
* __CERTFILE__: HTTPS certificate file path.
|
||||
* __KEYFILE__: HTTPS key file path.
|
||||
* __CORS_ALLOWED_ORIGINS__: Comma-separated list of origins permitted to make cross-origin requests to the MeTube API. When unset or empty, all cross-origin requests are denied. Set to `*` to allow all origins. This must be configured for [browser extensions](#-browser-extensions), [bookmarklets](#-bookmarklet), and any other browser-based tools that contact MeTube from a different origin. For browser extensions use `*` (see below); for bookmarklets you can list specific sites, e.g. `https://www.youtube.com,https://www.vimeo.com`.
|
||||
* __CORS_ALLOWED_ORIGINS__: Comma-separated list of origins permitted to make cross-origin requests to the MeTube API; `*` allows all. When unset or empty, all cross-origin requests are denied. Required for browser extensions and bookmarklets — see [Sending links to MeTube](#-sending-links-to-metube).
|
||||
* __ROBOTS_TXT__: A path to a `robots.txt` file mounted in the container.
|
||||
|
||||
### 🏠 Basic Setup
|
||||
|
||||
* __PUID__: User under which MeTube will run. Defaults to `1000` (legacy `UID` also supported).
|
||||
* __PGID__: Group under which MeTube will run. Defaults to `1000` (legacy `GID` also supported).
|
||||
* __UMASK__: Umask value used by MeTube. Defaults to `022`.
|
||||
* __DEFAULT_THEME__: Default theme to use for the UI, can be set to `light`, `dark`, or `auto`. Defaults to `auto`.
|
||||
* __LOGLEVEL__: Log level, can be set to `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`, or `NONE`. Defaults to `INFO`.
|
||||
* __ENABLE_ACCESSLOG__: Whether to enable access log. Defaults to `false`.
|
||||
|
||||
## 🎛️ Configuring yt-dlp options
|
||||
|
||||
MeTube lets you customize how [yt-dlp](https://github.com/yt-dlp/yt-dlp) behaves at three levels, from broadest to most specific:
|
||||
@@ -229,47 +229,27 @@ In case you need to use your browser's cookies with MeTube, for example to downl
|
||||
* After upload, the cookie indicator should show as active.
|
||||
* Use **Delete Cookies** in the same section to remove uploaded cookies.
|
||||
|
||||
## 🔌 Browser extensions
|
||||
## 🔗 Sending links to MeTube
|
||||
|
||||
Browser extensions allow right-clicking videos and sending them directly to MeTube. If you're on an HTTPS page, your MeTube instance must be behind an HTTPS reverse proxy (see below) for extensions to work.
|
||||
Several integrations let you send URLs to MeTube from wherever you are, instead of pasting them into the UI. The browser-based ones make cross-origin requests, so they require `CORS_ALLOWED_ORIGINS` to be set; and if you're on an HTTPS page, your MeTube instance must be served over HTTPS too (with `HTTPS=true` or behind an HTTPS reverse proxy — see below).
|
||||
|
||||
Since browser extensions make requests from their own origin (`chrome-extension://...` or `moz-extension://...`), you must set `CORS_ALLOWED_ORIGINS=*` for them to work.
|
||||
__Browser extensions__ allow right-clicking videos and sending them directly to MeTube. Since extensions request from their own origin, set `CORS_ALLOWED_ORIGINS=*`.
|
||||
* __Chrome:__ contributed by [Rpsl](https://github.com/rpsl) — install from the [Chrome Webstore](https://chrome.google.com/webstore/detail/metube-downloader/fbmkmdnlhacefjljljlbhkodfmfkijdh) or [from sources](https://github.com/Rpsl/metube-browser-extension).
|
||||
* __Firefox:__ contributed by [nanocortex](https://github.com/nanocortex) — install from [Firefox Addons](https://addons.mozilla.org/en-US/firefox/addon/metube-downloader) or get sources [here](https://github.com/nanocortex/metube-firefox-addon).
|
||||
|
||||
__Chrome:__ contributed by [Rpsl](https://github.com/rpsl). You can install it from [Google Chrome Webstore](https://chrome.google.com/webstore/detail/metube-downloader/fbmkmdnlhacefjljljlbhkodfmfkijdh) or use developer mode and install [from sources](https://github.com/Rpsl/metube-browser-extension).
|
||||
__Bookmarklets__ send the currently open page to MeTube with one click. Add the origins of the sites where you use them to `CORS_ALLOWED_ORIGINS`, e.g. `https://www.youtube.com,https://www.vimeo.com`. The code (Chrome and Firefox variants, contributed by [kushfest](https://github.com/kushfest) and [shoonya75](https://github.com/shoonya75)) is in the [Bookmarklets wiki page](https://github.com/alexta69/metube/wiki/Bookmarklets).
|
||||
|
||||
__Firefox:__ contributed by [nanocortex](https://github.com/nanocortex). You can install it from [Firefox Addons](https://addons.mozilla.org/en-US/firefox/addon/metube-downloader) or get sources from [here](https://github.com/nanocortex/metube-firefox-addon).
|
||||
__iOS Shortcut:__ [rithask](https://github.com/rithask) created an [iOS shortcut](https://www.icloud.com/shortcuts/66627a9f334c467baabdb2769763a1a6) for sending URLs to MeTube from Safari's share menu; it prompts for your instance address on first use.
|
||||
|
||||
## 📱 iOS Shortcut
|
||||
__Raycast:__ [dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) for adding videos to MeTube directly from Raycast.
|
||||
|
||||
[rithask](https://github.com/rithask) created an iOS shortcut to send URLs to MeTube from Safari. Enter the MeTube instance address when prompted which will be saved for later use. You can run the shortcut from Safari’s share menu. The shortcut can be downloaded from [this iCloud link](https://www.icloud.com/shortcuts/66627a9f334c467baabdb2769763a1a6).
|
||||
## 🎵 Pairing with a music tagger
|
||||
|
||||
## 🔖 Bookmarklet
|
||||
MeTube deliberately stops once the file is written — tagging and library organization belong to dedicated tools. Point one at your audio download folder (`AUDIO_DOWNLOAD_DIR`):
|
||||
|
||||
[kushfest](https://github.com/kushfest) has created a Chrome bookmarklet for sending the currently open webpage to MeTube. Please note that if you're on an HTTPS page, your MeTube instance must be configured with `HTTPS` as `true` in the environment, or be behind an HTTPS reverse proxy (see below) for the bookmarklet to work.
|
||||
|
||||
Since bookmarklets run in the context of the current page (e.g. youtube.com), the requests they make to MeTube are cross-origin. You must add the origins of sites where you use the bookmarklet to the __CORS_ALLOWED_ORIGINS__ environment variable, otherwise the browser will block the requests. For example, to use the bookmarklet on YouTube and Vimeo: `CORS_ALLOWED_ORIGINS=https://www.youtube.com,https://www.vimeo.com`.
|
||||
|
||||
GitHub doesn't allow embedding JavaScript as a link, so the bookmarklet has to be created manually by copying the following code to a new bookmark you create on your bookmarks bar. Change the hostname in the URL below to point to your MeTube instance.
|
||||
|
||||
```javascript
|
||||
javascript:!function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.withCredentials=true;xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function(){if(xhr.status==200){alert("Sent to metube!")}else{alert("Send to metube failed. Check the javascript console for clues.")}}}();
|
||||
```
|
||||
|
||||
[shoonya75](https://github.com/shoonya75) has contributed a Firefox version:
|
||||
|
||||
```javascript
|
||||
javascript:(function(){xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function(){if(xhr.status==200){alert("Sent to metube!")}else{alert("Send to metube failed. Check the javascript console for clues.")}}})();
|
||||
```
|
||||
|
||||
The above bookmarklets use `alert()` for notifications. This variant shows a toast instead (Chrome — for Firefox, replace the `!function(){...}()` wrapper with `(function(){...})()`):
|
||||
|
||||
```javascript
|
||||
javascript:!function(){function notify(msg) {var sc = document.scrollingElement.scrollTop; var text = document.createElement('span');text.innerHTML=msg;var ts = text.style;ts.all = 'revert';ts.color = '#000';ts.fontFamily = 'Verdana, sans-serif';ts.fontSize = '15px';ts.backgroundColor = 'white';ts.padding = '15px';ts.border = '1px solid gainsboro';ts.boxShadow = '3px 3px 10px';ts.zIndex = '100';document.body.appendChild(text);ts.position = 'absolute'; ts.top = 50 + sc + 'px'; ts.left = (window.innerWidth / 2)-(text.offsetWidth / 2) + 'px'; setTimeout(function () { text.style.visibility = "hidden"; }, 1500);}xhr=new XMLHttpRequest();xhr.open("POST","https://metube.domain.com/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function() { if(xhr.status==200){notify("Sent to metube!")}else {notify("Send to metube failed. Check the javascript console for clues.")}}}();
|
||||
```
|
||||
|
||||
## ⚡ Raycast extension
|
||||
|
||||
[dotvhs](https://github.com/dotvhs) has created an [extension for Raycast](https://www.raycast.com/dot/metube) for adding videos to MeTube directly from Raycast.
|
||||
* [beets](https://beets.io) — `beet import` matches tracks against MusicBrainz, fixes tags, and files them into an Artist/Album library; headless and scriptable.
|
||||
* [MusicBrainz Picard](https://picard.musicbrainz.org) — GUI tagger with acoustic fingerprinting.
|
||||
* [Lidarr](https://lidarr.audio) — full music library manager; add the folder as an import path.
|
||||
|
||||
## 🔒 HTTPS support, and running behind a reverse proxy
|
||||
|
||||
@@ -293,11 +273,7 @@ services:
|
||||
- KEYFILE=/ssl/key.pem
|
||||
```
|
||||
|
||||
MeTube can also run behind a reverse proxy for HTTPS termination or authentication. When serving under a subdirectory, set `URL_PREFIX` accordingly.
|
||||
|
||||
The [linuxserver/swag](https://docs.linuxserver.io/general/swag) image includes ready-made snippets for MeTube in [subfolder](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subfolder.conf.sample) and [subdomain](https://github.com/linuxserver/reverse-proxy-confs/blob/master/metube.subdomain.conf.sample) modes, plus Authelia for authentication.
|
||||
|
||||
### 🌐 NGINX
|
||||
MeTube can also run behind a reverse proxy for HTTPS termination or authentication. When serving under a subdirectory, set `URL_PREFIX` accordingly. MeTube uses WebSocket for real-time updates, so the proxy must pass the `Upgrade`/`Connection` headers, as in this NGINX example:
|
||||
|
||||
```nginx
|
||||
location /metube/ {
|
||||
@@ -309,41 +285,7 @@ location /metube/ {
|
||||
}
|
||||
```
|
||||
|
||||
Note: the extra `proxy_set_header` directives are there to make WebSocket work.
|
||||
|
||||
### 🌐 Apache
|
||||
|
||||
Contributed by [PIE-yt](https://github.com/PIE-yt). Source [here](https://gist.github.com/PIE-yt/29e7116588379032427f5bd446b2cac4).
|
||||
|
||||
```apache
|
||||
# For putting in your Apache sites site.conf
|
||||
# Serves MeTube under a /metube/ subdir (http://yourdomain.com/metube/)
|
||||
<Location /metube/>
|
||||
ProxyPass http://localhost:8081/ retry=0 timeout=30
|
||||
ProxyPassReverse http://localhost:8081/
|
||||
</Location>
|
||||
|
||||
<Location /metube/socket.io>
|
||||
RewriteEngine On
|
||||
RewriteCond %{QUERY_STRING} transport=websocket [NC]
|
||||
RewriteRule /(.*) ws://localhost:8081/socket.io/$1 [P,L]
|
||||
ProxyPass http://localhost:8081/socket.io retry=0 timeout=30
|
||||
ProxyPassReverse http://localhost:8081/socket.io
|
||||
</Location>
|
||||
```
|
||||
|
||||
### 🌐 Caddy
|
||||
|
||||
The following example Caddyfile gets a reverse proxy going behind [caddy](https://caddyserver.com).
|
||||
|
||||
```caddyfile
|
||||
example.com {
|
||||
route /metube/* {
|
||||
uri strip_prefix metube
|
||||
reverse_proxy metube:8081
|
||||
}
|
||||
}
|
||||
```
|
||||
Apache, Caddy, and [linuxserver/swag](https://docs.linuxserver.io/general/swag) (with Authelia) examples are in the [Reverse proxy configurations wiki page](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations).
|
||||
|
||||
## 🔄 Updating yt-dlp
|
||||
|
||||
@@ -358,9 +300,11 @@ docker exec -ti metube sh
|
||||
cd /downloads
|
||||
```
|
||||
|
||||
Common issues and their fixes are collected in the [Troubleshooting FAQ](https://github.com/alexta69/metube/wiki/Troubleshooting-FAQ) on the wiki.
|
||||
|
||||
## 💡 Submitting feature requests
|
||||
|
||||
MeTube development relies on community contributions. If you need additional features, please submit a PR. Create an issue first to discuss the implementation — some PRs may not be accepted to reduce bloat. Feature requests without an accompanying PR are unlikely to be fulfilled.
|
||||
MeTube development relies on community contributions. If you need additional features, please submit a PR. Create an issue first to discuss the implementation before writing code — MeTube's scope is deliberately narrow: it downloads well and stops once the file is written. Features that improve the download itself are welcome; post-download file management (tag editing, metadata lookups, library organization) is out of scope regardless of implementation quality — see [AGENTS.md](AGENTS.md) for the full policy. Feature requests without an accompanying PR are unlikely to be fulfilled.
|
||||
|
||||
## 🛠️ Building and running locally
|
||||
|
||||
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
# Security Policy
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Please report vulnerabilities privately via
|
||||
[GitHub private vulnerability reporting](https://github.com/alexta69/metube/security/advisories/new)
|
||||
(Security tab → "Report a vulnerability"). Do **not** open a public issue for
|
||||
security problems.
|
||||
|
||||
You can expect an initial response within a few days. Please include a
|
||||
reproduction and the MeTube release version (visible in the UI footer).
|
||||
|
||||
## Supported versions
|
||||
|
||||
MeTube is continuously released; only the **latest release** is supported.
|
||||
Update to the current Docker image before reporting.
|
||||
|
||||
## Scope notes
|
||||
|
||||
MeTube ships **without authentication by design** — it is intended to run on a
|
||||
trusted network or behind an authenticating reverse proxy (see the
|
||||
[wiki](https://github.com/alexta69/metube/wiki/Reverse-proxy-configurations)).
|
||||
Reports that reduce to "the UI is reachable without a login" are expected
|
||||
behavior, not vulnerabilities.
|
||||
@@ -49,6 +49,7 @@ sys.modules.setdefault("yt_dlp.postprocessor", fake_postprocessor)
|
||||
sys.modules.setdefault("yt_dlp.postprocessor.common", fake_postprocessor_common)
|
||||
sys.modules.setdefault("yt_dlp.utils", fake_utils)
|
||||
|
||||
import ytdl
|
||||
from ytdl import (
|
||||
Download,
|
||||
DownloadInfo,
|
||||
@@ -405,11 +406,16 @@ class ProgressThrottleTests(unittest.TestCase):
|
||||
|
||||
|
||||
class CancelProcessGroupTests(unittest.TestCase):
|
||||
def test_cancel_kills_group_when_child_is_group_leader(self):
|
||||
# cancel() now sends SIGINT first (so yt-dlp/ffmpeg can finalize the
|
||||
# partial file) and schedules a SIGKILL escalation via the event loop
|
||||
# after ytdl._CANCEL_GRACE_SECONDS, instead of SIGKILLing immediately.
|
||||
|
||||
def test_cancel_sends_sigint_to_group_and_schedules_sigkill_escalation(self):
|
||||
# Child successfully ran os.setpgrp(): its pgid equals its own pid.
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
dl.status_queue = types.SimpleNamespace(put=lambda _item: None)
|
||||
dl.loop = MagicMock()
|
||||
|
||||
with patch.object(Download, "running", return_value=True), \
|
||||
patch("ytdl.os.getpgid", return_value=4321) as mock_getpgid, \
|
||||
@@ -417,38 +423,109 @@ class CancelProcessGroupTests(unittest.TestCase):
|
||||
dl.cancel()
|
||||
|
||||
mock_getpgid.assert_called_once_with(4321)
|
||||
mock_killpg.assert_called_once_with(4321, signal.SIGKILL)
|
||||
mock_killpg.assert_called_once_with(4321, signal.SIGINT)
|
||||
dl.loop.call_later.assert_called_once_with(ytdl._CANCEL_GRACE_SECONDS, dl._kill_if_alive)
|
||||
self.assertTrue(dl.canceled)
|
||||
|
||||
def test_cancel_does_not_killpg_parent_group_kills_child_only(self):
|
||||
def test_cancel_does_not_killpg_parent_group_signals_child_only(self):
|
||||
# Child has NOT become its own group leader yet (pgid != pid, e.g. it is
|
||||
# still in the server's process group). killpg must NOT be called — that
|
||||
# would SIGKILL the whole server — and we fall back to proc.kill().
|
||||
# would signal the whole server — and we fall back to os.kill(pid, SIGINT).
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321, kill=MagicMock())
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
dl.status_queue = types.SimpleNamespace(put=lambda _item: None)
|
||||
dl.loop = MagicMock()
|
||||
|
||||
with patch.object(Download, "running", return_value=True), \
|
||||
patch("ytdl.os.getpgid", return_value=999), \
|
||||
patch("ytdl.os.killpg") as mock_killpg:
|
||||
patch("ytdl.os.killpg") as mock_killpg, \
|
||||
patch("ytdl.os.kill") as mock_kill:
|
||||
dl.cancel()
|
||||
|
||||
mock_killpg.assert_not_called()
|
||||
dl.proc.kill.assert_called_once()
|
||||
mock_kill.assert_called_once_with(4321, signal.SIGINT)
|
||||
dl.loop.call_later.assert_called_once_with(ytdl._CANCEL_GRACE_SECONDS, dl._kill_if_alive)
|
||||
self.assertTrue(dl.canceled)
|
||||
|
||||
def test_cancel_falls_back_to_proc_kill_when_getpgid_unavailable(self):
|
||||
def test_cancel_falls_back_to_pid_signal_when_getpgid_unavailable(self):
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
dl.status_queue = types.SimpleNamespace(put=lambda _item: None)
|
||||
dl.loop = MagicMock()
|
||||
|
||||
with patch.object(Download, "running", return_value=True), \
|
||||
patch("ytdl.os.getpgid", side_effect=OSError("no such process")), \
|
||||
patch("ytdl.os.kill") as mock_kill:
|
||||
dl.cancel()
|
||||
|
||||
mock_kill.assert_called_once_with(4321, signal.SIGINT)
|
||||
dl.loop.call_later.assert_called_once_with(ytdl._CANCEL_GRACE_SECONDS, dl._kill_if_alive)
|
||||
self.assertTrue(dl.canceled)
|
||||
|
||||
def test_cancel_kills_immediately_when_signal_delivery_fails(self):
|
||||
# Neither killpg nor os.kill succeed (process already gone): cancel()
|
||||
# must fall through to an immediate SIGKILL attempt instead of
|
||||
# scheduling a pointless escalation.
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321, kill=MagicMock())
|
||||
dl.status_queue = types.SimpleNamespace(put=lambda _item: None)
|
||||
dl.loop = MagicMock()
|
||||
|
||||
with patch.object(Download, "running", return_value=True), \
|
||||
patch("ytdl.os.getpgid", side_effect=OSError("no such process")):
|
||||
patch("ytdl.os.getpgid", side_effect=OSError("no such process")), \
|
||||
patch("ytdl.os.kill", side_effect=ProcessLookupError()):
|
||||
dl.cancel()
|
||||
|
||||
dl.loop.call_later.assert_not_called()
|
||||
dl.proc.kill.assert_called_once()
|
||||
self.assertTrue(dl.canceled)
|
||||
|
||||
def test_cancel_schedules_escalation_even_without_running_loop(self):
|
||||
# dl.loop is None (e.g. cancel() called before start()'s run_in_executor
|
||||
# set it up): _kill_if_alive() must run synchronously instead of being
|
||||
# scheduled, since there's no loop to schedule it on.
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
dl.status_queue = types.SimpleNamespace(put=lambda _item: None)
|
||||
self.assertIsNone(dl.loop)
|
||||
|
||||
with patch.object(Download, "running", side_effect=[True, True]), \
|
||||
patch("ytdl.os.getpgid", return_value=4321), \
|
||||
patch("ytdl.os.killpg") as mock_killpg:
|
||||
dl.cancel()
|
||||
|
||||
# First SIGINT, then _kill_if_alive() ran inline and sent SIGKILL.
|
||||
mock_killpg.assert_has_calls([
|
||||
unittest.mock.call(4321, signal.SIGINT),
|
||||
unittest.mock.call(4321, signal.SIGKILL),
|
||||
])
|
||||
self.assertTrue(dl.canceled)
|
||||
|
||||
|
||||
class KillIfAliveTests(unittest.TestCase):
|
||||
def test_kill_if_alive_sigkills_running_process(self):
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
|
||||
with patch.object(Download, "running", return_value=True), \
|
||||
patch("ytdl.os.getpgid", return_value=4321), \
|
||||
patch("ytdl.os.killpg") as mock_killpg:
|
||||
dl._kill_if_alive()
|
||||
|
||||
mock_killpg.assert_called_once_with(4321, signal.SIGKILL)
|
||||
|
||||
def test_kill_if_alive_noop_when_process_already_exited(self):
|
||||
dl = _make_test_download()
|
||||
dl.proc = types.SimpleNamespace(pid=4321)
|
||||
|
||||
with patch.object(Download, "running", return_value=False), \
|
||||
patch("ytdl.os.killpg") as mock_killpg, \
|
||||
patch("ytdl.os.kill") as mock_kill:
|
||||
dl._kill_if_alive()
|
||||
|
||||
mock_killpg.assert_not_called()
|
||||
mock_kill.assert_not_called()
|
||||
|
||||
|
||||
class ConvertSrtToTxtTests(unittest.TestCase):
|
||||
def test_basic_conversion(self):
|
||||
|
||||
+45
-20
@@ -47,6 +47,10 @@ _MP_CTX = (
|
||||
else multiprocessing.get_context()
|
||||
)
|
||||
|
||||
# Grace period between SIGINT (lets yt-dlp/ffmpeg finalize the partial file,
|
||||
# e.g. when cancelling a livestream) and SIGKILL escalation.
|
||||
_CANCEL_GRACE_SECONDS = 15
|
||||
|
||||
_LIVE_CHECK_INTERVAL = 60
|
||||
_LIVE_MAX_CHECK_INTERVAL = 3600
|
||||
# Consecutive probe failures (network blips, rate limits, transient extractor
|
||||
@@ -704,29 +708,50 @@ class Download:
|
||||
self.status_queue.put(None)
|
||||
await self.status_task
|
||||
|
||||
def _signal_group(self, sig):
|
||||
"""Send *sig* to the download's process group, falling back to the
|
||||
process itself. Returns True if a signal was delivered.
|
||||
|
||||
Only signal the whole group when the child actually became its own
|
||||
group leader via os.setpgrp() in _download() — that sets its pgid
|
||||
equal to its own pid. If it hasn't run setpgrp() yet, or setpgrp()
|
||||
failed, its pgid is still the SERVER's group and killpg would signal
|
||||
the entire MeTube process (PID 1 in Docker). Fall back to signalling
|
||||
just the child process by pid in that case.
|
||||
"""
|
||||
try:
|
||||
pgid = os.getpgid(self.proc.pid)
|
||||
if pgid == self.proc.pid:
|
||||
os.killpg(pgid, sig)
|
||||
return True
|
||||
except (OSError, AttributeError):
|
||||
pass
|
||||
try:
|
||||
if sig == signal.SIGINT:
|
||||
os.kill(self.proc.pid, sig)
|
||||
else:
|
||||
self.proc.kill()
|
||||
return True
|
||||
except Exception as e:
|
||||
log.error(f"Error signalling process for {self.info.title}: {e}")
|
||||
return False
|
||||
|
||||
def _kill_if_alive(self):
|
||||
if self.running():
|
||||
log.info(f"Escalating cancel to SIGKILL for: {self.info.title}")
|
||||
self._signal_group(signal.SIGKILL)
|
||||
|
||||
def cancel(self):
|
||||
log.info(f"Cancelling download: {self.info.title}")
|
||||
if self.running():
|
||||
killed_group = False
|
||||
try:
|
||||
pgid = os.getpgid(self.proc.pid)
|
||||
# Only kill the whole group (yt-dlp + any ffmpeg children) when
|
||||
# the child actually became its own group leader via
|
||||
# os.setpgrp() in _download() — that sets its pgid equal to its
|
||||
# own pid. If it hasn't run setpgrp() yet, or setpgrp() failed,
|
||||
# its pgid is still the SERVER's group and killpg would SIGKILL
|
||||
# the entire MeTube process (PID 1 in Docker). Fall back to
|
||||
# killing just the child process by pid in that case.
|
||||
if pgid == self.proc.pid:
|
||||
os.killpg(pgid, signal.SIGKILL)
|
||||
killed_group = True
|
||||
except (OSError, AttributeError):
|
||||
pass
|
||||
if not killed_group:
|
||||
try:
|
||||
self.proc.kill()
|
||||
except Exception as e:
|
||||
log.error(f"Error killing process for {self.info.title}: {e}")
|
||||
# SIGINT first so yt-dlp/ffmpeg can finalize the partial file
|
||||
# (livestream recordings stay playable); SIGKILL after a grace
|
||||
# period if the process ignores it.
|
||||
interrupted = self._signal_group(signal.SIGINT)
|
||||
if interrupted and self.loop is not None:
|
||||
self.loop.call_later(_CANCEL_GRACE_SECONDS, self._kill_if_alive)
|
||||
else:
|
||||
self._kill_if_alive()
|
||||
self.canceled = True
|
||||
if self.status_queue is not None:
|
||||
self.status_queue.put(None)
|
||||
|
||||
Reference in New Issue
Block a user