Managed YouTube Downloader

yt-dlp (previously youtube-dl) is very good for downloading YouTube videos, but deciding which videos to download is tricky to automate. This download manager checks your YouTube channel subscriptions and downloads videos based on the criteria you set.

All configuration is done using YAML, and can be set for individual channels or globally.

Reasons it's interesting:

1. Uses the YouTube Data API (with OAuth 2.0 credentials)
2. Represents and constructs (dumps and loads) custom classes in YAML
3. Uses YAML as a way to get and set properties of a program
4. Contains a quick how-to guide for setting up your own app's OAuth 2.0 credentials with Google. If you think this shouldn't be too difficult or strange, boy do I have a few wasted hours of my life to share with you...

---

Installation

Install the pyhon project requirements like normal

pip3 install -r requirements.txt

Credentials

Obtaining the YouTube API credentials is an awkward process (described by Google here), but I've tried to simplify it below. This app just needs read-only access to the channels you're subscribed to (which can be seen in the fetcher.py file), but the process is awkward all the same.

Enable the YouTube Data API for your account

Log in to to your Google API console and enable the YouTube Data API.

Obtain OAuth 2.0 credentials

You need to set up your OAuth consent screen. That's right, you have to create a consent screen for yourself... I just called my version "App", and clicked save, but you can go nuts if you'd like.

Go to the credentials section of the API console, click the Create Credentials button and choose OAuth Client ID. Make it a Desktop app and give it a name.

Back on the credentials screen, you can download the json file containing the client secrets. Keep this somewhere safe.

---

Operations

Initialisation

Create the YAML config file using the command:

python3 managedYoutubeDL init /path/to/secrets.json config.yaml

where /path/to/secrets.json is the client-secrets file obtained in the previous step, and config.yaml is the name you want to call your YAML config file.

Google will display a link to go to in order to authorise the application. Paste the code into the console, and you're done forever with authorisation. The manager will get a list of your channel subscriptions and create your YAML configuration file.

Download new videos

To process the config.yaml file and download the new videos in your subscribed channels, run:

python3 managedYoutubeDL download-new config.yaml

Update channel list

If your subscription list has changed, you can add new channels and remove old ones from config.yaml using:

python3 managedYoutubeDL update-channels config.yaml

Manual video download

To download one or more YouTube videos directly:

python3 managedYoutubeDL manual-download config.yaml "videoURL1 videoURL2 videoURL3"

where the videoURLs are the direct links to the YouTube videos. Note, this command does not use the YouTube API, and therefore does not cost API credits.

---

Configuring your download preferences

An example YAML configuration file is shown here:

!Manager
clientSecretsFile: /path/to/secrets.json
downloadDirectory: /path/to/download/directory
ffmpegLocation: /path/to/ffmpeg
pickledCredentials: <LONG ASCII STRING>
globalMinVideoDate: 2020-06-01 12:00:00
globalMaxVideoDate: null
globalIncludeFilter: null
globalExcludeFilter: null
globalMaxVideoLength: !timedelta '7200s'
globalMinVideoLength: !timedelta '60s'
channelList:
- !Channel
  title: Fermilab
  id: UCD5B6VoXv41fJ-IW8Wrhz9A
  ignore: false
  publishedAt: 2019-06-17 02:36:07.310000
  excludeFilter: null
  includeFilter: null
  minVideoDate: null
  maxVideoDate: null
  minVideoLength: !timedelta '0s'
  maxVideoLength: null
seenChannelVideos: {}

General properties, such as downloadDirectory, are listed at the top. The clientSecretsFile and pickledCredentials entries are set at initialisation, so you don't need to worry about them.

Filters

There are global filters, applied to all videos, and local filters that are applied to each channel individually. The minVideoDate of each channel is automatically updated to reflect the past videos that have been downloaded.

regex include:

• only download videos whose title contain at least one match to the regular expression X

regex exclude:

• only download videos whose title contain no matches to the regular expression X

minimum date:

• only download videos published on or after date

minimum video length:

• only download videos that are at least S seconds long

maximum video length:

• only download videos with a length no longer than S seconds

Note: All regular expressions have the MULTILINE and IGNORECASE flags set.

Video conversion

YouTube's highest quality video and audio are often stored separately, and so yt-dlp requires FFmpeg to combine them together. If you don't already have it, you can download FFmpeg via the link.

Setting your configuration file's ffmpegLocation property to the location of FFmpeg on your system, or if the location FFmpeg is already in your PATH, will allow yt-dlp to download videos at the highest possible quality.