handle youtube #110
handle youtube #110
Conversation
mruwnik
requested review from
ccstan99,
henri123lemoine,
Aprillion and
Thomas-Lemoine
| "source_type": "youtube", | ||
| "date_published": self._get_published_date(video), | ||
| "authors": self.extract_authors(video), | ||
| }) |
There was a problem hiding this comment.
Did you find that we were able to get fairly decent metadata this way?
There was a problem hiding this comment.
yes. The main missing thing is the authors, as it just does the channel name, but even that should be ok.
| return None | ||
| except TranscriptsDisabled: | ||
| logger.error(f'Transcripts disabled for https://www.youtube.com/watch?v={video_id} - skipping') | ||
| return None |
There was a problem hiding this comment.
Were there many that had transcriptions disabled or unavailable? Would something like this help? https://huggingface.co/spaces/SteveDigital/free-fast-youtube-url-video-to-text-using-openai-whisper
There was a problem hiding this comment.
yes. Or some kind of alternative. Issue added - #112
| 'PLAPVC5uNprwY0q4_nyeeHqIT07wZqwjGO', | ||
| 'PLCRVRLd2RhZTpdUdEzJjo3qhmX3y3skWA', | ||
| 'PLTYHZYmxohXpn5uf8JZ2OouB1PsDJAk-x', | ||
| ] |
There was a problem hiding this comment.
Nice! Each channel is a dataset and all the playlists are a dataset? So to add an individual video, we add it to a playlist we're already tracking? Might have quite a few duplicates since all of Rob's videos will also be in a playlist too? And are we skipping "Robert Miles 2" channel since it's only a handful of videos and more off topic?
There was a problem hiding this comment.
Duplicates are fine - they will be ignored. This is part of the reason why I put all the playlists together :D
That's part of the reason I skipped it - I just wanted the main body of this to be merged, as later additions will be simple
No description provided.