Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(community): update YoutubeLoader implementation #7477

Merged
merged 2 commits into from
Jan 9, 2025

Conversation

sinedied
Copy link
Contributor

@sinedied sinedied commented Jan 7, 2025

Changes

  • Removes youtube-transcript dependency
  • Updates youtubei.js dependency and made peer dependency versioning lenient
  • Refactored YoutubeLoader to use youtubei.js Innertube API instead or relying on youtube-transcript web scraping (may solve some issues regarding YT IP banning)
  • Added test for YoutubeLoader

As YT transcript loading is often broken and requires frequent package updates, this change will (hopefully) simplify the maintenance by now relying on a single package that can be updated independently of @langchain/community package.

Copy link

vercel bot commented Jan 7, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Jan 9, 2025 4:40am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Jan 9, 2025 4:40am

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. auto:refactor A large refactor of a feature(s) or restructuring of many files labels Jan 7, 2025
Copy link
Collaborator

@jacoblee93 jacoblee93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@dosubot dosubot bot added the lgtm PRs that are ready to be merged as-is label Jan 9, 2025
@jacoblee93
Copy link
Collaborator

I think we will also need to update docs but I can do that in a followup

@jacoblee93 jacoblee93 changed the title refactor(community): update YoutubeLoader implementation fix(community): update YoutubeLoader implementation Jan 9, 2025
@jacoblee93 jacoblee93 merged commit d92af44 into langchain-ai:main Jan 9, 2025
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:refactor A large refactor of a feature(s) or restructuring of many files lgtm PRs that are ready to be merged as-is size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants