Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proof of concept: optional features for Invidious through optional shards #5075

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@
/invidious
/sentry
/config/config.yml

/src/addons/enabled.txt
2 changes: 2 additions & 0 deletions shard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ dependencies:
http_proxy:
github: mamantoha/http_proxy
version: ~> 0.10.3
extendious-reddit-comments:
github: "syeopite/extendious-reddit-comments"

development_dependencies:
spectator:
Expand Down
19 changes: 19 additions & 0 deletions src/addons/extract-addons.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
require "yaml"

shardyml = File.open("shard.yml") do |file|
YAML.parse(file).as_h
end

# Finds all dependencies prefixed with extendious
raw_addons = shardyml["dependencies"].as_h.keys.select(&.as_s.starts_with?("extendious"))
addons = [] of String

raw_addons.map do |addon_name|
addon_name = addon_name.as_s
addon_module = addon_name.lchop("extendious-")
addon_module = addon_module.split("-").map!(&.capitalize).join

addons << "#{addon_name},#{addon_module}"
end

File.write("src/addons/enabled.txt", addons.join("\n"))
18 changes: 14 additions & 4 deletions src/invidious.cr
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ HMAC_KEY = CONFIG.hmac_key
PG_DB = DB.open CONFIG.database_url
ARCHIVE_URL = URI.parse("https://archive.org")
PUBSUB_URL = URI.parse("https://pubsubhubbub.appspot.com")
REDDIT_URL = URI.parse("https://www.reddit.com")
YT_URL = URI.parse("https://www.youtube.com")
HOST_URL = make_host_url(Kemal.config)

Expand Down Expand Up @@ -207,9 +206,9 @@ end

# Routing

before_all do |env|
Invidious::Routes::BeforeAll.handle(env)
end
# Custom handlers actually has a higher priority than the handler defined via
# before_all
add_handler TrueBeforeAllHandler.new

Invidious::Routing.register_all

Expand Down Expand Up @@ -243,6 +242,17 @@ Kemal.config.host_binding = Kemal.config.host_binding != "0.0.0.0" ? Kemal.confi
Kemal.config.port = Kemal.config.port != 3000 ? Kemal.config.port : CONFIG.port
Kemal.config.app_name = "Invidious"

# Loads optional extensions for Invidious
# Essentially just glorified shards with a set naming scheme
# and load method

{{run("./addons/extract-addons.cr", "--minified")}}
{% for addon in read_file("src/addons/enabled.txt").lines %}
{% shard_name, module_name = addon.split(",") %}
require {{shard_name.id.stringify}}
{{module_name.id}}Ext.invidious_load
{% end %}

# Use in kemal's production mode.
# Users can also set the KEMAL_ENV environmental variable for this to be set automatically.
{% if flag?(:release) || flag?(:production) %}
Expand Down
41 changes: 0 additions & 41 deletions src/invidious/comments/reddit.cr

This file was deleted.

57 changes: 0 additions & 57 deletions src/invidious/comments/reddit_types.cr

This file was deleted.

50 changes: 0 additions & 50 deletions src/invidious/frontend/comments_reddit.cr

This file was deleted.

8 changes: 8 additions & 0 deletions src/invidious/helpers/handlers.cr
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,14 @@ class Kemal::ExceptionHandler
end
end

class TrueBeforeAllHandler < Kemal::Handler
def call(env)
return call_next(env) unless env.route_found?
Invidious::Routes::BeforeAll.handle(env)
call_next env
end
end

class FilteredCompressHandler < Kemal::Handler
exclude ["/videoplayback", "/videoplayback/*", "/vi/*", "/sb/*", "/ggpht/*", "/api/v1/auth/notifications"]
exclude ["/api/v1/auth/notifications", "/data_control"], "POST"
Expand Down
31 changes: 0 additions & 31 deletions src/invidious/routes/api/v1/videos.cr
Original file line number Diff line number Diff line change
Expand Up @@ -353,37 +353,6 @@ module Invidious::Routes::API::V1::Videos
end

return comments
elsif source == "reddit"
sort_by ||= "confidence"

begin
comments, reddit_thread = Comments.fetch_reddit(id, sort_by: sort_by)
rescue ex
comments = nil
reddit_thread = nil
end

if !reddit_thread || !comments
return error_json(404, "No reddit threads found")
end

if format == "json"
reddit_thread = JSON.parse(reddit_thread.to_json).as_h
reddit_thread["comments"] = JSON.parse(comments.to_json)

return reddit_thread.to_json
else
content_html = Frontend::Comments.template_reddit(comments, locale)
content_html = Comments.fill_links(content_html, "https", "www.reddit.com")
content_html = Comments.replace_links(content_html)
response = {
"title" => reddit_thread.title,
"permalink" => reddit_thread.permalink,
"contentHtml" => content_html,
}

return response.to_json
end
end
end

Expand Down
19 changes: 0 additions & 19 deletions src/invidious/routes/watch.cr
Original file line number Diff line number Diff line change
Expand Up @@ -89,25 +89,6 @@ module Invidious::Routes::Watch
begin
comment_html = JSON.parse(Comments.fetch_youtube(id, nil, "html", locale, preferences.thin_mode, region))["contentHtml"]
rescue ex
if preferences.comments[1] == "reddit"
comments, reddit_thread = Comments.fetch_reddit(id)
comment_html = Frontend::Comments.template_reddit(comments, locale)

comment_html = Comments.fill_links(comment_html, "https", "www.reddit.com")
comment_html = Comments.replace_links(comment_html)
end
end
elsif source == "reddit"
begin
comments, reddit_thread = Comments.fetch_reddit(id)
comment_html = Frontend::Comments.template_reddit(comments, locale)

comment_html = Comments.fill_links(comment_html, "https", "www.reddit.com")
comment_html = Comments.replace_links(comment_html)
rescue ex
if preferences.comments[1] == "youtube"
comment_html = JSON.parse(Comments.fetch_youtube(id, nil, "html", locale, preferences.thin_mode, region))["contentHtml"]
end
end
end
else
Expand Down
Loading