Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Tocs #22

Open
5 of 6 tasks
mytlogos opened this issue Aug 2, 2020 · 0 comments
Open
5 of 6 tasks

Enhance Tocs #22

mytlogos opened this issue Aug 2, 2020 · 0 comments
Labels
crawler Specific to the Crawler Database Specific to the Database

Comments

@mytlogos
Copy link
Owner

mytlogos commented Aug 2, 2020

PRO:

  • Less changes from contradictory tocs
  • Delete releases (and maybe episodes) which do not occur in toc anymore
    • Against spam episodes
    • against invalid scraped chapters which were fixed later (no more manual removals!)

CON:

  • How to resolve the it when displaying in client? (which available properties wins?)
  • More resources needed to save it (up to 300% of media space, atm + ~7MB?)
  • More resources needed to enforce it (should be doable, as there is enough ram and maybe cpu)

TODO:

  • Add Properties of medium to medium_toc
  • Add new Primary Key to medium_toc and previous to unique index
  • Add referenced toc_id column to episode_release
  • Updates medium_toc instead of medium when scraping tocs
  • User updates medium only, not medium_toc properties (only the crawler should do that)
  • tocs releases are validated every toc job
    • releases are assigned a toc id if first seen
    • releases with toc ids which do not occur in a toc job anymore are removed (and their episode maybe too)
    • what is with toc releases without toc id which are currently invalid?
@mytlogos mytlogos added crawler Specific to the Crawler Database Specific to the Database labels Aug 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crawler Specific to the Crawler Database Specific to the Database
Projects
None yet
Development

No branches or pull requests

1 participant