Add source "hackernews"

[telotortium]

Uses the dogsheep module from HPI

Fix karlicoss/HPI#222

[purarue]

Ah, this is likely failing because dogsheep is such a new module, the module isn't included on the release on pypi (which CI here installs from)

I think @karlicoss usually resolves this by doing a new rolling release on HPI, so CI here works properly

[purarue]

Also, could you add this line to the top index -- is just some shared code for promnesia sources with use HPI modules. Incase the user doesnt have HPI installed, it warns them

[telotortium]

Also, could you add this line to the top index -- is just some shared code for promnesia sources with use HPI modules. Incase the user doesnt have HPI installed, it warns them

Done

[karlicoss]

hey @telotortium , thanks for the contribution! Sorry just got to check this out.

Agree it would be better to rename this to hackernews.py instead. Maybe we'll have a dogsheep module one day, but it'd probably need to talk to Dogsheep directly instead and be more or less service agnostic (this one is just reusing Dogsheep code for data retrieval)

l also left a few comments -- fairly minor, and I'm happy to merge as it is, but if you have some time to fix, would be grateful!

Also yeah @seanbreckenridge is right about the CI failing -- I've released a new HPI version and rerun. Would be nice to solve it somehow so it works against master as well, but going to be tedious... P.S. it still fails with mypy issues -- so @telotortium let me know if you need help resolving

[telotortium]

Please review and approve.

[telotortium]

Now that you pointed out how Visit is constructed, and what the difference is between url and locator (by the way, you should make that more clear in the comments on the Visit class), I'm going to try to make a change upstream to add the story each comment was posted on to the Dogsheep database, so that we can use that info here: dogsheep/hacker-news-to-sqlite#4. However, it's probably best to merge this first.

[karlicoss]

Looks good, thanks!

CI failed with mypy, I suggested a change that should help. Btw here's how you can run type checks locally https://github.com/karlicoss/promnesia/blob/master/doc/DEVELOPMENT.org#running-tests--mypy-checks

Also some errors are irrelevant (related to fbmessengerexport/twitter), not sure how it happened, but not your fault -- I'll take a look at the meantime

[karlicoss]

thanks for your work & fixes!