Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ECO-277/find-upvs-urls-with-sparql #138

Merged
merged 3 commits into from
Jun 18, 2024

Conversation

tomasdrga
Copy link
Contributor

  • Vytvoril som nové joby, ktoré hľadajú aktuálne URL pre sťahovanie UPVS datasetov
  • Tieto joby pošlú najdenú URL už do existujúcich fetch jobov
  • Zmenil som UPVS rake tasks aby volal tieto nové joby, ktoré potom volajú fetch
  • Upravil som pôvodne fetch joby a ich testy
  • Pridal som testy pre nové joby
  • V jobe FindPublicAuthorityActiveEdesksListJob, sú tam vždy publikované dve verzie, ktoré majú rovnaký čas vytvorenia aj modifkácie, jeden je však formát .xlsx a druhý .csv, preto je tam táto kontrola

Ešte tu možno priložím web, kde som našiel návod dokumentácia

…t for these jobs, changed upvs rake task, adjusted upvs jobs and tests accordingly
@tomasdrga tomasdrga requested a review from luciajanikova June 12, 2024 18:37
Comment on lines 29 to 30
csv_dataset_url = response.body.split("\n")[1..].map(&:strip)
.find { |url| Faraday.get(url).headers['Content-Disposition']&.include?('.csv') }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Za mna vsetko dobre, len uvazujem ci tento sposob identifikacie spravnej distribucie nedame aj na zvysne joby (i ked tam zatial ine moznosti nie su, ale casom mozu pribudnut a boli by sme aspon na to ready).

@tomasdrga tomasdrga requested a review from luciajanikova June 18, 2024 12:42

SET_URL = 'https://data.gov.sk/set/8572f288-0186-4bc2-8d12-9eb324ff47bd'
BASE_URL = 'https://data.slovensko.sk/api/sparql'
NUMBER_OF_DATASET_FORMATS = 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Toto je potrebne udavat?

@tomasdrga tomasdrga merged commit 07da3b3 into master Jun 18, 2024
3 checks passed
@tomasdrga tomasdrga deleted the ECO-277/find-upvs-urls-with-sparql branch June 18, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants