Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IOP Spider: improve and add tests #206

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

IOP Spider: improve and add tests #206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
eec9f7a
parsers: create an NLM parser
szymonlopaciuk Jan 16, 2018
724ce2d
tests: fix invalid DOI in IOP tests
szymonlopaciuk Jan 16, 2018
7b925d0
parsers.nlm: add mapping from 'Republished'
szymonlopaciuk Jan 16, 2018
6b96a6c
parsers.nlm: document return value on bulk_parse
szymonlopaciuk Jan 16, 2018
ad95e0b
parsers.nlm: add case to material extraction
szymonlopaciuk Jan 16, 2018
d1e4cbe
parsers.nlm: use PartialDate.from_parts
szymonlopaciuk Jan 16, 2018
2b87bfe
parsers: add NLMParser to __init__ of parsers
szymonlopaciuk Jan 17, 2018
8cdf47a
parsers.nlm: fix bug when no PublicationType
szymonlopaciuk Jan 17, 2018
d745f78
parsers.nlm: allow lack of publication date
szymonlopaciuk Jan 17, 2018
3106759
parsers.nlm: handle maths in abstract
szymonlopaciuk Jan 17, 2018
bc8d5d4
IOP Spider: improve and add tests
szymonlopaciuk Jan 12, 2018
9573ab5
tohep: rm _filter_affiliation, handled in builder
szymonlopaciuk Jan 12, 2018
c2757ab
IOP Spider: make use of NLM parser
szymonlopaciuk Jan 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ env:
- SUITE=functional_desy
- SUITE=functional_cds
- SUITE=functional_pos
- SUITE=functional_iop

matrix:
fast_finish: true
Expand Down
7 changes: 7 additions & 0 deletions docker-compose.test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,13 @@ services:
http-server.local:
condition: service_healthy

functional_iop:
<<: *service_base
command: py.test -vv tests/functional/iop
depends_on:
scrapyd:
condition: service_healthy

unit:
<<: *service_base
command: bash -c "py.test tests/unit -vv && make -C docs clean && make -C docs html && python setup.py sdist && ls dist/*"
Expand Down
158 changes: 0 additions & 158 deletions hepcrawl/extractors/nlm.py
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions hepcrawl/parsers/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@
from __future__ import absolute_import, division, print_function

from .jats import JatsParser
from .nlm import NLMParser
Loading