Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[max-length] Fix ROA parsing issue #868

Closed
wants to merge 1 commit into from
Closed

[max-length] Fix ROA parsing issue #868

wants to merge 1 commit into from

Conversation

tomhrr
Copy link
Contributor

@tomhrr tomhrr commented Aug 25, 2018

The ROA update and generate methods were parsing the ipv4 and ipv6
instance variables as if they were resource sets, rather than ROA
prefix sets. This works as expected when max-length is not being
used, but when max-length is used, parsing does not succeed and affected
ROAs cannot be generated or updated. Parsing each variable as if it's a
ROA prefix set, and then converting the result to a resource set,
appears to fix the problem.

I would have added a test for this, but I had some issues running the existing
CA tests. make parse-test failed with:

/usr/bin/python xml-parse-test.py
Traceback (most recent call last):
  File "xml-parse-test.py", line 99, in <module>
    parser = rpki.up_down.msg,
AttributeError: 'module' object has no attribute 'msg'
Makefile:56: recipe for target 'parse-test' failed
make: *** [parse-test] Error 1

make yamltest failed with:

/usr/bin/python xml-parse-test.py
Traceback (most recent call last):
  File "xml-parse-test.py", line 99, in <module>
    parser = rpki.up_down.msg,
AttributeError: 'module' object has no attribute 'msg'
Makefile:56: recipe for target 'parse-test' failed
make: *** [parse-test] Error 1

rm -rf yamltest.dir rcynic-data
/usr/bin/python yamltest.py smoketest.1.yaml
Initializing RIR
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/rpki.conf
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/rsyncd.conf
Creating certificates for RIR RRDP HTTPS server
Running Django setup for RIR
Operations to perform:
  Apply all migrations: rpkidb
Running migrations:
  Rendering model states... DONE
  Applying rpkidb.0001_initial... OK
  Applying rpkidb.0002_root... OK
Running Django setup for RIR
Operations to perform:
  Apply all migrations: pubdb
Running migrations:
  Rendering model states... DONE
  Applying pubdb.0001_initial... OK
  Applying pubdb.0002_auto_20160221_0617... OK
  Applying pubdb.0003_remove_delta_xml... OK
Running Django setup for RIR
Operations to perform:
  Synchronize unmigrated apps: staticfiles
  Apply all migrations: rcynicdb, sessions, app, auth, contenttypes, gui_rpki_cache, routeview, irdb
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying routeview.0001_initial... OK
  Applying irdb.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying app.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying gui_rpki_cache.0001_initial... OK
  Applying gui_rpki_cache.0002_auto_20160411_2311... OK
  Applying gui_rpki_cache.0003_auto_20160420_2146... OK
  Applying rcynicdb.0001_initial... OK
  Applying rcynicdb.0002_auto_20160227_2003... OK
  Applying rcynicdb.0003_auto_20160301_0333... OK
  Applying sessions.0001_initial... OK
Running "rpkic -i RIR initialize_server_bpki"
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/ca.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/rpkid.key
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/rpkid.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/irdbd.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/irbe.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/ca.crl
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/pubd.key
Writing /root/rpki.net/ca/tests/yamltest.dir/RIR/pubd.cer

Initializing R0
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/rpki.conf
Running Django setup for R0
Operations to perform:
  Apply all migrations: rpkidb
Running migrations:
  Rendering model states... DONE
  Applying rpkidb.0001_initial... OK
  Applying rpkidb.0002_root... OK
Running Django setup for R0
Operations to perform:
  Apply all migrations: pubdb
Running migrations:
  Rendering model states... DONE
  Applying pubdb.0001_initial... OK
  Applying pubdb.0002_auto_20160221_0617... OK
  Applying pubdb.0003_remove_delta_xml... OK
Running Django setup for R0
Operations to perform:
  Synchronize unmigrated apps: staticfiles
  Apply all migrations: rcynicdb, sessions, app, auth, contenttypes, gui_rpki_cache, routeview, irdb
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying routeview.0001_initial... OK
  Applying irdb.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying app.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying gui_rpki_cache.0001_initial... OK
  Applying gui_rpki_cache.0002_auto_20160411_2311... OK
  Applying gui_rpki_cache.0003_auto_20160420_2146... OK
  Applying rcynicdb.0001_initial... OK
  Applying rcynicdb.0002_auto_20160227_2003... OK
  Applying rcynicdb.0003_auto_20160301_0333... OK
  Applying sessions.0001_initial... OK
Running "rpkic -i R0 initialize_server_bpki"
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/ca.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/rpkid.key
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/rpkid.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/irdbd.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/R0/irbe.cer

Initializing Alice
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/rpki.conf
Running Django setup for Alice
Operations to perform:
  Apply all migrations: rpkidb
Running migrations:
  Rendering model states... DONE
  Applying rpkidb.0001_initial... OK
  Applying rpkidb.0002_root... OK
Running Django setup for Alice
Operations to perform:
  Apply all migrations: pubdb
Running migrations:
  Rendering model states... DONE
  Applying pubdb.0001_initial... OK
  Applying pubdb.0002_auto_20160221_0617... OK
  Applying pubdb.0003_remove_delta_xml... OK
Running Django setup for Alice
Operations to perform:
  Synchronize unmigrated apps: staticfiles
  Apply all migrations: rcynicdb, sessions, app, auth, contenttypes, gui_rpki_cache, routeview, irdb
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying routeview.0001_initial... OK
  Applying irdb.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying app.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying gui_rpki_cache.0001_initial... OK
  Applying gui_rpki_cache.0002_auto_20160411_2311... OK
  Applying gui_rpki_cache.0003_auto_20160420_2146... OK
  Applying rcynicdb.0001_initial... OK
  Applying rcynicdb.0002_auto_20160227_2003... OK
  Applying rcynicdb.0003_auto_20160301_0333... OK
  Applying sessions.0001_initial... OK
Running "rpkic -i Alice initialize_server_bpki"
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/ca.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/rpkid.key
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/rpkid.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/irdbd.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Alice/irbe.cer

Initializing Bob
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/rpki.conf
Running Django setup for Bob
Operations to perform:
  Apply all migrations: rpkidb
Running migrations:
  Rendering model states... DONE
  Applying rpkidb.0001_initial... OK
  Applying rpkidb.0002_root... OK
Running Django setup for Bob
Operations to perform:
  Apply all migrations: pubdb
Running migrations:
  Rendering model states... DONE
  Applying pubdb.0001_initial... OK
  Applying pubdb.0002_auto_20160221_0617... OK
  Applying pubdb.0003_remove_delta_xml... OK
Running Django setup for Bob
Operations to perform:
  Synchronize unmigrated apps: staticfiles
  Apply all migrations: rcynicdb, sessions, app, auth, contenttypes, gui_rpki_cache, routeview, irdb
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying routeview.0001_initial... OK
  Applying irdb.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying app.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying gui_rpki_cache.0001_initial... OK
  Applying gui_rpki_cache.0002_auto_20160411_2311... OK
  Applying gui_rpki_cache.0003_auto_20160420_2146... OK
  Applying rcynicdb.0001_initial... OK
  Applying rcynicdb.0002_auto_20160227_2003... OK
  Applying rcynicdb.0003_auto_20160301_0333... OK
  Applying sessions.0001_initial... OK
Running "rpkic -i Bob initialize_server_bpki"
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/ca.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/rpkid.key
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/rpkid.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/irdbd.cer
Writing /root/rpki.net/ca/tests/yamltest.dir/Bob/irbe.cer

Running "rpkic -i RIR create_identity RIR"
Wrote /root/rpki.net/ca/tests/yamltest.dir/RIR/RIR.identity.xml
This is the "identity" file you will need to send to your parent
Running "rpkic -i R0 create_identity R0"
Wrote /root/rpki.net/ca/tests/yamltest.dir/R0/R0.identity.xml
This is the "identity" file you will need to send to your parent
Running "rpkic -i Alice create_identity Alice"
Wrote /root/rpki.net/ca/tests/yamltest.dir/Alice/Alice.identity.xml
This is the "identity" file you will need to send to your parent
Running "rpkic -i Bob create_identity Bob"
Wrote /root/rpki.net/ca/tests/yamltest.dir/Bob/Bob.identity.xml
This is the "identity" file you will need to send to your parent
Running "rpkic -i RIR configure_root"
Generating root for resources ASN: 0-4294967295, V4: 0.0.0.0/0, V6: ::/0
Wrote /root/rpki.net/ca/tests/yamltest.dir/RIR/RIR.RIR.repository-request.xml
This is the file to send to the repository operator

Running daemons for RIR
Running /root/rpki.net/ca/irdbd --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/RIR/irdbd.log for RIR: pid 14956 process <subprocess.Popen object at 0x7fce6b706210>
Running /root/rpki.net/ca/rpkid --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/RIR/rpkid.log for RIR: pid 14957 process <subprocess.Popen object at 0x7fce6b706bd0>
Running /root/rpki.net/ca/pubd --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/RIR/pubd.log for RIR: pid 14958 process <subprocess.Popen object at 0x7fce6b7062d0>
Running rsyncd for RIR: pid 14959 process <subprocess.Popen object at 0x7fce6b706890>
Running httpsd for RIR: pid 14960 process <subprocess.Popen object at 0x7fce6b706650>

Running daemons for R0
Running /root/rpki.net/ca/irdbd --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/R0/irdbd.log for R0: pid 14961 process <subprocess.Popen object at 0x7fce6b706350>
Running /root/rpki.net/ca/rpkid --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/R0/rpkid.log for R0: pid 14962 process <subprocess.Popen object at 0x7fce6b706750>

Running daemons for Alice
Running /root/rpki.net/ca/irdbd --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/Alice/irdbd.log for Alice: pid 14963 process <subprocess.Popen object at 0x7fce6b706990>
Running /root/rpki.net/ca/rpkid --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/Alice/rpkid.log for Alice: pid 14964 process <subprocess.Popen object at 0x7fce6b706a50>

Running daemons for Bob
Running /root/rpki.net/ca/irdbd --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/Bob/irdbd.log for Bob: pid 14966 process <subprocess.Popen object at 0x7fce6b7068d0>
Running /root/rpki.net/ca/rpkid --foreground --log-level debug --log-destination file --log-filename /root/rpki.net/ca/tests/yamltest.dir/Bob/rpkid.log for Bob: pid 14967 process <subprocess.Popen object at 0x7fce6b706a10>

Giving daemons time to start up

Shutting down

Politely nudging pid 14956
Politely nudging pid 14958
Politely nudging pid 14959
Politely nudging pid 14960
Politely nudging pid 14961

Program pid 14956 <subprocess.Popen object at 0x7fce6b706210> returned -15
Program pid 14957 <subprocess.Popen object at 0x7fce6b706bd0> returned 1
Program pid 14958 <subprocess.Popen object at 0x7fce6b7062d0> returned -15
Program pid 14959 <subprocess.Popen object at 0x7fce6b706890> returned 0
Program pid 14960 <subprocess.Popen object at 0x7fce6b706650> returned -15
Program pid 14961 <subprocess.Popen object at 0x7fce6b706350> returned -15
Program pid 14962 <subprocess.Popen object at 0x7fce6b706750> returned 1
Program pid 14962 <subprocess.Popen object at 0x7fce6b706750> returned 1
Program pid 14963 <subprocess.Popen object at 0x7fce6b706990> returned 1
Program pid 14964 <subprocess.Popen object at 0x7fce6b706a50> returned 1
Program pid 14966 <subprocess.Popen object at 0x7fce6b7068d0> returned 1
Program pid 14967 <subprocess.Popen object at 0x7fce6b706a10> returned 1
Blowing out on exception
Traceback (most recent call last):
  File "yamltest.py", line 926, in <module>
    assert all(p.poll() is None for p in progs)
AssertionError
Makefile:92: recipe for target 'yamltest' failed
make: *** [yamltest] Error 1

At 'giving daemons time to start up', there was a pause of about 20s.

Running smoketest.py directly produced:

$ python smoketest.py smoketest.1.yaml
Traceback (most recent call last):
  File "smoketest.py", line 46, in <module>
    import rpki.http
ImportError: No module named http

Any pointers as to test writing would be appreciated.

@tomhrr
[max-length] Fix ROA parsing issue
47a5ff4 
The ROA update and generate methods were parsing the ipv4 and ipv6
instance variables as if they were resource sets, rather than ROA
prefix sets.  This works as expected when max-length is not being
used, but when max-length is used, parsing does not succeed and ROAs
cannot be generated or updated.  Parsing each variable as if it's a
ROA prefix set, and then converting the result to a resource set,
appears to fix the problem.
@tomhrr
Copy link
Contributor Author

tomhrr commented Aug 25, 2018

Also, this may be related to the problem reported in #853.

@sraustein
Copy link
Contributor

Thanks, looks reasonable, cherry picked to master.

Yeah, some of the test code was way out of date, sorry about that. Somebody else asked about that this weekend too.

I agree that this might be related to #853.

@sraustein sraustein closed this Aug 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tomhrr @sraustein