Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add evaluateKinetics.py script to testing folder #117

Merged
merged 16 commits into from
Jul 27, 2016
Merged

Add evaluateKinetics.py script to testing folder #117

merged 16 commits into from
Jul 27, 2016

Conversation

connie
Copy link
Member

@connie connie commented Jul 22, 2016

This is a revised and updated script of Nathan's old kinetics evaluation script containing 3 main functionalities:

  • obtainKineticsFamilyStatistics: gathers and saves data about families into a .csv files containing info on the number of exact rate rules, number of groups per tree in the groups
  • leaveOneOut: performs a leave one out test by deleting an exact node (excludes averaged ones), averaging the tree and then trying to estimate that tree. Creates parity plots for each family and creates a .csv of the root mean square error.
  • compareNIST: compares the family estimated kinetics against the exact kinetics from the NIST depository within each family. Also creates parity plots and .csv of error data.

connie added 8 commits July 21, 2016 21:58
Raise errors if depository isn't found.  Remove
averaging and adding of training reactions in the family,
assumes you have done it outside the function.

Compares reverse kinetics if the NIST reaction is in the
reverse direction from the family template
retrieves data involving the family
Now shows the top node names, and also the
number of rules
csv file has headers now.
Does not average the families or add training data.
Assumes any type of manipulation has been done outside the folder.

Also, don't load any kineticsLibraries or thermoLibraries
by default except the primaryThermoLibrary.  But do
load all the depositories
@mention-bot
Copy link

@connie, thanks for your PR! By analyzing the annotation information on this pull request, we identified @jwallen and @stroiano to be potential reviewers

Is not quite the original leaveOneOut test as it was done before.
The original test takes extremely long to run because it
must average up the family after each exact rate rule.
In this method, we remove the original rate rule, and see if
using an average that (should incorporate the original) does a
good job of estimating that rate rule.

The option to use the original version of the algorithm can still be
used by setting averaging=True

Write data to csv file after each family rather than after entire set of families
so that they are in alphabetical order

The original family should not contained averaged nodes when starting out. The
leave one out test should be performed only for original exact matches.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update comment here explaining about the averaging boolean

@nyee
Copy link
Contributor

nyee commented Jul 26, 2016

Maybe leaveoneout and compareNIST should be combined into one function with more input arguments because they seem to do almost the same thing

@connie
Copy link
Member Author

connie commented Jul 27, 2016

I think leaveOneOut and compareNIST are pretty different. One compares rate rules internally, the other compares NIST against RMG estimates. So I wouldn't separate them. There should be something similar to compareNIST that is capable of comparing reaction libraries against RMG estimates eventually.

connie added 3 commits July 27, 2016 12:12
This used to excise parity data, it's better
to keep them for complete statistical analysis
…MG kinetics in parityData

User gets to choose whether it wants all kinetics in the
parity Data (both exact and non-exact), or only the exact ones
@nyee nyee merged commit 7a4207d into master Jul 27, 2016
@nyee nyee deleted the leaveOneOut branch July 27, 2016 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants