Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Commit Scoring Logic #129

Open
Shwetakale opened this issue Sep 19, 2016 · 4 comments
Open

Commit Scoring Logic #129

Shwetakale opened this issue Sep 19, 2016 · 4 comments
Labels

Comments

@Shwetakale
Copy link
Contributor

Currently while scoring commits bugspots is used and in bugspot each file is rated based on historical commit messages on that file and timestamp of commit message. File whose historical commit message has max number of "fix" or "close" word gets higher weightage.

Incase of aasm highest weightage is for README and also CHANGELOG has higher weightage.

    Hotspots:
        5.9860 - README.md
        2.5098 - lib/aasm/core/transition.rb
        2.5010 - lib/aasm/instance_base.rb
        1.9753 - spec/unit/transition_spec.rb
        1.9007 - lib/aasm/persistence/active_record_persistence.rb
        1.7792 - lib/aasm/base.rb
        1.6691 - spec/unit/inspection_spec.rb
        1.5949 - lib/generators/active_record/templates/migration_existing.rb
        1.5705 - CHANGELOG.md

Incase of mongoid highest weightage is to CHANGELOG

    Hotspots:
        2.9300 - CHANGELOG.md
        0.8669 - lib/mongoid/contextual/mongo.rb
        0.7303 - spec/mongoid/scopable_spec.rb
        0.7236 - lib/mongoid/scopable.rb
        0.7053 - lib/mongoid/persistable/settable.rb
  • Here while calculating score bugspot_score of file is used so if person do typo fix in readme update gets higher score than actual code changes incase of aasm.
    • If person adds totally new file for some feature will get 0 marks as bugspots_scores[file.filename] will be 0
  • Here total_score is divided by number_of_files in the commit so if user has done good change and if test case file has less weightage then that commit will be scored less. If person commits test cases seperately and code seperately he will get more marks than person commiting both things together. but I think this can't be handled.

Thoughts?

@gautamrege
Copy link
Member

I agree we need some more innovation around the scoring algorithm. Ideas are welcome.

@gautamrege
Copy link
Member

Update: we are working on some ideas here.

Interestingly, we have seen instances where people look at the scoring algorithm and try to "tweak" their code to manipulate the scoring. Though I am a proponent of keeping the scoring algorithm open, I want some ideas about how we "hide" the logic from prying eyes. Remember, we spend real money on rewarding people, so we have to be extra careful that this is not mis-used.

@prasadsurase
Copy link
Contributor

We can use the same technique as secrets.yml. Move the scoring logic to a file which isn't committed. Load the file containing the scoring logic when the application starts.

@thecodersblock
Copy link

Hey, just wondering. What's the rationale behind dividing the total score with the number of files?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants