This is a Ruby on Rails application that will mine the United States Chess Federation website for data and present it.
When I was a more serious chess player, I would always do things like this for myself. I kept meticulous records of all my games and would keep track of how I did under various circumstances against various people, look at my trends, and try to use them to understand myself better. I reveled in my biggest upsets and best performances while being upset by my worst losses. I compared myself to my friends and rivals using various criteria and invented goals for myself.
I am hoping to apply these same ideas and experiences to build an app that a lot of people can use.
9/22:
-Initiated project -Learned about Nokogiri and Mechanize via RailsCasts -Built web scraper to parse the USCF website and get data using regular expressions and CSS selectors -Create UscfWebsite class to hold common methods for accessing the USCF website
9/23:
-Learned about Ajax in Rails (annoying because a lot changed in Rails 3.1 so all the tutorials were outdated) -Used instance variables in partials with Ajax calls to dynamically update the page using a Ruby object -Wrote code to collect all the opponents someone has ever played and sort them by rating -Made MySQL database, pushed project to Heroku, sorted out little bugs in transitioning -Doing it all in one call caused Heroku to timeout for players with long histories, so I went through and made the entire thing a collection of Ajax calls that pass JSON objects and arrays back and forth to collect all the data -Added some jQuery to make the user interface a little more friendly and show a progress indicator with % complete and the date of the last tournament processed
9/24:
-Worked on the Stanford machine learning course
9/25:
-Worked on the Stanford machine learning course
9/26:
-Built a good user interface -Wrote an intro to the app to show to people
9/27:
-Spent a long time fighting with the poorly-formatted USCF website -Learned that Ruby regular expressions don't recognize some of the UTF-8 blank space characters
9/28:
-Talked to GoDaddy and learned how to properly redirect naked domains through the Heroku proxy to the main page. -Put a link to the website on chess.stackexchange.com
9/29:
-Added a bunch of methods to the API -Did some bug fixing -Played in a tournament and invited people to visit the website
9/30:
-Redesigned the code for the demo to make it better -Added a lot of new columns -Made the user interface a lot better -Let users choose between regular and quick history -Put sorting on the columns
10/1:
-Identified problems in the current demo design -Learned about Redis -Implemented a Redis NoSQL cloud database and set up the application to work with it -Rewrote various parts of the application to use the new database -Fixed some bugs -Set the type and USCF ID to be session variables
10/2:
-Fixed some small bugs
10/3:
-Worked on the Stanford machine learning course
10/4:
-Went to hack night with the Research Triangle Ruby meetup group -Built the get_results_against_opponents_from_tournament method in the USCF Website API
10/5:
-Vacation to visit my father, didn't work on stuff
10/6:
-Still on vacation -Poked at the FANN neural network library
10/7:
-Still on vacation -Learned some D3JS -Built draft of delta chart
10/8:
-Worked on learning D3JS
10/9:
-Added D3Js axis to delta chart -Did some math to scale the locations and sizes of the axis and bubbles according to the specified width/height. -Made the bubbles say who you played -Updated the interface to make it a little better and let users input the ID of who to check out
10/10:
-Usability testing, someone noticed a major bug in the deltas chart in later versions of Firefox. -Opened issue, found solution and fixed it.
10/11:
-Busy watching the debate
10/12:
-Busy doing other things
10/13:
-Did some research and discovered locally weighted logistic regression -Read Ph.D thesis paper and tried to implement a library for the algorithm in Ruby -Got irritated with poorly written paper -Made the deltas example a collection of AJAX calls similar to the demo example -UI/UX changes to the deltas example
10/14:
-Fixed a defect in the deltas page -UI/UX changes to deltas page -Looked up the mathematical papers behind the USCF rating system to get a better understanding of it -Opened up the JavaScript for the performance calculator on the USCF website and read it -Implemented a binary search algorithm similar to that on the USCF website in Ruby to calculate performances -Learned how to do line charts in D3JS -Learned how to do time scales in D3JS -Built a line chart that shows performance ratings and regular ratings over time -Added buttons to deltas and performance charts to add gridlines.
10/15:
-Found out there's a tiny book on D3JS, read about fifty pages of it -Added some animations and hovers to the charts
10/16:
-Finished D3JS book -Added cool D3JS things to the charts -Better hovers on the performance chart
10/17:
-Made the performance chart example a collection of AJAX calls -Used D3JS's algorithms to make a force-directed graph to represent the results of a tournament
10/18:
-Busy doing other things
10/19:
-Busy doing other things
10/20:
-Studied linear regression -Built a structure to add a linear regression model to the performance chart using the normal equation -Let users choose features -Users can choose arbitrary data and use machine learning to get an expected performance
10/21:
-Fixed a bug, did some testing
10/22:
-Added a new method to the USCFWebsite class to count how many games someone played in a tournament -Started building a chart to show rating/performance vs number of games played -The USCF has blocked my localhost from accessing their website, so I'll have to develop from a VPN from now on.
10/23:
-Made some cosmetic changes
10/24:
-Made some more cosmetic changes -Updating routing a bit
10/25 - 11/29:
-Took a month-long break -Studied public speaking and developed foreign policy ideas for the U. Penn Model U.N. conference -Visited Amelia in Massachusetts
11/30:
-Built a logistic regression class that implements a basic logistic regression algorithm. -Added two logistic regression classifiers to the deltas page -Predict what rating difference marks the point where a player has a better chance of winning/drawing by rating.
12/1:
-Built a linear regression class to implement a basic linear regression algorithm -Added two linear regression lines, with features log(rating) and log^2(rating), to the performances page. -Made various parts of the website work better and look better.
12/2:
-Found a problem with the logistic regression classifier. -Spent a while exploring various solutions, couldn't find a consistent one, asked a question on StackOverflow. -Made a lot of cosmetic changes to the site.
12/3:
-Updated what the description says on the homepage.