Introduction

Opinoid is a simple web application which syndicates news from a set of known RSS/Atom sources and provides visitors with a sorted list of articles (like Reddit), based on user’s votes and comments. It uses TornadoWeb, MongoDB, jQuery and Bootstrap.

A live version is available at alpha.opinoid.com (hosted on OpenShift). Please report issues and ideas here on github!

Hacking

The web app file are in the webapp folder, some additional utilities are in the utils folder. The utils initialize the db and and update the list of articles periodically.

webapp

The python modules required to get the web app running:

$ pip freeze
pep8==0.6.1
pycurl==7.19.0
pygeoip==0.2.2
pymongo==2.0.1
six==1.0.0
textile==2.1.5
tornado==2.1.1
wsgiref==0.1.2

If you’re using python <= 2.6 you will also need to install the argparse module. You’ll also need a working install of mongodb.

$ ./opinoid.py --help
Usage: ./opinoid.py [OPTIONS]

Options:
  --help                           show this help information
  --log_file_max_size              max size of log files before rollover
  --log_file_num_backups           number of log files to keep
  --log_file_prefix=PATH           Path prefix for log files. Note that if you are running multiple tornado processes, log_file_prefix must be different for each of them (e.g. \
include the port number)
  --log_to_stderr                  Send log output to stderr (colorized if possible). By default use stderr if --log_file_prefix is not set and no other logging is configured.
  --logging=info|warning|error|none Set the Python log level. If 'none', tornado won't touch the logging configuration.
  --db_host                        DB host
  --db_pass                        DB pass
  --db_port                        DB port
  --db_user                        DB user

You will need to run such a populate.py script at least once, before starting the web app, to populate the db. We suggest you to set a cronjob to spawn it hourly.

15 * * * * /home/release/virtualenv/bin/python /home/release/opinoid/utils/populate.py >> /tmp/populate.log 2>&1

utils/populate.py

A python script which parses the feeds and populates the db accordingly.

$ ./populate.py
2012-04-06 14:15:02,164 INFO STARTED populate
2012-04-06 14:15:02,165 INFO working on http://www.ilfattoquotidiano.it/feed/
2012-04-06 14:15:03,396 INFO entries received 20, skipped 6, added 14
[...]
2012-04-06 14:15:16,024 INFO STARTED cleanup
2012-04-06 14:15:16,025 INFO deleting articles older than 1331036102
2012-04-06 14:15:16,029 INFO articles deleted 1, comments 0
2012-04-06 14:15:16,029 INFO STARTED logrun
2012-04-06 14:15:16,029 INFO last full run at 1333714502

It will work with any python2 version providing the following modules:

$ pip freeze
feedparser==5.0.1
pep8==0.6.1
pymongo==2.0.1
tldextract==0.3.1
wsgiref==0.1.2

populate.py also provides a —h flag to specify some additional options.

utils/init.mongo

This is the mongodb initialization file. It simply adds a short list of urls (feeds) to be parsed, including their country and category for better categorization. Assuming your mongodb is running on localhost, you can do the initial population with:

$ mongo 127.0.0.1 < init.mongo

License

Opinoid is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Opinoid is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with Opinoid. If not, see <http://www.gnu.org/licenses/>.

Copyright 2012 Giulio Fidente