Energy Wars Display

RPNL

Currently, this is how we're getting data from RPNL.

1. RPNL sends an email containing CSV to testcs128@cs.earlham.edu
2. testcs128 forwards it to ghcrows13@cs.earlham.edu in its .forward file
3. ghcrows13@cs.earlham.edu executes mail.py (in home directory) from its .forward file
4. mail.py scrapes off CSV attachments
5. mail.py pipes text into parse.py
6. parse.py scrapes off data, copies tmp.csv to ghcrows13@proto.cs.earlham.edu:/home/users/energy/contest/daily/tmp.csv
   • uses RSA keys to bypass passwords
   • user names must be the same for RSA keys to work
7. parse.py executes a sql command that pipes this csv file into /home/users/energy/contest/daily/parse_houses.py
8. parse_houses.py strips off csv data (again!) and derives the energy change
9. parse_houses.py then copies the data into the electrical_energy_rpnl database

Display

The display is written using google charts.

Data

Overview

• Data is generated by ~energy/contest/daily/execute.bash
  • executed daily by energy's crontab
• Temporary csv files are stored in ~energy/contest/daily/tmp_data/
  • useful for desktop testing
• final csv files are stored in ~energy/public_html/development/EnergyWars/
• execute.bash has some options that you can change, including:
  • start and end dates for the baseline and the contest
  • the target database
  • name -- the prefix for CSV files
  • street -- that's the string that RPNL uses

Specifics

• execute.bash generates the daily, baseline, cumulative, and cumulative baseline
• each of these are a different .bash script
• it does this for buildings, national road, and college avenue
• an overview of what each script does:
  1. generates appropriate file names
  2. executes a dynamic SQL statement that's based on arguments
  3. stores the SQL response in a file
  4. reformats the data using ~energy/contest/daily/reformat.py
  5. stores the reformatted data in ~energy/contest/daily/tmp_data/
  6. copies the reformatted data to the destination