SE2006:group bar:minutes

Feb 15

Possible meeting time: Working M/W/F in the evening for 1 or 2 hours starting the 27th. Toby, Kevin, and I are going to be working this weekend on clustcomp projects and then going to a conference on Feb 22-26.

• Languages:
  • php - most of us are familiar. big libraries. web stuff.
  • java with tomcat - web stuff. good development environment.
  • python - learn a new language.
  • perl - most of us are familiar. less good, but good web stuff.
  • ruby - learn a new language. there may be other reasons but we don't know what they are because we don't know ruby.

• Caching Location data:
  • with memcache (http://www.danga.com/memcached/)
  • postgres
  • mysql

• Communication
  • using a mailing list for now.
  • developing new methods as they seem fit (wiki,cvs,bulletin board).

• Group name:
  • kartgoolers.
  • cartooglers.
  • pwners
  • colin's team.
  • group-b
  • group bar

• Assignment:
  • send us your group name choice to list. we accept write ins. if anyone feels super strong about a name then tell us why.
  • submit your leader (self?)nominations.

Feb 28

• We have decided on some of the roles:
  • Aybars: Libarian
  • Alex & ColinC: Developers
  • Tobias: Leader
  • Kevin: Architect

• We have roughly agreed on meeting Monday, Tuesday and Wednesdays.

• We have discussed which language to use, and decided that we'll decide on that later after we break the project up a little bit more (possibly tomorrow).

• Tomorrow we will discuss the documentation.

• We will look at the lab 2 solutions soon.

• Error checking?

Mar 1

• We worked on breaking this project into smaller pieces.

• For database we are going to use SQL and for the rest of the project, we will probably use Java.

• We will use Javadoc for documentation of the API

• Next meeting we are going to talk about:
  • Lab 2 solutions
  • Time budgeting
  • Eclipse review
  • Java prep
  • Kevin will bug Skylar about getting Java API installed on Quark.
  • We have two solutions so far on the table
  • We will talk more about documentation

Mar 6

• we went over our lab 2 solutions:

We compared three of our Kerasotes lab scripts. We apparently all wrote them in PHP. The best innovations we found were Toby's data point definition abstraction and test suite. In essence, the data point definition abstraction allows one to describe a data source and data points that one would like to harvest in a general manner. The test suite is a basis for more general testing of the software that we will write. Also of note, were some of our findings as far as regular expressions. For instance, as a group we learned how to make a regular expression not greedy and how to not save a group (?).

Attached you will find an image of our general model thus far. We have already incorporated some of the ideas discussed in this session, including a more general data point abstraction and streamlined data flow between the programmer API, database, and WUI. As part of our plan, we intend to use Java as the base language, XML for temporary data storage and transfer, PostgreSQL or MySQL for data storage, and XHTML Transitional and JavaScript for the WUI.

• eclipse overview and java prep held till next mtg.
• java API + doc has been installed on quark

Mar 7

• Toby went over Eclipse basics with us
• We set up a CVS repository on Quark in /clients/groups/bar/cvs/
  • So far, we're planning on using the repository through Eclipse's interface
• We went over Java basics, and described the Javadoc idea

• On the boiler plate:
  • Have a discussion and get in writing exactly what our individual jobs entail.
  • Show how Javadoc works
    • Develop our group's commenting style
  • Think about our coding style, especially as regards code formatting
  • CVS plumbing (CVS spam, bugzilla, etc.)

Mar 8

• We decided on the roles. SE2006:group_bar:roles

• We went over using Javadoc feature of Eclipse.

• Next time we are going to:
  • Flesh out the model.
  • Write the stubs of the code

Mar 13

• Today we worked on finding some sources. Next time we will be discussing about either to use one of these or look for another one.

• Sources
  • [fact finder]
  • [Surface Condition Weather Forecasting for Air Sports Aviators]
  • [Energy Information Administration]
  • [Geostat Center: Collections]
  • [US Census Bureau: International Database (IDB)]
  • [CIA World Factbook GDP per capita ranking of countries]

• Perhaps useful sources
  • [Demographic Data Viewer]

Mar 14

• Today we worked on the stubs for the scraping API (scrAPI) and had some preliminary discussions about how to create the SQL schemas and connect them to this API.

• Tomorrow we are planning to work more on this API and possibly improve its relations with our SQL schemas.

Mar 15

• We worked more on scrAPI and added more regular expressions into it. We have some new ideas emerging about some new data types. We are making good progress on scrAPI.

• We have found some more standardized websites that allow us to automatically scrap them. Moreover, we also tried to find ideas on how to scrap some of these websites that we have found. We have communicated them to each other either by e-mail or by talking.

• Next meeting we are going to try to incorporate the structure of these new websites we have found into our scrAPI design.