Difference between revisions of "SE2006:group bar:todo"
Jump to navigation
Jump to search
(clean up, add api item) |
|||
| Line 9: | Line 9: | ||
** superfund sites - almost done (Aybars & Kevin) | ** superfund sites - almost done (Aybars & Kevin) | ||
*** we ran into trouble with non-uniform data that made parsing a headache. will try to resolve or find another source. | *** we ran into trouble with non-uniform data that made parsing a headache. will try to resolve or find another source. | ||
| − | |||
* scrAPI: | * scrAPI: | ||
** API to data store | ** API to data store | ||
*** need to add normalization procedures to SourceDefinition classes | *** need to add normalization procedures to SourceDefinition classes | ||
| − | *** may serve up normalized XML data via tomcat (this way we can take leverage all the source definition/management that we've already done in Java). PHP | + | *** may serve up normalized XML data via [http://tomcat.apache.org/ tomcat] (this way we can take leverage all the source definition/management that we've already done in Java). PHP or Perl are also possibilities (simpler but less robust). |
** testing suite - colin & alex | ** testing suite - colin & alex | ||
*** done (Source,Schema) needs more coverage | *** done (Source,Schema) needs more coverage | ||
*** move test source(s) to cvs & setup bar web directory | *** move test source(s) to cvs & setup bar web directory | ||
*** need documentation for testing suite! | *** need documentation for testing suite! | ||
| − | |||
** proper javadoc - Aybars - done a bunch needs more coverage | ** proper javadoc - Aybars - done a bunch needs more coverage | ||
** exception handling | ** exception handling | ||
*** (asserts, etc.) | *** (asserts, etc.) | ||
| − | |||
** Source Management - toby - done | ** Source Management - toby - done | ||
***Scheduler - toby - done | ***Scheduler - toby - done | ||
***sourceManager - toby - done | ***sourceManager - toby - done | ||
***sourceDefinition - toby - done | ***sourceDefinition - toby - done | ||
| − | |||
** source stream - toby - done | ** source stream - toby - done | ||
| − | |||
** single place for var name to sql type - done | ** single place for var name to sql type - done | ||
| − | |||
** figure out group skipping - done | ** figure out group skipping - done | ||
*** to skip a group, put a '?' as the first character in that group, e.g., (?.+) | *** to skip a group, put a '?' as the first character in that group, e.g., (?.+) | ||
*** see this [http://www.amk.ca/python/howto/regex/regex.html#SECTION000530000000000000000 awesome regexp tutorial] (specific to Python but talks about Perl as a baseline too) | *** see this [http://www.amk.ca/python/howto/regex/regex.html#SECTION000530000000000000000 awesome regexp tutorial] (specific to Python but talks about Perl as a baseline too) | ||
| − | |||
* database | * database | ||
** input - done | ** input - done | ||
| − | |||
* geocoding | * geocoding | ||
** waiting on database from group fu | ** waiting on database from group fu | ||
Revision as of 11:41, 31 March 2006
- 5 data sources
- energy - done(ish)
- resultant database has many specific date columns
- water - done(ish)
- only gets 1990 data. need more dates.
- occupancy - done(ish)
- could be generalized to get more census data
- only gets state data not county, region, etc.
- superfund sites - almost done (Aybars & Kevin)
- we ran into trouble with non-uniform data that made parsing a headache. will try to resolve or find another source.
- energy - done(ish)
- scrAPI:
- API to data store
- need to add normalization procedures to SourceDefinition classes
- may serve up normalized XML data via tomcat (this way we can take leverage all the source definition/management that we've already done in Java). PHP or Perl are also possibilities (simpler but less robust).
- testing suite - colin & alex
- done (Source,Schema) needs more coverage
- move test source(s) to cvs & setup bar web directory
- need documentation for testing suite!
- proper javadoc - Aybars - done a bunch needs more coverage
- exception handling
- (asserts, etc.)
- Source Management - toby - done
- Scheduler - toby - done
- sourceManager - toby - done
- sourceDefinition - toby - done
- source stream - toby - done
- single place for var name to sql type - done
- figure out group skipping - done
- to skip a group, put a '?' as the first character in that group, e.g., (?.+)
- see this awesome regexp tutorial (specific to Python but talks about Perl as a baseline too)
- API to data store
- database
- input - done
- geocoding
- waiting on database from group fu
