Difference between revisions of "SE2006:group bar:todo"

From Earlham CS Department
Jump to navigation Jump to search
Line 1: Line 1:
 
* 5 data sources
 
* 5 data sources
 +
 
* scrAPI:
 
* scrAPI:
 
** proper javadoc
 
** proper javadoc
Line 6: Line 7:
 
*** to skip a group, put a '?' as the first character in that group, e.g., (?.+)
 
*** to skip a group, put a '?' as the first character in that group, e.g., (?.+)
 
*** see this [http://www.amk.ca/python/howto/regex/regex.html#SECTION000530000000000000000 awesome regexp tutorial] (specific to Python but talks about Perl as a baseline too)
 
*** see this [http://www.amk.ca/python/howto/regex/regex.html#SECTION000530000000000000000 awesome regexp tutorial] (specific to Python but talks about Perl as a baseline too)
** single place for var name to sql type
 
 
** exception handling
 
** exception handling
 
*** (asserts, etc.)
 
*** (asserts, etc.)
 
** auto (periodic) fetching of data
 
** auto (periodic) fetching of data
 
*** make ScrAPI run as a daemon?
 
*** make ScrAPI run as a daemon?
 +
** single place for var name to sql type - done
 
** thread support (scrape multiple sources simultaneously)
 
** thread support (scrape multiple sources simultaneously)
 +
 
* database
 
* database
 +
 
* geocoding
 
* geocoding

Revision as of 10:26, 30 March 2006

  • 5 data sources
  • scrAPI:
    • proper javadoc
    • testing suite
    • figure out group skipping
      • to skip a group, put a '?' as the first character in that group, e.g., (?.+)
      • see this awesome regexp tutorial (specific to Python but talks about Perl as a baseline too)
    • exception handling
      • (asserts, etc.)
    • auto (periodic) fetching of data
      • make ScrAPI run as a daemon?
    • single place for var name to sql type - done
    • thread support (scrape multiple sources simultaneously)
  • database
  • geocoding