Difference between revisions of "Ivan-cs488-paper"
(→Solution) |
(→Software functionality) |
||
| (17 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | <span style="color:#FF0000"> '''Inexpensive, portable and | + | <span style="color:#FF0000"> '''Inexpensive, portable and extendable open source environmental monitoring platform with integrated geocoding ''' </span> |
= Abstract = | = Abstract = | ||
| Line 14: | Line 14: | ||
== Main reason for high cost of monitoring devices == | == Main reason for high cost of monitoring devices == | ||
| − | A simple water quality meeter, with closed software, inability to add any other sensor probe to it and without any possibility of directly geocoding collected data costs around $ | + | A simple water quality meeter, with closed software, inability to add any other sensor probe to it and without any possibility of directly geocoding collected data costs around $3,000. An example is YSI 650 MDS Datalogger water quality meeter (Picture 1) whose cost is \$3.600. This multi thousands dollars worth sensor, YSI 650 MDS, does not have a USB port neither a way of data geocoding. If we are planing to monitor water, air and soil the total device cost will add up close to $8,000. Taking in consideration the fact that none of those devices is able to geocode generated data, we have to add jet another device, a Global Positioning System (GPS). Having four different monitoring platforms, dealing with inconsistency in their underlying architectures, operating systems and geocodeless readings they generate is more than inconvenient and extremely expensive. |
| − | The problem is more than obvious, every sensor has its unique computer platform attached to it whose main job is to communicate with a sensor probe and take readings from it. In the case of water quality meeter, platform to where water probe is connected costs 5 times more than probe itself. Cost is high because every basic computer platforms has its screen, memory, battery... | + | PICTURE HERRREEEE |
| + | |||
| + | The problem is more than obvious, every sensor, as seen on the picture one, has its unique computer platform attached to it whose main job is to communicate with a sensor probe and take readings from it. In the case of water quality meeter, platform to where water probe is connected costs 5 times more than probe itself. Cost is high because every basic computer platforms has its screen, memory, battery... | ||
=Solution = | =Solution = | ||
| Line 27: | Line 29: | ||
Very important factor in this decision had integrated GPS receiver. Before selecting Nexus 7 as the base platform I implemented a couple of tests of its GPS precision and accuracy and compared it with other devices such as iPhone, Germini Extreme and Motorola drone device. This experiment was conducted outside on predefined circle area whose diameter is 6 meters. First I determined center of the circle and placed all 4 devices on the center. Than I recorder latitude and longitude from all four devices. After that I moved them away for a minute and placed them back on the very same center and recorded the values again. I repeated this proces 4 times and averaged the results which I latter placed on the Google Earth map. The results vere very satisfying and have proved that Nexus 7 has GPS precise and accurate enough for the application it is going to be used for. The following picture shows the visualization of collected data. | Very important factor in this decision had integrated GPS receiver. Before selecting Nexus 7 as the base platform I implemented a couple of tests of its GPS precision and accuracy and compared it with other devices such as iPhone, Germini Extreme and Motorola drone device. This experiment was conducted outside on predefined circle area whose diameter is 6 meters. First I determined center of the circle and placed all 4 devices on the center. Than I recorder latitude and longitude from all four devices. After that I moved them away for a minute and placed them back on the very same center and recorded the values again. I repeated this proces 4 times and averaged the results which I latter placed on the Google Earth map. The results vere very satisfying and have proved that Nexus 7 has GPS precise and accurate enough for the application it is going to be used for. The following picture shows the visualization of collected data. | ||
| − | My research showed that stand alone USB water, air and soil probes are available on market and their costs vary from $100 to $500. | + | [[File:Filemoze_gps.png|700px|thumb|center|alt=Checking the precision of GPS receiver of Nexus 7|Checking the precision of GPS receiver of Nexus 7]] |
| + | |||
| + | |||
| + | My research showed that stand alone USB water, air and soil probes are available on market and their costs vary from $100 to $500. One of the example would be a NexSens Smart Sensor capable of measuring water quality, pH and temperature. Its documentations claims hat temperature accuracy is +/-0.2 C and taht pH Range is 0-14 pH. The sensor does not need any external energy source because it is powered my USB. The price is $299 and it can se seen on the following picture. | ||
| + | |||
| + | [[File:Filemoze_water_quality.png|300px|thumb|center|alt=NexSens Smart Sensor|NexSens Smart Sensor]] | ||
=Geocoded data = | =Geocoded data = | ||
| − | + | Some data sets that I came across in the last few years were geocoded. This process is not specifically defned neither correct definition exists. It is usually in the form of a ZIP Code or a mailing address. To place it on a map accurately one must know its coordinates. Generally it means recording the Latitude (the angular distance, in degrees, minutes, and seconds of a point north or south of the Equator) and Longitude (the angular distance, in degrees, minutes, and seconds, of a point east or west of the Prime (Greenwich) Meridian) of the place where measurement was taken. Geocoding could also be seen as metadata of reading. Some research papers that I came across, in addition to latitude and longitude had another parameters such as altitude, temperature, date and time in their metadata. Surprisingly, none had all of them. | |
| + | |||
| + | I have learned from the people I work with and later confirmed by my personal experience that we are great at collecting huge quantities of data but at the same time we are not good at making sense of those data and taking the full advantage of them. One way of taking better advantage of collected date would be to visualize them. Having automatically geocoded date would make the process of visualization much easier, more precise and more interesting. | ||
| − | + | Enriching data with another dimension (latitude, longitude and altitude) is crucial if one wants to have very useful and good quality readings. Therefore I came up with special algorithm, which can be see on the Figure 3, for data geocoding on our sensor monitoring platform (Nexus 7). | |
| − | |||
| + | [[File:Filemoze_only_gps.png|700px|thumb|center|alt=Algorithm No1|Algorithm No1]] | ||
| − | |||
| − | + | As we can see on the algorithm diagram, everything starts with the button Collect which will take three reading of latitude and longitude, average them up and store one final value in csv file column number 1 and 2. Once this process is complete, three reading of altitude will be taken and average will be stored in the csv file in column number 3. Furthermore, column number 4 will be populated with magnetic field data. The date and time will be stored in the column number 5 and 6. This part is going to happen every time one press the collect button regardless of which sensor probe is plugged to the monitoring platform. | |
| − | |||
| − | + | = Base sensor = | |
| − | { | + | My sample sensor will be {\bf Yocto-Meteo}, a device developed in Switzerland capable of measuring via USB the current humidity, pressure, and temperature. Designer of this product claims that device has accuracy of 0.3$^\circ$C for temperature, about 4\% for the humidity rate, and 1mbar for the atmospheric pressure (my personal test of precision and accuracy of Yocto-Meteo will be documented in 5.1). Furthermore, Yocto-Meteo has integrated data logger capable of recording data on its internal flash for later retrieval when connected again by USB (this function wont be used for my preliminary device test stage). |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| Line 71: | Line 71: | ||
[[File:precision.png|800px|thumb|center|alt=Estimating Precision of Yocto-Meteo device|Estimating Precision of Yocto-Meteo device]] | [[File:precision.png|800px|thumb|center|alt=Estimating Precision of Yocto-Meteo device|Estimating Precision of Yocto-Meteo device]] | ||
| − | Based on this graph, we can conclude that Yocto-Meter are very precise because | + | Based on this graph, we can conclude that Yocto-Meter are very precise because two of them have almost the same measurements. |
| − | One of the ways to check | + | One of the ways to check {\bf accuracy} is to use multiple different thermometers, place them in the same environment, take readings at the same time and compare them to Yocto-Meteo readings. |
| − | For the purpose of this experiment, I have used | + | For the purpose of this experiment, I have used eight individual termometers and record their values at the same time. I waited few minutes before taking the temperature measurements because of sensor adaptation to extreme temperature change (from swimming poll to the Earlham Heart) The following picture demonstrates my set up of all eight devices. |
[[File:setup_4.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | [[File:setup_4.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | ||
| − | This graph shows all | + | This graph shows all eight termometers and we can see that values are very close, they are all in the window of 5$^\circ$C difference. From the Figure, it is clear that Arduino setup has the biggest variations and difference when compared to all other sensors, one of the reasons for this could be a low end thermometer. |
[[File:accuracy_1.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | [[File:accuracy_1.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | ||
| − | Purple line shows average of all termometers other than Arduino (because of its deflection from all other devices) and Yocto-Meteo RED and BLUE. This graph is a prof that Yocto-Meteor devices are vert accurate because their line is almost identical to average of other 6 thermometers. | + | Purple line shows average of all termometers other than Arduino (because of its deflection from all other devices) and Yocto-Meteo RED and BLUE. This graph is a prof that Yocto-Meteor devices are vert accurate because their line is almost identical to average of other 6 thermometers. The 2.5$^\circ$C difference between average and Ypcto-Meteo in the CCG Machine Room can be explained with shorter period of sensors adaptation in the CCH Machine Room setup. |
[[File:average.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | [[File:average.png|900px|thumb|center|alt=A cartoon centipede reads books and types on a laptop.|The Wikipede edits]] | ||
| Line 116: | Line 116: | ||
*Take a look at the cvs file | *Take a look at the cvs file | ||
*Upload a cvs file to MicroFe which will visualize it in Gnuplot | *Upload a cvs file to MicroFe which will visualize it in Gnuplot | ||
| + | |||
| + | |||
| + | Final algorithm is shown in the following figure: | ||
| + | |||
| + | [[File:Filemoze.png|1000px|thumb|center|alt=Algorithm No1|Algorithm No1]] | ||
| + | |||
| + | |||
| + | As we can see on the algorithm diagram, everything starts with the button Collect which will take three reading of latitude and longitude, average them up and store one final value in csv file column number 1 and 2. Once this process is complete, three reading of altitude will be taken and average will be stored in the csv file in column number 3. Furthermore, column number 4 will be populated with magnetic field data. The date and time will be stored in the column number 5 and 6. This part is going to happen every time one press the collect button regardless of which sensor probe is plugged to the monitoring platform. The column number 7 will have a unique key which will correspond to the person who is currently using the monitoring platform. The cvs column number 8 will contain name and the ID of the device (water, air, soil, temperature...) and after column 8 other instances of data will contain specific parameters generated by different probes. In the case of Yocto-Meteo, the output will look like this. | ||
| + | |||
| + | After pressing Collect, this output should be stored to memory: | ||
| + | '''39.82427190, -84.91212248, 299, 334-834, 10.26.2012, 2:25, IvanBabic001688412, Yocto-Meteo-1001, 16, 34, 90''' | ||
| + | |||
| + | This is how unique keys are matching with csv values: | ||
| + | |||
| + | {| class="wikitable" style="text-align:center" | ||
| + | |- | ||
| + | ! Latitude !! Longitude !! Altitude !! Magnetic Field !! Date !! Time !! Collectors ID !! Device ID !! Temperature !! Humidity !! Presure | ||
| + | |- | ||
| + | | 39.82427190 || -84.91212248|| 299 || 334-834 || 10.26.2012 || 2:25 || IvanBabic001688412 || Yocto-Meteo-1001 || 16 || 34 || 90 | ||
| + | |} | ||
| + | |||
| + | |||
Latest revision as of 12:45, 10 December 2012
Inexpensive, portable and extendable open source environmental monitoring platform with integrated geocoding
Contents
Abstract
The aim of my senior project is to designe an inexpensive, flexible, open and portable platform for environmental monitoring sensors. In order to achieve this goal, my paper will deeply explore both hardware and open source software solutions with a final goal of creating a simple monitoring platform consisting of one environmental sensor and a frame support software. A very strong emphasis will be placed on integrated geocoding of data for sensors reading conducted by this monitoring platform.
Introduction
Everything our civilization have learned so far about the planet we all live in is from observing and monitoring the environment around us. Understanding how our planet works and predicting its behavior is very important and one of the underlying ideas of modern science. In order to implement successful experiments and observations, a complete understanding of complicated environment sensors/instruments is crucial. I personally believe that complexity of monitoring sensors devices such as different operating systems, variety unstandardized data formats they generate and particularly high cost are creating significant friction for many field scientist (Biologists, Chemists, Geologists, Physicists, Geochemists, Archeologists, Environmental Scientists and many others) to do the actual science.
My inexpensive and portable platform aims to remove friction related to extremely steep learning time curve (which is not useful) for every individual monitoring device, their operating systems and variety of different data ?les they generate. The entire project is going to be part of the open source community which will make it easy for everybody to integrate any other sensor with USB interface in to this system.
Main reason for high cost of monitoring devices
A simple water quality meeter, with closed software, inability to add any other sensor probe to it and without any possibility of directly geocoding collected data costs around $3,000. An example is YSI 650 MDS Datalogger water quality meeter (Picture 1) whose cost is \$3.600. This multi thousands dollars worth sensor, YSI 650 MDS, does not have a USB port neither a way of data geocoding. If we are planing to monitor water, air and soil the total device cost will add up close to $8,000. Taking in consideration the fact that none of those devices is able to geocode generated data, we have to add jet another device, a Global Positioning System (GPS). Having four different monitoring platforms, dealing with inconsistency in their underlying architectures, operating systems and geocodeless readings they generate is more than inconvenient and extremely expensive.
PICTURE HERRREEEE
The problem is more than obvious, every sensor, as seen on the picture one, has its unique computer platform attached to it whose main job is to communicate with a sensor probe and take readings from it. In the case of water quality meeter, platform to where water probe is connected costs 5 times more than probe itself. Cost is high because every basic computer platforms has its screen, memory, battery...
Solution
The easiest way to significantly reduce the cost of monitoring devices it to have one platform with one battery, one screen and one memory capable of accepting variety of different sensor probes such as water, air and soil quality probes.
My plan is to use an Android Nexus 7 whose cost is $199,0 for base platform. The main reason behind this decision is its low cost, integrated GPS device (Integrated Monolithic GPS Receiver - BCM4751) as well as few more sensors (Gyro sensor, Magnetometer, Accelerometer), well documented USB interface, quad-core processor, open source operating system and great reviews.
Very important factor in this decision had integrated GPS receiver. Before selecting Nexus 7 as the base platform I implemented a couple of tests of its GPS precision and accuracy and compared it with other devices such as iPhone, Germini Extreme and Motorola drone device. This experiment was conducted outside on predefined circle area whose diameter is 6 meters. First I determined center of the circle and placed all 4 devices on the center. Than I recorder latitude and longitude from all four devices. After that I moved them away for a minute and placed them back on the very same center and recorded the values again. I repeated this proces 4 times and averaged the results which I latter placed on the Google Earth map. The results vere very satisfying and have proved that Nexus 7 has GPS precise and accurate enough for the application it is going to be used for. The following picture shows the visualization of collected data.
My research showed that stand alone USB water, air and soil probes are available on market and their costs vary from $100 to $500. One of the example would be a NexSens Smart Sensor capable of measuring water quality, pH and temperature. Its documentations claims hat temperature accuracy is +/-0.2 C and taht pH Range is 0-14 pH. The sensor does not need any external energy source because it is powered my USB. The price is $299 and it can se seen on the following picture.
Geocoded data
Some data sets that I came across in the last few years were geocoded. This process is not specifically defned neither correct definition exists. It is usually in the form of a ZIP Code or a mailing address. To place it on a map accurately one must know its coordinates. Generally it means recording the Latitude (the angular distance, in degrees, minutes, and seconds of a point north or south of the Equator) and Longitude (the angular distance, in degrees, minutes, and seconds, of a point east or west of the Prime (Greenwich) Meridian) of the place where measurement was taken. Geocoding could also be seen as metadata of reading. Some research papers that I came across, in addition to latitude and longitude had another parameters such as altitude, temperature, date and time in their metadata. Surprisingly, none had all of them.
I have learned from the people I work with and later confirmed by my personal experience that we are great at collecting huge quantities of data but at the same time we are not good at making sense of those data and taking the full advantage of them. One way of taking better advantage of collected date would be to visualize them. Having automatically geocoded date would make the process of visualization much easier, more precise and more interesting.
Enriching data with another dimension (latitude, longitude and altitude) is crucial if one wants to have very useful and good quality readings. Therefore I came up with special algorithm, which can be see on the Figure 3, for data geocoding on our sensor monitoring platform (Nexus 7).
As we can see on the algorithm diagram, everything starts with the button Collect which will take three reading of latitude and longitude, average them up and store one final value in csv file column number 1 and 2. Once this process is complete, three reading of altitude will be taken and average will be stored in the csv file in column number 3. Furthermore, column number 4 will be populated with magnetic field data. The date and time will be stored in the column number 5 and 6. This part is going to happen every time one press the collect button regardless of which sensor probe is plugged to the monitoring platform.
Base sensor
My sample sensor will be {\bf Yocto-Meteo}, a device developed in Switzerland capable of measuring via USB the current humidity, pressure, and temperature. Designer of this product claims that device has accuracy of 0.3$^\circ$C for temperature, about 4\% for the humidity rate, and 1mbar for the atmospheric pressure (my personal test of precision and accuracy of Yocto-Meteo will be documented in 5.1). Furthermore, Yocto-Meteo has integrated data logger capable of recording data on its internal flash for later retrieval when connected again by USB (this function wont be used for my preliminary device test stage).
Validating of sensors precision and accuracy
Sensors precision and accuracy is the most important factor of any research. I am using temperature probe as my first sensor because I have the ability to test both its precision and accuracy.
The best way to check devices precision is to have multiple Yocto-Meteo devices, put them in the same environment and record their data at the same time. Based in this, I went to 6 different environments on campus whose temperature varies greatly. Place where I took measurements are as follow: Wellness Center, The Heart, 4th Floor Out, 4th Floor In, CCG Machine Room and Barrette.
Based on this graph, we can conclude that Yocto-Meter are very precise because two of them have almost the same measurements.
One of the ways to check {\bf accuracy} is to use multiple different thermometers, place them in the same environment, take readings at the same time and compare them to Yocto-Meteo readings.
For the purpose of this experiment, I have used eight individual termometers and record their values at the same time. I waited few minutes before taking the temperature measurements because of sensor adaptation to extreme temperature change (from swimming poll to the Earlham Heart) The following picture demonstrates my set up of all eight devices.
This graph shows all eight termometers and we can see that values are very close, they are all in the window of 5$^\circ$C difference. From the Figure, it is clear that Arduino setup has the biggest variations and difference when compared to all other sensors, one of the reasons for this could be a low end thermometer.
Purple line shows average of all termometers other than Arduino (because of its deflection from all other devices) and Yocto-Meteo RED and BLUE. This graph is a prof that Yocto-Meteor devices are vert accurate because their line is almost identical to average of other 6 thermometers. The 2.5$^\circ$C difference between average and Ypcto-Meteo in the CCG Machine Room can be explained with shorter period of sensors adaptation in the CCH Machine Room setup.
Software functionality
- Launch Application
- Enter your ID: First, Last name and student ID
- Press "Collect"
- Record Latitude, Longitude and Altitude (3 fields in cvs)
- Record Magnetic Reading (1 field in csv)
- Record Date and Time (2 fields in csv)
- Record ID (1 field in csv)
- if sensor detected
- Record name and ID of sensor
- if Yocto-Meteo
- Record temperature, humidity and pressure values (3 fields in csv)
- if x Sensor
- Record x (x fields in scv)
- if Yocto-Meteo
- Move to another line in cvs file
- Record name and ID of sensor
- Move to another line in cvs file
Other functionalities:
- Take a look at the cvs file
- Upload a cvs file to MicroFe which will visualize it in Gnuplot
Final algorithm is shown in the following figure:
As we can see on the algorithm diagram, everything starts with the button Collect which will take three reading of latitude and longitude, average them up and store one final value in csv file column number 1 and 2. Once this process is complete, three reading of altitude will be taken and average will be stored in the csv file in column number 3. Furthermore, column number 4 will be populated with magnetic field data. The date and time will be stored in the column number 5 and 6. This part is going to happen every time one press the collect button regardless of which sensor probe is plugged to the monitoring platform. The column number 7 will have a unique key which will correspond to the person who is currently using the monitoring platform. The cvs column number 8 will contain name and the ID of the device (water, air, soil, temperature...) and after column 8 other instances of data will contain specific parameters generated by different probes. In the case of Yocto-Meteo, the output will look like this.
After pressing Collect, this output should be stored to memory: 39.82427190, -84.91212248, 299, 334-834, 10.26.2012, 2:25, IvanBabic001688412, Yocto-Meteo-1001, 16, 34, 90
This is how unique keys are matching with csv values:
| Latitude | Longitude | Altitude | Magnetic Field | Date | Time | Collectors ID | Device ID | Temperature | Humidity | Presure |
|---|---|---|---|---|---|---|---|---|---|---|
| 39.82427190 | -84.91212248 | 299 | 334-834 | 10.26.2012 | 2:25 | IvanBabic001688412 | Yocto-Meteo-1001 | 16 | 34 | 90 |
Visualization
Timeline
- Done:
- Create android development platform (software chain)
- Make a simple Hello World android application
- Make an application that will read latitude and longitude
- Learn about different sensors
- Decide which sensor to use first
- Contact people from Yoctopus about their sensor
- Contact people from water sensor company
- Learn more about USB and its way of communication
- Learning more abut Yocto-Meteo
- Testing its accuracy and precision
- Document its capabilities, max and mins and error
- Working on now:
- Figuring out the shape of graph output (final product)
- To do soon:
- Add altitude, # of satellites, precision, time and date to the application
- Figure out how to make android device communicate with sensor probe
- Make application store data in to the csv file (together with geocoded info) every time collect button is pressed
- Create an option allowing upload of collected data to another server or cloud
- Create a changeable timer that will initiate collect button every x minutes
- Design s simple web interface using tool such as Gnuplot to automatically visualize uploaded temperature data
- Note to myself, learn how to spell without spellchecker, one day you might use paper again...
