Duluth Laboratories & Administration 5013 Miller Trunk Highway Duluth, Minnesota 55811 Coleraine Laboratories One Gayley Avenue P.O. Box 188 Coleraine, Minnesota 55722
By Michael Joyce, Ph.D. Ron Moen, Ph.D. Report Number: NRRI/TR -2018/28, Release 1.0
SUMMARY
One of the main factors that affect GPS location accuracy is the type of GPS receiver being used. In general, more expensive receivers (e.g., mapping-grade or survey-grade receivers) provide better accuracy, and GPS users must balance GPS receiver cost with location accuracy when determining which receiver to use. Applications of GPS often require use of GPS receivers in less than ideal conditions while GPS manufacturers often report accuracy specifications that can be expected under ideal conditions. Forest canopies reduce GPS accuracy by interfering with signal transmission between GPS satellites and the GPS receiver and causing multipath errors. When GPS receivers are to be used in forest conditions and accuracy thresholds must be met, it is important to conduct accuracy testing in forest conditions rather than relying on accuracy specifications provided by the manufacturer.
We tested the accuracy of the SXBlue II + GNSS, a modular, mapping-grade GPS receiver, under forest canopies in northeastern Minnesota. We estimated cumulative accuracy to evaluate the relationship between collection period and accuracy. GPS test sites covered a range of canopy conditions. We compared accuracy among sites to determine how canopy closure influenced location accuracy. Finally, we compared post-hoc methods to evaluate accuracy based on characteristics of the sites and acquired GPS fixes. The SXBlue II + GNSS receiver typically provided meter or sub-meter accuracy, even under forest canopy. Maximum accuracy was achieved after 10- 30 minutes. Accuracy was lower at sites with higher canopy closure values. In sites with canopy closure >65%, maximum accuracy was reduced to 1.5 m. Post-hoc filtering to remove outliers did not improve accuracy. There was a strong, positive relationship between 50% CEP, a measure of location precision, and accuracy, suggesting that 50% CEP can be used for post-hoc accuracy assessment. Our results suggest that the SXBlue II + GNSS provides sufficient accuracy for a wide range of applications, including those that require GPS location measurement in forest conditions
This report and any future updates can be downloaded from the University of Minnesota Digital Conservancy (https://conservancy.umn.edu). For questions related to NRRI reports, contact us @ nrrireports@umn.edu. Web site: http://www.nrri.umn.edu ©2018 by the Regents of the University of Minnesota
INTRODUCTION
With the increasing availability of LiDAR data for forestry and wildlife applications, precise geographic positioning is critical to ensure features of interest (e.g., field plot locations, animal locations, etc.) can be compared directly with the corresponding LiDAR data or derived products. Spatial overlap is affected by both global positioning system (GPS) accuracy and horizontal accuracy of LiDAR data. GPS accuracy is usually a bigger source of positional error (White et al. 2013). LiDAR data often have horizontal accuracy within 1 meter. Relatively inexpensive recreation-grade (also known as consumer-grade) GPS receivers typically have THE accuracy of about 9 meters when used under closed forest canopy (Wing and Eklund 2007, Wing 2008).
Survey-grade GPS receivers can achieve centimeter-level accuracy, but tend to be cost-prohibitive for many applications (Laes et al. 2011, White et al. 2013). Guidelines for forest inventory modeling using LiDAR typically recommend using mapping-grade GPS receivers capable of obtaining locations with sub-meter accuracy under forest canopy (Laes et al. 2011). Mapping-grade receivers cost less than survey-grade receivers, and they can typically achieve sub-meter to 2 m accuracy (White et al. 2013). Modular mapping-grade GPS receivers now available are less expensive but still as accurate.
GPS position error is typically caused by interference with the signal being broadcast from the satellite and received by the GPS unit. Given that the GPS satellites are about 20,000 km above the earth, it is not surprising that interference occurs. Forest canopies obstruct the signal, especially when moisture is present (Johnson and Barton 2004, Edson and Wing 2012), and can also reflect the signal and cause multipath interference in which the receiver has difficulty identifying the signal amongst the noise (Wing 2008). For these reasons, using GPS in forested environments is often associated with reduced accuracy. One solution to improve accuracy is to use differential correction with a base station. If a base station is located at a known location the error in the position can be calculated, although base stations are not typically located under a forest canopy. If the same satellites are used, the error should be the same at the unknown location where a GPS unit is, and the result of this differential correction is higher precision. Differential GPS approaches include both real-time differential correction, for which the GPS unit receives corrections in real time from a base station, and post-processing when corrections are applied after the GPS data have been acquired.
Another solution to reduce GPS error is the use of Space-Based Augmentation Systems (SBAS), such as the Wide Area Augmentation System (WAAS) that covers Central and North America. SBAS utilizes a network of ground reference stations with known locations which provide information to a master station that calculates corrections that can be applied over a wide area. SBAS calculates separate correction factors for different error sources (e.g., ionospheric errors, GPS satellite timing errors, GPS satellite orbit errors) rather than calculating the total effect of these factors. Corrections are broadcast using a constellation of geostationary satellites, allowing use of SBAS for real-time correction without the need for communicating with a differential GPS base station.
Our objectives were to test the accuracy of a modular, mapping-grade GPS receiver under forest canopies in northeastern Minnesota. The GPS receiver we tested is capable of sub-meter GPS accuracy under ideal conditions but has not been tested in conditions under a forest canopy. We (1) evaluated the relationship between the length of the data collection period and accuracy, (2) identified the effect of tree canopy closure on the accuracy, and (3) tested potential post-hoc methods to evaluate accuracy based on site characteristics or GPS data. Our results are specific to the receiver and software that we used, but could logically be extended to other GPS units in similar conditions.
METHODS
Field Testing
We tested the horizontal accuracy of the SXBlue II + GNSS receiver (Geneq Inc., Montreal, Quebec, Canada), a compact Global Navigation Satellite System (GNSS) receiver. In ideal conditions with an unobstructed view of the sky, the SXBlue II + GNSS should provide sub-meter horizontal accuracy 95% of the time (Geneq Inc., 2014). In many locations the view of the sky is obstructed by trees, hillsides, or other structures. We determined expected accuracies when using the SXBlue II receiver under forest canopy.
The SXBlue II + GNSS receiver uses conventional real-time differential corrections obtained from a Space Based Augmentation System (SBAS) to improve position accuracy. The SXBlue II + GNSS unit receives location information from both GPS and GLONASS satellite constellations. Use of both satellite systems improves accuracy and reduces the chance that poor satellite geometry will reduce position accuracy by increasing the number of satellites that are available to determine the position.
The SXBlue II + GNSS receiver is one component of a modular system to collect location information at a field site (Fig. 1). The two other required components of the system are data acquisition hardware and data collection software. Many types of computers can be used as the acquisition hardware, including smart phones, laptops, PDA, and tablets. There are also options for data collection software, including free mobile applications, ArcGIS Collector, and Microsoft Windows-compatible software. Data collection software acquires data from one of 3 available communication options: (1) Bluetooth port (Class 1), (2) USB Port (Type B, female port), and (3) RS-232 Serial Port.
Figure 1. Components of the system we used:
We used a tablet (Samsung Galaxy Tab A) and mobile applications to collect location data at test sites using the SXBlue II + GNSS receiver. We used the ‘Bluetooth GPS’ application (Version 1.3.7, GG MobLab) to establish a Bluetooth connection between the receiver and the tablet, and the ‘GPSlogger’ application (Version 91, Mendhak) to collect location information. Both mobile applications are available free of charge from Google Play. We set GPSlogger settings to record a point every 2 seconds. With this setting, points were actually recorded every 3 seconds. We could not analyze the effect of the number of satellites or dilution of precision measurements on location error because GPSlogger does not store those values.
We conducted stationary tests at 9 georeferenced survey markers (Fig. 2) from October 2016- November 2017. We obtained information on survey markers from St. Louis County’s Survey Explorer website (http://gis.stlouiscountymn.gov/gisviewers/surveyexplorer.aspx). We viewed corner reports of potential sites and selected sites that were georeferenced with reported horizontal accuracies ≤0.05 m. The final set of sites included a range of forest conditions, from open, non-forested sites to mature, closed-canopy sites.
Figure 2. Map of GPS test sites at georeferenced survey markers in St. Louis County, Minnesota.
We used a standard protocol at each site to ensure location data were collected consistently. The SXBlue II receiver was assembled and turned on in an area with a clear view of the sky (i.e., with no vegetation directly overhead). We kept the receiver and antennae in an area with a clear view of the sky until the receiver indicated it had achieved a differential position and it had obtained an SBAS lock (i.e., indicator lights ‘DGPS’ and ‘DIFF’ were illuminated). The SXBlue II antenna was then attached to a tripod and positioned directly over the survey monument. We measured the height of the antenna above ground to account for variations in antenna height among sites. Once the receiver was in position over the survey monument, we allowed the receiver to track satellites and acquire SBAS corrections for 5 minutes before beginning to collect location data. We collected location data for at least one hour at each site (Table 1). Mean test duration was 161 minutes (SE = 38, minimum = 60 min, maximum = 370 min).
We measured canopy closure over each survey monument with a convex densitometer using Strickler’s (1959) modification. Mean canopy closure at test sites was 57% (SE = 11%; minimum = 0%; maximum = 97%). Five of 9 test sites had canopy closures >65% (Table 1). We also recorded site (forest cover type, tree species, relative age or size, etc.) and weather (cloud cover, wind conditions, etc.) conditions.
Data Analysis
For each test, we calculated the cumulative average x- and y-coordinates following the addition of each new position fix. We calculated cumulative accuracy over time as the straight-line distance between each cumulative average coordinate and the “true” site coordinates obtained from the corner report. This allowed us to evaluate accuracy at specific time intervals and determine how long it took to achieve a sub-meter location. We limited cumulative accuracy calculations to the first hour of testing and pooled data among sites to calculate mean cumulative accuracy and 95% confidence interval. We used simple linear regression to test for a relationship between accuracy and canopy closure, antennae height, or precision. Individual models were fit for each variable because the sample size was too small to fit multivariate models. We used 50% circular error probability (CEP) as the measure of precision.
We calculated direction and angular dispersion for each site (i) at 5-minute intervals over the first hour of testing to evaluate directional bias. Direction was calculated as the angle from the true location to the cumulative average coordinate for a given time interval. Angular dispersion was calculated using equations given by Zar (1984). Dispersion is a measure of concentration of angles that ranges from zero to one. Values near one indicate high concentration and, therefore, directional bias.
Cumulative accuracy estimates described above do not capture variation in location accuracy at any given site. To evaluate variation in SXBlue II accuracy over time, we changed the start time for each test and re-calculated cumulative accuracy over time for individual one-hour sample intervals. For each site, we used a total of 5 different sample intervals. For tests lasting ≥5 hours (n = 2), sample intervals started at 0, 1, 2, 3, and 4 hours. For tests lasting >1 but <5 hours (n = 4), sample intervals started every 15-30 minutes. For tests lasting 1 hour (n = 3), samples started at 0, 5, 10, 15, and 20 minutes, and each test duration got progressively shorter. We used the 5 sample intervals to calculate mean and 95% confidence intervals for cumulative accuracy over time. We then used site-level mean accuracies to compute grand mean cumulative accuracies and 95% confidence intervals.
We evaluated whether post-hoc filtering of obvious outliers could improve location accuracy. Visual inspection of test data revealed that points were occasionally recorded in linear tracks outside the main cluster of points and generally farther away from the “true” location. We calculated 95% CEP for each test site using all points collected over the first 30 minutes. We used the average x- and y-coordinates of test points rather than the “true” location to calculate 95% CEP. We then removed points outside 95% CEP and compared error of unfiltered and filtered data using a paired t-test. The alternative hypothesis for the paired t-test was that errors of filtered data were less than errors from unfiltered data.
RESULTS
Cumulative accuracy was sub-meter within 0.75 minutes when 9 sites were averaged, after which it remained sub-meter for the duration of the testing (Fig. 3). The minimum average cumulative accuracy was 0.64 m, which occurred at 9.05 minutes. The upper 95% confidence interval was above 1 m until about 20 minutes, after which the upper 95% confidence interval remains consistently at about 1 m.
Figure 3. Mean cumulative accuracy over time. The black solid line represents the average cumulative accuracy among 9 test sites. Red dashed lines represent the 95% confidence interval. The cumulative accuracy at each test site is shown in Fig. 4.
Patterns of cumulative accuracy over time varied among test sites (Fig. 4). At 4 test sites, cumulative accuracy decreased to about 0.5 m after about 5 minutes and remained fairly stable for the remainder of the test. For the other 5 sites, cumulative accuracy remained at or above 1 m until about 20-25 minutes, after which cumulative accuracy for all but one test remained below or near 1 m for the remainder of the test duration. After 5 minutes, cumulative accuracy for all test sites was generally less than 1.5 m, with a maximum cumulative accuracy of 1.62 m for times >5 minutes.
Differences in accuracy among sites seemed to be affected by canopy closure. The 3 sites with canopy closure < 40% reached sub-meter accuracy within 10 minutes and remained at sub-meter accuracy for the duration of the test (Fig. 4). In contrast, only 1 of 6 sites with >50% canopy closure had reached sub-meter accuracy within 20 minutes. After 40 minutes, 4 of 6 sites with >50% canopy closure had reached sub-meter accuracy and remained at sub-meter accuracy for the duration of the 1-hour test. The test with the largest error had 97% canopy closure.
Figure 4. Cumulative accuracy over time for individual test sites. Each line represents a different test site. Labels in the right-hand margin indicate canopy closure at the corresponding site.
Error at 30 minutes increased (i.e., lower accuracy) with increasing canopy closure (Fig. 5a), but the effect was only marginally significant (F1,7 = 4.62, P = 0.07). This regression model suggested that on average, the SXBlue II provides sub-meter accuracy until ~90% canopy closure. However, canopy closure only explained about 40% of the variation in error among sites (R2 = 0.398).
Figure 5. Relationship between accuracy at 30 minutes and canopy closure (a) or precision (b) among test sites. Precision was measured as 50% CEP at 30 minutes. Red lines indicate linear regression models. Expected accuracy is sub-meter (blue line) at <90% canopy closure and <1.02 m 50% CEP.
An alternative method that could be used without the need to collect field canopy closure measurements is to determine if characteristics of the GPS locations are correlated with location error. For example, a characteristic like 50% CEP can be calculated from points collected at any site. When 50% CEP was compared to location error from the survey marker, error at 30 minutes increased with increasing 50% CEP (Fig. 5b; F1,7 = 17.6, P = 0.004, R2 = 0.72). This regression model suggested that on average, the SXBlue II provides sub-meter accuracy when 50% CEP at 30 minutes is <1.02 m.
There was a positive correlation between canopy closure and 50% CEP at 30 minutes (r = 0.87). Although either measure could be used to estimate accuracy of unknown locations, the potential for measurement variability is higher for canopy cover, and 50% CEP is a property of the collected data that is relatively easy to calculate. We did not find a significant relationship between accuracy and antenna height (F1,7 = 0.002, P = 0.97)
Both direction and angular dispersion were fairly constant over time (Figs. 6 and 7, Table 2). Direction and dispersion values suggest persistent directional bias of points to the northwest, west, and southwest of the true location.
Figure 6. Directional bias of average x/y coordinate over time. The SXBlue II + GNSS receiver shows persistent bias to the northwest/west relative to the true location
Figure 7. Angular dispersion of the cumulative average x/y coordinate over time
Table 2. Canopy closure, accuracy, mean angle, and dispersion for each test site calculated from points collected over the first 30 minutes of operation. Values are given for both full (unfiltered, "u") and filtered ("f") datasets. Filtering did not increase accuracy or reduce directional bias.
Filtering to remove outliers did not improve accuracy (Table 2; t = 0.64, df = 8, P = 0.27). Filtering changed the mean angle (t = 3.6, df = 8, P = 0.004) and increased dispersion (t = -2.63, df = 8, P = 0.015). Varying the start time changed the patterns of cumulative accuracy over time at each test site (Fig. 8), but the overall results were consistent with our analysis of the first hour of testing (Fig. 3, Fig. 4). Submeter accuracy was achieved after about 8 minutes for 7 of the 9 sites. At one site, accuracy was not sub-meter until ~49 minutes, although it was close to 1 m after about 35 minutes. For the other site, accuracy varied between 1.4 and 1.8 meters and did not improve over time. In general, accuracy was better for open sites and those with lower canopy closure values compared to sites with closed canopies.
Figure 8. Effect of start time on pattern of cumulative accuracy over time. Each line represents the cumulative accuracy at a test site averaged over 5 samples with different start times. Labels in the righthand margin indicate canopy closure at the corresponding site.
DISCUSSION
The SXBlue II + GNSS receiver generally provided sub-meter accuracy in closed canopy conditions, which means that it can be used for applications such as LiDAR forest inventory modeling that require meter or sub-meter accuracy. At sites with dense canopy (>65% canopy closure), the accuracy of the SXBlue II + GNSS receiver was reduced to ~1.5 m. This accuracy level is considerably better than that typically provided by consumer-grade GPS in forest conditions (9 m; Wing 2008) and is still sufficient for many applications. One potential strategy to improve accuracy when working in deciduous forest types would be to record locations during the leaf-off season, when canopy vegetation density is reduced. This strategy may not be suitable for all applications, but when feasible could provide enhanced accuracy at sites with closed canopies.
Location precision estimated using 50% CEP was a better predictor of location accuracy than canopy closure, suggesting that calculation of 50% CEP may provide a robust means of post-hoc accuracy assessment. Although forest canopy reduces GPS accuracy, canopy closure does not provide information on other factors (e.g., GPS satellite geometry) that also influence GPS accuracy. The 50% CEP likely had a stronger relationship with accuracy because CEP indirectly accounts for canopy closure and other factors that influence GPS accuracy. Using CEP to evaluate accuracy may be especially useful when using software (such as the GPSlogger application we used) that does not record HDOP, the number of satellites used to calculate a fix, or other information that would allow for a more direct measure of GPS performance.
When we varied the start time, our results were largely consistent with those obtained from the first hour of testing, suggesting that our cumulative accuracy estimates were fairly representative of the accuracy that can be expected using the SXBlue II + GNSS receiver. Regardless of the start time, most tests had stabilized at the meter or sub-meter accuracy after 30 minutes of location collection, further emphasizing that 30-minute collection intervals may be preferred when using the SXBlue II + GNSS receiver under forest canopy and sub-meter accuracy is needed.
When designing GPS location protocols, users should determine collection duration by considering the tradeoff between field-data collection efficiency and desired location accuracy. In general, maximal accuracy was achieved after 10 minutes, but waiting 20-30 minutes may provide better accuracy in areas where canopy closure is >65%. If sub-meter accuracy is not essential, shorter collection intervals may be desirable to improve time efficiency.
Filtering locations to remove outlying points did not improve accuracy because there were relatively few outlying points. For shorter collection durations it may be a worthwhile consideration, but for 30-minute locations with fixes recorded every 3 seconds, most fixes were concentrated around the true location. Consequently, the relatively few outlying points did not have much influence on the average location computed from the hundreds of fixes recorded.
Our results indicate slight directional bias in locations collected using the SXBlue II + GNSS, which could limit the applications of location data obtained with this receiver. Because directional bias was associated with relatively low positional errors (typically ≤1 m), directional bias is important to be aware of but is probably not a major concern for most applications.
Performing additional tests in which locations are collected at different sites, or for the same sites on different days or in different seasons, would provide additional information to evaluate the effect of variation in satellite constellation geometry, site conditions, and weather on location accuracy. This would also make it possible to further test the relationship between CEP and accuracy. However, it appears that sub-meter to 1.5 m accuracy is possible when using the SXBlue II + GNSS receiver under forest canopy. At sites with canopy closure <65%, sub-meter accuracy is likely for locations collected for 10-30 minutes or more.
ACKNOWLEDGEMENTS
We would like to thank Josh Kircher for help collecting field data. MJJ was supported by a Doctoral Dissertation Fellowship from the University of Minnesota. The receiver was purchased by the Natural Resources Research Institute. Additional support was received from the MN Department of Natural Resources and the Minnesota Environment and Natural Resources Trust Fund (ENRTF) as recommended by the Legislative‐Citizen Commission on Minnesota Resources (LCCMR). The Trust Fund is a permanent fund constitutionally established by the citizens of Minnesota to assist in the protection, conservation, preservation, and enhancement of the state’s air, water, land, fish, wildlife, and other natural resources. Currently, 40% of net Minnesota State Lottery proceeds are dedicated to growing the Trust Fund and ensuring future benefits for Minnesota’s environment and natural resources.