Lab 4: Quality assurance and quality control (QA/QC)

Goals and Background:

This lab consisted of practicing performing synoptic QC measures including producing a point cloud density image, and producing statistics for local areas in the point cloud data using a stamp tool. Also practiced in this lab were relative accuracy assessment procedures, and absolute accuracy assessment procedures. Relative accuracy assessment consisted of creating a delta Z (dZ) ortho image raster and determining swath-to-swath accuracy in non-vegetated terrain as a root-mean-square difference (RMSDz). Absolute accuracy consisted of obtaining non-vegetated vertical accuracy and horizontal accuracy. Finally, manual cleanup was performed. The dataset worked on in the past three labs was the dataset assessed in this lab. 

Quality assurance (QA) and quality control (QC) are two different processes that ensure that overall accuracy goals of a LiDAR data are met or exceeded. Quality assurance consists of care taken with events taking place up to the end of any processing of the dataset. This includes activities taken before a flight mission to ensure quality needed by a client is met, and also all pre and post-processing operations. Certain parameters need to be considered such as the time range on large and small timescales taking into consideration leaf-on and leaf-off conditions and weather events, ground conditions relating to weather, point density, and vertical and horizontal accuracy. Quality control consists of identifying the status of the dataset after all processing. This consists of identifying errors in a dataset and either fixing them or flagging them. It also consists of creating statistics, assessing vertical and horizontal accuracy, and assessing classification accuracy. QC is divided into synoptic, relative, and absolute accuracy.

Methodology:

     Part 1: Synoptic QA/QC

          Section 1: Point cloud density analysis

The previously used Lake LAS Statistics task was copied and named AOI stamp statistics. This new task was used to get statistics using the execute by stamp tool and output those statistics into a shapefile (Figure 1). The statistics could then be read by opening them in ArcMap or using Feature Analyst in LP360. This was used mainly to find point density for specific areas of interest.

Figure 1: Capturing statistics by stamp

A point density image was created to check data coverage and to screen for voids or potential low confidence areas. This was created using the Export Wizard in the stand alone LP360 application for Windows. A surface was exported using only first returns and filtered based on scan angles (min: -13.5, max: 13.5). The Point Insertion method was used with a Cell Edge length that was two times the NPS determined in the Lab 1 statistics shapefile. Density was the attribute that was exported, and the inverse of the cell area was used for the point density value entered (calculated from the previously found cell edge length). Units were set to the unit of the data, feet, and an interval of 4 was used. The result was calculated for the entire LAS dataset area and was brought into LP360 for viewing. The export wizard was opened for a legend to be displayed for the data classification.

         Section 2: Point cloud spatial distribution analysis

Using the raster calculator in the Model Building in ArcCatalog, an output integer image was created with three values: 0: voids, -255: meets specifications, and 255: below specification. The raster calculator equation used was band1 - band2 - band3. a new field was then created for the output raster image which was calculated to achieve the percentage of total pixels (count/total *100). 

Figure 2: Configuring the raster calculator in the ArcCatalog Model Builder

     Part 2: Relative accuracy assessment

          Section 1: Creating and analyzing a Dz ortho image

The LAS files and the NAIP imagery for the area were brought into LP360 and the QA/QC toolbar was enabled. The LAS data was displayed by Dz. Another surface was exported, but this time with all returns, a cell edge length of 10, a Dz image output, a Dz interval size of 0.04, and an interval of 5. Again the point insertion surface method was used, and the total area of the LAS data. 

          Section 2: Swath-to-swath analysis

Two adjacent polylines were created in areas of swath overlap on non-vegetated areas. The areas that were most chosen for polyline creation were flat areas of roads. Theses seamlines were then brought into the seamline analysis tool in LP360. This tool was setup with a sample distance of 5 and a search radius of 1, no-data areas were omitted from outputs, and ground points were chosen as the source points (this process necessitated correctly classified ground points). The process was run, and the resultant data was brought into the viewer.

The residuals (samples) were viewed and were symbolized by quantities as graduated symbols. Five classes were created and the symbols were classified as red, orange, yellow, green, and blue. The red and blue outliers were identified.

Figure 3: Seamline analysis in LP360

     Part 3: Absolute accuracy assessment

          Section 1: Non-vegetated vertical accuracy

A checkpoints file was supplied with checkpoints and was opened in ArcMap with points displayed using the XY coordinates. The vertical points were then exported to a shapefile that was brought into LP360 into the QA/QC toolbar as the Target Control Points file. The points were now viewed in the profile window and map. The control points report dialog was now opened and the source points class was set to ground points. The surface interpolation method was set to TIN and the Z probe location was set to Control XY. The statistics associated with the accuracy of the points could now be viewed.

          Section 2: Horizontal accuracy

A shapefile for the horizontal GCPs was created with the points that were supplied. The shapefile was brought into LP360 just as the vertical accuracy GCPs were. The control points report dialog was then opened. The source points were set to all points, the interpolation method was set to TIN, and the Z probe location was set to control XY. The statistics were then calculated. Because no imagery was supplied with the lab that showed any points on the ground with which to match to the horizontal accuracy, points nearby the GCPs were selected to see how LP360 would generate statistics.

     Part 4: Manual QA/QC of classification errors

A filter was created for QA/QC purposes and a shapefile was also created. The shapefile was then set as the target shapefile in the QA/QC toolbar. Using the jump amount and a standardized map scale the LAS dataset was screened for data voids and errors. Issues were then flagged with the tools in the QA/QC toolbar on the profile. The various selection tools were used on the profile window to edit the point classifications.

Results:

     Part 1: Synoptic QA/QC

          Section 1:

The stamp tool that was used to get local statsistics revealed areas that needed attention. This tool was even more useful after an entire LAS dataset image was created showing areas that needed more attention. The tool was able to be used in small areas of the image where it was obvious that more data was needed. A subset of the image for the entire study area is shown in Figure 4. The entire image can be seen in Figure 5. Areas of swath overlap showed much more comprehensive coverage. The image was classified so that black indicated data voids, oranges indicated less complete coverage in areas, and green indicated the most complete coverage.

Figure 4: NPS raster image assessment

Figure 5: NPS raster image assessment

          Section 2:

The raster image generated with the raster calculator showed that 9% met specifications, and that 9% was below specification. Lack of ground manual cleanup must have resulted in ground points with Z values with highly variable distribution causing the scan angle filter to skip over the points.

     Part 2: Relative accuracy assessment

          Section 1:

The Dz image was viewed and overlapping areas were viewed. There are many areas that are above the standard elevation difference in non-vegetated areas. This could be due to time-dependent features such as cars, or possibly due to data noise or small areal features such as light poles or other features. The error also could be due to calibrations made in pre-processing or calibrations that did not hold up between flight lines.

          Section 2:

Seamline analysis outliers were viewed in ArcMap. The seamline sample outliers were then viewed to see where the source of error could be in every instance.

Figure 6: Seamline analysis outliers

     Part 3: Absolute accuracy assessment

          Part 1: Non-vegetated vertical accuracy

Figure 7: Non-vegetated vertical accuracy

          Part 2: Horizontal accuracy

Because horizontal accuracy was only performed for practice as GCPs were not visible in the imagery supplied, there is no need to see the results.

Sources:

• Lab instruction was provided by Dr. Cyril Wilson
• LAS data was sourced from Lake County on the Illinois Geospatial Data Clearinghouse.
• NAIP imagery is from the USDA Geospatial Data Gateway.