Quality assessment report

"This report summarizes a formal quality assessment of the National Ceiling and Visibility Analysis product (NCVA), a gridded analysis that evaluates reported ceiling and visibility information for the purpose of improving flight planning process. On behalf of the Federal Aviation Administration's Aviation Weather Research Program, and in support of an Aviation Weather Technology Transfer (AWTT) D4 (operational) decision point, this study was carried out to examine the following: The quality of the NCVA product with respect to a baseline analysis; The effect on the NCVA of utilizing a satellite-based cloud mask; The potential value of NCVA's frequent update-cycle to the flight planning process; NCVA's performance compared to the operational Weather Depiction Analysis, a product specifically referenced in the NCVA Concept of Use. Constrained to the use of METARs for verification, this assessment employs a cross-validation technique to create independence between the observational set and the input set that is utilized by the NCVA algorithm. Cross-validation statistics produced from METAR data that are withheld from the NCVA should not be interpreted as measures of the algorithm's skill at METAR sites, but rather as a measure of performance of the NCVA grid at points away from METAR sites. The NCVA algorithm retains information from all operationally available METARs used as input, never altering data at grid points associated with available METAR reports. In the absence of a secondary gridded product that represents current operational planning information, the analysis team created a proxy baseline product for comparison. This product, referred to in this report as the Nearest Neighbor Analysis (NN-A), simply uses the METAR closest to each evaluation point in determining ceiling, visibility, and flight category values. The study period consists of the summer months of 2008 and the winter months of 2008-09, chosen to provide significant data for stratification of results by season and time of day"--Executive summary (p. viii).