Improved Doppler Radar/Satellite Data Assimilation
Improved Doppler Radar/Satellite Data Assimilation
This project’s goal is to develop a short-term high-resolution data assimilation capability that can provide the Navy with improved analyses and forecasts of atmospheric conditions with sufficient detail and accuracy for supporting the Navy mission in threat detection, weapons deployment, and weather safe operations. The data assimilation system will utilize all available weather data, such as Doppler radar, in situ, and remotely sensed observations. The system will run efficiently and generate a detailed analysis of the atmosphere with sufficient accuracy to predict target area weather conditions. This information can then be fed back to weapon system operators to improve detection and strike capabilities.
The objective of this research is to build an advanced high-resolution data assimilation system for the Navy Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS®) for on-demand use in COAMPS-OS® and at the same time, investigate the impact of high-resolution data assimilation on short-term mesoscale numerical weather prediction. This data assimilation scheme will be able to analyze mesoscale and storm-scale weather by applying sophisticated analysis procedures capable of ingesting the information from Doppler radar, satellite, and other remote sensors. The primary focus of this effort will be to design a system that optimally utilizes the available weather data such as DoD Doppler radar data for initializing COAMPS.
The Naval Research Laboratory (NRL), through collaborations with universities and other government agencies, is developing a high-resolution data assimilation system. The system takes advantages of the full resolution, full volume, and frequently updated observations from Doppler radars that provide detailed three-dimensional dynamical and microphysical structures inside storms. The radar data processing, quality control, and compression algorithms recently developed at NRL are used for data acquisition, processing and quality assurance for assimilation. These algorithms have been extensively tested at NRL with radar observations from the nation’s Weather Surveillance Radar-1988 Doppler (WSR-88D) and the DoD Doppler radars.
The high-resolution data assimilation system includes a variational approach for Doppler radial velocity and reflectivity assimilation that uses the background fields provided by atmospheric predictions from COAMPS at non-synoptic times and/or analyses from the NRL operational Atmospheric Variational Data Assimilation System (NAVDAS) at synoptic times. The variational approach with a simplified adjoint method is used to achieve the high computational efficiency needed to assimilate high-resolution non-conventional sensor data from Doppler radars (including DoD radars on ships and at forward-deployed locations). An innovative approach has also been proposed to provide the radar assimilation with flow-dependent background error covariance estimated from COAMPS ensemble forecasts with an ensemble Kalman filter (EnKF) recently developed at NRL. The assimilation time window is synchronized with COAMPS integration time steps and radar volumetric scans to enhance the coupling of the model with the data. To compliment the radar assimilation system, the cloud information from high-resolution geostationary satellite observations (such as the infrared and visible imagery), and surface cloud observations is also fused and assimilated to enhance the cloud analyses in the model initial fields.
Scientists involved in this project include Dr. Allen Zhao, the PI who works on radar data assimilation, Dr. Paul Harasti, a UCAR visiting scientist at NRL working on radar data processing, quality control, and nowcasting, Dr. David Pan, a professor of the University of Alabama and an ONR Summer Faculty Researcher at NRL who works on UF radar data compression, and Mr. Dan Geiszler from the Science Applications International Corporation, who provides COAMPS supports on radar data assimilation. Collaborators include those from National Severe Storms Laboratory (Dr. Qin Xu), The Space and Naval Warfare Systems Center (Dr. Lee J. Wagner), Penn State University (Dr. Fuqing Zhang), the University of Oklahoma (Dr, Ming Xue), and the University of Utah (Dr. Zhaoxia Pu).
During the fiscal year 2009, research and development efforts were continued to enhance NRL capability in acquisition, processing, quality control, and assimilation of real-time DoD shipboard radar data into COAMPS for nowcasting and forecasting high-impact weather events that affect Navy operations over oceans. In addition, research resulting in improved impacts of assimilated data on hazardous weather prediction was also accomplished by developing an ensemble Kalman Filter (EnKF) data assimilation system that increase the system’s ability to deal with strong non-linearity at meso- and storm-scales and in estimating flow-dependent background error covariance.
1. The improved NRL capability in acquisition, processing, quality control and assimilation of the real-time SPS-48E shipboard radar data from the previous year was further advanced to a new level. The Universal Format Data Compressor (UFZIP), an innovative technique developed at NRL in 2008 that compresses the raw (level-II) SPS-48E Doppler radar universal format data and allows the transfer of near real-time radar data from ships to Fleet Numerical Meteorology and Oceanography Center (FNMOC) for assimilation into COAMPS with limited bandwidth, was further improved, finalized and transitioned to the Space and Naval Warfare Systems Command (SPAWAR) Hazardous Weather Detection and Display Capability (HWDDC) for operational implementation. The UFZIP technique was also extended and standardized for applying to other DoD Doppler radars such as SPY-1 and the Supplemental Weather Radar (SWR).
2. Studies were also conducted to estimate the storm-derived winds from the radar reflectivity observations from SPS-48E with the use of Tracking Radar Echoes by Correlation (TREC) technique to provide wind nowcasting for the battle ships. TREC was extensively tested with the [MISSING FROM ORIGINAL]
The most significant accomplishment of this project in the past year was the development of the Ensemble Kalman Filter (EnKF) at NRL. This effort adds a new component and capability to the NRL data assimilation systems to estimate flow-dependent background error covariance and to provide initial conditions for the COAMPS ensemble forecasts. The EnKF has been extensively tested with both simulated and real observations of conventional and satellite data. An adaptive inflation algorithm has been developed to ensure appropriate ensemble spreads that cover all model forecast errors. A case study was also conducted to compare the EnKF analyses with those from the current operational Naval Atmospheric Variational Data Assimilation System (NAVDAS) 3D-Var. The assimilation capability of radar observations of reflectivity and Doppler radial velocity from both fixed and moving platforms is to be added to the EnKF.
1. Compression of UF data files with the improved and standardized UFZIP
The high-resolution data assimilation system developed at NRL takes advantage of Navy meteorological Doppler radars forward deployed and tactical Doppler radars with meteorological data capabilities onboard US Navy battle ships. These radars generate UF data files with sizes ranging from ~5 MB for SPS-48E, ~13 MB for SPY-1 to ~17 MB for SWR. To minimize the load on the operational bandwidth, the UF files have to be significantly compressed before being transmitted to the Fleet Numerical Meteorology and Oceanography Center (FNMOC) for data assimilation, requiring a high compression rate that even the state-of-the-art data compressors cannot provide. To overcome this challenge, several innovative techniques were developed at NRL and integrated into UFZIP to losslessly compress the SPS-48E UF data. In the past year, several additional improvements were implemented to UFZIP to further enhance the compression ratio. UFZIP was also standardized to extend its applications to the UF data files from other DoD Doppler radars. Table 1 shows the most recent test results with SPS-48E and SWR UF data files.
Table 1. Test results of the improved and standardized UFZIP.
2. Storm-derived winds from TREC
The Marine Meteorology Division of the NRL is working with Basic Commerce and Industries and SPAWAR to implement TREC as a wind nowcasting tool within the HWDDC. A new version of TREC has been developed, specifically designed for use by the HWDDC at sea. Allowances have been made for ship movement, mast reflection artifacts, ground and sea clutter, and an improved wind smoothing technique has also been developed. The HWDDC creates base reflectivity data in NEXRAD NIDS format at 60 second intervals, while the adjunct TREC algorithm processes these data every 5 minutes to allow sufficient time to resolve reflectivity echo movement between analysis grid points. The HWDDC displays the TREC winds as conventional wind barbs superimposed upon the imagery of the most recent reflectivity data, which ship personnel will use along with other data to enhance the near-term ship and flight course planning, operations and safety.
Figure 1 shows examples, before and after quality control, of TREC applied to NIDS reflectivity data files created at 1527 UTC and 1532 UTC from the 22 February 2006 when the USS PELELIU encountered lines of showers and thunderstorms associated with a low pressure center and trough located ~300 km north of Hawaii. The maximum slant range of reflectivity data shown is 230 km, which corresponds to an altitude of ~4 km ASL. Prior to quality control, note the false TREC winds embedded within 1) the large annulus of sea clutter within 22 km of the ship located at the center of the image, and 2) the ground clutter from Maui. The TREC quality control removed all the false winds. Moreover the cyclonic flow in the TREC wind field near the top of the image seems to successfully capture the cyclonic flow around the low pressure center observed north of Hawaii.
Figure 1. NIDS reflectivity data with TREC winds superimposed as conventional wind barbs for the 1532 UTC 22 February 2006 analysis time, before (left) and after (right) quality control.
3. Development of NRL ensemble data assimilation system
The EnKF recently developed at NRL is under extensive test. One of the important parameters we tested was the ensemble spread that measures the effectiveness of the EnKF in accounting for the uncertainties in the mode forecasts during the data assimilation. According to previous studies, the ratio, R, of the actual ensemble spread to the optimal ensemble spread should be close to 1.0 in order to ensure that the analyses from the EnKF have appropriate ensemble spreads that cover all model forecast errors. We tested the ensemble spreads for different types of observations and the results were given in Table 2. What we found during the test was that the ensemble spreads were scaled back too much during the data assimilation for most types of observations. To increase the ensemble spread, an adaptive inflation algorithm has been developed and implemented into the EnKF. The EnKF with the adaptive inflation algorithm was tested again with the same types of observations and the results are given in the last row of Table 2. As you can see, the ensemble spreads have been increased for all the observation types and the values of R with the inflation are closer to 1.0 than those from the EnKF without inflation.
Table 2. Values of the ratio of actual ensemble spread to the optimal ensemble spread without and with adaptive inflation for different types of observations.
One case study was also conducted to compare the analyses between the EnKF and the current operational NAVDAS (a 3D-Var system) for COAMPS. In the ensemble analyses, 32 members of COAMPS ensemble forecasts were used. The analyses of the ensemble mean and those from the NAVDAS were verified against all sounding observations over US. The domain-averaged analysis errors (analyses – observations) for u, v and T are given in figure 2 and the root-mean-square (RMS) errors are shown in figure 3. For this particular case, the EnKF over performed the NAVDAS, indicating the potential to improve analyses with ensemble-based data assimilation. However, one should keep in mind that this is just one case study. The EnKF still needs some enhancements. More observations, especially those from satellite and Doppler radars, will be added to the EnKF. More comprehensive studies will be carried out to demonstrate the advantages of the EnKF data assimilation system at meso-and storm-scales and with non-conventional sensor data.
The high-resolution data assimilation system developed at NRL for both conventional meteorological observations and high-resolution sensor data from radars, satellites and other remote sensors for the COAMPS model will provide the Navy with near real-time, three-dimensional cloud and wind analyses and short-term (0-48 hours) theater-scale weather forecasts in any region of interest to support the Navy’s mission. The technology was demonstrated during Fleet Battle Experiment – Juliet with products providing up-to-date, detailed information to tactical decision makers about the three-dimensional atmospheric battlespace conditions. The high-resolution winds from both the data assimilation system and the COAMPS model forecast are also used to drive chemical/biological (CB) dispersion models, which are used for assessing contamination avoidance and decontamination strategies. While focusing on battlespace environmental applications, this work also establishes a scientific framework for utilizing radar-derived meteorological information in nowcasting and numerical weather prediction applications.
Figure 2. Domain-averaged analysis errors of temperature T (K) and horizontal winds u and v (ms-1) as a function of pressure (mb) from EnKF and NAVDAS verified against sounding observations over US valid at 00UTC 23 June 2006.
Figure 3. Same as figure 2 except for RMS errors.
1. UFZIP/ UFUNZIP (Version 1.0): Compresses and uncompresses UF files. Transitioned to SPAWAR/HWDDC, 19 June 2009.
2. TREC (Version 1.0): Creates U and V wind estimates from 5-minute reflectivity data, which HWDDC superimposes on latest, lowest tilt, reflectivity data. Transitioned to SPAWAR/HWDDC, 19 June 2009.
Related NRL base projects include 6.2 tasks BE-435-047, Advanced Assimilation of Non-conventional Data for Improved High-Impact Weather Prediction, and BE-435-003, Probabilistic Prediction of High Impact Weather. Other related projects include Radar Data Quality Control and Assimilation At the National Weather Radar Testbed (NWRT) (ONR, N000140410312), 6.4 Reach-Back Doppler Radar Data Assimilation (PMW-120, X2341) and 6.4 On-Scene Tactical Atmospheric Forecast Capability (PMW-120, X2342).
Xu, Q., K. Nai, L. Wei, and Q. Zhao, 2009: An Unconventional Approach for Assimilating Aliased Radar Radial Velocities. Tellus A [in press, refereed]
Pan, W. D., P. R. Harasti, M. Frost, Q. Zhao, J. Cook, T. Maese, L. J. Wagner, 2009: Efficient Reduction and Compression of Weather Radar Data in Universal Format. Extended abstract 11A.1, 25th Conference on International Interactive Information and Processing Systems (IIPS’09) for Meteorology, Oceanography, and Hydrology, 9 pp, Phoenix, AZ, 10-16 January 2009, http://ams.confex.com/ams/89annual/techprogram/paper_144487.htm [publis
Access original "Improved Doppler Radar/Satellite Data Assimilation" document here.