Improved diagnostics for NWP verification in the tropics
The root-mean-square error (RMSE) is often used to verify forecasts. However, its strong dependence on the observation variability makes it unsuitable for comparing model performance between regions where observation variability is much different, e.g., across vertical levels or between the midlatit...
Saved in:
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
2012
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/95388 http://hdl.handle.net/10220/8233 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-95388 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-953882020-09-26T21:34:25Z Improved diagnostics for NWP verification in the tropics Koh, Tieh Yong. Ng, Jun Siang. DRNTU::Science::Geology The root-mean-square error (RMSE) is often used to verify forecasts. However, its strong dependence on the observation variability makes it unsuitable for comparing model performance between regions where observation variability is much different, e.g., across vertical levels or between the midlatitudes and the tropics. The alpha index based on the tensor variance of forecast-observation discrepancy was formulated to improve on RMSE (and the closely related bias-corrected RMSE). An “error ellipse” was used to represent the random error in vector wind, yielding two other diagnostics: eccentricity and orientation. These diagnostics were applied to verify Naval Research Laboratory's limited-area model, Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS), for the first time in Southeast Asia. COAMPS forecasts were verified against radiosonde data from South China Sea Monsoon Experiment (SCSMEX), May–June 1998. Results revealed falling model performance as forecast time increases but little difference between forecasts at 18-km and 54-km resolution. Systematic errors in the model dynamics were suggested from the biases. The alpha indices show that (after bias correction) COAMPS performs best for wind, followed by temperature and then by dew point depression. In this tropical region, 1-day persistence forecasts were only outperformed by the model for wind predictions between 400 mb and 850 mb at forecast times less than 24 hr. The RMSE diagnostic was shown to sometimes yield misleading evaluation of the model's performance. The wind error ellipses revealed that the random error tended to align more with the background flow than with the model bias, possibly indicating a dynamical reason for its existence. Published version 2012-06-21T04:30:51Z 2019-12-06T19:13:53Z 2012-06-21T04:30:51Z 2019-12-06T19:13:53Z 2009 2009 Journal Article Koh, T. Y. & Ng, J. S. (2009). Improved Diagnostics for NWP Verification in the Tropics. Journal of Geophysical Research, 114. https://hdl.handle.net/10356/95388 http://hdl.handle.net/10220/8233 10.1029/2008JD011179 en Journal of geophysical research © 2009 American Geophysical Union.This paper was published in Journal of Geophysical Research and is made available as an electronic reprint (preprint) with permission of American Geophysical Union. The paper can be found at the following official URL: [http://dx.doi.org/10.1029/2008JD011179]. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper is prohibited and is subject to penalties under law. 13 p. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
country |
Singapore |
collection |
DR-NTU |
language |
English |
topic |
DRNTU::Science::Geology |
spellingShingle |
DRNTU::Science::Geology Koh, Tieh Yong. Ng, Jun Siang. Improved diagnostics for NWP verification in the tropics |
description |
The root-mean-square error (RMSE) is often used to verify forecasts. However, its strong dependence on the observation variability makes it unsuitable for comparing model performance between regions where observation variability is much different, e.g., across vertical levels or between the midlatitudes and the tropics. The alpha index based on the tensor variance of forecast-observation discrepancy was formulated to improve on RMSE (and the closely related bias-corrected RMSE). An “error ellipse” was used to represent the random error in vector wind, yielding two other diagnostics: eccentricity and orientation. These diagnostics were applied to verify Naval Research Laboratory's limited-area model, Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS), for the first time in Southeast Asia. COAMPS forecasts were verified against radiosonde data from South China Sea Monsoon Experiment (SCSMEX), May–June 1998. Results revealed falling model performance as forecast time increases but little difference between forecasts at 18-km and 54-km resolution. Systematic errors in the model dynamics were suggested from the biases. The alpha indices show that (after bias correction) COAMPS performs best for wind, followed by temperature and then by dew point depression. In this tropical region, 1-day persistence forecasts were only outperformed by the model for wind predictions between 400 mb and 850 mb at forecast times less than 24 hr. The RMSE diagnostic was shown to sometimes yield misleading evaluation of the model's performance. The wind error ellipses revealed that the random error tended to align more with the background flow than with the model bias, possibly indicating a dynamical reason for its existence. |
format |
Article |
author |
Koh, Tieh Yong. Ng, Jun Siang. |
author_facet |
Koh, Tieh Yong. Ng, Jun Siang. |
author_sort |
Koh, Tieh Yong. |
title |
Improved diagnostics for NWP verification in the tropics |
title_short |
Improved diagnostics for NWP verification in the tropics |
title_full |
Improved diagnostics for NWP verification in the tropics |
title_fullStr |
Improved diagnostics for NWP verification in the tropics |
title_full_unstemmed |
Improved diagnostics for NWP verification in the tropics |
title_sort |
improved diagnostics for nwp verification in the tropics |
publishDate |
2012 |
url |
https://hdl.handle.net/10356/95388 http://hdl.handle.net/10220/8233 |
_version_ |
1681058662394626048 |