|
How does RisQuant measure efficacy? [Note: This section is undergoing major improvements.]
There are two basic comparisons that we make to ensure model efficacy.
Integrity Test: How well did the models perform, given actual fuel and weather? For each and every modeled location, we compare actual outcomes with modeled outcomes given actual spot month fuel prices and actual weather. We execute this test of model integrity to measure accuracy and bias while controlling for inputs. We further examine whether the experienced model error is consistent with the expected model error.
Forward Test: How well did the models perform in comparison with an established forward market? For locations that have at least one year reported forward, we assess the modeled values compared with the forward values for bias and accuracy. We compare these outcomes at 1, 15, 30, 45 and 60 days prior to delivery. PJM Western Hub, Cinergy Hub, NYJ, NYG are implemented. We examine both the modeled power price and the modeled implied heat rate (power price/natgas price).
Are RisQuant Power Nominals tests conducted 'out of sample'? Modeled values are always estimated strictly 'out of sample'. That is, test data occurred subsequent to the data used to estimate model parameters. If we were to ever report modeled values 'in sample' or 'pre sample', we would very clearly identify them as such along with the reasons for doing so.
Why does Power Nominals examine both power price and implied heat rate in the 'Forward Test'? The highly volatile fuel markets in recent years have driven enormous volatility in the power markets. A forecast error that is due to fuel price movement doesn't say much.
|
Power Nominals 
