A matter of skill
The key to our approach is an assessment of model skill
based on validation data. LCM models the relationship between land cover
transitions and explanatory variables (such as proximity to roads, slopes and
soils) by using examples of areas known to have transitioned in the past as
training areas. In LCM’s MLP neural network procedure, half of the training
data are held back for validation. Thus it learns on the half designated for
training and tests the skill of the model based on the half reserved for
validation. With these validation cases we know what transition the land
actually went through as well as the values of the explanatory variables at
those locations. However, the model was never trained on the validation data.
Thus it is a true test.
Using the validation data, the MLP procedure in LCM now
reports:
- The overall skill of the model (a value that ranges from -1 to +1)
- The skill in predicting each transition (a change from one land cover to another)
- The skill in predicting persistence (cases where the transitions were eligible to happen, but did not)
- The contribution of each explanatory variable to the overall model skill.
- A backwards stepwise model assessment procedure which permits a quick assessment of the most parsimonious model (the model with the greatest power while using the least resources -- the one with the most bang for the buck).
The skill measure used expresses the accuracy of the model, based on the validation data, compared to the expected accuracy that would occur by chance. A skill of 0 indicates that the model has no better than chance agreement with reality while a skill of 1 would indicate a perfect prediction. It should be noted that this is a measure of the skill of the model to predict what happened in the past (i.e., the period over which it trained). Thus, truly, it is a hindcast skill measure rather than a forecast skill. However, to the extent that one can assume that business as usual conditions will persist, it is a reasonable statement of the expected skill of the model in the future.
Try it out!
New to Clark Labs software? Haven’t upgraded to IDRISI Selva yet? Try out a free evaluation copy today.
No comments:
Post a Comment