CAUSAL MODELLING USING RECURSIVE BAYESIAN ESTIMATION AND KALMAN FILTERING ON DATA OF VARIABLE PRECISION Alan Solomon School of Management Thesis submitted for Ph.D. Cranfield Institute of Technology School of Management Thesis submitted for Ph.D. Alan Solomon CAUSAL MODELLING USING RECURSIVE BAYESIAN ESTIMATION AND KALMAN FILTERING ON DATA OF VARIABLE PRECISION Supervised by Dr. F. Fishwick Presented 1st November 1983 Dedication To Angela, who set my deadline and to Jennifer, who did her best to make me miss it. Thanks are due to : The International Wool Secretariat and British Petroleum, who provided computer time, data and money. Dr. Fishwick, who encouraged and advised me. Librarians all over the country, who chased up my references. Typists, who struggled with my handwriting and my algebra. Elizabeth Sheffer, who will never know what she started. Most of all, Susan, who did everything I should have been doing. ABSTRACT Five problems are described; they are concerned with the treatment of preliminary data, imprecise data, prior information, old data and residuals. Conventional solutions to these problems are described. From Bayes theorem, an estimation technique called recursive Bayesian estimation is examined, and it is shown how the five problems can be treated. It is shown that recursive Bayesian estimation is, with certain restrictions, equivalent to simple regression, but more general. This technique is used to estimate a wool consumption model, and it is shown that the model fits better than a model estimated using ordinary least squares or weighted least squares. Recursive Bayesian estimation is then generalized to cope with multiple explanatory variables. The Kalman filter is introduced, and its relationship to recursive Bayesian estimation is demonstrated. The meaning of the various matrices used by the Kalman filter is explained. The historic development of Kalman filtering is reviewed, and various papers that use the Kalman filter to estimate a causal model are examined. A comparison is made between Kalman filter estimated models and ordinary least squares estimated models, using synthetic data; the Kalman filter models are better, but not dramatically so. A model of energy demand is estimated using ordinary least squares and the Kalman filter. The Kalman filter is used with various different values of the matrix governing the rate at which data become irrelevant. The result is a model which is a rather better fit in the residential sector, but a slightly worse fit in the industrial sector. A model of wool demand is estimated, in which many of the features offered by the Kalman filter are used. The model is a simultaneous equation system, with parameter restrictions across the equations. Variable data precision is incorporated, treating pre-1970 data, post-1970 data and preliminary data as having different precisions. Various ways of describing the rate at which data become irrelevant are tried, and prior information is incorporated. The model is a much better fit than a model estimated using ordinary least squares. TABLE OF CONTENTS 1. INTRODUCTION 1.1 The five problems to be treated 1.1.1 Preliminary data 1.1.2 Imprecise data 1.1.3 Incorporation of other information 1.1.4 The irrelevance of old data 1.1.5 Residuals 1.2 Conventional solutions to the five problems 1.2.1 Preliminary data 1.2.2 Imprecise data 1.2.3 Incorporation of other information 1.2.4 The irrelevance of old data 1.2.5 Residuals 1.3 The five problems together 1.4 Coping with uncertainty 1.5 The importance of causal models 1.6 A guide to this thesis 2. BAYES THEOREM AND ITS IMPLICATIONS 2.1 Bayes Theorem 2.2 The Bayes/frequentist controversy 3. RECURSIVE BAYESIAN ESTIMATION; ONE DEPENDENT AND ONE EXPLANATORY VARIABLE. 3.1 A simple model 3.2 The recursive Bayesian estimation system 3.3 Recursive least squares estimation; one dependent and one explanatory variable 3.4 The relationship between recursive Bayesian estimation and recursive least squares 3.5 Forecasting 3.6 How the five problems can be treated 3.6.1 Preliminary data 3.6.2 Imprecise data 3.6.3 Incorporation of other information 3.6.4 The irrelevance of old data 3.6.5 Residuals 3.7 Non-normal distributions 4. RECURSIVE BAYESIAN ESTIMATION, ONE DEPENDENT AND SEVERAL EXPLANATORY VARIABLES 4.1 The need for more variables 4.2 The matrix inversion lemma 4.3 The recursive estimation procedure 4.4 Equivalence of this result to recursive Bayesian estimation for one independent variable 4.5 Relaxation of assumptions about parameters 5. RECURSIVE BAYESIAN ESTIMATION, ONE DEPENDENT AND ONE EXPLANATORY VARIABLE; SOME RESULTS 5.1 The measurement of the fit of a model - Dynamic sum of squared errors 5.2 The model 5.3 The model estimations 5.4 The results 5.4.1 The model fit 5.4.2 The price elasticity 5.5 Discussion of results 6. RECURSIVE BAYESIAN ESTIMATION, SEVERAL DEPENDENT AND SEVERAL EXPLANATORY VARIABLES 6.1 The Kalman filter 6.2 An explanation of how the Kalman filter works 6.3 The relationship of the Kalman filter to recursive Bayesian estimation and ordinary least squares 6.4 Forecasting 6.5 The relevance of V and W 6.5.1 V 6.5.2 W 6.6 The relevance of H 6.7 The relevance of G 6.7.1 G adjusts the parameter estimates 6.7.2 G adjusts the precision attached to the parameter estimates 6.8 Other benefits of the Kalman filter 6.8.1 Multicollinearity 6.8.2 Computational efficiency 7. PAST APPLICATIONS OF THE KALMAN FILTER 7.1 Historic development of Kalman filtering 7.1.1 Filtering 7.1.2 Least squares 7.1.3 Time series forecasting 7.1.4 Econometrics 7.2 Harrison and Stevens, 1976 7.3 Borooah and Chakravarty, 1978 7.4 Meade, 1979 7.5 Johnston and Harrison, 1980 7.6 Hughes, 1980 7.7 Burmeister and Wall, 1982 7.8 McWhorter, Narasimham and Simonds, 1977 7.9 Mitchell, 1982 7.10 McNelis and Neftii, 1982 7.11 Aoki, 1982 7.12 Kure, 1983 7.13 Fildes, 1983 and Fildes, 1982 7.14 Makridakis et al., 1982 7.15 Harvey, 1983 7.16 McWhorter et al., 1976 7.17 Summary 8. THE BENEFITS OF KALMAN FILTER MODELS COMPARED TO CONVENTIONAL METHODS; A MONTE CARLO STUDY 8.1 The model 8.2 The results 8.3 Conclusions 9. A KALMAN FILTER ESTIMATED MODEL OF ENERGY DEMAND; ONE DEPENDENT AND MULTIPLE EXPLANATORY VARIABLES 9.1 The model 9.2 The data 9.3 Estimating the model with OLS 9.4 Estimating the model with the Kalman filter 9.4.1 W = 0 9.4.2 W =/ 0 9.4.2.1 W = 10-6 9.4.2.2 W = 10-5 9.4.2.3 W = 10-4 9.4.2.4 Conclusions from W /= 0 9.5 Conclusions about the model parameters and the standard errors attached to them 9.6 Conclusions about the Kalman filter estimation 10. A KALMAN FILTER ESTIMATED MODEL OF WOOL DEMAND; MULTIPLE DEPENDENT AND MULTIPLE EXPLANATORY VARIABLES 10.1 The model 10.2 The data and data precision 10.3 The estimates of Solomon, 1980 10.4 The model with W = 0 10.4.1 V set to post-1970 values 10.4.2 V set to values for final data 10.4.3 V set to values for final and preliminary data 10.5 The model with W=/0 10.5.1 W = 10-7, 10-6, 10-5, 10-4 10.5.2 W = All 10-3 10.5.2.1 Some parameters have drifted 10.5.2.2 The standard errors are larger 10.5.2.3 The income elasticities are more stable than the price elasticities 10.5.2.4 The forecasts are good 10.5.2.5 The parameters are still good 10.5.3 W = 10-3, 10-3, 0, 10-3, 10-3 10.5.4 W = 10-3, 10-3, non-zero, 10-3, 10-3 10.6 The model with non-constant W 10.6.1 U = 0.5, V set to values for post-1970 data 10.6.2 U = 0.5, V set to values for final and preliminary data 10.6.3 U = 0.05, V set to values for post-1970 data 10.6.4 U = 0.05, V set to values for final and preliminary data 10.6.5 U = 0.5, 0.05, 0.05, V set to values for post-1970 data 10.6.6 U = 0.5, 0.05, 0.05, V set to values for final and preliminary data 10.6.7 W = 10-3, 10-3, 10-3; U = .05, .05 10.6.8 W = 10-3 except for one parameter; U = .05 10.6.8.1 W = 10-3 except for income elasticity 10.6.8.2 W = 10-3 except for wool price elasticity 10.6.8.3 W = 10-3 except for synthetic fibre price elasticity 10.6.9 W = 10-3 except for the parameter D; U = 0.05 or 5.0 10.6.9.1 U = 0.05 10.6.9.2 U = 5.0 10.6.10 W = 10-3 except in 1973, when W = 3*10-3 10.7 Conclusions from the estimations without prior information 10.8 Using prior information 10.8.1 The prior information, case 1 10.8.1.1 Prior data case 1, W = 10-3 10.8.1.2 Prior data case 1, W = 10-2 10.8.1.3 Prior data case 1, W = 10-4 10.8.1.4 Prior data case 1, W = 0 10.8.2 The prior information, case 2 10.8.2.1 Prior data case 2, W = 10-3 10.8.2.2 Prior data case 2, W = 10-2 10.8.2.3 Prior data case 2, W = 10-4 10.8.3 The prior information, case 3 10.8.3.1 Prior data case 3, W = 10-3 10.8.4 The prior information, case 4 10.8.4.1 Prior data case 4, W = 10-3 10.8.4.2 Prior data case 4, W = 10-2 10.8.4.3 Prior data case 4, W = 10-4 10.8.5 The prior information, case 5 10.8.5.1 Prior data case 5, W = 10-3 10.8.5.2 Prior data case 5, W = 10-2 10.8.5.3 Prior data case 5, W = 10-4 10.9 Conclusions from the estimations using prior information 10.10 Conclusions 11. IDEAS FOR FURTHER WORK 11.1 Using the Kalman filter program 11.2 Systematically varying parameters 11.3 Forecasting 11.4 Forecasting with uncertainty in the explanatory variables 11.5 Estimation with uncertainty in the explanatory variables 11.6 Estimating V 11.7 Estimating W 11.8 Non-linear models 11.9 Non-normal errors 11.10 Backward Kalman filtering 11.11 Decision making 11.12 Incorporation of the Kalman filter into packages 11 13 Redefinition of DSSE 12. SUMMARY, DISCUSSION, MAIN CONCLUSIONS AND RECOMMENDATIONS 12.1 Summary 12.2 Discussion 12.3 Main conclusions 12.4 Recommendations Appendix A Abbreviations Appendix B Notation Appendix C The multiple dynamic weighted least squares program Appendix D The Kalman filter program (as used in chapter 9 and chapter 10) Appendix E The Kalman filter Appendix F The data used Appendix G The recursive Bayesian estimation programs on the Texas Instruments programmable calculator Appendix H References LIST OF FIGURES Figure 1 page 18 Figure 2 page 90 Figure 3 page 115