CAUSAL MODELLING USING RECURSIVE BAYESIAN ESTIMATION AND
KALMAN FILTERING ON DATA OF VARIABLE PRECISION




Alan Solomon

School of Management

Thesis submitted for Ph.D.


Cranfield Institute of Technology

School of Management

Thesis submitted for Ph.D.

Alan Solomon







                             
CAUSAL MODELLING USING RECURSIVE BAYESIAN ESTIMATION AND



                             
KALMAN FILTERING ON DATA OF VARIABLE PRECISION











Supervised by Dr. F. Fishwick

Presented 1st November 1983











Dedication


To Angela, who set my deadline
and to Jennifer, who did her best to make me miss it.









Thanks are due to :
       The International Wool Secretariat and British 
       Petroleum, who provided computer time, data and 
       money.
       Dr. Fishwick, who encouraged and advised me.
       Librarians all over the country, who chased up
       my references.
       Typists, who struggled with my handwriting and 
       my algebra.
       Elizabeth Sheffer, who will never know what she 
       started.
       Most of all, Susan, who did everything I should 
       have been doing.



                         ABSTRACT



Five problems are described;  they are concerned with the 
treatment of preliminary data, imprecise data, prior 
information, old data and residuals.  Conventional 
solutions to these problems are described. 

From Bayes theorem, an estimation technique called 
recursive Bayesian estimation is examined, and it is 
shown how the five problems can be treated. It is shown 
that recursive Bayesian estimation is, with certain 
restrictions, equivalent to simple regression, but more 
general. 

This technique is used to estimate a wool consumption 
model, and it is shown that the model fits better than a 
model estimated using ordinary least squares or weighted 
least squares. Recursive Bayesian estimation is then 
generalized to cope with multiple explanatory variables. 

The Kalman filter is introduced, and its relationship to 
recursive Bayesian estimation is demonstrated.  The 
meaning of the various matrices used by the Kalman filter 
is explained. The historic development of Kalman 
filtering is reviewed, and various papers that use the 
Kalman filter to estimate a causal model are examined. 

A comparison is made between Kalman filter estimated 
models and ordinary least squares estimated models, using 
synthetic data;  the Kalman filter models are better, but 
not dramatically so.  

A model of energy demand is estimated using ordinary 


least squares and the Kalman filter.  The Kalman filter 
is used with various different values of the matrix 
governing the rate at which data become irrelevant.  The 
result is a model which is a rather better fit in the 
residential sector, but a slightly worse fit in the 
industrial sector. 

A model of wool demand is estimated, in which many of the 
features offered by the Kalman filter are used.  The 
model is a simultaneous equation system, with parameter 
restrictions across the equations.  Variable data 
precision is incorporated, treating pre-1970 data, 
post-1970 data and preliminary data as having different 
precisions.  Various ways of describing the rate at which 
data become irrelevant are tried, and prior information 
is incorporated.  The model is a much better fit than a 
model estimated using ordinary least squares.       



                    TABLE OF CONTENTS


1.  INTRODUCTION 
    1.1  The five problems to be treated 
         1.1.1 Preliminary data 
         1.1.2 Imprecise data 
         1.1.3 Incorporation of other information 
         1.1.4 The irrelevance of old data 
         1.1.5 Residuals 
    1.2  Conventional solutions to the five problems 
         1.2.1 Preliminary data 
         1.2.2 Imprecise data 
         1.2.3 Incorporation of other information 
         1.2.4 The irrelevance of old data 
         1.2.5 Residuals 
    1.3  The five problems together 
    1.4  Coping with uncertainty 
    1.5  The importance of causal models 
    1.6  A guide to this thesis

2.  BAYES THEOREM AND ITS IMPLICATIONS 
    2.1  Bayes Theorem 
    2.2  The Bayes/frequentist controversy 



3.  RECURSIVE BAYESIAN ESTIMATION; ONE DEPENDENT AND ONE 
    EXPLANATORY VARIABLE. 
    3.1  A simple model 
    3.2  The recursive Bayesian estimation system 
    3.3  Recursive least squares estimation; one          
         dependent and one explanatory variable 
    3.4  The relationship between recursive Bayesian          
         estimation and recursive least squares 
    3.5  Forecasting 
    3.6  How the five problems can be treated 
         3.6.1 Preliminary data 
         3.6.2 Imprecise data 
         3.6.3 Incorporation of other information 
         3.6.4 The irrelevance of old data 
         3.6.5 Residuals
    3.7  Non-normal distributions 

4.  RECURSIVE BAYESIAN ESTIMATION, ONE DEPENDENT AND
    SEVERAL EXPLANATORY VARIABLES 
    4.1  The need for more variables 
    4.2  The matrix inversion lemma 
    4.3  The recursive estimation procedure 
    4.4  Equivalence of this result to recursive          
         Bayesian estimation for one independent variable 
    4.5  Relaxation of assumptions about parameters 

5.  RECURSIVE BAYESIAN ESTIMATION, ONE DEPENDENT AND ONE 
    EXPLANATORY VARIABLE; SOME RESULTS 
    5.1  The measurement of the fit of a model - Dynamic
         sum of squared errors
    5.2  The model 
    5.3  The model estimations 
    5.4  The results 
         5.4.1  The model fit 
         5.4.2  The price elasticity 
    5.5  Discussion of results 



6.  RECURSIVE BAYESIAN ESTIMATION, SEVERAL DEPENDENT AND 
    SEVERAL EXPLANATORY VARIABLES 
    6.1  The Kalman filter 
    6.2  An explanation of how the Kalman filter works 
    6.3  The relationship of the Kalman filter to          
         recursive Bayesian estimation and ordinary least 
         squares 
    6.4  Forecasting 
    6.5  The relevance of V and W 
         6.5.1  V 
         6.5.2  W 
    6.6  The relevance of H 
    6.7  The relevance of G 
         6.7.1  G adjusts the parameter estimates 
         6.7.2  G adjusts the precision attached to the 	 
       	        parameter estimates 
    6.8  Other benefits of the Kalman filter
         6.8.1  Multicollinearity
         6.8.2  Computational efficiency



7. PAST APPLICATIONS OF THE KALMAN FILTER 
    7.1  Historic development of Kalman filtering 
         7.1.1  Filtering 
         7.1.2  Least squares 
         7.1.3  Time series forecasting 
         7.1.4  Econometrics 
    7.2  Harrison and Stevens, 1976 
    7.3  Borooah and Chakravarty, 1978 
    7.4  Meade, 1979 
    7.5  Johnston and Harrison, 1980 
    7.6  Hughes, 1980 
    7.7  Burmeister and Wall, 1982
    7.8  McWhorter, Narasimham and Simonds, 1977
    7.9  Mitchell, 1982
    7.10 McNelis and Neftii, 1982
    7.11 Aoki, 1982
    7.12 Kure, 1983
    7.13 Fildes, 1983 and Fildes, 1982
    7.14 Makridakis et al., 1982
    7.15 Harvey, 1983
    7.16 McWhorter et al., 1976
    7.17 Summary

8.  THE BENEFITS OF KALMAN FILTER MODELS COMPARED TO         
CONVENTIONAL METHODS; A MONTE CARLO STUDY 
    8.1  The model 
    8.2  The results 
    8.3  Conclusions 

9.  A KALMAN FILTER ESTIMATED MODEL OF ENERGY DEMAND; ONE         
DEPENDENT AND MULTIPLE EXPLANATORY VARIABLES 
    9.1  The model
    9.2  The data
    9.3  Estimating the model with OLS
    9.4  Estimating the model with the Kalman filter
         9.4.1  W = 0
         9.4.2  W =/ 0


       	        9.4.2.1  W = 10-6
       	        9.4.2.2  W = 10-5
       	        9.4.2.3  W = 10-4
       	        9.4.2.4  Conclusions from W /= 0
    9.5  Conclusions about the model parameters        	
         and the standard errors attached to them
    9.6  Conclusions about the Kalman filter estimation

10. A KALMAN FILTER ESTIMATED MODEL OF WOOL DEMAND;         
MULTIPLE DEPENDENT AND MULTIPLE EXPLANATORY VARIABLES 
    10.1  The model
    10.2  The data and data precision
    10.3  The estimates of Solomon, 1980
    10.4  The model with W = 0
          10.4.1  V set to post-1970 values
          10.4.2  V set to values for final data
          10.4.3  V set to values for final and        	   
       	          preliminary data
    10.5  The model with W=/0
          10.5.1  W = 10-7, 10-6, 10-5, 10-4
          10.5.2  W = All 10-3
       	          10.5.2.1  Some parameters have drifted
       	          10.5.2.2  The standard errors are        	
       	       	            larger
       	          10.5.2.3  The income elasticities are
       	       	            more stable than the price
       	       	            elasticities
       	          10.5.2.4  The forecasts are good
       	          10.5.2.5  The parameters are still good
          10.5.3  W = 10-3, 10-3, 0, 10-3, 10-3
          10.5.4  W = 10-3, 10-3, non-zero, 10-3, 10-3
    10.6  The model with non-constant W
          10.6.1  U = 0.5, V set to values for post-1970 
       	          data
          10.6.2  U = 0.5, V set to values for final and 
       	          preliminary data


          10.6.3  U = 0.05, V set to values for post-1970 
       	          data
          10.6.4  U = 0.05, V set to values for final and 
       	          preliminary data
          10.6.5  U = 0.5, 0.05, 0.05,  V set to values 	 
       	          for post-1970 data
          10.6.6  U = 0.5, 0.05, 0.05,  V set to values 	 
       	          for final and preliminary data
          10.6.7  W = 10-3, 10-3, 10-3; U = .05, .05
          10.6.8  W = 10-3 except for one parameter;       
       	          U = .05
       	          10.6.8.1  W = 10-3 except for income
       	       	            elasticity
       	          10.6.8.2  W = 10-3 except for wool        	
       	       	            price elasticity
       	          10.6.8.3  W = 10-3 except for synthetic
       	       	            fibre price elasticity
          10.6.9  W = 10-3 except for the parameter D;
       	          U = 0.05 or 5.0
       	          10.6.9.1  U = 0.05
       	          10.6.9.2  U = 5.0
          10.6.10 W = 10-3 except in 1973, when W =
       	          3*10-3
    10.7  Conclusions from the estimations without prior 
          information
    10.8  Using prior information
          10.8.1  The prior information, case 1
       	          10.8.1.1  Prior data case 1, W = 10-3
       	          10.8.1.2  Prior data case 1, W = 10-2
       	          10.8.1.3  Prior data case 1, W = 10-4	
       	          10.8.1.4  Prior data case 1, W = 0
          10.8.2  The prior information, case 2
       	          10.8.2.1  Prior data case 2, W = 10-3
       	          10.8.2.2  Prior data case 2, W = 10-2
       	          10.8.2.3  Prior data case 2, W = 10-4	
          10.8.3  The prior information, case 3


       	          10.8.3.1  Prior data case 3, W = 10-3
          10.8.4  The prior information, case 4
       	          10.8.4.1  Prior data case 4, W = 10-3
       	          10.8.4.2  Prior data case 4, W = 10-2
       	          10.8.4.3  Prior data case 4, W = 10-4	
          10.8.5  The prior information, case 5
       	          10.8.5.1  Prior data case 5, W = 10-3
       	          10.8.5.2  Prior data case 5, W = 10-2
       	          10.8.5.3  Prior data case 5, W = 10-4	
    10.9  Conclusions from the estimations using prior 
          information
    10.10 Conclusions

11. IDEAS FOR FURTHER WORK 
    11.1  Using the Kalman filter program
    11.2  Systematically varying parameters
    11.3  Forecasting
    11.4  Forecasting with uncertainty in the explanatory
          variables
    11.5  Estimation with uncertainty in the explanatory
          variables
    11.6  Estimating V
    11.7  Estimating W
    11.8  Non-linear models
    11.9  Non-normal errors
    11.10 Backward Kalman filtering
    11.11 Decision making
    11.12 Incorporation of the Kalman filter into 
          packages    
    11 13 Redefinition of DSSE

12. SUMMARY, DISCUSSION, MAIN CONCLUSIONS AND 
RECOMMENDATIONS 
    12.1  Summary
    12.2  Discussion
    12.3  Main conclusions
    12.4  Recommendations


Appendix A   Abbreviations 
Appendix B   Notation 
Appendix C   The multiple dynamic weighted least squares 
             program
Appendix D   The Kalman filter program (as used in              
             chapter 9 and chapter 10)
Appendix E   The Kalman filter 
Appendix F   The data used
Appendix G   The recursive Bayesian estimation programs 	 
             on the Texas Instruments programmable              
             calculator 
Appendix H   References 




LIST OF FIGURES

Figure 1       	       	       	       page 18
Figure 2       	       	       	       page 90
Figure 3       	       	       	       page 115