Friday 20 August 2010

Reverse causality, age, and scientists

According to the BBC, "Fear of falling 'boosts elderly's fall risk'" - that is, the more worried an old person is about falling over, the more likely they are to actually do so. The full text of the study is freely available, for once.

The reporting is actually a reasonable summary of the paper's conclusions, for once - though it as usual misses out the numerous caveats written into the paper and presents the conclusion as established unambiguous fact.

Of course, the paper's conclusions are somewhat dubious.

The research assessed 500 people aged between 70 and 90 and without a fairly long list of disabilities and health conditions to calculate an estimated fall risk based on their physiology (various assessments of proprioception, balance, visual senses, muscle strength, and so on). It then asked them about their perceived fall risk.

The study then compared what the researchers' thought their chance of falling over was with what the individuals themselves thought, and divided them into four groups for "researchers think" high or low versus "individuals think" high or low. The study talks as if these were very distinct groups, but if you look at Figure 2 in the paper, there is a lot of clustering around the centre, and you could shift the boundaries quite significantly with relatively small changes in where you defined the centre point.

Over a year of follow-up study they discovered that of those people who believed they had a low risk of falling, but the researchers' assessment categorised them as high risk, only 30% had actually fallen. Meanwhile, of those people who believed they had a high risk of falling, but who were assessed by the researchers as having a low risk, 40% had fallen.

The obvious explanation, to me, is that the physiological assessment for risk of falling is missing at least one major factor that makes it inaccurate in a number of cases. This is mentioned as a possibility in the paper:

It is possible that the anxious group had a residual physiological fall risk not encompassed by the physiological profile assessment. Thus, although the group’s low physiological profile assessment score indicated that the integrity of individual sensorimotor components was adequate, their relatively poor performance on the coordinated stability test suggests that the integration of each of these physiological systems in the maintenance of dynamic balance control was impaired.22 However, the disparity between physiological and perceived fall risk in the anxious group seemed to be strongly related to psychological factors.

...but dismissed without much of an explanation as to why.

As far as I can tell, they're getting cause and effect completely backwards.

They're assessing people who have been living for some time, and presumably have fallen over during that time. In the 69-89 years prior to this study, they've presumably each built up a good picture of how often they individually fall over. Those who fall over often presumably are more worried about doing so again (or at least, what's actually being measured, think they are more likely to do so again whether or not this worries them).

When you study them for a year - no surprises: past experience is a good predictor of future experience in this area.

The results they've got are exactly the results you'd expect for any event the frequency of which can be predicted reasonably well by the people it happens to, but not as well by a statistical assessment over the population. You could conclude for any of those that "expecting X causes X" but it wouldn't actually be justifiable.

Dismissing this possibility with a vague statement that "the disparity between physiological and perceived fall risk in the anxious group seemed to be strongly related to psychological factors" doesn't actually disprove it, especially when actual fall risk matches perceived fall risk quite well.

But I guess it's easier to disbelieve old people than to believe your "physiological assessment" is flawed.