Collecting and interpreting rape statistics

[trigger warning]

Since some of this isn't completely obvious, and so I want to write this down for my own use later if nothing else, here's a post on the difficulties of collecting statistics on the prevalence of rape. A lot of this applies, in other forms, to collecting statistics on just about anything that happens to humans, but because it's working against privileged structures, statistics about rape get criticised more for the same inevitable problems.

A lot of the problems aren't as important as they're made out to be, though, from the point of view of having useful statistics. (Furthermore, despite the problems, the statistics are all relatively consistent to within an order of magnitude)

I'm discussing the problems here from the perspective of statistics on victims; statistics on perpetrators have basically the same sort of problems.

Definitions

The first difficulty is that the colloquial and legal definitions of rape vary considerably. Colloquially rape is sex without consent. Legally, this could be any of "rape", "assault by penetration", "sexual assault" or "legal". Furthermore, the same sexual activity could be rape if A does not consent, but "only" sexual assault if A consents and B does not, because of the asymmetry in the law discussing penetration.

This is to an extent an inevitable problem with the law. The Sexual Offences Act 2003 defines "rape" and "assault by penetration" very precisely. Everything else, whether it would generally be considered rape or not, is "sexual assault". This matters, because the maximum sentence for sexual assault is ten years, whereas rape and assault by penetration have life sentences. Providing a loophole-free legal definition of rape is the same problem as providing a loophole-free definition of sex and adding "without consent" to the end of it.

Add to this the "reasonable belief" exemption in law that means many things that the victim will call rape the law will call "legal" or "no crime".

The problem comes, then, when you try to do any sort of measures of prevalence. People will correctly say that they were raped when it wasn't rape in legal terms. There seems to be some attempt to deal with this in the British Crime Survey's figures on sexual assault, which split "serious sexual assaults" (which include rape and assault by penetration) away from "less serious" (indecent exposure, sexual touching, sexual threats, etc: note that "sexual touching" is in law the same crime as raping someone by forcing them to penetrate you)

If you classify according to the strict legal definition, then you inevitably lose quite a few rapes in there buried in the sexual assault and no crime categories. If you classify according to the colloquial definition, then it becomes much harder to do comparisions with the statistics produced by the criminal justice system (and apologists will claim that your statistics are worthless because they don't only include government-approved rapes)

There's also a question of whether to include the "attempted" types of crimes within the statistics. I think that one should, because the difference between the two is largely down to random circumstances, and either way there's a rapist to apprehend: that they didn't get quite as far as raping this victim (but far enough to make it clear that they were going to try) shouldn't be considered.

Reporting

The second problem is that being a victim of rape has an extremely strong stigma associated with it, and even if it didn't, as a traumatic act it's something that some victims block out of their minds. (Associated with this, misconceptions about consent aren't just confined to rapists, so it's common for people not to define what was done to them as rape until much later, even though they're dealing with the psychological consequences immediately)

So it becomes very difficult to find out if people have been raped by surveying them. The British Crime Survey tries to do this by asking questions about actions rather than about legal definitions, and this does help - around twice as many people will say that someone did [action(s) constituting rape] to them than will say that someone raped them. However, you then inevitably have the situation that the survey is only as good as the questions are exhaustive (and even then despite good methodology people may decline to answer).

We can use the surveys to establish some upper and lower bounds, at least. The 2009/10 BCS data gives 0.4% of women and 0.1% of men, in the preceding year, have been subject to rape, assault by penetration, "serious sexual assault" and/or an attempt at either. We can't necessarily scale up from this to a lifetime prevalence (if we for simplicity assume the likelihood of being raped doesn't change with age¹ it gives a likelihood of 17.6% of women and 4.4% of men, not all of whom will be raped in the legal sense).

The NSPCC survey (page 66 onwards) gives figures for sexual violence of around 27% for girls, 16% reporting that they were pressured into intercourse (6% with physical force). There's no particular attempt to match these up to legal categories, but it's obvious here that the risk to 13-16 year old girls (not varying much by age within that) is considerably higher than the average risk for the 16-59 year old adults covered by the BCS. The NSPCC survey notes that figures between 4% and 78% have been found by other surveys of children, with - as with adults - there being a significant gender split in perpetrators and victims.

Meanwhile the Havens survey says that 41% of 18-25 year old Londoners have felt pressured into unwanted sex. 9% of women in the sample had said no and been ignored, and 25% of women (almost certainly a strongly overlapping set) had said nothing and been ignored².

Measuring the legal system

It's relatively easy, within the provisos of the definition problems, to get statistics on the legal process - convictions, prosecutions, arrests, reports. Relying solely on those statistics is a mistake: the attrition rate from report to conviction has worsened considerably since the 1970s, while the number of convictions has increased. What's happening is that rapes that wouldn't previously have been reported - and marital rapes that were legal until 1991, for that matter - are now being reported ... and the justice system hasn't caught up.

There isn't any comparable prevalence survey going back that far - the BCS only started asking about sexual violence in 2004 - but despite everything, and despite appearances, the justice system is probably better now (with its 90%+ attrition rate, and widely reported failings) than it was in the 1970s with a much lower attrition rate - because most of the attrition was occuring before reporting.

Measures from the justice system aren't useful for measuring incidence of rape, but they are useful for measuring the (in)effectiveness of the justice system (and hopefully improving it). It's important, however, to note that the definitions problem makes it very difficult to compare prevalence statistics with justice system statistics (which makes getting meaningful figures about reporting rates - beyond "very low" - very difficult indeed).

Another problem is that the categorisations used for reporting and police activity - the report to charge stage - use one set of categories (managed by the Home Office), but the categories at the charge to conviction stages use a different set (managed by the Ministry of Justice). This makes sense, because the police often won't know exactly what crime has occurred until after they've investigated, whereas the CPS and courts do know the details of the charges, but it makes comparisions tricky. Kelly, Lovett and Regan's attrition study dealt with this by following cases right through the report to conviction process (or as far through the process as they got, anyway).

The need for statistics

On the one hand, it doesn't really matter at this stage. It's very clear from the statistics that the (lifetime) chances of being raped (in the colloquial sense) are somewhere between 1 in 20 and 1 in 2 for women (and most probably around the 1 in 4 figure generally quoted), and lower (but still probably higher than most people would guess) for men. Wherever it falls within that range, it's still a massive problem (we view murder as a serious problem at the far lower 1 in 1000³ lifetime prevalence, and rape is sentenced similarly)

On the other hand, if it's not to remain a problem, there's a need for accurate statistics to monitor things over time, so that it's possible to tell if actions to deal with the problem (by bringing rapists to justice and more importantly because we don't have space for all of them by educating people so they don't become rapists).

Fortunately, for the purposes of accuracy, it doesn't matter than much exactly what definitions you use, as long as you're consistent over time. You'll always only be asking about - and being told about - a particular subset of sexual violence, but you should be able to measure trends in it. The way that rape culture works, it's vanishingly unlikely that one particular form of sexual violence that you're surveying will disappear or expand while the rest remain unchanged.

As long as you remember that changes in methodology are likely to give changes in result significantly larger than any change in the underlying facts, then changes can be seen (and yes, this means sticking with methodology you know is flawed, at least until you've run it in parallel with the improvements for a few survey cycles to see what difference it makes).

A final thing to note - and bear with me here - is that accuracy can be overrated. For year-to-year comparisions about the scale of the problem, a repeatable survey that's not too vulnerable to random noise is needed. For surveys to establish the existence of a problem within a particular context, it's not. There was a recent NUS Women's Campaign survey looking at female university students' experience of sexual violence, which gave the predictable results. It wasn't at all statistically sound: self-selected sample, no attempt to normalise it demographically, massive difference in response rates between universities, etc. but that doesn't actually matter for establishing the existence of the problem.

Now, if universities were to take the NUS survey seriously and start doing (actually useful⁴) things to reduce sexual violence on campus, then a survey less sensitive to random noise would be needed. But if they were going to take it seriously they'd fund their own surveys for that purpose, and if they're not going to take it seriously a "yes, this is still a problem, what did you expect?" survey is all that's needed.

Footnotes

¹ I believe it decreases with age, but I don't have the figures for that.

² This brings us back to definitions. Silence is not consent, but the law usually takes it as such. But you'd never find out about this set of colloquial rapes/sexual assaults if you didn't ask that specific question.

³ Massive variation with gender, cis/trans status, age, race, class, sexuality, disability, location, etc. As a society we don't view the elevated number of murders of certain non-default individuals as a problem, even if the murder rate as a whole is considered a problem.

⁴ Putting up "have you considered not getting raped?" posters, for instance, is fairly common and massively counter-productive.

Semi-Equality Act: EU directives, and views from the Minister

So, the message from the Equalities Office referred to four European Directives, claiming that they meant that the harassment protections were necessary to implement EU law - with the nasty implication, of course, that if it weren't for those directives, it would be legal to harass people in a racist, sexist or disablist fashion, provided that it didn't constitute direct or indirect discrimination.

• 2000/43/EC: article 2, paragraph 3 defines harassment as a type of discrimination, and article 3 sets out various areas in which it applies.
• 2000/78/EC: article 2, paragraph 3 again defines harassment, with articles 1 and 3 setting out that this applies to age, sexuality, disability and religion or belief, in contexts relating to employment only.
• 2002/73/EC: this is similar to 2000/78/EC but for gender discrimination and harassment in employment. It states that sexual harassment is to be considered harassment and that marital and family status cannot be used as 'covers' for discrimination.
• 2006/54/EC (PDF): extends 2002/73/EC slightly.

The Equalities Office's claim is that because of these EU regulations, the prohibitions on harassment are not redundant as they implement EU regulations.

However, the Equalities Act rightly goes beyond the requirements of the EU regulations - it forbids harassment on the grounds of gender, age and disability in many contexts other than employment.

There seems little point in raising this with the Equalities Office itself, however. It's fairly clear that their job is to defend the existing law as if it were perfect, until such time as it is changed by Parliament into another perfect form. To an extent that's even a reasonable requirement to place upon the civil service, but it does mean that getting answers to "why" questions out of them is near-impossible.

Meanwhile, the Equalities Minister, Lynne Featherstone MP (Lib Dem, Hornsey & Wood Green) was celebrating the Equality Act becoming active on 1 October. There were several comments criticising the reversals in equality for trans people that the Act caused, which she promised to look into, while expressing doubt (like her office: in opposition to what the Act's own text and Explanatory Notes says) that the Act actually would have that effect. Hopefully there she'll get some good legal advice and be in a position to repeal¹ some of those exemptions, which are designed to allow the provision of a service for transphobic individuals at the expense of trans people.

On the harassment exemptions her comment was:

Yes – Evan [Harris, then-MP for Oxford West & Abingdon] and I fought very hard over the differential discrimination in terms of LGB at school – and Labour took no notice. EU is looking at this too at the moment. Will say more when I know more.

So it looks like the EU route may be the best way to get these provisions repealed¹, though not a particularly quick one.

¹ Her Conservative coalition partners are planning a "Great Repeal Bill", though repealing clauses of legislation for the purpose of strengthening it - especially equality legislation - is probably not what they have in mind. Still, once that bill gets to first reading I'll suggest it.

Reverse causality, age, and scientists

According to the BBC, "Fear of falling 'boosts elderly's fall risk'" - that is, the more worried an old person is about falling over, the more likely they are to actually do so. The full text of the study is freely available, for once.

The reporting is actually a reasonable summary of the paper's conclusions, for once - though it as usual misses out the numerous caveats written into the paper and presents the conclusion as established unambiguous fact.

Of course, the paper's conclusions are somewhat dubious.

The research assessed 500 people aged between 70 and 90 and without a fairly long list of disabilities and health conditions to calculate an estimated fall risk based on their physiology (various assessments of proprioception, balance, visual senses, muscle strength, and so on). It then asked them about their perceived fall risk.

The study then compared what the researchers' thought their chance of falling over was with what the individuals themselves thought, and divided them into four groups for "researchers think" high or low versus "individuals think" high or low. The study talks as if these were very distinct groups, but if you look at Figure 2 in the paper, there is a lot of clustering around the centre, and you could shift the boundaries quite significantly with relatively small changes in where you defined the centre point.

Over a year of follow-up study they discovered that of those people who believed they had a low risk of falling, but the researchers' assessment categorised them as high risk, only 30% had actually fallen. Meanwhile, of those people who believed they had a high risk of falling, but who were assessed by the researchers as having a low risk, 40% had fallen.

The obvious explanation, to me, is that the physiological assessment for risk of falling is missing at least one major factor that makes it inaccurate in a number of cases. This is mentioned as a possibility in the paper:

It is possible that the anxious group had a residual physiological fall risk not encompassed by the physiological profile assessment. Thus, although the group’s low physiological profile assessment score indicated that the integrity of individual sensorimotor components was adequate, their relatively poor performance on the coordinated stability test suggests that the integration of each of these physiological systems in the maintenance of dynamic balance control was impaired.22 However, the disparity between physiological and perceived fall risk in the anxious group seemed to be strongly related to psychological factors.

...but dismissed without much of an explanation as to why.

As far as I can tell, they're getting cause and effect completely backwards.

They're assessing people who have been living for some time, and presumably have fallen over during that time. In the 69-89 years prior to this study, they've presumably each built up a good picture of how often they individually fall over. Those who fall over often presumably are more worried about doing so again (or at least, what's actually being measured, think they are more likely to do so again whether or not this worries them).

When you study them for a year - no surprises: past experience is a good predictor of future experience in this area.

The results they've got are exactly the results you'd expect for any event the frequency of which can be predicted reasonably well by the people it happens to, but not as well by a statistical assessment over the population. You could conclude for any of those that "expecting X causes X" but it wouldn't actually be justifiable.

Dismissing this possibility with a vague statement that "the disparity between physiological and perceived fall risk in the anxious group seemed to be strongly related to psychological factors" doesn't actually disprove it, especially when actual fall risk matches perceived fall risk quite well.

But I guess it's easier to disbelieve old people than to believe your "physiological assessment" is flawed.