The Lusk Study - A Critical Assessment

In 2011, a study was published online in the BMJ journal 'Injury Prevention'. The study, 'Risk of injury for bicycling on cycle tracks versus in the street' by Anne C. Lusk et al, got a lot of press (as cycling safety studies go), and it is regarded with respect by many in the cycling community. In fact, I think it's probably the single most referenced study of cycling safety in current circulation. But a more critical assessment shows significant flaws.

----------------------------------------

Flaws in the 2010 Lusk Montreal Study

The Lusk study measures bicyclist injury rates on cycle tracks and compares the results with results from supposedly similar reference streets. This was done for six cycle tracks and comparable reference streets. Here I will list the streets that were compared and try to illustrate the flaws I found in terms of the nature of the streets being compared.

1. Rue de Brebeuf Cycle Track vs. Rue St. Denis between Rachel and Laurier.

Rue de Brébeuf Cycle Track

Rue St. Denis

These streets are not comparable.

Brebeuf (which has a cycle track) is a narrow 40kph slow-moving one-way residential street with one traffic lane and one parking lane.

Rue St. Denis (which has no cycle track) is a six-lane (two lanes often taken up by parking) 50kph limit two-way highway in a commercial area with lots of stores and distractions.

It seems to me that more accidents will naturally occur on the six-lane highway with a faster speed limit. It's unsurprising then that the study did indeed find a statistically significant advantage in terms of safety for Rue de Brébeuf. However, I would argue that this has nothing to do with the safety of the cycle track and everything to do with the very different nature of the roads compared. 

2. Rue Rachel Cycle Track vs. Avenue du Mont Royal between St. Urbain and Marquette.

Rue Rachel Cycle Track

Avenue du Mont Royal

Do these streets look at all alike?

Rue Rachel (which has a cycle track) is a four-lane 50kph limit commercial street with a lane apparently open to parking on both sides. On one side, the two-way cycle track is separated from the general traffic lane by a physical barrier and a line of parked cars. Along a third of its length, Rachel has a park next to it, reducing the potential for crossing traffic on this cycle track.

For two thirds of its length, Avenue du Mont Royal (which has no cycle track) is indeed a similar 50kph commercial street with parking both sides. But there the similarity ends. Avenue du Mont Royal has no park and has many intersecting streets along its entire length.

One would think that, if the cycle track was safer, Rue Rachel, with the reduction in crossing conflicts - an unfair advantage - conferred by the park, would show it clearly. Yet the differences in terms of injuries between Rue Rachel and Avenue Mont Royal were statistically insignificant. 

3. Rue Berri Cycle Track vs. Rue St. Denis between Cherrier and Viger.

Rue Berri Cycle Track

St. Denis

These streets are not comparable.

Rue Berri (which has a cycle track) is a 50kph limit divided highway along 1/3rd of its length with the cycle track removed from busy intersections by an underpass, so cyclists are naturally removed from the possibility of intersection accidents.

Along this stretch of Rue St. Denis, the road (which has no cycle track) is a one-way street with a 50kph speed limit. However it is a much busier road than Rue Berri in terms of people doing their business somewhere along that stretch, with a relatively narrow street and lots of intersections and distractions in the form of little shops and cafes along the whole route.

Rue Berri showed a statistically significant reduction in injuries compared with its reference street. However, more accidents are bound to occur where there are lots of intersections and where drivers are likely to be distracted. It seems reasonable that the advantage in terms of reduced injury results on Rue Berri derive from the very different nature of the roads compared and not from the presence of a cycle track.

4. Boulevard de Maisonneuve Cycle Track vs. Rue Sherbrooke and Rue Ste. Catherine between Claremont and Wood.

Blvd de Maisonneuve Bike Track

Blvd de Maisonneuve Bike Track

Rue Sherbrooke

Rue Ste. Catherine

One of these streets is not like the others. Can you spot the differences?

Boulevard de Maisonneuve (which has a cycle track) is a quiet 30kph one-way two lane residential street along much of its length. The bike track goes through a park for 1/5th of its length, thus removing any possibility of intersection conflicts in that area. The presence of the park effectively reduces the chance of traffic collisions by 20%.

Sherbrooke (which has no cycle track) is a downtown 40kph commercial street with four lanes of moving traffic and parking on both sides. It has numerous business distractions along its length. It should be noted that a recent study found that Sherbrooke is the single most dangerous route in Montreal for cyclists. Ste. Catherine (which also has no cycle track) is a similar downtown street, but with a 30kph limit and just two lanes of moving traffic and a lane for parking on both sides.

The idea that these streets are comparable on anything but the most superficial level (i.e. they are streets) is a joke. It is ridiculous, in my view, to attribute a reduction of injuries on Boulevard de Maisonneuve to the presence of a cycle track, when the streets being compared are not at all similar - and when the street with the cycle track has obvious and significant advantages in terms of safety that are unrelated to the bicycle track itself. 

5. Avenue Christophe Columb Cycle Track vs. Rue St. Hubert between Gouin and Jarry and Avenue Christophe Columb between Villeray and Rosemont.

Ave Christophe Columb Cycle Track

St. Hubert

Ave Christophe Columb

These streets do seem comparable.

Avenue Christophe Columb (which has a cycle track) is a 4-5 lane 50kph limit highway along much of its length with one or two extra lanes for parking. The cycle track is completely segregated from the road except at intersections.

Rue St. Hubert appears to be a somewhat quieter 4-5 lane residential street with one or two extra lanes for parking and no cycle track. Avenue Christophe Columb between Villeray and Rosemont (which also has no cycle track) is a 4-5 lane highway along much of its length with a 50kph limit and one or two extra lanes for parking, much like the portion with the cycle track.

I would say this street study would be the most fair in terms of the nature of the streets being compared. So it's interesting that the overall injury rate on these streets showed comparison results that were statistically insignificant. However, it should be noted that in terms of relative danger from vehicular traffic, Avenue Christophe Columb (the street with the cycle tracks) had a much worse (and statistically significant) danger factor. I personally think the way they calculated the danger factor is bogus, but it's interesting that, in this case, it works out to favor the street without the bike path. Also interesting is that the study's authors make no mention of this in their conclusions.

6. Boulevard René Lévesque Cycle Track vs. Rue Sherbrooke between Lorimier and St. Hubert.

Blvd René Lévesque Cycle Track

Rue Sherbrooke

These streets are not comparable.

Boulevard René Lévesque (which has a cycle track) is a very wide six lane divided highway with a 50kph limit and very good visibility along its length. On one side, the two-way cycle track is separated from the general traffic lane by a physical barrier.

Rue Sherbrooke (which has no cycle track) appears to be a slightly narrower undivided 50kph limit road, six lanes wide, with one of the lanes used for parking. The area around the street is somewhat more built-up and closed-in than René Lévesque, so sight lines are not as good. Again, it should be noted that, in choosing Sherbrooke for a reference street, the study authors selected the most dangerous route in Montreal for cyclists.

Rue Sherbrooke has slight disadvantages compared to Boulevard René Lévesque in terms of cyclist safety. However, these are related to the nature of the roads themselves, and not to the presence or absence of bicycle tracks. Again, even though the road with the bicycle track had an unfair advantage, the study found no statistically significant differences in terms of injuries on these roads.

----------------------------------------

Discussion

The study claims:

"Contrary to AASHTO's safety cautions about road-parallel paths and its exclusion of cycle tracks, our results suggest that two-way cycle tracks on one side of the road have either lower or similar injury rates compared with bicycling in the street without bicycle provisions."

Yes, the results do indeed suggest that. But is it any wonder that the Lusk study finds that cycle tracks are safer than street cycling? I don't think so. With the exception of #5, the streets appear to have been chosen in such a way that safety is maximized on the streets with bicycle facilities, while risk is maximized on the so-called 'reference' streets. In my opinion, given the selection of streets used, the study's conclusions could have been foretold before the first cyclist in the study put his foot on a pedal. With the overwhelming nature of the bias in the study, I find it amazing that three of the six street comparisons still showed no safety advantage to cycle tracks!

Here we have what seems to me to be a clear case of selection bias, and selection bias so blatant that I find it hard to believe that the people doing the study did not have a point they wanted to prove at the outset. In fact, the study itself all but admits this agenda, when it almost lapses into rant mode. Note the use of emotive or dismissive language and a very telling admission of a pro- cycle track agenda:

"Cycle track construction has been hampered in the USA by engineering guidance in the American Association of State Highway and Transportation Officials (AASHTO) ‘Guide for the development of bicycle facilities'... A long-standing, and yet not rigorously proved, philosophy in the USA has suggested instead that ‘bicyclists fare best when they behave as, and are treated as, operators of vehicles.’ "

 In my opinion, the integrated (or 'vehicular') cycling philosophy has been rigorously proved (as much as anything can be) by the very studies that Lusk et al go on to dismiss as 'conflicting':

"...crash rate comparisons between the USA and The Netherlands have been dismissed by vehicular cycling proponents, with arguments of non-transferability to the American environment. Cycle tracks have been controversial, especially due to conflicting studies with warnings of increased crash rates."

Contrary to this assertion of conflict, the studies are interesting in that (apart from a very few exceptions) they don't conflict, and that so many of them agree on this specific point: that bike facilities increase the risk of intersection collisions.

Lusk et al go on to say:

"Public health and bicycling advocates in the USA have faced a dichotomy, believing from surveys and European experience that cycle tracks encourage more bicycling, yet being warned that they lead to higher crash and injury rates. Our results suggest that cycle tracks lessen, or at least do not increase, crash and injury rates compared with the street. The construction of cycle tracks should not be discouraged."

So Lusk et al knew that the weight of expert opinion was against bicycle facilities, and they knew that health and bicycling advocates in the US and Canada were finding it hard to move ahead with their objectives while bicycle facilities were known to be hazardous. So it seems they wanted to find a way to redress the 'imbalance'. In short, they seem to have had a pro- bicycle facility agenda.

In an attempt to find other studies that back up their results, Lusk et al attempt to use Wachtel and Lewiston's 1994 study 'Risk factors for bicycle-motor vehicle collisions at intersections', yet they seem to assume that all non-intersection collisions must be on the roadway. Since driveways are not intersections, and since there will be interactions between intersections even for cycle tracks, I'm not sure they can extrapolate so simplistically. Even if they can, they admit that the extrapolated results merely show that separated paths are "at least no more dangerous than bicycling in the street". Hardly a ringing endorsement of cycle tracks. And let's not forget that it was Wachtel and Lewiston who said, in that very same study:

"Bicyclists on a sidewalk or bicycle path incur greater risk than those on the roadway (on average 1.8 times as great), most likely because of blind conflicts at intersections... intersections, construed broadly, are the major point of conflict between bicycles and motor vehicles. Separation of bicycles and motor vehicles leads to blind conflicts at these intersections."

Anne Lusk, judging from the information available on her website at the Harvard School of Public Health, is keen on the health benefits of cycling. I am too. But I don't think misrepresenting data in a way that makes hazards look safer is a good way to help people to become healthy.

Whether the bias was intentional or merely subconscious, the results are the same, and we are talking about people's lives here.

 ----------------------------------------

Implications for Cyclists

At 6:50am on July 24, 2012, a 33 year-old man became the first cyclist (and the first cycle track user) to be killed on any of the streets in the Lusk study, since the study was published. He died on the Christophe Columb cycle track where it intersects Rue Mistral. The driver who killed him may have been dazzled by the morning sun, which would have been in his eyes, and the cyclist may have been partially hidden by the signs on the traffic light post as he approached the intersection. This image shows the driver's view: the cyclist would have been approaching the camera on the cycle track.

The cyclist was killed in a classic intersection turning conflict of the type that bicycle facility experts have been trying to warn cyclists and transportation engineers about for at least four decades. He most likely thought he was perfectly safe - protected by a bicycle track that was physically separated from the roadway. What he almost certainly did not know was that the Lusk study, which supports the facility he was using, admits:

"the American Association of State Highway and Transportation Officials (AASHTO) ‘Guide for the development of bicycle facilities’... cautions against building two-way paths along, but physically separated from, a parallel road. AASHTO states that sidewalk bikeways are unsafe and implies the same about shared-use paths parallel to roads, listing numerous safety concerns and permitting their use only in special situations."

Also, it should be noted that bicycle lane use is no longer mandatory in the province of Quebec. So this cyclist could have legally avoided this bike lane death trap, if he was aware of the change in the law, which happened in 2011. Before then, cyclists were required to use bike lanes.

So far, no roadway cyclists have been killed on any of the study's reference streets. This, despite Quebec's law that forces cyclists to ride on the far right hand side of the road. Presumably this is so that they will be less visible to motorists and therefore more easily killed by them - that way, bike lanes might seem that much less dangerous when everyone finally figures out how hazardous they actually are.

As I said at the beginning, the Lusk study is regarded with respect by many in the cycling community who see it as a confirmation of what we want to hear - that the facilities we expect to keep us safe are indeed as safe as they can be. The problem is, these facilities pose a serious hazard to cyclists, and sadly the Lusk study serves to keep us unaware of that fact. I hope that this assessment serves as a wake-up call to those of us who rely so uncritically on such facilities.

 ----------------------------------------

Since posting this, John Allen alerted me to the fact that there are other critiques of the Lusk study - specifically by M Kary and by Wayne Pein. They both go into far more detail than mine and while mine is merely an amateur critique that addresses only a few of the more obvious (to me) concerns, theirs are far more scholarly and convincing. They can be found here: 

http://john-s-allen.com/reports/montreal-kary.htm

http://bicyclingmatters.files.wordpress.com/2011/03/montreal12.pdf

http://injuryprevention.bmj.com/content/17/2/131/reply