Death to the jammer lap point: Interview with a murderer

Today everyone got word, “jammer lap points will be eliminated.” We are all getting ready to wave goodbye to those 5-point passes. Super jams.

Before we do, I want to introduce you to the man who helped get us here. Code Adam is a long-time, high-level referee who now bench coaches Arch Rival All Stars. He is also the [proud] murderer of the jammer lap point (JLP).

In 2016 Adam conducted a thorough, unscientific study about the JLP and how no one gets them right. You can read his full analysis here, or his abridged version here. To try to summarize for you:  To track JLPs, a person needs to be able to track the location of two jammers at the same time — a relatively impossible task given everything else a jammer referee is responsible for. This, plus some funky situations leads to no one getting JLPs right. Adam’s study broke down the 2016 playoffs and champs, a place with many certified, high-level officials and found no pair of jam refs got these points consistently correct, with 130 uncorrected errors in the games — 18 of which were won by only 1 to 4 points. More on this below.

What’s it feel like, being a murderer? (I kid, I kid). But seriously, how does it feel to have the WFTDA (a vote by leagues all over the world) back you in your conclusions and take action on killing the JLP?

I’m very, very happy this happened. It is something I have been thinking and talking about for close to 3 years, much to the annoyance of those around me. I’m so glad I was able to effectively communicate my point of view.

Were you surprised that the derby community was relatively quick to make this change, once you proved that so many 5-point-passes are mis-awarded?

I was extremely surprised that skaters voted to get rid of them. I felt the only people who were on my side were officials, and they — rightfully — only get one collective vote among hundreds cast for rules changes. If any change was going to take place, I had to convince most skaters that this thing that had been with the sport since its inception was actually way more complex than any sport’s method of scoring should be. I acknowledged from the start that my study wasn’t scientifically rigorous, and I meant it more as just a jumping off point for others to start their own conversations about it. It seems like that was successful.

Why did you do this study in the first place?

Back in 2016, my officiating peers and I were putting a lot of effort into getting jammer points right: we developed methods of communication and thought processes that would allow us to track who was lapping whom and when. I found that this wasn’t enough, though. The chaos of the game crept into our best-laid plans and caused mistakes. As much thought as I put into it, there was no way to guarantee a point wouldn’t be missed or erroneously added. The more I thought about it, the more I realized what a complex thing jammer referees were being tasked with doing.

I didn’t have any real goal in mind when I started logging the mistakes I saw (and produced myself) at WFTDA Playoffs in 2016, but once I got rolling with it, I wanted to capture everything and share with the community a problem that I feel no one outside of my peers knew existed.

You beautifully outlined how it’s nearly impossible to get JLPs correct in a 29-page study. For people who didn’t read it — or had difficulty understanding it — could you summarize?

I wanted to shed light on the fact that jammers being scored on by other jammers was an extremely difficult thing to track.

1. I laid out all the reasons from a rules/game perspective why tracking these kinds of points was hard not just for referees, but for spectators, too.
2. I showed the results of a survey/quiz I distributed to all kinds of members of the derby community — from highly accomplished officials to skaters on the world’s best teams — in which they watched a single jam as many times as they wanted to identify how many Jammer Lap Points each jammer scored. Answers were all over the place, and only a small percentage of people got it right.
3. I watched all 116 games from WFTDA 2016 Playoffs and Champs to log every single mistake with jammer points and the situations that led to them. There were a lot of mistakes, simply because there are so many different situations that can lead to them.

It all comes down to the fact that jammers can move independently of the pack, so referees and spectators need to keep track of where each jammer is relative to one another at all times. This can prove difficult, especially for a Jammer Referee who needs to keep their eyes on a single jammer the whole time.

I am a jammer. I am also a jammer-ref. As a jammer, I want to tell you that this elimination makes me incredibly sad. As a jammer referee, your study changed my entire way of thinking. The day before your study came out, I would have told people I 100% understand how to award jammer lap points and do so correctly most of the time. The day your study came out, I knew I was entirely wrong (and so was everyone else). What feedback did you get from the derby community upon posting this? 

I first posted this on the WFTDA forums in January 2017 (immediately after a new set of rules came out, so I wasn’t expecting anything to immediately happen with it), and I got generally positive responses then. Most people had not thought about the craziness that is jammer lap points (JLP) before, and it helped people evaluate the actual value they brought to the sport relative to the amount of mental energy it takes to track them.

So while the people on the forums were very positive, it’s important to note the forums are made up of people who volunteer to be their league’s representative on there: that gave me a very small, specific niche of the derby community that probably is not a great bellwether of what the community as a whole might feel about a large challenge to an existing institution. Given the timing with the recent rules release, I simply hoped the educational aspect would help reduce mistakes until the next time we got the opportunity to change the rules.

This last summer, wanting to generate more support for my cause before the imminent vote, I posted a slightly abridged version of my original write-up on Medium. I felt this got me a slightly wider range of responses, though the negative ones were never really more than expressing the opinion “it’s always been that way.”

The biggest shock to me was your playoffs/champs stats. You wrote, “In the 118 games at 2016 Playoffs/Champs, 191 mistakes with Jammer Points were recorded, though 61 were corrected after the jam. [80 missed, 50 added – uncorrected]” Which is just incredible. Did that shock you? 

Because of the number of mistakes I myself was making despite dedicating way too much mental energy to tracking them, it wasn’t surprising to me that others who weren’t thinking about it as much were messing it up, too. In my study, I laid out 13 different ways something can go wrong when figuring out if a point should be awarded for scoring on the other jammer. On a long enough timeline, you’re going to see a lot of mistakes being made.

What was ultimately the most surprising to me was the results of the survey I distributed. Even when tasked with ignoring all other parts of roller derby, people who should be experts in our sport were not able to identify how many JLPs jammers were scoring. This illustrated the main issue to me: JLPs made the scoring rules of our sport inaccessible to anyone less than an expert, and no sport should have points being scored unbeknownst to everyone watching. It is apparent to me that the only way I can consistently track them as a spectator is if I ignore almost everything else; otherwise, I am just trusting the jammer referee to get it right.

JLPs made the scoring rules of our sport inaccessible to anyone less than an expert

I know others have discussed ideas like, “4 points only when the jammer is out of the engagement zone” or “1 point per pass” to simplify scoring. Was this the solution you wanted?

It was.

When toying with the idea of axing JLPs, I had considered the idea of earning a 5th point when the jammer completes a scoring trip and exits the Engagement Zone. This would give the other team incentive to force a call-off with the lead jammer still in the pack even if she has scored on everyone already. I never felt that strongly about it, though, and I wanted the focus to be on the thing that I felt was detrimental to the sport. The community was bound to only accept one large change to scoring at a time, so maybe this can come in the future.

There was a small push from some to keep JLPs but remove the idea that lapping position was “reset” when a penalized jammer returned from the box. (The current/old rules dictated the first time a jammer passed the other previously penalized jammer, a point would not be scored.) While this concept did cause confusion among a lot of people, this solution wasn’t sufficient for me. Most officiating errors with JLPs had nothing to do with this weird rule or penalized jammers, and the survey/quiz I distributed that returned wildly inconsistent answers among the community had them watch a jam with no jammer penalties. Had this change happened, we still would have seen lots of confusion regarding JLPs.

Not everyone would do what you did. Do you do data analysis for work? Or is this just a side passion?

You are being a bit kind by just saying “not everyone” would be weird enough to spend hours going through footage of 116 roller derby games to provide evidence for some kind of point. Working with data is the basis for my day job, and it informs a lot of how I operate elsewhere in my life. I knew challenging the status quo of scoring in this sport would require a lot of evidence to back up my claim; I couldn’t just offer my own personal experiences. I ultimately decided a complete list of everything from the most popular games of the year would be the most powerful message.

You laid out the reasons JLP needs to die, but are there any good arguments for it to stay? As a greedy jammer, I want 5 points for passing 5 people. Do you think it’s enough of a benefit that the other jammer is stuck?

I get that people want credit for every single person they pass, but it’s mostly arbitrary that they get an additional point for the jammer.

What effect do you think eliminating the JLP will have on the sport?

Most importantly, the sport will be easier to officiate and easier to spectate. It will be a lot more obvious when a jammer is scoring a point, and there is no need to allocate mental space to remember which jammer passed the other one more recently. Beyond that, I don’t think there will be much of a change in gameplay or outcomes.

From a gameplay standpoint, right now we just see a very small number of times each game the lead jammer will continue the jam a few seconds longer to catch the other jammer out of the Engagement Zone to score one final point. Eliminating the ability for that extra point will mean a few jams get called off a few seconds earlier. With the opposing jammer still in the pack, not scoring a JLP doesn’t change anything about what a jammer is trying to do: get out of the pack so they can make it around again for an additional chance to score points.

As far as the outcome of games, a referee from Gotham named Ref in Peace released a follow-up study to mine that answered the question, “What would happen to recent games if we changed all the 5-point passes to 4-point passes?” From his data set of 394 games, he found 2.5% of games had their outcome change, and all those games had an original margin of victory of 5 or fewer points. This essentially means if you had won your game by 6 or more points with JLPs, you would win the game without them. And if you’re winning the game by 5 or fewer points, one small thing could have gone differently in that game that would have made you lose. So they are not really helping the better team win, nor are they helping the opponent stay in the game. They’re just this arbitrary thing that does not add enough actual value to keep around.

Are you worried about these complications happening all over again when we get into star passes (jammer turns blocker = point)?

Not particularly. The Rules Committee should be able to handle the limited number of scenarios that can happen with star passes. The rules now have the casebook in which they can spell out specific scenarios in a way that still has them universally applicable. I am confident everything will be easier.

What else needs to change in roller derby?

I don’t feel as strongly about anything else as I did about JLPs, as there is nothing that is as widely misunderstood as JLPs were.

One thing related to knowledge of scoring that came on my radar recently was score reporting as the clock ticks down near the end of the game. In A Coruña, there was a final jam that ended with 5 seconds left in a very close game with both teams sitting on clock stoppages. The jammer referees put up the score immediately, but the final score tally from the jam was not on the scoreboard until long after the period time had hit 0:00. There is always going to be a delay in getting points visible on the scoreboard (what if a jammer referee had to talk to a referee on the outside to get feedback on whether points were earned, delaying the reporting even more?), but I think teams should not be forced to guess as to whether they should be using a timeout before they know if they are winning or not. An idea I came up with after seeing this game—and something that could be easily built into any scoreboard application — is if a jam ends with less than 30 seconds on the period clock, the period clock will stop until the scoreboard operator hits a button that indicates the score from the previous jam is fully entered, at which time the period clock and normal 30-second lineup clock will start up. This would give teams the opportunity to see what the score actually is so they can decide whether or not to use a timeout.

As a sort of aside, but I’m very curious: How did your officiating background translate to coaching? 

The fact that I have never played the game gives me these large blind spots into so many parts of derby, including how skaters think and what reasonable expectations are for on-track awareness. I offer a very niche point of view coming at it from a long-time official’s perspective, and I acknowledge it’s a very limited scope. It makes me feel like a football team’s low-level assistant coach who is just in charge of training the long-snapper on how to hike the ball to the punter: it’s an important role that can give a team a small competitive advantage, but even if that long-snapper only does his job half as well as I taught him, it’s not going to have much of an effect on whether the team wins.

Are the jammer refs at Arch’s games terrified? I would be.

Those jammer referees are around me all the time, so they have suffered through my complaints about JLPs off the track enough to know how to get it right on the track.

Has coaching taught you anything new about the sport? 

It would have been incredibly arrogant of me to come into coaching thinking I had nothing to learn about the sport when I have only officiated it. I have learned a lot about things as simple as what happens on a bench during a game to the complex nature of how specific jam-start strategies work. And whereas early in my career I was just the stupid referee congratulating himself for calling some small technical penalty that no one else in the venue saw and which didn’t offer the team any real advantage, I now fully understand how terrible penalties are and how detrimental they can be to a team.

Are you retired from officiating?

Unless something large changes, I won’t be traveling to officiate derby anymore. I largely achieved what I wanted to, and I don’t want to take a tournament spot away from a person who deserves and probably wants it more at this point. I still help out with local play for Arch Rival, though I imagine this will be my last season.

Wanting to get back into soccer but realizing I am now terrible at playing it after so many years away, I recently started officiating that. While the pay is better, I know it will never give me the community that roller derby did.

Code Adam has officiated roller derby since 2009, working as a skating official at three World Cups, 18 WFTDA Playoff tournaments, and 5 WFTDA Championship tournaments. Since 2017, he has served as a bench coach for the Arch Rival All Stars.