I recently received two pieces of internal Stack Overflow communication about the "moderator action" that started on June 5. Obviously I'm no longer an employee by Stack Overflow, so these documents have been leaked to me. It's natural to wonder who leaked them and why. Some might guess it's someone I know from my time there who is sympathetic to the moderators' cause. That guess is wrong. It wasn't anyone on the community team or a developer. I don't know why this person contacted me other than they had read my recent blog posts.

Deciding to publish these documents wasn't easy. I worry about the negative side-effects of releasing this material, which was never intended for public consumption. In particular, I worry about my friends at the company being wrongly accused, about driving the wedge between community and company deeper, and harming Stack Exchange.1

On the other hand, the strike has been organized on a Discord channel that employees have access to. Non-employees don't have access to the company's side of the discussion, so there is an information imbalance. Unfortunately, it's aligned with the substantial power differential. Moderators have quite a bit of influence over the community, but their administrative power is given and taken away by the company.

The documents below combined with recent company actions lead me to believe the company will not be negotiating in good faith. The crux of the prisoner's dilemma is that the parties can't (or simply don't) communicate. If they could coordinate, each would get a better deal. But even when the parties are able to communicate and do come to an agreement, there's still a risk one party will defect anyway in order to get the better deal and stick the other party with a booby prize. Reading the documents below changed my estimate of the odds that Stack Exchange Inc will ignore the results of negotiation when it is convenient for them.

Email to managers

I'm going to post both documents in full first so that you can evaluate them yourself. Later in this post, I'll share my thoughts. First up, we have an email sent to Stack Overflow managers. I believe it was sent on June 5 because that's the date on the FAQ and also when approximately 11% of moderators had signed the strike letter.2

Hi Managers,

As you may have heard, some moderators on our sites are beginning a time of shutdown or slowdown on site, in an attempt to protest some recent decisions that we've made and out of disagreement with the way that they were characterized.

It's important to note—for a sense of scale—that about 11% of the roughly 600 moderators that we currently have active on site have joined this action.

As you might imagine, this is a delicate situation, so we've put together some FAQs for Stackers, which are attached to this email.

We do not know what it will take to end this action, nor do we know how long they are prepared to engage in this, but we will do our part to bring things to a speedy resolution; one that's best for our company, our community, and our customers.

The FAQs attached are meant for you to distribute to your teams if they have any questions about what's going on. Please take just a moment to read them yourself, so that you are familiar with the contents. If you have any questions, please feel free to ask me or to ask any SLT member. If any of your teams would like more information about what's going on and how to handle particular situations, I'm happy to speak to them or to your team meetings if that would be useful.

Best,
Philippe

FAQ attachment

The email mentioned an attached FAQ. It had a June 5 date on it. I've redacted the names of people who have not made public statements about the strike. I've also reformatted the document, but not in a way that changes the meaning.

What's happening? What's this about a "moderator strike"?

Moderators on Stack Overflow and some of our network sites have announced their disagreement with some policies that have recently been rolled out, and have announced what they are referring to as a "general moderation strike" beginning on June 5, 2023.

What does this mean?

First, please note that we are not referring to it as a moderator strike for legal reasons. Officially, this is a moderator action. During this action, moderators who join will not engage in moderation activity on our sites, including:

What happens next?

The Community Management team will step in to handle any needed moderation on Stack Overflow and any impacted Exchange sites. Not all mods are participating in the action, so many sites will not need additional moderation support. The Community Product Enablement team has been working with the Community Management team to prepare options that will mitigate the loss of these community run bots.

This may impact Stackers in that the Community Management team must pause all non-essential tasks to focus on moderating the sites, and other teams may need to adjust priorities to quickly redeploy to replace functions previously done by moderators. Please excuse us if we're slower to respond to inquiries or liaison tasks.

Philippe is partnering closely with Prashanth and the SLT on progressing the situation, and is working with the moderators to come to a reasonable and rapid conclusion.

Does this place us in direct opposition to the mods? Will we be "firing" them?

Not all of the moderators are planning on participating in this action. Philippe and other members of the Community Management team have already been in conversation with the moderators. We'll continue to work together with the moderators to come to a resolution that they can live with, without compromising what is best for the community, the company, and our customers.

We are not removing moderator diamonds from anyone unless they request to step down or violate our Code of Conduct or Moderator Agreement.

It's important to remember that these mods are doing what they believe is best in a tough situation, just like us. Our mods have cumulatively contributed tens of thousands of hours to the network sites and simply have a different perspective than us on some key issues. Nonetheless, we are certain our course of action (i.e. pausing the usage of highly flawed ChatGPT detectors) best serves the company, the community, and our customers.

How are we messaging this? Who is allowed to post and respond to questions and comments on Meta, chat, social media, etc?

The Community Leadership Team ([redacted]) are working together in close coordination with Marketing ([redacted]) on comms. They will post and respond to questions on-site. Unless you are specifically tapped to respond to something please do not engage. It is best to avoid commenting on anything related to this action on site, even if you think you have something helpful to add. Please get review and approval from Philippe prior to posting on site, or from [redacted] if you are approached off-site.

What if the Media contacts me?

If someone from the Media contacts you, do not respond. Forward the email to press@stackoverflow.com and someone from the Comms team will handle it directly.

Has anything of this nature happened before?

There was an action of this nature a few years ago, about unrelated events. It was settled fairly quickly, and the network moved on.

How big a deal is this really?

This is not our ideal state of affairs, but it's also not the end of the world. It will work out, and we anticipate a speedy resolution. This is also an opportunity for us to work with the mods to advance some of our own ideas about creating a friendlier and more welcoming site. In the meantime, while things could be a bit bumpy in the short term, we believe this will be quickly settled and we can move forward.

Will we be "caving in to their demands"? And what are their demands, anyway?

We have not been presented with any demands, and don't anticipate that we will until the action begins. At that time, we will evaluate their requests and determine the best course forward for our company, our community, and our customers. If they have reasonable requests that move us forward, we will certainly evaluate them.

I have other questions, or I need something unrelated from the community team. Who do I ask?

For other questions about these events that aren't answered here, please see Philippe and [redacted].

If there is something that requires immediate action or input from the Community Management team that is not related to the moderator action, reach out to [redacted].

My interpretation

SLT stands for Senior Leadership Team, which probably includes these people. I don't know who else, if anyone, is on that team. It appears Philippe, VP of Community, is taking on internal communication and probably is the primary author of these documents. It's hard to know how much influence he has over the policy at the center of the strike.

Nothing in the email itself is surprising. It's a watered down view of the situation that seems designed to reassure employees that the matter is in hand. There's no point in alarming the majority of the company who doesn't interact with the community each day. But this sentence stands out:

We do not know what it will take to end this action, nor do we know how long they are prepared to engage in this, but we will do our part to bring things to a speedy resolution; one that's best for our company, our community, and our customers.

By June 5, the strike demands were made clear in the strike letter (emphasis mine):3

Until Stack Overflow, Inc. retracts this policy change to a degree that addresses the concerns of the moderators, and allows moderators to effectively enforce established policies against AI-generated answers, we are calling for a general moderation strike, as a last-resort effort to protect the Stack Exchange platform and users from a total loss in value.

It's possible this line in the email was meant to signify that company leadership isn't sure what it would take to address moderators concerns. Or maybe it was allowing for the possibility that moderators could add more demands during negotiation. But when you add in the FAQ, I think it's more likely the company doesn't want to give employees a complete picture of what the strike is about:

We have not been presented with any demands, and don't anticipate that we will until the action begins.

Since the FAQ directly quotes from the strike letter (to answer "What does this mean?"), there is no possibility that the document had been written before the strike letter was made public. And since at least one demand (to retract the policy change that is the subject of the letter) was made, this answer is incorrect. Or, to be crass, the FAQ is lying to employees.

Strangely, the section about "firing" moderators obliquely refers to the key demand:

Nonetheless, we are certain our course of action (i.e. pausing the usage of highly flawed ChatGPT detectors) best serves the company, the community, and our customers.

Of course the moderators aren't demanding ChatGPT detectors, but being able to take any action against ChatGPT content at all.4 The public policy implies moderators can use the "low-quality post" standard. In practice, that's no help because GPT-generated answers look reasonable and only reveal themselves to be low quality when the content fails to answer the question. As Donald Knuth notes:

The most immediate impression is the quality of the wordsmithing. It's way better than 99% of copy that people actually write. It's definitely not like a Markov model that uses the most predictable way to continue what's already been said.

If moderators are unable to remove machine-generated content that happens to be correct, they must either leave it for unsuspecting visitors to get tripped up on or test each and every answer on the site for correctness. Non-programmers rarely appreciate the degree to which questions turn on details such as the exact version of languages and libraries. Moderating is time consuming enough without a requirement to create unit tests for every answer.

I mentioned the quality standards in point six of my steel man argument. People new to the network naïvely assume moderators are subject matter experts. It's easy to see why since they are invariably experts on some site subjects. Nobody can be an expert on every subject asked about on Stack Overflow. Highly upvoted and unanswered questions5 suggest nobody is an expert on some questions. Unfortunately, LLMs happily pretend they know the unknowable.

It's not surprising the company is avoiding the word "strike". As I noted previously, volunteer moderators are not currently classified as employees by the Department of Labor. If the company used "strike" language, that would indicate it considers them employees rather than volunteers. This is also why "firing" moderators is put in quotes.

The FAQ references the pronoun affair in order to dismiss it. This suggests the company failed to learn the lesson it should have three years ago. Despite being a company closely associated with a community, leadership was focused on other things (particularly an exit strategy for investors). When the Eye of Sauron leadership focus returned temporarily to the community, it assumed volunteer moderators can be directed as if they were employees.

It's hard to sympathize with the company needing to replace community-operated systems. For years leadership studiously ignored community concerns so that they were only addressed by individual developers (which buys a lot of community trust) or members of the community. Now that the community is on strike and their tools aren't working for the company, it'd be pretty handy to have the staff it recently laid off.

This is also an opportunity for us to work with the mods to advance some of our own ideas about creating a friendlier and more welcoming site.

The company has been working on this problem for almost 11 years. I'm not saying to give up on the idea, but maybe spend some time trying to understand why the problem persists. Conflict is inevitable in mature communities, so the important thing is giving communities tools to manage it. Stack Overflow has always had options besides blaming moderators, but that's the path of least resistance.

Over and over, these documents use the formula:

best for our company, our community, and our customers

There are small variations, but "best" invariably applies to those three parties. It sounds good since those three parties are pretty important in this ecosystem. The trick is that "company" and "customer" are fairly well-defined, but "community" isn't. The email and FAQ focus on moderators without mentioning the majority of the signatories who the letter categorizes as "contributors and users". That, combined with the mention of becoming more "welcoming", suggests leadership considers itself to be a rescuer in the drama triangle and the moderators as persecutors. But the company can't solve all (or perhaps any) of the community's problems on its own. Unfortunately, leadership lacks flexibility when it comes to outcomes.

To make matters worse, these internal documents promise "quick" and "speedy" resolutions. That's setting up employees to blame the moderators for an extended strike. I joked that "extrapolating the trend, Stack Overflow will have negative questions in 16 months or so". It's probably less funny6 for the company since it hits the bottom line. In public Philippe takes a more dire tone:

If there is no future for a network where mods can’t assess posts on the basis of GPT authorship (where we would be after this policy), and if there is no future for a network where mods can assess posts on the basis of GPT authorship (where we are today), then there is no future for the network at all.

This is a classic false dichotomy. One possible future (as of a couple of weeks ago) was for the company to discuss their data with moderators and collaborate on a solution that didn't involve sparking a "moderator action". It's clear time is not on the company's side, but it's not clear that the community is under any time pressure at all.

Previous posts about the strike:

  1. Both the communities and their host. ↩

  2. I went into detail about why that 11% number is misleading in a previous post. If we take the email's estimate of 600 total moderators (only off by 62), 11% of moderators is ~66 total. On June 5 at 08:16:18 GMT, 67 moderators had signed the strike letter. By 14:29:37, 78 had signed so we can narrow down the time to shortly after 0800 GMT when the email got its final draft. ↩

  3. Indeed the first draft of the letter makes clear retracting the network policy was the core of the strike's demands. ↩

  4. The most prolific moderator in terms of handling ChatGPT flags doesn't use detection tools. ↩

  5. There was an answer to my question which was deleted by a moderator as a ChatGPT answer. It seemed reasonable enough, but I couldn't understand it. I also couldn't test the answer myself since it'd been over 4 years and I no longer had access the machine where my code was running. The core suggestion was use another library, which might work, but doesn't answer my question. And yes, it was a ChatGPT answer since the user credited https://chat.openai.com/chat in the answer. ↩

  6. Assuming it's funny at all to anyone. ↩