Ansh Arora

Did You Miss My Comment or What?

Papers We Love (FOSS edition)

“Did You Miss My Comment or What?” Understanding Toxicity in Open Source Discussions by Courtney Miller, Sophie Cohen, Daniel Klug, Bogdan Vasilescu, Christian Kästner

Originally posted on forum.fossunited.org

I came across this paper a little while ago after having a frustrating discussion on the FOSS United telegram. I am also a part of discussion channels of a few open source projects where I regularly notice a lot of not-polite back and forth among maintainers, users and contributors, which from my experience is unique only to OSS forums.

In this paper the authors try to understand online toxicity in open source communities. The kinds, reasons and effects of toxicity across the internet are well documented, but toxicity specifically within OSS communities is not very well understood.

To this end, we curate a sample of 100 toxic GitHub issue discussions combining multiple search and sampling strategies. We then qualitatively analyze the sample to gain an understanding of the characteristics of open-source toxicity. We find that the pervasive forms of toxicity in open source differ from those observed on other platforms like Reddit or Wikipedia.

From an interview -

They used a toxicity and politeness detector developed for another platform to scan nearly 28 million posts on GitHub made between March and May 2020. The team also searched these posts for "code of conduct" — a phrase often invoked when reacting to toxic content — and looked for locked or deleted issues, which can also be a sign of toxicity.

In our sample, some of the most prevalent forms of toxicity are entitled, demanding, and arrogant comments from project users as well as insults arising from technical disagreements. In addition, not all toxicity was written by people external to the projects; project members were also common authors of toxicity.

image|540x292

Open Source and Toxicity

What is toxicity in the context of online discussions?

Toxicity, defined here as “rude,disrespectful, or unreasonable language that is likely to make some-one leave a discussion” is a huge problem online

Open source communities are not immune to toxicity. While the term “toxicity” as defined above has only recently started being used in the open-source literature, the presence of behaviors “likely to make someone leave” have long been documented by researchers and practitioners in this space. For example, the Linux Kernel Mailing List is notorious for having discussions with a tone that “tends to discourage people from joining the community”

Linus Torvalds acknowledges that he has at times been "overly impolite" and that is "a personal failing."

Toxicity is also a major threat to diversity and inclusion: prior work has found that it can especially impact members of certain identity groups, particularly women , who are already severely underrepresented.

I remember wanting to contribute to open source a little more than a year back and the tone of the day to day discussions would be a major factor for me to choose which project I would be like to be involved with.

Identified characteristics of open source toxicity image|690x252

Unlike some other platforms where the most frequent types of toxicity are hate speech or harassment, we find entitlement, insults, and arrogance are among the most common types of toxicity in open source. We also learned that many of the ways projects address toxicity are closely connected to the GitHub interface itself and open source culture more broadly, such as locking issues as too heated or invoking a project’s code of conduct.

I tried finding if there are any blogs from github on the measures they take to handle toxicity but couldn't find anything

Edit: https://docs.github.com/en/site-policy/github-terms/github-community-guidelines

Another ex-open source community leader, when explaining why they quit, described how “I had been told that I needed a ‘tough skin’ to work in the community, and I needed to ‘not take it personally’ when developers were abrasive during code review”.

Toxicity in open source is often written off as a naturally occurring if not necessary facet of open source culture. The aforementioned community leader describes how “When I complained about the toxic environment, I was told it was ‘tradition’ to be blunt and rude in order to have a truly open dialogue.”

Four high ranking Perl community members stepped down due to community- related issues. One of them, when elected as community leader in April 2016, set the goal to make the mailing list “a place on which we can have technical conversations without worrying about abusive language or behavior” however, in April 2021, he stepped down explaining how the “chain of continuous bullying and hostility I’ve been receiving” has caused him “significant emotional distress”.

(Types of) Toxicity observed on Github

Insults

Over half of our sample contained insults (55 cases), i.e., disrespectful or scornful expressions, often using curse words or intentionally offensive language. Toxic insulting comments tend to be targeted at people rather than at the code itself.

This is interesting because I've heard a lot of seasoned open-sourcerers say to new contributors that they should never take criticism personally and it's always about the code not the person. The contributor convenant which is one of the most widely adopted COCs explicitly states that criticism should be constructive.

For example, a user of a GUI crypto wallet with a built-in crypto miner noticed the presence of the miner and interpreted it as malware (a misunderstanding, the presence, deactivated by default,was mentioned as an intentional feature in the readme). The user threw explicit curse words at the maintainers of the project and accused them of being “criminal crooks” for trying to “infect other computers with malware”

A project member was unhappy with the colors of a project, reporting “colors are horrible for […], just look at this s**t” . Even after a contributor provided a link to the documentation, the user remained unsatisfied and unapologetic.

Entitled

Entitled comments make demands of people or projects as if the author had an expectation due to a contractual relationship or payment.

A user, upon being told that their suggestion was based on a misunderstanding of the project, began aggressively criticizing the contributor for how they addressed the issue, saying “Like just add the flavor text or show me how to or something. Don’t just fu**ing close people’s tickets they would like some help on”

Arrogant

We consider comments as arrogant when the author imposes their view on others from a position of perceived authority or superiority (earned or not) and demands that others act as recommended

One of the users in the discussion was unfamiliar with some of the legislation being discussed and asked for more information, a second user responded saying “Never hear about [standard]? A baseline for developers. Use Google.”

Trolling

For example, a user was generally unhappy with a project and wrote “Worst. App. Ever. Please make it not the worst app ever. Thanks” (I2), followed by a pull request that deleted all the code in the repo; after the main-tainer closed the issue, the user responded “Merge my PR damnit” and nothing else happened.

Unprofessional

Comments that are not overly toxic but nonetheless create an unwelcoming environment.

Examples include self directed pejorative terms (e.g., “It seems like I have been acting like a re**rd. Sorry. […]” ), self-deprecating humor, and jokes and puns with explicit vocabulary or terms broadly perceived as politically incorrect or unacceptable in a professional setting.

Triggers of toxicity

Failed Use of Tool/Code or Error Message

Some comments actually report the problem in some detail to help the project or receive help with their immediate problem, but still include toxicity, typically expressing frustration.

For example, as one such user of a popular library puts it, “I just tried reinstalling your buggy, sh**ty software for the third time. Maybe you guys can get one that works right and stick to it without changing it all the time” .

Yet, in other cases, users simply vent about problems without seeking help or any attempt to provide constructive feed- back to the project.

In some cases, the users respond with toxic messages when asked for more information or asked to follow the issue template,

for example “Yeah, not really sorry i’m lazy, and it’s more to help you then me. It’s simple to understand: […]. don’t need a ret**ded format to understand that! thanks”

Toxicity triggered by failed tool use is often entitled, insulting, unprofessional, or just trolling e.g., “It doesn’t work. F*** this”

Politics/Ideology

We fairly frequently observed toxicity arising over politics or ideology differences, e.g., referring to specific beliefs about open source culture, processes, or the involvement of specific companies (especially Microsoft was a frequent target in our sample)

For example, a user wrote a hostile issue in a Microsoft project titled “WHY ⁉⁉⁉⁉” which simply said “Revenue. F**k you guys”

Past Interactions

Finally, we observed several cases where toxic comments were posted that referred to past interactions of the author with the project, without continuing to discuss the previous technical issue, but shifting to personal attacks, complaints, or meta discussions about process.

For example, a user was unsatisfied with the response time on an existing issue so they created a new one asking “did you miss my comment or what?” These comments were often posted in a new issue after the old one was not answered or closed, and they often occur in the opening comment of the new issue

Authors of toxicity

image|451x197

Project Characteristics

Project size

Several toxic comments in small projects appeared to be trolling or jokes among friends, e.g., “Dear Mr. [project owner name], Could you perhaps please get your s**t together and reincorporate the brilliant switch statement once again, bitch. XoXo, [author]”

Project domain

After toxicity: harms and reactions

Tools to curb discussions

Reactions

When maintainers invoked the code of conduct, the author of the toxic comment usually did not engage any further. However, there were also a few cases where the author pushed back on being policed in their speech

"Again. No discussion allowed. No critique allowed. Just pushing fingers into the ears and singing. To avoid hearing about the impending doom, to avoid hearing the truth about the quality of this project”

In one case, a user called out for violating the code of conduct responded insisting “I will neither change my language, nor my tone or style. Both, language and tone, are perfectly valid, given the circumstances. I will remain myself, and will repel this attack to my individuality”

referring to invoking the code of conduct as “CoC-Fascism,” upon which projects members banned the user.

Discussion and Implications

Liked this post? Subscribe to my blog via email or RSS feed


#foss #notes #summary