Last week, former Starbucks CEO and 2020 presidential hopeful Howard Schultz drew outrage—not over policy positions or campaign slogans or even his company’s chronically burnt French roast, but word choice. In a clip from a January CNBC Q&A that surfaced on Twitter, Schultz was asked whether he thought that billionaires had too much influence on American public life. He responded, “The moniker billionaire now has become the catchphrase. I would rephrase that, and I would say that people of means have been able to leverage their wealth and their interest in ways that are unfair, and I think that … directly speaks to the special interests that are paid for by people of wealth and corporations who are looking for influence.”

To me, this quote and Schultz’s larger statement show him relatively clearly, if weakly, responding that rich people do have too much influence and that fixating only on billionaires would be too narrow of a focus. But that was not the consensus of the progressive internet. Instead, many came away with the impression that Schultz thinks that billionaire is a pejorative, and that we should all be nicer to folks like him by using the softer people of means.

Regardless of what Schultz really meant, it’s worth considering why we are so ready to hear people of means as a slimy obfuscation rather than as a neutral rephrasing or expansion of the category of person in question. Because that readiness speaks to a larger linguistic problem that has implications far beyond the primaries.

The “people of/with x” formulation—wherein people who have some quality, like size or disability, are condensed into a solid noun—has become increasingly common (particularly on the left) since the 1990s. The sentiment behind that semantic shift is the same one that underlies the move from terms like “victims of HIV” and “homeless” to ones like “people living with HIV” and “living unhoused,” respectively. Or the move from “disabled people” or “handicapped people” to “people with disabilities.” It is a euphemistic linguistic model that intends to center humanity separate from situation or identity, and it is a model that creates new terms that are supposed to, as John McWhorter wrote for Slate in 2016, “rise above pejorative connotations that society has linked to the thing in question.” Its biggest success story might be the phrase people of color.

While people of color may not have risen to the same level of prominence as minorities just yet, there is a growing sense that the former should replace the latter when specifically referring to people who aren’t white. Minority, along with nonwhite, necessarily defines people by a negative, as lacking some quality that would place them in the majority category. (And as American demographics continue to change, there’s a possibility that minorities will become as inaccurate nationally as it has always been globally.) Similarly, the now-passé colored people held associations with the state-sanctioned apartheid of the Jim Crow South, where roles in public life were defined by whether one was colored or not. Enter people of color. While the phrase has existed long before its current heyday (appearing as far back as 1807 in legal records), it seems to have begun its modern ascent in the late 1980s. A 1988 New York Times piece on the phrase describes a comic strip that suggests people of color as a “new-age” replacement for colored people. “Politically, [people of color] expresses solidarity with other nonwhites, and subtly reminds whites that they are a minority,” wrote columnist William Safire.

These are noble origins. But for all the good intentions behind it, the success of people of color has brought with it a strong potential for misuse. In our modern discourse, the phrase has come to be thought of as both the most courteous way to refer to a nonwhite person and a signal that its user is down for the cause of racial justice. It has become depressingly common for a well-meaning white person to, despite my fairly conspicuous self-identification as black, refer to me as a woman or writer of color. In that choice lies an uneasiness, either with referring to me as black—despite its accuracy—or with the potential of misidentification of my race. In either case, person of color on some level serves to make the (typically white) speaker feel better, rather than me, the person whom the terminology is theoretically for.

In many spaces, the term functions now as performative fauxgressive politeness—as one of the many buzzwords such as intersectionality or systemic that one can drop, with little understanding, to display her wokeness. In its presence, more accurate terminology is forgone because it feels easier and safer (mainly for white people) to just say people of color.

Take, for example, Rolling Stone describing Sen. Tim Scott, in a recent article on Republican support of noted racist Rep. Steve King, as a person of color rather than as a black man. This is a choice that, at the very least, creates a lack of journalistic clarity since, as the only black Republican senator, Scott’s toeing of the party line with regard to King’s racism is particularly newsworthy. What’s needed here is specificity, not genteel ease—and that’s not the only case where people of color elides crucial detail. For example, using people of color when discussing the history of chattel slavery or police brutality flattens the specificities of anti-black racism in America. Using people of color when referring to the genocide of native and indigenous people in America obfuscates particular histories of colonial violence. Suggesting that newsrooms or corporate boards need to hire more people of color when there are specifically no Latino people or Southeast Asians on the payroll suggests that any nonwhite person will do, that we are all the same and bring the same experience to the table.

The swift and intense reaction to Schultz’s “people of means” suggests, to me at least, that we have become sensitive to the misuse of this formulation. We know, on some level, that people of color and its cousins have evolved from a compassionate shift in our linguistic paradigms to tools that people in positions of racial or other kinds of power can use to appear politically sensitive while doing little, if any, of the actual work of social justice. In POC’s case, what was partially meant to support a sense of radical solidarity between different marginalized communities has been so watered down as to be comfortable in the mouth of someone who either wore blackface or a Klan hood, or thought either was worthy of appearing in his yearbook page. It’s a term that, in many ways, still centers whiteness and suggests that anti-blackness doesn’t exist in Latino communities or that anti-immigrant sentiments don’t exist in black American ones. A term that has happily been co-opted by vice presidents of diversity who think there is a way to make a space welcome to nebulous “people of color” without addressing issues specific to different communities.

This is not to say that people of color has no place in our lexicon anymore. There are times and places where it is the most accurate term—when discussing the need for diversity in the largely white publishing world, for example. But we cannot allow people of color to erase specificity for the sake of ease, to suggest that calling someone black is somehow impolite or to allow those uncomfortable with blackness to obscure their discomfort behind “progressivism.” There’s no question that the move toward the “people of/with x” formulation was meant to confer humanity onto those who have been dehumanized. But now, it’s increasingly apparent that the communities this linguistic shift was supposed to dignify might no longer be the primary beneficiaries of it. The pendulum of sensitivity feels like it’s swinging away from marginalized communities and toward the comfort of the powerful. That’s partially why Schultz’s word choice, whatever he actually meant, rankled so many. The only way to bring it back in the proper direction is to get rid of the hiding places in the language we use and to use the words we really mean.