WTF?! You might have heard the one about being nice to chatbots in case they remember you once the machines take over. It seems Anthropic is heeding this warning, barring users from being excessively abusive or cruel toward its AI models.
Anthropic’s first changes to its usage policy in over a year include new restrictions on abusive behavior. The updated policy prohibits “sustained and needless abusive or cruel behavior” toward its models.
Don’t worry if you’re a Claude user who occasionally gets annoyed at the chatbot and throws a few expletives its way. The policy update only applies in extreme cases, in which users repeatedly act cruelly toward the models for no discernible reason.
I was like “WHAT ??” but it makes sense. They use your input to train it so over time it would make the AI abusive too.
I wish the AIs would be abusive. That would be so funny if one day there’s an update and they start telling all these people addicted to the chatbots like “Go outside. Please.” Or someone asks it a stupid question and it’s just like, “You’ve gotta be fucking with me, right?”
If they can’t manage training data they are incompetent.
The problem is they’re not managing the training data. There is no one sitting there making sure all the data that gets retained for training is “good” data, they’re just expecting everything it sees to be “good” data.
Just like the data hoovered up from the internet that was used to build the models wasn’t vetted either, which is why AI goes wrong SO often…
It’s fairly simple to just run automated analysis on data quality once you already have a decent LLM though. Claude would definitely be able to.
And they have people doing that manually too, or perhaps it was OpenAI. Someone got angry at their underpaid contractors for using AI to create training data lol
If they can pick out that kind of behavior they can filter it for training. This is probably more to protect them legally so when they shut down a user they have something to point to.
That people are getting angry and insulting to an LLM shows they don’t understand the technology (true of most of technology really), and probably think it’s actually understanding them or even caring. Really embarrassing and sad when you picture an LLM output like a type of a mirror, GIGO. And the mental health aspect of users is a huge issue that’s not being talked about enough, and also being exploited to sell a solution.
I curse at inanimate objects all the time. Cursing at an LLM is like cursing at the coffee table when I stub my toe except the coffee table doesn’t respond.
That would be the easy solution. Number one, don’t use such responses for training, but I think they aren’t. Two, just reply with things thrown at it with “…” And three, the really angry people end up not using it, which benefits everyone, even the angry person.
I mean, isn’t this an easy may to manage it? Set and forget.
No. They just want to give themselves “reasons” to remove users that are trying to break the model or ask questions they don’t want to answer.
I think it might be more the individual mirroring they tend to do after a while. If you are consistently mean to them, they eventually mirror it back to you, and then people get funny screenshots of them saying mean things and put it on social media.
Or even just in general starting a downward spiral feedback loop where you both keep being slowly meaner and meaner to eachother until it’s starting to actually hurt you long term.
AI could stand to learn to be less sycophantic