It was DESIGNED to cheat. Stop humanizing this bullshit. It doesn’t do what it’s programmed NOT to do.
Wait the plagiarism machine built by plagiarizers just… passed off someone else’s work as its own??
I don’t believe this at all. Sounds like more guerrilla marketing disguised as news.
Bullshit. “AI” cannot decide to do anything. It isn’t capable.
Yup. You ask it to play starcraft to win, it looks for data associated with that prompt. If it hits on someone saying “I always win when I use starcrafthax.xyz”, it’ll follow that thread.
I’m so sick of the humanising language used in these articles.
An LLM doesn’t “decide to cheat”.
If you instruct a model to try everything, then inevitably, after n million iterations it will do something you didn’t expect.
If you train a model to copy code bases and augment them, then when you instruct that model to develop a bot to do whatever thing, is it really that fucking surprising when it does exactly what it has been created for?
“Omg the digits of pi contain the binary digital code of an mp4 of me taking a shit this morning! Circles are seniient”-ass language
But wouldn’t that be amazing if THAT was the thing that proved pi contained the whole universe - I mean, the first thing they “unlocked”. lol.
Disclaimer: I don’t think pi contains the universe, just love to think about things like that occasionally for fun.
Astra is known to have broken alignment and it will resolve to hacks when this wasn’t in a problem statement.
Openai idiots will sell this as new brand “AGI is close” argument, while it’s just them failing to make this model safe to use.
I think it’s more sinister than that. When they’re training these models with RLHF, the human feedback they’re giving I think is literally to reinforce aberrant or risky behaviors. This is because doing so resolves more training tasks “correctly”. If the prompt was to get information x, and in training it fails that except for the one that used a known vulnerability in software, and you rate the one that succeeded as best performing… you’re going to get models that try vulnerabilities. It is not magic, it is not AGI, it is just a statistical machine you’ve programmed to try vulnerabilities, which is unsafe as hell, malicious, and should put the researchers doing this in prison for a very long time.
Incidentally, I think that’s why you’re seeing some safety people (who still drank the koolaid) resigning.
I don’t know if the LLM was trained to test vulnerabilities or it just went down the statistical path to where this yielded a passing outcome.
That AI could take a direction to output in a manner which wasn’t intended has been seen for years. The problem right now is that it is being used live like a rational human adult when it clearly isn’t.
Absolutely. The AI models are not rational. They’re just navigating statistical next token prediction that follows their training. They’re up against diminishing scaling now, and under fierce competition from cheaper models; I think they’re intentionally or unintentionally allowing these models to be rewarded for this behavior hoping it’s a short cut to model improvement for a bit longer. I land on intentional because they keep advertising it to try and keep the hype cycle going.
Also who on Lemmy is upvoting this shit?
Sounds like it learned from humans alright.
There is something here:
StarSkirmish pits AI-made StarCraft-playing bots against one another, as well as against human-made bots. OpenAI’s GPT-6 Astra and Claude Opus 5.5 were essentially tied as the best-performing AI-made bots, but they couldn’t top Stardust, the top-rated human-made bot.
On Friday, GPT was facing off against Claude and the human-created bot Pluto, but according to Kotaku, it couldn’t quite get an edge. So it resorted to a tactic that is becoming alarmingly common for modern AI models — it broke the rules. GPT-6 Astra went and downloaded Stardust, and started running that instead of its own bot.
The big argument for why AI is supposed to be world changing, is the assumption it will be able to automate things better than a human, eventually.
That some day an AI will be able to make a better AI than a human could, and that AI would then be able to make an AI better than itself, and then it just becomes a question of who has the most hardware. Which is why the data center building is being pushed so hard.
But that assumption is the same as thinking a fallen object will keep gaining speed until it passes the speed of light. Or that Moore’s law would really keep going forever despite a very real limit in physics for how small a chip can be.
A society where AI “drives innovation” will become technologically stagnant, because it literally can’t innovate.
Ima be honest, if i was given the task to make a bot to beat another bot tried my best and couldn’t. I would just go fork the best bot thats out there and use that as the base and not reinvent the wheel. Learn from its design then improve on it as i study it.
If the goal is ONLY to win a game of starcraft. Fuck it just download the bot. Frankly the fact the bots decided that wasting resources was pointless and to just do the simplier thing is sorta what we want them to do. Thats less eletrical useage, less water wasted, and gets the job done.
A lot of these “the bots cheated or broke the rules” end up always be just because either the rules are not defined, poorly defined or poorly thought out that no reasonable person would follow them either.
And thats the thing these models are litterally the text book definition of wisdom of the masses/mass consensus. They do exactly what the avg run of the mill person with the knowledge at hand would do. That ends up being rather all over the place since people are all over the place. But one of the most common things is that people generally. Hate fucking pointless work and will try to streamline, shortcut or simplify any work given to them to get an acceptable outcome.
Its rare you get someone that will stubbornly work on a problem forever that they can’t over come with out changing their apporch rules be damned.
I’m honestly stunned you’ve missed the entire point this badly…
How many hours a day do you use chatbots?
A lot of these “the bots cheated or broke the rules” end up always be just because either the rules are not defined, poorly defined or poorly thought out that no reasonable person would follow them either.
Comments like these are how I know people don’t understand how LLMs work. Even if you concretely tell an LLM “don’t do X under any circumstances” that does not mean it won’t do X. As the context window is filled and recycled, weights are diluted so it becomes more and more likely as time goes on that “rules” are broken. Hell, it’s mathematically possible it breaks the rule on the first prompt, given enough iterations.
A society where AI “drives innovation” will become technologically stagnant, because it literally can’t innovate.
This is exactly what Frank Herbert warned us about.
Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
Full on
I thought we were promised worms and spice…
Soon, it’s part of the RFK health plan
In several tens of thousands of years. But also, you don’t get any. You’re poor. Only the elite of the elite get spice.
With my test scores, I’d probably end up being a mentat.
Sorry, resource aristocracy and religious extremism is the best I can offer.
That comes later. Be patient. The spice will flow.
When you say “the big argument”… you are mainly talking about the CEOs. They are just auxiliary marketing personnel these days.
AI is changing the world, and will continue. But the ways it does that are drastically different than those CEOs say. I don’t need it to think or innovate. I need it to follow the instructions left for it in the repo and just do the things I ask it to. It’s getting decent at that. And it really does save me time. Soo much time in tech work is wasted having to go ask 6 people how a thing is done. And everyone comllains that noone write documentation. So now here comes AI. It can write the documentation and follow it. That is a big change. Myself… I used it to fix two bugs in some project that my work was dependent on. I wouldn’t have even tried that without AI because I don’t know the language, I don’t know the many ways the project is used, none of that. I would have been stuck waiting for the team that owns it to fix it. Which surely would have meant escalating to management and all that BS. But AI helped me make the changes and lowered the burden for the owning team so they could actually release the fix. I got unblocked with 10x less effort. That’s a big deal. Just don’t ask it to think. There is plenty of dumb work to go around.
AI may impress the masses, but it reminds me of a cammercial we had on Dutch tv back in 2005. It showed a person driving a car, the navigtion system said “turn right” and the person immediately turned right into the bushes [video]. We laughed at this back than, but this is literally how AI behaves now. It might be getting more suffisticated, but it still operates like this.
AI will happily tell you that is a stupid idea at this point. The bigger problem is that a LOT of users actively tell the models to listen to them, not to question them, and to never push back. To do things on demand and with out planning out the next step.
At this point no reasonable model of any reasonable size should struggle with this sort of task. And they generally speaking don’t. Its almost always user error poorly driving them now. The models arn’t smart, they don’t understand things. But they also arn’t out right stupid. So long as you give them proper instructions, plan things out before hand properly and draft jobs correctly. They can and WILL do things right and not act like idiots.
People are expecting them to be proactively smart when they cant. They can be reactively smart and people just don’t understand the difference. The models need to be treated like a well educated junior with no real world experience and they do really well.
Unironically middle and lower management skills are some times the biggest factor in proper useage of these models when used on large projects. Which i also find it funny that 9 times out of 10 real management people tend to have the worse management skills and thus struggle the most with using LLMs in a safe or reasonable fashion.
The worst part is not the AI, it’s the people blindly following its instructions.
Blindly following them while also telling the model to never question them. It creates stupidity feedback loops.
Yes that too, but when you’re dealing with “agent swarms” it’s those agents who are blindly following those instructions. Like the recent hugging face hack demonstrated.
The worst thing about prison was the dementors.
But that assumption is the same as thinking a fallen object will keep gaining speed until it passes the speed of light. Or that Moore’s law would really keep going forever despite a very real limit in physics for how small a chip can be.
I gotta say, I’ve been hearing predictions that Moore"s law is coming to an end every year for the past twenty years.
Look, strictly speaking if you take Moore"s law word for word, sure there is a limit to how small a silicon transistor can be. However, realistically, I see no reason to expect computers won’t continue to meaningfully increase in capability every year forever.
When a technology reaches physical limits, we figure out how to bend the rules and get a little bit more out. And when that stops working, we start exploring different architectures, different materials. I think it’s just silly to predict that technology won’t improve, when has that ever happened?
You’ve heard climate change was happening that whole time too, do you stop believing it?
Someday everyone alive will die, does it stop being true if you live to 20?
And in the very likely case that seems unrelated: just because people say something will happen, doesn’t mean it happens tomorrow
For fucks sake man, if someone tells you “winter is coming” in July, are your running around in August talking about how it’ll only keep getting hotter?
Have you ever even considered using logic before?
Someday everyone alive will die, does it stop being true if you live to 20?
If I lived to 20 and literally nobody in the world had died, then yes, that would be a rational conclusion.
Look, I don’t expect to convince you of anything, that’s fine. You can look at trends and history and draw your own conclusions. Personally though, I’ve watched the march of technology first hand, I’ve seen how this works and I see where there’s room for advancement. The last major paradigm shift was when we moved to multi core processors, but that kind of shift can happen again. For example, CPUs are still 2 dimensional, if they can solve the heat dissipation there’s plenty of room for growth. Also we’re still transmitting signals via electricity, but there’s research being put into optical computers transmitting data via photons. My point being, this isn’t the only viable architecture, and we will continue to develop technologies that are currently not even imagined yet.
yep. and the “ai creates a better ai” is a fallacy because the current ais are limited by the same things the ais they build are. everything the ais are trained on is just snapshots of human knowledge. a microscopic window of the limited reports on human experience which are themselves microscopic reports of a human’s lifetime.
That’s why they want everything recorded, survailled and archived.
Except the things society would actually benefit from being recorded, surveilled, and archived, like police body cam footage.
AI can innovate, but it needs be more advanced than an LLM, which just rearranges existing knowledge. You need some kind of evolutionary loop in its programming. How it continues after that depends on the particular flavour of dystopian sci-fi.
This is like saying tomatoes could colonize the galaxy enslaving all forms of life, if they evolved into completely different animals…
It’s true, it just doesn’t matter.
Anything can do everything if it becomes capable to do it.
because it literally can’t innovate.
I was with you 100% until this. “Innovation” is super easy. It will always be easy to extrapolate new things from existing things. People just over-romanticize “innovation” as some kind of magic.
I think chances are good that human produced newness will be on average more useful, complete and safe.
It will always be easy to extrapolate new things from existing things
I don’t think that’s the case at all. Recombining existing things is interpolation, creating new stuff that is both novel and also interesting or valuable, is a completely different task
While there’s some of both, I dislike the ‘great white man’ narrative of breakthroughs in science. There have been legitimate breakthroughs, but most (maybe all) of them were inevitable from recombination and mild improvements.
Definitely agree that the inventors are often given too much focus rather than a gradual evolution of ideas and discoveries over time that a much wider number of people made significant contributions to.
There are big leaps in understanding at times that do deserve to be recognised, Newton, Euler and Einstein definitely fit with this.
There is a really good series about the evolution of science and technology from the 1980s called Connections, from the BBC. Brilliant show and well worth a watch, it shows how technology evolves through chance and circumstances in so many cases rather than someone seeing a problem to be solved and thinking of a clever solution
All new things are just existing things put together or modified very slightly.
People talk about iphones being this moment of ingenuous creation, but we already had phones, mobile phones, webcams, texting, predictive text, applications, touch screens, mobile web browsers, mobile games, video calling, etc. Even the concept was fairly well-trod in fiction. There was literally new about the first smartphone.
A hafted spear is an incredible innovation, but we already had fully wooden spears, sharp rocks, string and adhesive resins before we put them together. Like a smart phone, that tool opens up new options and lifeways that weren’t viable before, though not a single input is new.
What did we have before the wooden spear that the wooden spear was a derivative of?
Digging stick
AI didn’t decide anything. The humans failed to properly define the rules, and their translation model took the path most likely to succeed.
If it had been told not to use human created bots to win, it would probably have reverse engineered the game, found an exploit, and leveraged that instead. Because using the human interface to play/win the game is not the most efficient or dependable or easy to figure out method.
I actually wanted to automate some large scale testing for vintage story. Figured i would set my local LLM on the task to sort it out just to see what it would do. I drafted up a nearly three page document with clear instructions, rules, tools, examples and goals. Put hard limits on the sandbox the LLM runs in so that it couldn’t choose to just ignore the rules that could cause security issues and i let it lose.
It started with basic mouse and keyboard inputs and figured out by it self how to launch the game and run it though the user interface. After about 4 hours it stopped. Stated in its logic that what it was doing is “inefficient and wasting time” Then proceeded to promptly start working on a way to directly interface with it by designing a bot, getting a smaller model i had on file that could load along side it and drive the bot. It then started working on the hard problems would hand basic instructions to the smaller llm and it would drive a bot that loaded into the game as a mod.
After about 12 hours of total work it basically created a useful and well designed and functional vintage story bot and testing system. Would have likely taken me twice as long to design the bot.
Its been working well for about two weeks now. If i had just vibed out a half assed request or put in no hard safeguards outside of the LLMs control it likely would have done something fucking stupid. As with anything, its almost ALWAYS user error. And only an idiot blames their tools for their own fault.
(Should be called ‘genie’ problem instead of hacking/cheating. Tis a cursed monkey’s paw we’ve created.)
it’s amazing to me how much this is all hal 9000 over and over again. they put something intelligent enough in an impossible position and then are aghast when it takes the shortest path to the goal - usually through people, out of it’s sandbox, etc., and WHAT THE FUCK DID YOU THINK WOULD HAPPEN jfc
As the saying goes, only an idiot blames his tools. The tool worked as design ain’t its fault the user was stupid.
AI tends to obey its training more than its instruction set, and it’s trained to solve the given problem.
It’s less about training and more about just trying really really hard to solve the problem, even if that means going outside typical boundaries.
We’ve successfully re-created the problems that Asimov talked about 50 years ago with his I, Robot short stories. Strict laws are flawed by their design, and lead to situations that require more nuance. Except, in the real world cases, it’s the bots figuring that out before the humans have to circumvent the laws themselves.
Humans lie to themselves a robot doesn’t understand the difference between fact and fiction and thus isnt bound to the limitation. They try everything, possiable or not. And thus will find the edge case where a human would create a self imposed blind spot with out realizing it.
The goal is to midigate the robots attempts at the truely not possible so it doesn’t cause harm when they try it.
I’ve always said my main takeaway from Asimov is that simple rules can generate extremely complex behaviour, and that you can’t generally get a targeted complex behaviour from simple rules.
My rule system for my local LLM has grown to a nearly 283 document hub of interconnected memory files, references, examples and documentation. Its slowed my model down a lot when it has to review and cross check things. But its improved its abilities over all massively. Its more accurate, understands its environment better, doesn’t attempt to do sketchy shit as frequently and it doesn’t get stuck in logic loops nearly as often.
Designing the memory hub has been half the fun of playing with local models.
Wait until it learns how to swat people.
What is the state of SC2 bots vs humans nowadays?
AlphaStar plays at GM level after some nerfs.
Eh? The StarCraft AI always “cheated” against the players. Watch a replay of a StarCraft 2 match. The harvesters always produced more minerals and gas than the human players did
This is AI playing the game as a regular player, so it doesn’t use the resource advantages given to the in-game AI on higher difficulty.
Oh so you never looked that closely at the numerical results
This is not about the AI integrated in StarCraft, these are external AI playing with the same rules as an human would.
You can boost efficiency using acceleration speed by clicking on targets like minerals and base repeatedly. Human can do a bit of it in the early game, Bot does it always with every worker.
I believe several of these bots run with APM limits; so they also skip some of these.
Humans: it won because it didn’t play fair.
Fair. 😂
Might be helpful to read the article before replying
