After all that hype, AI scanner found one low-severity cURL flaw

cURL developer Daniel Stenberg has seen Anthropic’s Mythos, a model the AI biz has suggested is too capable at finding security holes to release publicly, scan his popular open source project. But after the system turned up just a single vulnerability, he concluded the hype around Mythos was “primarily marketing” rather than a major AI security breakthrough.

Stenberg explained in a Monday blog post that he was promised access to Anthropic’s Mythos model - sort of - through the AI biz’s Project Glasswing program. Part of Glasswing involves giving high-profile open source projects access via the Linux Foundation, but while Stenberg signed up to try Mythos, he said he never actually received direct access to the model. Instead, someone else with access ran Mythos against curl’s codebase and later sent him a report.

“It’s not that I would have a lot of time to explore lots of different prompts and doing deep dive adventures anyway,” Stenberg explained. “Getting the tool to generate a first proper scan and analysis would be great, whoever did it.”

  • nomad@infosec.pubEnglish
    111·
    16 hours ago

    As far as i understand it, its not all hype. Its a little bit like having a really competent security researcher go deep through your complete codebase just really fast and with improved recall.

    So no black magic, just stuff regular security reviews would find. Firefox is just a huge codebase and once a bug got past review it might stay there forever.

    So this will be abused if released publicly sooner or later. This way is a little bit like responsible disclosure. This will make the initial wave hurt way less. And obviously it doesn’t hurt marketing.

    Anybody working with software knows marketing people promise the world and understand nothing. Pretty sure they just heard “black magic” and ran with it.

    • MalReynolds@slrpnk.netEnglish
      9·
      12 hours ago

      Its a little bit like having a really competent security researcher go deep through your complete codebase just really fast and with improved recall.

      I doubt that, more of a force multiplier for security researchers at this stage (perhaps always for LLMs without an architecture leap) IMO. Otherwise I generally agree. It’s responsible to take this approach perhaps, but mostly marketing. Still let’s not kid ourselves it isn’t happening at scale already. Plenty of open weights models can also force multiply a competent security researcher, either black or white hat. Mythos isn’t a quantum leap or anything, just 4.7.

      Anybody working with software knows marketing people promise the world and understand nothing. Pretty sure they just heard “black magic” and ran with it.

      Heh, yup.