• brucethemoose@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    1
    ·
    edit-2
    9 hours ago

    I thought this would be nonsense at first, but the methodology is interesting. It seems plausible. And they tested quite a good spread of open weights, like MiMo and LongCat (which usually fly under the radar), all the way back to Yi 34B.

    They missed Russia’s LLM, but that’s okay; it’s very obscure.

    It’s also “lucky” they found this correlation, and so many recent weights happened to be initialized with PyTorch, on the GPU: very specific conditions.

    …Also, I think the FP4 precision is a dead giveaway too, as Hopper wouldn’t be able to train that effectively, right? I assumed they were using at least some smuggled B200s.


    And the export restrictions were stupid anyway.