Last month, Washington insisted that training models on other people’s work without permission is so obviously fair that licensing requirements would be ruinous to national innovation. It also insisted that training models on other people’s work without permission is an act of industrial predation posing a grave threat to national security.
Both propositions cannot hold simultaneously. Either the outputs of these systems are protectable under the law, in which case the writers, musicians, artists, and programmers whose work American models were built on can hold a claim that the Justice Department has denied. Or they are not, in which case the six Chinese laboratories accused of “malicious and targeted” distillation attacks have not done anything that Elon Musk has not freely admitted to under oath, without consequence.
You clearly can, LLMs are just an extremely lossy method of compression. It does make one wonder at what point does a highly compressed version of an original work no longer qualify. If we compressed a licensed image enough that it was a blur that vaguely resembled the original, does that still count?