OpenAI’s Sora 2 shocks world; but is it safe?
In this DML Report…
OpenAI's Sora 2, an advanced AI video generation model, produces longer and more realistic clips than its predecessor, generating viral content including a video depicting CEO Sam Altman shoplifting GPUs with his permission. Integrated into a TikTok-style app launched in the U.S. and Canada, the tool requires an invite code for access and has topped Apple's App Store rankings despite the limited rollout. This follows OpenAI's recent $6.6 billion share sale, valuing the company at $500 billion and solidifying its position as the world's most valuable private entity. The model's rapid development reflects OpenAI's shift from a San Francisco research lab to a structured organization prioritizing speed, as seen in its quick release of two new models shortly after China's DeepSeek unveiled a competitor.
Sora 2 incorporates multiple safety layers, including prompt filtering, output moderation for video frames and audio transcripts, bans on explicit content, terrorist propaganda, self-harm promotion, and impersonation of likenesses, alongside visible watermarks. Despite these, users have bypassed restrictions to create deepfakes, such as unsettling depictions of cartoon characters like Pikachu and SpongeBob SquarePants. Internally, OpenAI grapples with balancing these guardrails against creative freedom, fearing over-censorship could stifle innovation or damage perceptions, though leadership deems strict measures essential. The company trains on copyrighted material unless rights holders opt out, a policy facing legal challenges. Altman addressed concerns on X, emphasizing transparency and commercial momentum to support broader artificial general intelligence goals.
(read more below)
Facing competition from Meta's Vibes, Google's Veo 3, and models from ByteDance and Alibaba, OpenAI plans $850 billion in infrastructure and next-generation model investments. Video synthesis expert Hao Li noted that AI systems must process visual and audio data to advance toward general intelligence, with his lab using synthetic videos to boost model performance through data feedback loops. Former OpenAI executive Zack Kass, in his upcoming book "The Next Renaissance: AI and the Expansion of Human Potential," argues that early releases of powerful tech, despite trade-offs, outweigh secrecy, enabling public adaptation over withheld or private development.