So remember this thread of mine?:
https://old.reddit.com/r/artificial/comments/1aws29z/will_hollywood_completely_cease_to_exist_very/
Well, when it comes to animated films, there were these articles about them and Sora:
OpenAI’s Sora Creates Minute-Long Photorealistic Animation From Text Prompts
Around this time a year ago, Stable Diffusion launched an early text-to-animation tool and the internet was obsessing over an AI-generated Seinfeld knockoff. This week, OpenAI has demonstrated how far animation-generating technology has come over the past 12 months with the unveiling of its new model Sora, which can create photorealistic and cg-style animation of up to one minute based on text prompts.
What is Sora? Sora is a generative AI model that uses text and image prompts to create videos and images of varying durations, aspect ratios, and resolutions, up to a full minute of high-definition video. The model can also take an existing video and extend it or fill in missing frames.
According to OpenAI:
Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt but also how those things exist in the physical world.
How does it work? Sora was built on past Dall-E and GPT models. It borrows Dall-E 3’s recaptioning technique in which the model generates highly descriptive captions for training data. When users enter prompts with similar language, the model knows to use corresponding training data when creating a new image or, in Sora’s case, a video. The Sora team says, as a result, “the model is able to follow the user’s text instructions in the generated video more faithfully.”
A more detailed explanation is available in a technical report published this week by OpenAI, available here:
https://openai.com/research/video-generation-models-as-world-simulators
What makes Sora different than other animation-generating models? According to OpenAI, Sora stands out for the impressive photorealism and length of the generated videos as well as its ability to closely adhere to user prompts. Sora can do this because the model plans out its videos many frames at a time, giving it greater foresight than previous programs and allowing it to remember what characters and objects look like through the duration of the generated videos, even if they leave the frame temporarily.
What are Sora’s shortcomings? Sora can struggle to accurately simulate physics in more complex scenes. OpenAI also says the model doesn’t have a strong understanding of cause and effect, which can create undesired or unrealistic outcomes. The model sometimes becomes confused regarding spatial details that require perspective, such as which way is up or down, left or right, and it can have trouble with timelines described by specific prompts.
How is OpenAI preventing this software from being used in harmful ways? The company says it is working with experts in misinformation, hateful content, and bias to test the model and build tools to detect misleading content, including a “detection classifier” that will indicate if a video was made by Sora. OpenAI says that Sora will have similar limitations to the company’s Dall-E software, which doesn’t allow prompts soliciting violence, sexual content, hateful imagery, representations of real people, or IPs belonging to other parties.
When will Sora be available to the public? Sora is currently only available to a few select creators and security experts who are checking it for safety vulnerabilities. OpenAI plans to make the model available to the public sometime in the future.
Examples: On the Sora website and accompanying technical report, numerous example videos and their corresponding prompts are listed. We’ve included a few here, but there are dozens more.
https://www.youtube.com/watch?v=Xl2o_iBCINk
Sora And The Future of Animation Industry
What is Sora (by OpenAi)?
OpenAI just rolled out another artificial intelligence powered tool called Sora and I have to say it is pretty mind-blowing. Picture this: you type up whatever scene pops into your head, hit enter, and bam, there’s your video. Turning thoughts into visuals with a few words. But as cool as it sounds, it’s got us in the creative industry doing a bit of a double-take. Sure, it’s a game-changer for making content but it should get us all creatives thinking and talking about where the lines are drawn.
How do we balance this incredible power to conjure up videos and animations from thin air and make sure it is used wisely?
Sora and Future of Creative Jobs
As the creative director of an animation company, I’ve been keeping a close eye on how AI is shaking things up in our field. One thing that’s been on my mind a lot is how technologies like Sora are changing the job landscape for folks in creative professions. Sure, AI has the chops to handle some tasks on its own, possibly nudging some jobs to the side.
But it’s important to remember that the spark of creativity and that gut feeling for art? Those are purely human. AI can’t mimic that completely. So, rather than seeing AI as a stand-in for human creativity, I think it’s better to view it as a tool that boosts what we can do. It lets us zero in on the really creative and subtle parts of our work, making our jobs not just easier, but more exciting, too.
Problem of Originality
There’s something about AI that’s been bothering many in the creative industry. It’s all about how it learns from existing pool of artworks (illustrations, animation, typography etc) to make something new. But does that really make it original or innovative? If AI just keeps going back to the same old stuff, aren’t we just going around in circles with the same ideas? That could really put a damper on true creativity and the birth of fresh styles. Imagine the impact on the variety and depth of artistic expression if we let that happen.
And let’s not forget about what makes art, well, art. It’s that human touch, the story of us, our dreams, our fears that are scribbled down, painted or animated. If we’re leaning too much on AI, do we start to lose that personal signature that makes art resonate so deeply with us?
Use Sora in The Right Place
Incorporating AI technologies such as Sora into professional software tools such as Adobe After Effects or Cinema 4D, offers a more balanced approach. By giving designers and animators the tools to quickly create and tweak 3D models or 2D animations, we can reduce the time needed for content creation.
For example, modeling a complex 3D object that takes 2 hours can be done in a matter of minutes using Sora added to a 3D software. Or creating a hand drawn frame-by-frame animation using Sora in Adobe After Effects or Adobe Photoshop that can be done in one day rather than one month.
This method does not take away from the essential value of human creativity but rather supports it. It ensures that creative community stays at the leading edge of innovation and artistic discovery. It’s like having an efficient assistant who takes care of the more repetitive tasks, allowing professionals to spend more of their time and energy on the creative parts of their work.
Project Ownership Problems with Sora
A major issue that arises with the use of AI in creative projects is the challenge of ownership and the authenticity of the generated content. When brands use Sora (or similar AI generators) to create content, they encounter some serious restrictions.
They will not own the project files because there is no project file to begin with; everything is generated by one big smart AI system. If they don’t own it and cannot remove the result from the AI pool, then anyone can use the same AI to generate the exact same thing.
This situation can lead to a weakening of brand identity and a loss of originality. It’s essential for those in the creative field to develop definitive guidelines and practices that safeguard the rights of creators and ensure that AI acts as a support to creativity rather than an obstacle.
Uncontrolled Sora: Great Threat to Aspiring Artists
To further elaborate on the points raised earlier, it’s essential to consider the broader impact of AI on the creative ecosystem. The democratization of content creation through AI like Sora has its merits. It is making high-quality video or animation production accessible to more people. However, this accessibility should not come at the cost of diminishing the value of professional creativity and craftsmanship.
The conversation around AI in creative industries must also include the perspective of aspiring artists. For newcomers, the prospect of competing with AI might seem daunting, potentially deterring them from pursuing careers in art and animation. It’s crucial to foster an environment that encourages learning, experimentation, and growth for these individuals. By promoting a culture that values both human creativity and technological advancement, we can ensure a vibrant and diverse future for the creative arts.
The Ball is in Policy Makers Court
Addressing the challenges of AI-generated content requires collaboration among technologists, artists and policymakers. Developing standards and ethical guidelines for AI in creative work can help mitigate concerns around originality and ownership. It makes sure that creators are credited and compensated for their hard work and talent. Additionally, exploring new business models that leverage AI to enhance human creativity (as suggested above), rather than replace it, could provide a sustainable path for the industry’s evolution.
Final Thoughts
Wrapping up our talk, it’s pretty obvious that OpenAI’s Sora is a game-changer at the crossroads of AI and creative work. As we dive deeper into this new era, we’ve got to keep a level head about these tech advances.
It’s all about seeing their huge potential to shake up how we create, while never forgetting the unbeatable value of human touch in art. By striking this balance, we’re set to open doors to fresh ways of storytelling, designing and expressing ourselves. We should make sure that tech boosts our creative spirit instead of overshadowing it.
https://www.kasradesign.com/sora-and-the-future-of-animation-industry/
A few thoughts on Sora
Yesterday, OpenAI announced Sora, a new product that generates realistic video from text prompts. The examples are remarkable.
https://www.youtube.com/watch?v=anmuklFtu8U
A TV writer friend texted me to ask “is it time to be petrified?”
I wrote back:
I don’t think you need to be petrified. It’s very impressive at creating video in a way that’s like how Dall-E does images. A huge achievement. For pre-viz? Mood reels? Incredible. We’ll see stuff coming out of it used in commercials first.
For longer, narrative stuff, there’s a real challenge moving from text generation (gpt-4 putting together something that looks like a script) to “filming” that script with these tools to resemble anything like our movies and television.
Writers, directors, actors and crew have a sense of why they’re doing what they’re doing, and what makes sense in this fictitious reality they’re creating. I don’t think you can do that without consciousness, without self-awareness, and if/when AI gets there, stuff like Sora will be the least of our concerns.
With a night to sleep on it, I think there are a few larger, more immediate concerns. Writers (and humans in general) should be aware of but not petrified by some of the implications of this technology beyond the obvious ones like deepfakes and disinformation.
Video as input. Like image generators, this technology can work off of a text prompt. But you can also feed it video and have it change things. Do you want A Few Good Men, but with Muppets? Done. Need to replace Kevin Spacey in a movie? No need to reshoot anything. Just let Sora do it.
Remake vs. refresh. Similarly, any existing film or television episode could be “redone” with this technology. In some cases, that could mean a restoration or visual effects refresh, like George Lucas did with Star Wars. Or it could be what we’d consider a remake, where the original writer gets paid. What’s the difference between a refresh and a remake, and who decides?
Animation vs. live action. How do we define the video material that comes out of Sora? It can look like live action, but wasn’t filmed with cameras. It can look like animation, but it didn’t come out of an animation process. This matters because while the WGA represents writers of both live action and animation, studios are not currently required to use WGA writers in animation. We can’t let this technology to be used as an end-run around WGA (and other guild) jurisdiction.
Reality engines. In a second paper, OpenAI notes that Sora could point to “general purpose simulators of the physical world.” The implications go far beyond any disruptive effects on Hollywood, and are worth a closer look.
It seems like a long way to go from videos of cute paper craft turtles to The Matrix, but it’s worth taking the progress they’ve made here seriously. In generating video, Sora does a few things that are really difficult, and resemble human developmental milestones.
Like all models, Sora is predictive, making guesses about what just happened and what happens next. But it feels different because it’s doing this in a 3D space that largely tracks with our lived experience. It remembers objects, even if they’re not on screen at the moment, and recognizes interactions between objects, such as paintbrushes leaving marks on the canvas.2
Sora makes mistakes, but the results surprisingly good for a system that wasn’t explicitly trained to do anything other than generate video. Those capabilities could be used to do other things. In a jargon-heavy paragraph, OpenAI notes:
Sora is also able to simulate artificial processes — one example is video games. Sora can simultaneously control the player in Minecraft with a basic policy while also rendering the world and its dynamics in high fidelity. These capabilities can be elicited zero-shot by prompting Sora with captions mentioning “Minecraft.”
Sora “gets” Minecraft because it’s ingested countless hours of Minecraft videos. If it’s able to create a simulation of the game that is indistinguishable from the original, is there really a difference? If it’s able to create a convincing simulation of reality based on the endless video it scapes, what are the implications for “our” reality?
These are questions for philosophers, sure, but we’re all going to be faced with them sooner than we’d like. Sora and its descendants are going to have an impact beyond the cool video they generate.
Sora is a great name, btw. It doesn’t mean anything, and doesn’t have any specific connotation, yet feels like something that should exist.
Not to dive too deeply into theories of human consciousness, but the ability to internally model reality and predict things feel like table stakes.
https://johnaugust.com/2024/a-few-thoughts-on-sora
I also felt like I should bring this back as a reference on why some people think that film industry will completely cease to exist very soon:
Doesn't matter if anyone can "produce" a "Hollywood tier" movie, because 90% won't be as good as dedicated movies, it'll be a flood of trash, which users won't try to sift through hoping for a good one. Also what value does a AI generated movie give when none will see it because Marvel or Disney's name isn't attached to it? There's a reason why 99.9% of YouTuber or shows or movies essentially don't exist, it's because they're not a brand. You could generate 1000's of hours of content, but they won't ever be seen by others.
I think you're looking at it wrong. It's not that people can make AI movies and then share with others - it's more that people will be able to create their OWN movies, on demand, - they don't need to wait for a studio to create the content they want - they simply ask AI to create a movie in a specific genre and with specific requests. I'm seeing this being a reality within 10 years. I think you're putting too much value in "the brand".
Given these and what you've seen from Sora, do you expect animation studios like Pixar Animation Studios, Walt Disney Animation Studios, DreamWorks Animation, Illumination, Sony Pictures Animation, and so on to completely cease to exist immediately once Sora is officially released this year? Why or why not?
P.S. I advise you all to read everything carefully before posting any comments.
[link] [comments]