I see lots of people say AI will not be able to do movies / TV because it hallucinates yada yada yada. But, movies / TV shows follow a clear script. Script being a generic formula that is taught to script writers the same way as music theory cheat sheet helps aspiring musicians to write songs.
Script. Script can be turned into machine readable format. You can add commands how to render a movie on the basis of it. For example in a script you could have #Jack, telling the thing reading the script that we are talking of actor #Jack meaning it should tap into assets about jack which would reside in folder Jack. Jack meanwhile could be rendered by sub ai to fit the part.
That helps us nail down the character so it wont be changing appearance wise in our script.
The AI part here comes from how Jack would be handled. "Acting" - ragdoll physics of Jack, a 3d character and how it behaves on the basis of the machine readable script, that would be delegated to sub AI. As would be the emotional tone of his voice which could also be implied in the script, as would be the look on his face, his pose. The way he behaves among other actors. This would be a sub AI. So a bunch of sub ai's that have been trained on possibly movies in which the character acting has been reduced to ragdoll physics and the way their faces look when they have certain looks on their faces.
Now this is not that much different from a car driving in traffic and observing what is happening around it. We are teaching AI's how to read the world as we are also teaching AI's sentiment analysis etc. All models that will be needed in a script rendering AI.
So it will not be a singular AI that will dream a movie, but it will be a whole bunch of AI's each dedicated to different parts of the script rendering.
So the script could be written by AI to be based on some movie for example and the AI could think about what is has learnt from the movie. The way the different sub AI's interpreted the acting, the sentiment, the scenes, the pacing. The overview and then if you want to do something similar in a different setting, it would do something similar to what chatgpt can do with text, or what runwayML can do with a video. Style transfer, in the way the person prompting requires.
But the script reading AI would act more like a director. Guiding the sub AI's to keep certain styles as asked by the promptee. This is how we humans do it too. A director does not a dream a movie in a single setting. A script writer writes it, a director sees it his own way, the cameraman films it the way he does, the choreographers do their thing as do the actors etc etc etc.
Until at the end of each filming day the director looks at the cuts of that particular day and thinks if it is ok, or not. If you have a strong willed director, he will over ride the will of the actor or the person in charge of the camera etc. These are basically the same as if you render a picture on basis of another picture, what sort of weight you give model.
So I think what we will end up is a bunch of highly specialized AI's that will be trained to do very specific things. Like people on the movie set.
I think this is how AI movies will be done. It might not be much in the beginning compared to masters of film making, but eventually, this sort of process will no doubt get better.
But I do not think we will see AI that will just dream up a hour 30 minute movie in a long while.
I think it will be a big business to train the models that can be used for movie making. Perhaps in the beginning they will be used more like DAW (Digital audio work station) tools on singular scenes of movies. But eventually. We might end up with Netflix AI that will write the movie we want.
[link] [comments]