AI Movies Prompt Wrangler: I feel like I’m watching the birth of a new creative role

Svetlovska

This guy’s tutorial, merging together a whole fistful of current AI tools to make an uncanny valley movie trailer, looks like the start of a new creative category, an evolution of - what? Video editor, photoshop artist, 3D animator, scriptwriter, set designer, voice coach, movie director… all of the above. Add in Chat GPT to write the script, and you have the whole, no humans required package.

Yes, it’s still schonky, but a whole lot less shonky than just a few months ago. Wow, but things are moving fast in this space:

What interested me most was the beginning of a formalisation of prompt and other techniques applied to the AI tools to get them to perform consistently, and persistently, through multiple frames of video action. I guess this is why all those folks in Hollywood are on strike right now. They can see the AI prompt wrangling on the wall…

AudioGus

Text prompts are like how we used to have to animate in 3D using a scripting language in that it won't last very long as an interface. Soon enough things will likely just morph in near realtime via more natural spoken language, which will make it all more accessible and shake things up even more. These things really are crazy, hard to imagine even three years from now.

mistercharlie

Yes, I think of text prompts as being like command-line interfaces on computers before GUIs and mice. This will all be sliders and menus soon enough, and either a) another deep tool for artists like Photoshop/Logic/Final Cut, or apps for bite-sized gimmicks—there are quite a lot of those gimmicky apps available already.

pedro

Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

Krupa

@pedro said:
Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

Yeah, I recently read a story about Kenyan workers being treated pretty badly by the contractors who were doing this work, not only with poor wages and hours, but some also traumatised after weeks and months of tagging images that were undesirable in the model (porn/violence/death etc)

AudioGus

@Krupa said:

@pedro said:
Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

Yeah, I recently read a story about Kenyan workers being treated pretty badly by the contractors who were doing this work, not only with poor wages and hours, but some also traumatised after weeks and months of tagging images that were undesirable in the model (porn/violence/death etc)

I think he was talking about the creation of the video and not people filtering image datasets.

boomer

To look at what @pedro said from a different angle, It still takes a creator to create.

I remember another technological revolution in music back in the 80s. That’s when musicians started to be replaced by sequencers and synths. It happened fast. I was just starting in the business and I knew many people who went from outrageous incomes as studio players to playing for beer inside of a few years. I personally was sitting in a theater pit reading an article about this new thing called MIDI. I remember thinking in that moment that my brand new career was about to implode. So I became one of the first people in town to own a wx7 and learn how to really play it. Adapting to the technology extended my viability as a musician for 10 or 15 years because keyboards just could not provide the level of expression I could. As samplers started getting good enough, I saw these skills becoming less valuable. So I got into sequencing shows for live performance in the early 90s. Again, adapting to the technology to remain viable. At that time, there were not that many people with both the musical and technical skills to pull that off. As computers got more and more user friendly, many more musicians gained access to music technology. But it still took a musician to make music.

Fast forward to now. I am once again a musician making my music like all of you are making your music. This little tablet can make any sound imaginable and infinitely more that have not been imagined. But I am still creating the music. “No humans involved” in this video is simply not true. This guy is a film maker who knows how to use technology to create a film. But I do believe @svetlovska is correct in that it’s time for us to learn something new.

Poppadocrock

Interesting stuff. I appreciate the Ai info and updates you’ve been sharing @Svetlovska

NeuM

@Svetlovska said:
This guy’s tutorial, merging together a whole fistful of current AI tools to make an uncanny valley movie trailer, looks like the start of a new creative category, an evolution of - what? Video editor, photoshop artist, 3D animator, scriptwriter, set designer, voice coach, movie director… all of the above. Add in Chat GPT to write the script, and you have the whole, no humans required package.

Yes, it’s still schonky, but a whole lot less shonky than just a few months ago. Wow, but things are moving fast in this space:

What interested me most was the beginning of a formalisation of prompt and other techniques applied to the AI tools to get them to perform consistently, and persistently, through multiple frames of video action. I guess this is why all those folks in Hollywood are on strike right now. They can see the AI prompt wrangling on the wall…

With the current rate of advances in "A.I."/machine learning systems I would not be surprised if it becomes possible over the next 5-7 years to co-author scripts with these systems, provide the barest of details about scenery, props, actors, music, lighting, etc. and be able to generate Hollywood-level (or Bollywood-level, if you like) final output which would previously would have required the participation of hundreds, if not thousands of people.

People are going to be shocked at how much they'll be able to do and they'll simultaneously find out how little their own profession is needed anymore. Today's stuff is just barely scratching the surface.

MadeofWax

@pedro said:
Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

It is definitely difficult to try to find the right concept, figure out a good set of prompts, find the right models and/ or extensions to create something worth watching. And then....editing. The video below (unlisted on Youtube, but viewable here) is something I sent Paul ( Paulieworld) after trying and failing to find a good concept for his excellent track "Rapture of the Deep". I was venting and got a bit silly with it.
I hope to finish it but I'm currently struggling with another project/ collaboration with him and GeoTony.
It also doesn't help that the technology keeps changing at a break neck pace and any update could potentially break everything. I've become quite the expert at reinstalling Stable Diffusion.
Anyway, for your amusement

Krupa

@AudioGus said:

@Krupa said:

@pedro said:
Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

Yeah, I recently read a story about Kenyan workers being treated pretty badly by the contractors who were doing this work, not only with poor wages and hours, but some also traumatised after weeks and months of tagging images that were undesirable in the model (porn/violence/death etc)

I think he was talking about the creation of the video and not people filtering image datasets.

I finally got round to watching the YouTube, it’s definitely a viable workflow but as pedro said, work intensive. I’m not sure for this kind of shot and result it’s in any way worth it, maybe needs a lot more hand work doing to make it commercially passable, but maybe you’d be as good doing it ‘properly’ for now. I have been slowly adding these machine learning tools to my kit, and finding limited uses for them, it’s a bit like trying to make house music with only modular, you can do it, and it’s useful as part of the process, but sticking with it exclusively is a road to hell, and with the broad toolset he’s using expensive, even just computationally (if not financially - gen2 runway is a monthly sub for serious use) for now…

GeoTony

Excellent video and music @MadeofWax and @Paulieworld 🙏

AudioGus

@Krupa said:

@AudioGus said:

@Krupa said:

@pedro said:
Not pictured: the truck-load of hours spent assembling that 1% of great content from the AIs into a cohesive whole. If only there was an AI for that

Yeah, I recently read a story about Kenyan workers being treated pretty badly by the contractors who were doing this work, not only with poor wages and hours, but some also traumatised after weeks and months of tagging images that were undesirable in the model (porn/violence/death etc)

I think he was talking about the creation of the video and not people filtering image datasets.

I finally got round to watching the YouTube, it’s definitely a viable workflow but as pedro said, work intensive. I’m not sure for this kind of shot and result it’s in any way worth it, maybe needs a lot more hand work doing to make it commercially passable, but maybe you’d be as good doing it ‘properly’ for now. I have been slowly adding these machine learning tools to my kit, and finding limited uses for them, it’s a bit like trying to make house music with only modular, you can do it, and it’s useful as part of the process, but sticking with it exclusively is a road to hell, and with the broad toolset he’s using expensive, even just computationally (if not financially - gen2 runway is a monthly sub for serious use) for now…

Yah a lot of these 'tools' right now are like random loot boxes. For most (that openly talk about using them anyway) it seems more like a gaming compulsion than actual production.