How AI Will Augment Human Creativity in Film Production

Every story needs a villain. For decades, artificial intelligence has been presented as a mysterious, powerful antagonist in Hollywood. HAL (2001: A Space Odyssey), Ava (Ex-Machina) and Ultron (Avengers) are all part of a broader narrative that paints AI as the personification of technologys threat to humankind.

With the arrival of generative AI, that threat has caromed off the screen onto the negotiating table between multiple creative guilds and the producers, studios and streamers as represented by the Alliance of Motion Picture and Television Producers. As the industry attempts to determine the role of AI, it has become a major sticking point. Last week, SAG-AFTRA joined the WGA on strike, marking the first double strike in over 60 years, with hundreds of thousands of Hollywood creatives now on the picket line.

AI is just one point of contention in the negotiations, but in the eyes of the guilds, it has the potential to create an extinction-level event to their craft. With both guilds at an impasse, Hollywoods production pipeline hangs in the balance.

SEE ALSO: Entertainment Pros Anxious About Gen AI

Advances in the generative capabilities of AI have produced a Cambrian-like explosion of possibilities for the applications of AI across all industries, but generative AI has special pertinence for Hollywood. Generative AI has seismically shifted the way we look at content creation, escalating tensions about AI-powered production becoming more of a reality, with the potential to detract human creativity from the film industry. Protecting human creativity, and the art of filmmaking in turn, is of paramount importance as the world collectively evaluates this technological paradigm shift.

Augmented Intelligence: From Existential Threat to Essential Tools

We believe the next paradigm of technological evolution will be defined by augmented intelligence, meaning companies that leverage software and machines to enhance the work, expertise and experience of a human user or worker. Lifting this veil reveals the narrative around AI in Hollywood should shift away from seeing AI as a threat and rather toward a new set of tools that can augment the capabilities of every filmmaker across the globe.

SEE ALSO: Will AI Supplant or Supplement Hollywood Workers?

Despite the wave of recent concerns and negative media, we believe AI will propel the film industry forward by giving more aspiring filmmakers access to previously unattainable tools, thus unlocking human creativity to previously unimaginable levels. As we dive deeper into understanding the capabilities of AI tools and their underlying models, it becomes clearer that AI can be a positive, democratizing force in Hollywood.

While focal issues of the strikes such as script generation and more robust generative video loom, the first wave of AI adoption in Hollywood will be most strongly felt in video editing and post-production, where AI tools come in to assist human productivity.

Today, filmmakers can use AI solutions in video editing to drastically reduce production times and decrease the budget needed to fulfill their creative visions. Films that previously needed massive budgets can now use novel technologies powered by large foundation models to broaden their creative faculties for multiple facets of their process, from storyboarding to visual effects and CGI on post-production editing.

SEE ALSO: How Gen AI Can Bring Efficiency to Production

The creative art of filmmaking remains protected but now, through augmented intelligence tools, we can democratize top-tier production capabilities to potentially thousands of filmmakers. Giving them the power to fulfill their creative visions without present constraints promises to usher in a new level of storytelling that will capture the hearts and minds of people across the globe.

So, what do these capabilities look like in action? Founded in 2018, Runway ML offers a suite of AI tools powered by models trained on quantities of video data for artists, filmmakers and creators. VFX artists have been using Runway to complete manual tasks that previously took days and costly professional software and equipment.

For example, with Runways inpainting tool, filmmakers can use AI to seamlessly remove objects or backgrounds in the video editing process. (Remember that pesky Starbucks coffee cup that somehow made it to Winterfell in Game of Thrones?) The companys suite of video editing AI tools also enables motion tracking and even greenscreen.

Source: Runway ML For filmmakers, transforming video footage is a critical aspect of the post-production workflow. Runways Gen-1 is video-to-video offering, a form of generative AI that uses words and images to turn out new videos based on existing ones. Gen-1 represents a drastically cheaper and more efficient way to create unique visual effects, stylize footage or even craft new scenes based on reusable footage.

Finally, text-to-video AI represents a significant technological leap in content generation. Runways Gen-2 is widely considered to be among the most sophisticated text-to-video AI models available. The technology is still in its infancy, but the current iteration allows users to make four-second videos simply from text prompts. While current tech capabilities are still not there for longer video outputs, Twitter user @TomLikesRobots created this movie trailer by stitching together generated clips.

Another company making waves in this space needs no introduction: Adobe has been an industry-leading player in design for decades. Iconic products such as Photoshop, Illustrator and After Effects have been longstanding staples in creatives repertoire and a pivotal engine for their workflows.

In March, Adobe launched Firefly, an embedded gen AI model that lives inside of Adobe products to enable all manner of new tasks, After Effects among them. Developed in 1993, the program ushered in a new era of creative capabilities for filmmakers.

Since then, it has been used consistently in post-production for films, TV and video games. Powered by Firefly, After Effects is once again poised to unlock a new wave of creativity as it grants access to a toolkit of resources previously only accessible to big-budget studios.

Now with After Effects, filmmakers can use AI-powered techniques to automate time-consuming manual tasks such as motion tracking and visual effects. The promise of Adobe Firefly is to dramatically accelerate pre-production, production and post-production workflows through natural language prompting to alter scenes and fulfill a filmmakers creative desires. With a simple prompt like, Make this scene feel warm and inviting, the time between imagination and final product can all but disappear.

These are just some snapshots of the AI-powered technologies fast becoming mission-critical in the filmmakers toolset.

While important issues such as actors digital likenesses and AIs role in screenwriting have occupied much of the oxygen in Hollywood to date, the less-heralded advancements in video editing and production indicate AIs first wave may crest elsewhere.

Through an augmented intelligence lens, we see that AI doesnt need to always be cast as the villain. It can have a positive impact by enhancing the art of filmmaking while unleashing unprecedented levels of efficiency that allow more time for the creativity that is the backbone of this industry.

Sunny Dhillon is an investor at Kyber Knight Capital, a $120M Silicon Valley and Los Angeles-based venture capital fund. Kyber Knights Ary Vaidya contributed to this article.

Does age impact sentiment on AIs onslaught? VIP+ digs in