Exactly what video editing software does Hollywood use for film and television? I have an update to our most popular episode from seven long years ago, and today we’ll reexamine the landscape and see how Hollywood has adapted since then, including a change from the big three solutions.
The original “The Truth About Video Editing Software” in 2017 got some attention.
Many years ago, I dissected the video editing landscape here in Hollywood and told you what most editors were using. But most importantly, why? Now, before we look at what’s changed, I do encourage you to check out the original The Truth About Video Editing Software in Hollywood from 2017, to see how we got to where we were…then.
Since 2017, desktop software has improved. New tools have now entered the market, and creating content on your phone now includes editing. There are just so many options out there, so I’ve reached out to various folks in the industry, both on the business and creative side. Some have been under FrieNDA (which is my favorite type of agreement) and some have been under the cloak of night in empty parking garages here in Southern California. But all have granular insight into the industry and where we are today.
Now you’ve got to consider the singularly unique place Avid was during 2017.
Media Composer looked…well, let’s face it, dated. It was designed using then-decades-old iconography and design, which was legacy from the days of helping film editors transition from analog editing to digital. Because most film and TV post-production pipelines within post facilities relied on Avid-centric workflows, updating the interface and workflows to attract new users and to grow Avid’s business heavily risked alienating existing users. As my grandmother used to say, “You’re damned if you do and you’re damned if you don’t.”
However, in 2019, Avid completed a new look for Media Composer, although they eventually acquiesced and offered the “Classic” look several years later. Avid has also since introduced UME, their new “Universal Media Engine,” which replaces AMA and thus successfully dodges another unfortunate product name (cough Avid ISIS cough).
After 30 years, Media Composer and Pro Tools finally have direct interoperability.
And after 30 years, Avid finally made the round trip workflow from Avid Media Composer to Avid Pro Tools and back easy with a direct session import and export. No AAFs needed!
When the pandemic hit, the industry had to quickly adapt while working remotely, and while Avid had existing products for these distributed teams, they were either expensive or better suited for broadcast news, or they were still in beta like Avid’s cloud editing product, Edit on Demand, which was both expensive and in beta.
Despite these product roadblocks for remote work, most productions stuck with good old-fashioned Media Composer. Work-from-home editors remoted into existing Avid workstations back at their facilities, which maintained environment familiarity and access to the usual shared storage. Avid rental facilities set up racks of systems to keep post-production moving. Alternative editing solutions to Media Composer didn’t offer huge cost savings for film and TV productions, whether it be doing everything in the cloud or by switching to another editing software platform.
But by and large, if you cut with Media Composer, you stuck with Media Composer.
A review of the Academy Award nominees and winners from 2017 until now also shows that a majority of the nominees and winners were cut with Avid Media Composer. At a business level, Avid reported 15,000 to 20,000 Media Composer Cloud subscriptions in late 2017 and into early 2018.
Now, that may seem low. I mean, how can you dominate one industry with only 20,000 seats of software? But keep in mind, seven years ago, not every Media Composer user had a cloud subscription. Many users still had legacy licenses, which were not yet counted within cloud subscriptions. Now, if we fast forward to late 2023, Avid reported over 150,000 cloud subscriptions for Media Composer prior to their acquisition by the private equity firm Symphony Technology Group.
It’s pretty clear Avid remains the standard for film and TV editing in 2024. Now, what will happen now that private equity owns Avid? Well, that’s a subject for another video.
And so in 2017, Premiere Pro was second, albeit distant, as a choice for film and TV editing. Since 2017, however, Adobe has continued to enhance its Productions workflow, which provides Premiere Pro editors with a collaborative experience similar to Avid’s Bin and Project Sharing, which is a mainstay in collaborative film and TV post-production. And this additional feature did gain Adobe some high-profile film and TV projects. The first Deadpool, Terminator: Dark Fate, and David Fincher’s projects like Gone Girl and his HBO mini-series Mindhunter were all edited with Premiere Pro, as was last year’s Best Picture Oscar winner Everything Everywhere All At Once. Now, if we look at Hollywood-adjacent productions like those at the Sundance Film Festival, we’ve seen Premiere Pro being used on a greater number of films over the years, and it’s now accounting for over half of all projects showcased at the festival. And even more projects use Adobe if you include other Adobe tools like Frame.io.
However, the fact remains that Premiere Pro in Hollywood has remained more unique than the standard. The general consensus is that Premiere Pro is still easily the main alternative to Avid Media Composer for film and TV work. But is that a bad thing? I mean, it’s important to remember that the market is relatively small compared to other, more lucrative media-rich markets, plus social media, where video content is produced at an exponentially faster rate than film and television. In fact, recent studies show that a majority of young Americans aspire to be social media influencers over most any other career. Film and TV are no longer the only media vehicles for visibility, let alone creative work. So for Adobe, why continue to focus heavily on a market that is no longer the main avenue that many creatives aspire to work in?
I think I’d rather take the bag of money from a faster-growing market over a case study and my logo in the credits of the film. Adobe doesn’t release much in the way of metrics for app usage. However, Adobe did report that it had approximately 12 million Creative Cloud subscriptions in 2017, and the subscriber count has grown to over 33 million in 2023. This year, however, Adobe has faced many challenges, including poor communication about how their AI is trained and emerging legal issues with the DOJ and FTC over early subscription termination fees and a complex cancellation process. As of now, it’s unclear how this will affect Adobe moving forward.
But I will mention a few key points as they’re important for context to understand where we are today.
Higher-profile film and TV post-production requires a solid tech stack and software interoperability. This includes multiple hardware components as well as a trained staff to maintain it. Often, complete turnkey systems for post-production are sold by what we call Systems Integrators or VARs (Value Added Resellers). Post facilities often buy from VARs because pricing is generally better, and the VAR can build, test, deploy, and support these solutions.
Putting the “value” in “Value Added Reseller”
Unlike Final Cut Pro Classic, Final Cut Pro X did not go through the usual sales channels. It was an App Store purchase. This meant that the VARs’ role was largely mitigated and thus professional adoption was slowed.
It would take Apple several years to get Final Cut Pro X to a level where it could be used in professional film and TV workflows. We had a perfect storm of factors that had most professional editors, and especially facilities, simply not interested in moving forward with Apple editing software. And as I said in 2017, it’s these hurdles that made Final Cut Pro X a distant third in Hollywood film and TV editing. And it hasn’t gotten any better.
In 2022, the editing community created a petition asking Apple to publicly stand by the use of Final Cut Pro for TV and film industries worldwide. Now, to their credit, Apple did respond, and they committed to the joint development of training and certification courses, plus creating an industry advisory panel and an increase in Hollywood-centric workshops for film and TV editors. So while Final Cut Pro X has matured to be a professional editing solution and has been used on a handful of high-profile projects, the window of opportunity to ascend or even eclipse the level of its predecessor has unfortunately closed. However, as a slight silver lining, Apple hardware was and still is a mainstay in the professional content creation industry, and I bet many of you have an iPhone instead of an Android.
But it’s Resolve’s color tools that have now become the industry standard for color grading. And it’s not just because of their intricate controls, affordability, and overall innovations, but because the existing color tool sets in Hollywood ten years ago were, let’s face it, lacking. Avid’s advanced color grading tool, Symphony, had very few feature updates at the time and required higher monthly costs. Lumetri, the Premiere Pro color tool, was also beginning to show its age following Adobe’s acquisition of SpeedGrade in 2011. Plus, the other color solutions on the market were either expensive or existed outside the video editing software that most creatives were familiar with, so there was a major deficit in accessible color tools, but not in creative video editing tools.
Avid and Adobe’s editorial tools had decades of nuanced development and were already affordable for most. Resolve’s video editing features were still evolving at the time. The flip side, however, is that even if Blackmagic did introduce revolutionary video editing tools within Resolve at the time, they still would have faced massive resistance from a Hollywood industry still wary of wholesale video editing changes following the failure of Final Cut Pro X to succeed. There simply wasn’t enough incentive for Hollywood to switch to Resolve for professional video editing.
But that’s not the full story. The free version of Resolve, or the one-time payment for Studio, is a no-brainer compared to continual, eternal paid subscriptions from other leaders like Adobe and Avid. Bundling the more powerful Resolve Studio with a Blackmagic camera purchase was also a fantastic marketing move. Resolve offers incredible value, especially for new editors, and it’s a well-established business tactic that getting buy-in from the younger, next generation of creatives is a fantastic way to boost sales down the road.
While the details on exact download numbers and usage are not publicized, in 2019, Blackmagic CEO Grant Petty did say that Resolve downloads topped 2 million, and I can’t see any reason why it still wouldn’t be growing. As an interesting side note, however, there’s a vast difference between the number of free users of Resolve and the number of paid users of Resolve Studio. The general consensus is that most professional film and TV creatives are going to need the features found in the paid Studio version, but most industry analysts estimate that only 5% or so of Resolve users actually pay for the Resolve Studio upgrade. So those professional creatives, at least those in Hollywood, are overwhelmingly using Resolve for color work and other side tasks, but they’re not using it as their primary video editor.
With Avid and Adobe taking the top spots as the creative editorial choices here in Hollywood, Resolve as a video editor faces steep competition, not unlike the smartphone market, with Apple and Android dominating the marketplace with no real challengers. So, however distant, that third place for video editing software in Hollywood that was previously occupied by Final Cut Pro can now easily be filled with the logo from Blackmagic Design. But like Adobe, this does beg the question: “Are the vanity bragging rights for Hollywood’s editors worth the windfall of cash to be found by capturing other markets?”
If anything, these two footnotes should reinforce that it’s not the tool, it’s the talent. Knowing the right tool makes you employable. Your talent is what keeps you employed. So what do you think we’ll see in the next seven years? Will predominantly mobile solutions like CapCut or open-source solutions make their way into the professional film and TV market? Or do you have a hot take on AI in Hollywood? I look forward to your thoughts.
Until the next episode, Learn more, do more. Thanks for watching.
Generative A.I. is the big sexy right now. Specifically creating audio, stills, and videos using AI. What’s often overlooked, however, is how useful Analytical A.I. is. In the context of video analysis, it would involve facial or location recognition, logo detection, sentiment analysis, and speech-to-text, just to name a few. And those analytical tools are what we’ll focus on today.
We’ll start with small tools and then go big with team and facility tools. But I will sneak in some Generative AI here and there for you to play with.
1. StoryToolkitAIStoryToolKitAI can transcribe audio and index video
Welcome to the forefront of post-production evolution with StoryToolKitAI, a brainchild of Octavian Mots. And with an epic name like Octavian, you’d expect something grand.
He understood the assignment.
StoryToolKitAI transforms how you interact with your own local media. Sure, it handles the tasks we’ve come to expect from A.I. tools that work with media like speech-to-text transcription.
But it also leverages zero-shot learning for your media. “What’s zero shot learning?” you ask.
Imagine an AI that can understand and execute tasks that it was never explicitly trained for. StoryToolKitAI with zero-shot learning is like trying to play charades. It somehow gets things right the first time. While I’m still standing there making random gestures, hoping someone will figure out that I’m trying to act out Jurassic Park and not just practicing my T-Rex impersonation for Halloween.
How Zero-Shot Learning Works. Via Modular.ai.
Powered by the goodness of GPT, StoryToolKitAI isn’t just a tool. It’s a conversational partner. You can use it to ask detailed questions about your index content, just like you would talk with chat GPT. And for you DaVinci Resolve users out there, StoryToolkitAI integrates with Resolve Studio. However, remember Resolve Studio is the paid version, not the free one.
Diving Even Nerdier: StoryToolkit AI employs various open-source technologies, including the RN50x4 CLIP model for zero-shot recognition. One of my favorite aspects of StoryToolkit is that it runs locally. You get privacy, and with ongoing development, the future holds endless possibilities.
Now, imagine wanting newer or even bespoke analytical A.I. models tailored for your specific clients or projects. The power to choose and customize A.I. models. Well, who does it like playing God with a bunch of zeros and ones, right? Lastly, StoryToolKitAI is passionately open-source. Octavian is committed to keeping this project accessible and free for everyone. To this end, you can visit their Patreon page to support ongoing development efforts (I do!)
On a larger scale, and on a personal note, I believe the architecture here is a blueprint for how things should be done in the future. That is, media processing should be done by an A.I. model – with its transparent practices – of your choosing and can process media independently of your creative software.
A potential architecture for AI implementation for Creatives
Or better yet, tie this into a video editing software’s plug-in structure, and then you have a complete media analysis tool that’s local and using the model that you choose.
2. Twelve LabsHave you ever heard of Twelve Labs?
Don’t confuse Twelve Labs with Eleven Labs, The A.I. Voice synthesis company.
Twelve Labs is another interesting solution that I think is poised to blow up… or at least be acquired. While many analytical A.I. indexing solutions search for content based on literal keywords, what if you could perform a semantic search? That is, using a search engine that understands words from the searcher’s intent and their search context.
This type of search is intended to improve the quality of search results, Let’s say here in the U.S., we wanted to search for the term “knob”.
Other English speakers may be searching for something completely different.
That may not actually be the best way to illustrate this. Let’s try something different.
“And in order to do this, you would need to be able to understand video the way a human understands video. What we mean by that is not only considering the visual aspect of video and the audio component but the relationship between those two and how it evolves over time, because context matters the most. ”
Travis Couture
Founding Solutions Architect
Twelve Labs
Right now, Twelve Labs has a free plan which is hosted by Twelve Labs. It’s fairly generous. However, if you want to take deeper advantage of their platform, they also have a developer plan. Twelve Labs tech can be used for tasks like ad insertion or even content moderation like figuring out which videos featuring running water depict natural scenes like rivers and waterfalls or manmade objects like faucets and showers.
Twelve Labs use cases
With no insider info, I’d wager that Twelve Labs will be acquired as the tech is too good not to be rolled into a more complete platform.
Next up, we have Curio Anywhere by GrayMeta, and it’s one of the first complete analytical A.I. solutions for media facilities. Now, this isn’t just a tagging tool. It’s a pioneering approach to using A.I. for indexing and tagging your content, using their localized models, and if you so choose, cloud models, too.
In addition to all of this, there’s also a twist. See, traditionally analytical A.I. generated metadata can drown you in data and options and choices, overloading and overwhelming you. GrayMeta’s answer is a user-friendly interface that simplifies the search and audition process right in your web browser. This means briefly being able to see what types of search results are present in your library across all of the models.
Is it a spoken word? Maybe it’s the face of someone you’re looking for. Perhaps it’s a logo. And for you Adobe Premiere users out there, you can access all of these features right within Premiere Pro via GrayMeta’s Curio Anywhere Panel Extension. Now let’s talk customization. Curio Anywhere allows you to refine models to recognize specific faces or objects.
Curio Anywhere Face Training Interface
Imagine the possibilities for your projects without days of training a model to find that person. Connectivity is key and Curio Anywhere nails it, whether it’s cloud storage or local data, Curio Anywhere has got you covered. And it’s not just about media files either. Documents are also part of the package. Here’s a major win: Curio Anywhere’s models are developed in-house by GrayMeta, and this means no excessive reliance on costly third-party analytical services.
But if you do prefer using those services, don’t worry Curio Anywhere supports those too, via API. For those of you eager to delve deeper into GrayMeta and their vision, I had a fascinating conversation with Aaron Edell, President and CEO of GrayMeta. We talk about a lot of things, including Curio Anywhere, making coffee, and if the robots will take us over.
Each gadget and tool on the belt represents a different analytical or generative A.I. function designed for specific tasks. And just like Batman has a tool for just about any challenge, CodeProject.AI Server offers a variety of A.I. tools that can be selectively deployed and integrated into your systems, all without the hassle of cloud dependencies. It includes object and face detection, scene recognition, text and license plate reading, and for funsies, even the transformation of faces into anime-style cartoons.
CodeProject.AI Server Modules
Additionally, it can generate text summaries and perform automatic background removal from images. Now you’re probably wondering, “how does this integrate into my facility or my workflow?” The server offers a straightforward HTTP REST API. For instance, integrating scene detection in your app is as simple as making a JavaScript call to the server’s API. This makes it a bit more universal than a proprietary standalone AI framework.
It’s also self-hosted, open source, and can be used on any platform, and in any language. It also allows for extensive customization and the addition of new modules to suit your specific needs. This flexibility means it’s adaptable to a wide range of applications from personal projects to enterprise solutions. The server is designed with developers in mind.
It’s for Developers!
It’s what developers use to collaborate and share code for almost any project you can think of. GitHub is where cool indie stuff like A.I. starts before it goes mainstream. Yes, you too can be a code hipster by using Pinokio and GitHub and you can experiment with various A.I. services before they go mainstream and become soooo yesterday. As you can imagine, if GitHub is for programmers, how can non-coding types use it?
Pinokio is a self-contained browser that allows you to install and run various analytical and generative AI applications and models without knowing how to code. Pinokio does this by taking GitHub code repositories -called repos – and automating the complex setups of terminals, git clones, and environmental settings. With Pinokio, it’s all about easy one-click installation and deployment, all within its web browser.
Diving into Pinoki ‘s capabilities, there’s already a list chock full of diverse A.I. applications from image manipulation with Stable Diffusion and FaceFusion to voice cloning and A.I. generated videos with tools like K and Animate Diff. The platform covers a broad spectrum of AI tools. Pinokio helps to democratize access to A.I. tools by combining ease of use with a growing list of modules as it continues to grow in various sectors.
Platforms like Pinokio are vital in empowering users to explore and leverage AI’s full potential. The cool part is that these models are constantly being developed and refined by the community. Plus, since it runs locally and it’s free, you can learn and experiment without being charged per revision.
Every week there are more analytical and generative A.I. tools being developed and pushed to market.
If I missed one, let me know what they are and what they do in the comments section below or hit me up online. Subscribing and sharing would be greatly appreciated.
5 THINGS is also available as a video or audio-only podcast, so search for it on your podcast platform du jour. Look for the red logo!
Until the next episode: learn more, do more.
Like early, share often, and don’t forget to subscribe.
PSA: This is not a paid promotion.
Is the embedded video not working? Watch on YouTube: https://youtu.be/UO6e_wMAp_0
In 2019, MovieLabs, a not-for-profit group founded by the 5 major Hollywood Studios released a whitepaper called “The 2030 Vision”. This paper detailed the 10 Principles that each of the studios agree is how they want to usher in the next generation of Hollywood content creation.
What are these 10 principles, why should you care, and who the heck is MovieLabs?
Let’s start there.
1. What is Movielabs?Paramount Pictures, Sony Pictures Entertainment, Universal Studios, Walt Disney Pictures and Television, and Warner Bros. Entertainment are part of a joint research and development effort called MovieLabs. As an independent, not-for-profit organization, Movielabs was tasked with defining the future workflows of the member studios and developing strategies to get there.
This led to their authoring of the “2030 Vision Paper”.
2030 Vision Overview
As Movielabs is also charged with evangelizing these 2030 goals, they’ve become the Pied Pipers for the future media workflows of the Hollywood studio system.
Now, what most folks don’t realize is that a vast majority of Hollywood is risk-averse.
Changing existing, predictable (and budgetable) workflows doesn’t happen until a strong case can be made for significant time and cost savings. Without a defined goal to work towards, studio productions would most likely continue on the path they’ve been on, and only iterating incrementally.
So, defining these goals and getting buy-in from each studio meant that there would be a joint effort to realize this 2030 vision. It also gave all the companies who provide technology to the studios a broad roadmap to develop against.
Essentially, everyone sees the blinking neon sign of the 2030 Vision Paper, and everyone is making their way through the fog to get to that sign.
The 2030 Vision Paper is directly influencing the way hundreds of millions of dollars are – and will – be spent.
So, what exactly does this manifesto say?
2. A New Cloud FoundationThe 10 core principles of the Movielabs 2030 Vision Paper can be organized into 3 broad categories: A New Cloud Foundation, Security & Access, and Software-defined Workflows.
The first half of these 10 principles relate directly to the cloud.
This means that any audio and video being captured go directly to the cloud – and stay there. And it’s not just captured content, it’s also supporting files like scripts and production notes. Captured content can either be beamed directly to the cloud or saved locally and THEN immediately sent to the cloud. This media can be a mix of camera or DIT-generated proxies and high-resolution camera originals, plus any audio assets.
As of now, an overwhelming majority of productions are saving all content locally. Multiple copies and versions are then made from these local camera originals. Then, the selected content is moved to the cloud. The cloud is usually not the first, nor primary repository for storage.
Having everything in the cloud is done for one big reason: if all the assets – from production to post-production – are in a place that anyone with permission in the world can access, then we no longer have islands of the same media replicated in multiple places, whether it be on multiple cloud storage pools or sitting on storage at some facility.
We’d save a metric [bleep]-ton of time lost by copying files, waiting for files to be sent to us, and the cost to each facility for storing everything locally.
Of course, having the media in the cloud does present some obvious challenges. How do we edit, color grade, perform high-resolution VFX, or mix audio with content sitting in the cloud?
That’s where Principle #2 comes into play.
Most local software applications for professional creatives are built on the concept that the media’s assets are local, either on a hard drive connected to your computer or on some kind of network-shared storage. Accessing the cloud for all media introduces increased lag and reduced bandwidth that most applications are just not built for.
Despite this, one of the many tech questions answered by the pandemic was “What is the viability of creatively manipulating content when you don’t have the footage locally?”
The answer is that in many cases, it’s totally doable. Companies like LucidLink thrived by enabling cloud storage with media to be used and shared by remote creatives in different locations. LucidLink was the glue that enabled applications to come to the media and not the other way around.
Now, in most cases, we still need to use proxies, rather than high-res files, for most software tools. Editing, iterating, and viewing high-res material that’s sitting in the cloud is still difficult unless you have a tailored setup. Luckily, we still have 7 years before 2030 – and advancements on this front are constantly evolving.
Working with everything while it’s in the cloud also means that when it’s time to release your blockbuster, you can simply point to the finished versions in the cloud – no need to upload new versions.
That’s where Principle #3 comes in.
This means no more waiting for the changes you’ve made on a local machine to export and then re-upload, then generate new links and metadata, as well as the laundry list of other things that can go wrong during versioning. The cloud will process media faster in most cases, as you can effectively “edit in place”.
The promise is that of time savings and less room for error.
Moving on, Principle #4 gets a bit wordy, but it’s necessary.
Archives generally house content that is saved for long-term preservation or for disaster recovery, and should rarely, if ever, be accessed. This is because archived material is typically saved on storage that is cheaper per Terabyte, but with the caveat that restoring any of that archived content will take time, and thus money.
This philosophy is as true for on-premises storage as it is for cloud storage. It can simply be cost-prohibitive to have massive amounts of content on fast storage when access to it is infrequent.
So, Principle #4 is saying that not everyone uses archives the same way, and if you do need to access that content in the cloud somewhere down the road, your archive should be set up in a way that makes financial sense for the frequency of access, while also ensuring that secure user permissions are granted to those who understand this balance.
Principle #4 also dovetails nicely into a major consideration with archived assets. Ya know, making more money!
Sooner or later, an executive is gonna get the great-and-never-thought-of-before idea to re-monetize archived assets. Whether it’s a sequel, reboot, or super-duper special anniversary edition, you’re going to need access to that archived content. Our next principle provisions for this.
Having content in a single location that you’re allowed to see is one thing, but how you accurately search for and then retrieve what you need is, for now, a challenging problem.
Let’s take this one step further: Years or even decades may have passed since the content was archived, and modern software may not be able to read the data properly.
Some camera RAW formats, as the paper points out, are proprietary to each camera manufacturer and need to be DeBayered using their proprietary algorithms.
So, do we convert the files to something more universal (for now) or simply archive the software that can debayer the files?
We simply don’t have a way to 100% futureproof every bit of content.
What I’m getting at here is that the content has to be fully accessible and actionable – all in the cloud.
Slow down, and take a breath, as Security and Access are one of the 3 areas of attention in the 2030 Vision paper.
Peeling the onion layers back, this will require the industry to devise and implement a mechanism to identify and validate every person who has access to an asset on a production, which we’ll call a “Production User ID”.
This would be for all creatives, executives, and anyone else involved in the production. This “Production User ID” would allow that verified person to access or edit specific assets in the cloud for that production.
This also paves the way for permissions and rights that are based on the timing of the project.
As the 2030 Vision Paper calls out, “A colorist may not need access to VFX assets but does need final composited frames; a dubbing artist may need two weeks’ access to the final English master and the script but does not need access to the final audio stems; and so on.”
As Hollywood frequently employs freelancers – and executives change jobs – this also allows for users to have their access revoked when their time on the production is over.
In theory, tying this to a “Production User ID” means all permissions for any Production you work on are administered to your single ID. You wouldn’t have to remember even more usernames and passwords, which should make all of you support and I.T. folks watching and reading this a bit happier.
As you can imagine, accessing the content doesn’t mean it’s 100% protected while you’re using it.
This is where I hope a biometric approach to security takes off sooner rather than later, as the addition of new security layers on top of aging precautionary measures is often incredibly frustrating. Multiple verification systems also increase the chance of systems not working properly with one another, which is a headache for everyone…and your IT folk.
Movielabs suggests “Security by design;’ which means designing systems where security is a foundational component of system design – not a bolt-on after the fact. There is also the expectation that safeguards will be deployed to handle predictive threat detection and that hopefully this will negate the need for 3rd party security audits, which is a massive headache and time suck.
The security model also assumes that any user at any time could be compromised. Yes, this includes you, executives. You are not above the law. This philosophy is also known as “Zero Trust”, and if you had my parents, you’d totally understand the concept of “zero trust”.
The last Principle in the “Security and Access” section is #8:
If you’ve done any kind of work with proxies, or managed multiple versions of a file, then you’ve dealt with the nightmare that is relinking files. Changes in naming conventions, timecode, audio channels, and even metadata can cause your creative application du jour to throw “Media Offline” gang signs.
And this isn’t just for media files. Supporting documents like scripts, project files, or sidecar metadata files can frequently have multiple versions.
This means that in addition to needing a “Production User ID” per person, we also need a unique way to identify every single media asset on a production; and every user with permission needs to be able to relink to that asset inside the application they’re using.
That’s a pretty tall order.
The upside is that you shouldn’t have to send files to other creatives – you simply send a link to the media, or even just to the project file, which already links to the media.
The goal is also to have this universal, relinking identifier functioning across multiple cloud providers. The creative application would then work in the background with the various cloud providers to use the most optimized version of the media for where and what the creative is doing.
Oh yeah, all of this should be completely transparent to the user.
This principle covers two huge areas. The first is a common “ontology” – that is, a unified set of terminology and metadata, as well as common industry API.
Consider this: You’ve been hired to work on The Fast and Furious 27, and there is, unpredictably, a flashback sequence.
You need access to all media from the 26 previous movies to find clips of Vin Diesel, err Dom, sitting in a specific car at night, smiling. Have the 26 previous movies been indexed second by second, so you can search for that exact circumstance? How do you filter search results to see if he’s sitting in a car or standing next to it? How do you filter out if the car is in pristine shape or riddled with bullet holes after a high-speed shootout with a fighter jet?
As an industry, don’t have “connected ontologies” meaning, we can’t even agree that the term for something, say, in a production sense is the exact same thing in a CGI sense. If we can’t even agree on unified terms for things, how we can label them so you can then search for them while working on the obvious Oscar bait Fast and Furious 27?
The second ask is a common industry API or “Application Programming Interface” that all creative software applications use.
This allows the software to be built in such a way that it can be slotted into modular workflows. This is meant to combat compatibility issues between legacy tools and emerging workflows…and reduce downtime due to siloed tech solutions. While each modular software solution can be specialized, there will also be a minimum set of data, metadata, and format support that other software modules in the production workflow can understand. This also means that because of this base level of compatibility, all creative functions will be non-destructive.
Wait, you may be thinking…how in the world is this accomplished?
All changes made during the production and post-production process will be saved as metadata. This metadata can be used against the original camera files. This provides not only the ultimate in media fidelity but also the ability to peel back the metadata layers at any time to iterate.
We wrap up Principle #10:
I get this; no one likes to wait.
When and if Principle #10 is realized, it means no more waiting for rendering, whether it be for on-set live visual effects, game engines, or even CGI in post. The processing in the cloud will negate the need to wait for renders. Feedback and iteration can be done in a significantly shorter amount of time. Creative decisions onset can be made in the moment rather than based on renders weeks or months later.
The 2030 Vision Paper also suggests this would limit post-production time – as well as budget – as more of each would be shifted towards visualizations in pre-photography.
There are a ton of moving parts – with both business and technology partners – that Movielabs annually checks the industry’s progress towards the 2030 vision; and Movielabs provides a gap analysis current state of the industry and the work that remains.
Without question, our industry has made advancements in areas like camera-to-cloud capture, VFX turnovers, real-time rendering, and creative collaboration tools that are all cloud-native. The Hollywood Professional Association – or HPA – Tech Retreat this year, which has become the de facto standard for must-attend industry conferences, showcased many Hollywood projects, and technology partners, like AWS, Skywalker, Disney Marvel.
The 10 principles need a foundation to build upon, and that means getting this cloud thing handled first. And that’s just what we saw.
Principles 1, 2, and 3 were the predominant Principles that large productions attempted to tackle first, with a smattering of “Software Defined Workflow” progress.
However, these case studies also reflected where we do need to make more progress, as we are already 4 years into this 2030 vision. These gaps include:
We also have gaps in operational support, where workflows are complex and often span multiple organizations, systems, or platforms.
If we take a look at Change Management, we do run into fundamental problems with a move of this scale.
I’m sure you have some input on one or more of these 5 THINGS. Let me know in the comments section. Also, please subscribe and share this tech goodness with the rest of your techie friends.
5 THINGS is also available as a video or audio-only podcast, so search for it on your podcast platform du jour. Look for the red logo!
Until the next episode: learn more, do more.
Like early, share often, and don’t forget to subscribe.
Editing & Motion Graphics: Amy at AwkwardAnthems.
Not too long ago, I posted on several social media social platforms asking what questions YOU had on AI.
Reddit | Facebook | Twitter | LinkedIn
I’ve taken all of the 100+ responses, put them in nice neat little (5!) categories, and – wow – do I have a deluge for you.
1. Current AI ToolsBy far, the most asked question was “what tools are available today?”
The easiest life is that of automated content analysis. For a hundred years, we’ve relied on a label on a film canister, a sharpie scrawl on the spine of a videotape case, or the name of a file sitting on your desktop.
Sure, while this tells you the general contents of that media, it’s not specific.
“What locations were they at?”
“What was said in each scene?”
“Were they wearing pants?”
This is where AI’s got your back. With content analysis, sometimes called automated metadata tagging, AI can analyze your media, recognize logos, objects, people, and their emotions, plus, transcribe what was said, and generate these time-based metadata tags. It’s a little bit like having Sherlock Holmes on your team, solving the mystery of the missing footage.
Because there is always a better take somewhere….right?
AI-assisted color grading is also starting to gain traction. With the help of AI models, colorists can quickly analyze the color palette of a video and create a consistent base grade.
This can save a ton of time when you’re dealing with various bits of media with different formats and color profiles. It does the balance pass so you can start to create …quicker.
AI can also suggest starter options for more creative color grades.
Like…adding a touch of “teal and orange”.
Colorists can also use existing content as an example to tell the AI the look you’re aiming for. Check out Colourlab.ai.
Additionally, AI tools are revolutionizing audio post-production, too. Noise reduction algorithms powered by AI can effectively remove unwanted background noise, enhance dialogue clarity, and improve overall audio quality. While this tech has been around for a while, adding AI models to existing algorithms is making these tools even more powerful.
These models also help with Frankenbiting.
Text-to-speech tools that also allow generated audio to sound like someone else – also called voice cloning – is a fantastic tool for Frankenbiting. For most of us commercially, it’s great right now for scratch narration or to replace a word or two, but it’s not quite at the point where you can easily adjust intonation. However, when that is possible, editors may possibly have a new skill to learn – crafting the performance of the voice-over in the timeline. Check out Elevenlabs.
Text-to-image and text-to-video products are rapidly being developed, just look at services like Runway, Midjourney, Dall-E, Kaiber, and a host of others. And while we can’t generate believable video b-roll on the fly (unless you like the current generative AI aesthetic) once that boundary is crossed, this will change the way editors work.
OK, instead of adding…can we subtract?
Rotoscoping tools are readily available in industry-standard tools like Photoshop. We can erase objects in near real-time and also use tools like Generative Fill for AI to “guess” what is missing right outside of the frame.
Now, Large Language Models – or LLMs – are the key for AI to understand what you want when you ask it a question. LLMs are also imperative for our next skill….text to code. Now, why is that important to you?
Many motion graphics software packages have an underlying code base you can use to script the actions you want. So, until text-to-video becomes more mature, you can use AI to generate code and scripts to tell the motion graphics tool what you want to do. Check out KlutzGPT.
All of the aforementioned model types are just a sampling of the broad categories of specific AI models. In fact, HuggingFace, a repository for various AI models and datasets, has nearly 40 categories of models for you to download, train, and experiment with to assist you on your next project.
2. Adapting to AI Evolution“It is not the strongest species that survive, nor the most intelligent but the ones most adaptable to change”, once wrote Leo Megginson. And while he’s not wrong, I much prefer my attention span appropriate “Adapt or Die”, by Charles Darwin.
AI and its many variants are exploding. AI has already become the fastest-adopted business technology in history.
So, how can you adapt to this?
Well, the first step is to not panic.
DON’T PANIC.
If you’ve used any type of AI, you can totally see the current deficits. Whether it’s factual hallucinations or relatives with 20 fingers and 37 toes, AI will be evolving, which gives you time to learn and use AI tools as your sidekick, and not your replacement.
As AI for the general public is still in its infancy, it’s critical for us to learn about the AI models we’re utilizing. This means what data was used to train them, and who has curated that same data the models were trained on? This level of openness not only empowers us to make better decisions when selecting model providers but also fosters a culture of responsibility. We’ll discuss this a bit more later.
As with any moment in the zeitgeist, it’s imperative that we understand the difference between what’s the real deal, and what’s part of the hype machine. We’re likely to see the “AI” label slapped on many tools. Now, you may not think this is a big deal, but it can be.
To start off, it’s a matter of investment and value. You want to ensure you’re getting the whizzbang AI capabilities you’ve paid for, and not just a repackaged widget marketed as AI.
Genuine AI tools can also perform complex tasks, adapt to new situations, and sometimes even learn from those experiences, leading to more efficient processes and smoother outcomes. Non-AI tools or overhyped plugins, however, may not deliver their promised results, leading to major disappointment and potential setbacks in your project or business.
Plus, understanding the capabilities of AI tools can lead to better usage. Knowing what an AI tool can and can’t do allows you to utilize it to its fullest potential.
On the other hand, blatantly mislabeling a non-AI tool as AI can lead to wasted time or underuse.
Lastly, it’s really about ethics and transparency. Misrepresenting a tool’s capabilities is deceptive marketing, plain and simple. This is already a major problem – and a rarely policed one at that – in the tech world. It erodes trust between providers and users and can lead to skepticism about the entire field of AI.
Now, speaking of Ethics….
3. Ethics in AI UsageAs we leverage AI to push the boundaries of creativity, we’re faced with new dilemmas, around privacy, security, bias, and yes, our core ethics.
These aren’t just questions for the tech wizards or the philosophical ponderers. They’re issues that each one of us, as contributors to the creative world, needs to grapple with.
Why?
Because the decisions we make today will shape the digital landscape of tomorrow.
If we start with your creative tools, such as the ability for voice cloning and face swapping – which do have legitimate applications in our industry – they also raise ethical issues such as privacy, consent, authenticity, and accountability.
For example, voice cloning and face swapping could be used to create fake news, deep fakes, or malicious impersonations that could harm the reputation or even the safety of the original speakers or actors. Plus, bad actors could undermine the trust and credibility of the creative content and its sources.
To solve this, we need explicit consent from the original person before their voice or face is used. No more unsolicited ‘borrowing’. These technologies aren’t a license to steal.
Next, we need transparency. Any content using these technologies should have some form of credit or disclaimer, to ensure the audience knows what’s up.
Finally, accountability and traceability are key. Keeping track of source data and synthetic outputs ensures responsible use. This means finally deciding on and implementing some form of chain of custody solution, such as the content authenticity initiative.
In essence, we’re talking about a culture of responsibility in AI usage, balancing the scales of creativity and ethics.
From a macro perspective, data privacy and security have become increasingly critical ethical concerns within the AI sphere. As AI systems extensively rely on vast amounts of your personal data, the issues of data ownership and protection have exploded. To handle this, it’s imperative to establish and enforce stringent data protection guidelines.
The problem is, we’ve already been using the internet for a few decades now, and a good chunk of your data is already out there. That forum terms-of-service that you just scrolled past and clicked I ACCEPT on, has your data, which they can potentially monetize in any number of ways – including using it to train new AI models.
Now, ethical online services could certainly provide users with a user-friendly opt-out option, particularly for AI services that are deemed excessively invasive. Similar to the CAN-SPAM Act in the US and GDPR regulation in the UK, this could allow users to opt out of the use of any of their data to be used in training AI models.
There are several generative AI cases currently making their way through the U.S. legal system, including lawsuits against Stability AI, Midjourney, and DeviantArt, who are accused of mass copyright infringement. Microsoft, which owns GitHub, and OpenAI are also being sued over Microsoft Copilot’s tendency for reproducing developers’ publicly posted, open-source, licensed code. Keep tabs on these cases as they will shape how the combined art you make with AI is recognized.
Another critical point to consider is the potential for bias and discrimination in AI. The danger here is that biased data can lead to AI systems that further perpetuate those biases. The key to breaking this cycle is ensuring that the data used to train AI is diverse and unbiased and captures multiple perspectives. It’s also essential to regularly monitor AI systems to identify and rectify any biases that may sneak in.
Mandating AI model providers to publish their datasets to the public won’t fly, which means some form of audit by a 3rd party. The immediate thought here is some form of regulatory body, which I really can’t see a way around.
Transparency in AI algorithms is also an integral part of ethical AI use. It’s important for creators to understand how and why AI systems make certain decisions. As AI for the masses is a relatively new technology, we all need to become educated on the models we’re using. This kind of transparency can lead to more informed model provider choices and quite frankly encourages a sense of accountability.
Update: OpenAI, Google, others pledge to watermark AI content for safety, White House says.
4. Societal Implications of AIThe broader societal implications of AI in the creative industry extend well beyond your editing fortress of solitude. As AI continues to transform workflows and redefine job roles, it is crucial to support your fellow creatives in adapting to these changes.
One way to support creatives is through continued education and upskilling programs. As AI tools become more prevalent, editors should embrace lifelong learning and acquire new skills to stay relevant in our industry. By developing a deeper level of understanding of AI technology and its applications, editors can leverage AI to their advantage and remain valuable contributors to the creative process.
As Phil Libin, creator of Evernote and mmhmm, and current CEO of All Turtles said: “AI won’t replace any humans, humans using AI will.”
Phil’s quote, but my presentation – LACPUG May 2023
Like it or not, Democratic governance of AI is also crucial to address potential risks and ensure responsible AI development and usage.
Transparent regulations and ethical guidelines can help shape the future of AI in a way that aligns with our societal values and prevents misuse.
I know, that last statement is loaded with several gotchas, but I really don’t see another option.
Washington has historically played catch up on all things technology, but the current administration has published the “AI Bill of Rights” which speaks to many of these topics.
However as of now, they are not enforceable by law, and there aren’t any federal laws that explicitly limit the use of artificial intelligence, or protect us from its harm.
5. AI Evolution & ImpactAt its core, what we do in post-production is all about storytelling, and AI, as fascinating as it is, is merely another tool in our toolbox. It’s true, AI can analyze data, identify patterns, and even suggest edits. Heck, it can generate content based on predefined parameters. But, no matter how advanced it becomes, it lacks the innate understanding of the human condition and the emotional and cultural context that you, as artists, possess.
To be clear: Don’t believe the job loss hype. AI is not about to take over our jobs. Please don’t fall for the classic “Lump of Labour” misconception that automation kills jobs. It’s simply not true. Technology serves as the spark for productivity enhancement, making people more efficient in their work. This increased efficiency triggers a chain reaction: it drives down the costs of goods and services and pushes up your wages. The net result is a surge in economic growth and job opportunities. But it doesn’t just stop there. It also inspires the emergence of fresh jobs and industries; ones that we couldn’t have imagined just a few short years ago.
In the face of evolving AI technology, the role of editors is not diminishing…but transforming. You are the bridge between the cold calculations of AI and the warmth of human connection. We infuse videos with a depth of storytelling that resonates with audiences, touches hearts, and sparks imaginations. AI cannot replace your creative intuition or your storytelling skills. It’s the human touch that adds the emotional depth, the nuanced transitions, and your profound connection with viewers. Remember that, my fellow creatives, and let’s shape the future of our industry together.
I’m sure you have some input on one or more of these 5 THINGS. Let me know in the comments section. Also, please subscribe and share this tech goodness with the rest of your techie friends.
Even if they are AI.
5 THINGS is also available as a video or audio-only podcast, so search for it on your podcast platform du jour. Look for the red logo!
Until the next episode: learn more, do more.
Like early, share often, and don’t forget to subscribe.
Editing & Motion Graphics: Amy at AwkwardAnthems.
Michael takes a deep dive into AI for Post Production. He'll answer YOUR top 5 Artificial Intelligence questions so you’re prepared for the creative new world order with AI.
Michael takes a deep dive into the various ways to edit remotely with Adobe Premiere Pro, when you or your team, plus your computers and media just can’t be in the same place at the same time.
All of you are asking the same thing. “How can I edit remotely or work from home?” Today we’ll look at the options you have with Avid Media Composer.
On this episode, we gonna get hiiiiiigh! In the clouds, with a primer on using the cloud for all things post-production. 1. Why use the Cloud for Post Production? 2. Transfer and Storage 3. Rendering and Transcoding and VFX 4. Video and Audio Editing and Finishing 5. Review and Approve
We’re checking out the new 2018 Mac Mini from Apple, and the new eGPU Pro offering from Blackmagic and …well, Apple. We’re also running benchmarks against your favorite NLEs!
I’ve got a few tricks that you may not know about that will help you upload, manage, and make YouTube do your bidding.
Have you ever thought about using a Hackintosh? Wondering how they perform? Or maybe you wanna build one? Fear not my tech friends, for in this episode, we’ve got you covered.
In this episode, we’re going to dive deep into live streaming. My friends at LiveU asked me to do a deep dive on their Solo streaming device. I suggested, “why not a shootout between the LiveU Solo and the Teradek VidiU Pro?”
Always dreamed of having your own TV Channel? With Roku's new "Direct Publisher" you can - and I'll show you how to with a no-coding walkthrough!
AJA has a brand new all-in-one portable, recording, and streaming device - the AJA Helo! 5 THINGS did a review.
Volume 1 was so popular that we've done a sequel! Special guest Vince Rocca rejoins Michael to set the story straight on 5 more post myths! 1. YouTube needs compressed video 2. Bare drives are safe for storage 3. You can’t create ProRes on Windows 4. Editors only need to know how to edit 5. Video needs to be action and title safe
Everyone has their favorite video editor. But what do Hollywood Film and TV editors use? What do others use? I break down all of your favorite NLEs here. 1. Avid Media Composer 2. Apple Final Cut Pro X 3. Adobe Premiere Pro 4. Everything Else 5. The Future
Help make the post audio person's day with these tips on prepping your audio tracks in your video editor correctly! 1. Track Layouts 2. Sync 3. Video 4. OMF and AAF Exports 5. Odds and Ends
Transcoding is required in many areas of post-production - which is why you should know about the process! 1. Onset and Editorial 2. Review and Approve, and Audio 3. VFX and Color 4. Distribution 5. Tips
Are you interested in live streaming, and want to go larger scale than just your smartphone - and don't know where to start? You start here! 1. What components do I need? 2. CDNs?! 3. Common software solutions 4. Common hardware solutions 5. Tips and considerations
There's a ton of misinformation out there on technology in post-production. Let's set the record straight on 5 of 'em! 1. Transcoding to a better codec will improve quality 2. Log formats are the same as HDR 3. You can grade video on a computer monitor 4. You can fix audio distortion 5. Storage is the same as capacity
Offline and Online workflows are used throughout the industry. Let's dive in and see if they'll work for your project! 1. What is an offline/online workflow? 2. What are some examples? 3. What are the benefits? 4. What are the downsides? 5. Is there a hybrid?
Is it time for a new edit system? Great! Let's dive into what components you're going to need to know about in order to get the most bang for your buck! 1. Mac, PC, Linux? 2. CPU 3. GPU and RAM 4. Boot "Drive" and Internal Storage 5. What would you choose?
Wanna improve your audio inside your video editing software? Michael brings 5 awesomely easy tips from his post-production audio editing days in this episode. 1. Dialogue Clarity 2. Noise Reduction 3. Reducing Reverb 4. Levels and Placement 5. Last Tips
Archiving media is not as easy as it seems! There are many options and considerations when choosing an archive strategy.
Let's face it - we're creating (and thus saving )more and more media. This means we need to search and retrieve it easily. Asset Management can do this - and a ton more!
VR and 360 videos are the new tech sexy. Let's take a look at how we shoot, edit, and watch VR! 1. What is VR? 2. How do I shoot VR? 3. How do I edit VR? 4. How do I watch VR? 5. What's the future for VR?
LTO is a cheap and secure way to store large amounts of media. Let's look into why LTO is a great fit for post-production. 1. What is LTO? 2. Why should I care? 3. What are the downsides of LTO? 4. What tech do I need to use LTO? 5. How much does LTO cost?
HDR is the next evolution of video. It can be difficult to work with - but less difficult after checking out this episode! 1. What is HDR? 2. How do I shoot HDR? 3. How do I edit HDR? 4. How do I view HDR? 5. What's the future of HDR?
In part 3 of a 3 part series, we look into shared storage for media. In this episode, we focus on management, permissions, and support. 1. Why do permissions matter? 2. How do I share projects and media? 3. What management features should I look for? 4. What kind of support do I need? 5. What shared storage solutions do you recommend?
In part 2 of a 3 part series, we look into shared storage for media. In this episode, we focus on drives, size, spindles, and protection.
In part 1 of a 3 part series, we look into shared storage for media. In this episode, we focus on SAN, NAS, bandwidth, and connections. 1. What is a SAN and NAS? 2. What is bandwidth and why is it important? 3. How much bandwidth does my video take up? 4. How many streams of video can I play? 5. What else do I need to know about bandwidth?
Today we're delving into the wonderfully geeky realm of codecs, the fundamentals of usage in post, and best practices. Let's roll up our sleeves and get ready for some acronyms. 1. What is a codec? 2. What do I need to be aware of when choosing a codec? 3. What codec should I edit with? 4. How do I convert from one codec to another? 5. How do I get the best look for _______ ?
Today we're looking at Dialogue Search from Nexidia. Indexing and searching for content, based on audio. 1. What is Dialogue Search? 2. Where is Dialogue Search used? 3. How does Dialogue Search work? 4. How accurate is it? 5. How much does it cost?
Today we're looking at UHD and 4K. What you need to know for production and post-production, and how you can use it.
On the inaugural episode of 5 THINGS, we'll be looking at one of my favorite new technologies: Adobe Anywhere. 1. What is Adobe Anywhere? 2. How does Adobe Anywhere work? 3. How does Adobe Anywhere perform? 4. How can I get Adobe Anywhere? 5. How much does Adobe Anywhere cost?