MTI Film’s has announced a new feature for Cortex Enterprise: a tool for frame-rate conversion. Scheduled for release this month, the tool leverages artifical intelligence-assisted machine vision technology to automate the process of converting content to a different frame rate while avoiding artifacts caused by camera motion.
“It’s a common task to convert media frame rates, for example from 30fps to 24,” said MTI Film CEO Larry Chernoff, “but you run into problems with shots that include lateral motion. If you simply remove one frame out of each five, you get a jumping effect.”
MTI Film began searching for a solution to this problem while remastering a television series from the 1990s. Source material from the show included a mix of 30- and 24-fps original camera negative that needed to be brought into alignment. The post team first tried eliminating every fifth frame from the 30fps media, but the camera motion resulted in too many jumps and other artifacts. They then tried an existing automation tool that used AI to identify frames that could be safely eliminated, but it produced too many errors and was too slow.
“It took 40 hours to process a 46-minute episode,” Chernoff recalled, “and that didn’t include the time it took in an edit bay to correct the artifacts, especially the ones occurring at each cut.”
So, MTI Film set about creating its own solution. It used an open-source AI machine vision software, originally developed at Beijing University, that essentially reconstructed frames through interpolation. This solution was successful in creating high quality frame interpolation, but it still produced artifacts at each cut and required a lot of processing time. MTI Film addressed that limitation through distributed processing.
“Using an [edit decision list], we parsed and distributed over 700 shots across ten machines and got them back in what resulted in real-time,” Chernoff said. “As part of the tool, we also created an algorithm that accounted for each shot’s time, thereby maintaining sync and producing clean cuts. After that, all that was left was QC and replacement of the few frames that needed additional attention.”
“There is a lot of content that started its life as 30fps but now needs to be 24fps,” Chernoff said. “Previously, it was simply too time-consuming and prone to error, but this tool makes it practical.”
MTI Film www.mtifilm.com