Needed AVISynth Plugins: TIVTC, RIFE, RemapFrames

Effectively what this technique does is generates the needed 29.97 -> 23.976 framerate conversion on the fly within an AVIsynth script using the Real-Time Intermediate Flow Estimation plugin. By assigning the conversion to a variable pre-decimation, we’re able to then ‘inject’ converted frame sections back into the decimated video sequence using the RemapFrames plugin after the decimatation. This greatly decreases the time needed to process a video file since the conversion doesn’t require an exported a full length clip for potentially a couple of short sequences.

Model 44 is the latest plugin I’ve tried and yields excellent results that is on par or better than what I’ve been able to get from exporting a converted framerate file from Topaz Video AI. Model 48 looks to the latest high quality model and may deliver better results but I haven’t tried it yet.

ReplaceFramesSimple() is the function we’ll be using to specify which sections need to be replaced with the converted framerate. It accepts a string input that can specify frames in two ways: single frame values, or a range of frames in the “[firstframe lastframe]” format. A valid string could be “100 [200 300]” which would replace frame 100 with the variable’s frame, and frames 200 to 300. Frame values are inclusive.

Here’s an example script to get started:

RIFEMappings="[101 154] [157 199]"

### Load Source ###
LSMASHVideoSource("videofile.mp4")

### Convert to 23.976 fps and assign to RIFE variable (Replace trailing ConvertBits and ConvertTo with the original file's formats) ###
RIFE=ConvertBits(32).ConvertToPlanarRGB(matrix="Rec709").RIFE(model=44, fps_num=24000, fps_den=1001, gpu_thread=1).ConvertBits(8, dither=1).ConvertToYUV420(matrix="Rec709")

### Decimate Duplicate Frames ###
tdecimate()

### Inject RIFE converted frames via the RIFE Mappings
ReplaceFramesSimple(RIFE, Mappings=RIFEMappings)

Frame values in “RIFEMappings” need to stop and start around scene changes. For example, if your 29.97 fps section is from frame 100 to frame 200, you’ll want to specify [101 199] as the frame values. I’ve found that RIFE doesn’t do a great job detecting scene changes, and in fact, RIFE’s scene change detection is disabled by default. If there’s a scene change (cut) within the 29.97 section (let’s say between frame 155 and 156), then you’ll want your mappings to be “[101 154] [157 199]”. By not doing this at the scene change, RIFE will create intermediate frame with a rather distracting warping effect.

RIFE requires a 32 bit planar RGB input to work. Simply convert back to your original format at the end of the RIFE= line.