Optical flow node for synthesising motion vectors for temporal denoising

[SimonHeggie]

Hi there,

I would like to share an idea I have for a workflow which would probably require this kind of addon be compatible with the optical flow node. The goal effect is to deflicker using a temporal denoising compositor workflow. Currently the bar of entry to proper temporal denoising is that you must render with cycles in order to generate motion vector passes. This would make temporal denoising available to everyone including film makers, 2D animators and AI animation artists.

https://www.youtube.com/watch?v=jyGQIKOPwGA

So by using optical flow vector maps; there may not be too much vector math required to convert this to something that works similarly to the vector motion pass required for temporal denoising.

I have not tested this but I wonder if anyone has thought of this yet?

In theory you would have 3 image sequences, each offset as is set up in the video. Then, in the comfyUI process; simply apply the optical flow node; and extract the optical flow data and transfer those three frames over to compostor: apply the standard temporal denoising using the 3 image sequences in compositor and their respective AI generated optical flow maps. Potentially some vector map is needed like normalise or something, not sure.

Does this seem like something I should try putting together and testing?

If Optical Flow node is not currently supported; could it kindly be added to make this kind of thing possible?

https://github.com/seanlynch/comfyui-optical-flow

Let me know what you think!

-S