Watch As NVIDIA Noise2Noise AI Magically Fixes Grainy Photos With Deep Learning
NVIDIA used a team of Tesla P100 GPUs along with the cuDNN-accelerated TensorFlow deep learning framework to train Noise2Noise using over 50,000 images from an ImageNet dataset using both noisy and clean versions of an image. To test the performance of the system post-training, a sample of original “clean” images served as a reference point to compare to later. That original image was then altered to add in randomized noise as you can see in the image below.
Noise2Noise would then use is denoising algorithms to generate a near-perfect result of what the system “thinks” the image should look like.
What’s really interesting is that the algorithm used for photos can also be applied to video footage. It can be used when scenes are being rendered for traditional movies or animated features. Instead of waiting for a frame to fully render, which can sometimes take hours depending on the complexity, it can be partially rendered and the de-noise algorithm can step in to quickly and efficiently clean up the noise.
“It is possible to learn to restore signals without ever observing clean ones, at performance sometimes exceeding training using clean exemplars,” wrote the researchers in their paper [PDF]. “[The neural network] is on par with state-of-the-art methods that make use of clean examples — using precisely the same training methodology, and often without appreciable drawbacks in training time or performance.”
Not only have NVIDIA and its academic partners used Noise2Noise to help restore grainy photos, but they are also using it for Magnetic Resonance Image (MRI) scans, which can be extremely beneficial in the medical sector. Nearly 5,000 images of human subjects from the IXI dataset were used to train Noise2Noise for MRI restoration.
The NVIDIA-led team will present their finding this week at the International Conference on Machine Learning which is being held in Stockholm, Sweden.
