Adobe has demonstrated Project Awesome Audio – an experimental audio tech that removes background noise from recordings.
The Sneaks session during its Adobe Max conference always shows off some cool tech that might appear in future versions of Photoshop and Adobe’s other Creative Cloud apps.
Everything shown off is at an experimental stage and isn’t anywhere ready for shipping. Some demos were just code at present while others barely worked.
However, Project Awesome Audio – created by Adobe researcher Zeyu Jin – was clearly at an advanced stage. It was able to take very dirty audio and clean it up. In one instance the narrator of an audiobook wasn’t picked up by her proper microphone, but the audio was instead recorded by a laptop’s built-in mic. This was no problem to clean up.
In the second demo, the recording started in one room and then moved to another. The second room had background noise. In one click, the software – running on top of Adobe Audition – was able to clean up the audio. The tech is powered by Adobe’s own Sensei AI to process the background noise and bring the audio up to the right level.
— Mike Russell (@imikerussell) November 6, 2019
Some of the other demos were less ‘wow’ but still highly impressive. One compared two photos to find the missing person – say someone took a photo of their family so they weren’t in the resulting shot. The demo, which was basically a bunch of code at present, was able to locate the missing person and paste them into the family shot.
Another cool demo – Project Sweet Talk – was able to animate any flat image to match an audio recording of someone’s voice. And a further example was able to track the movement of a person in a video and then apply that movement to an animated character.
— Amy Schlueter @ AdobeMAX (@atombombamy) November 6, 2019
Light Right enables you to analyse an image and change the position of the light (normally the sun or daylight) in an image – essentially re-lighting the image in a different way using the 3D geometry of an image and machine learning. We were also shown how the algorithm could potentially work in a video. It can also use the lighting from another photo online, say of a tourist spot, to re-light your own photo in a different way.
Finally, Image Tango can apply the texture from one photo to the shape of another – say with a sketch or painting to come up with completely new images. The demos we were shown saw the software use the look of a photo of a bird and apply the same colours to a sketch.
There’s a good chance of at least some of this tech making it into future Adobe products. The auto reframe feature that debuted in Premiere Pro at yesterday’s Adobe Max keynote was first shown off in last year’s Sneaks event. It helps when you’re cropping video to a smaller format – for Instagram, for example – but the subject keeps moving in and out of the frame. With auto-reframe, the video will keep following the subject.
And last week, Adobe showed off its awesome new Object Selection tool – it’s coming to Photoshop next year: