One of the most interesting sessions at Adobe MAX is traditionally the Sneaks keynote, where engineers from the company’s various units show off their most cutting-edge work. Sometimes, those turn into products. Sometimes they don’t. These days, a lot of the work focuses on AI, often based on the Adobe Sensei platform. This year, the company gave us an early look at Project Sweet Talk, one of the featured sneaks of tonight’s event.
The idea here is pretty straightforward, but hard to pull off: take a portrait, either a drawing or a painting, identify the different parts of the face, then animate the mouth in sync with a voice-over. Today, Adobe’s Character Animator (which you may have seen on shows like The Late Show with Stephen Colbert) does some of that, but it’s limited in the number of animations, and the result, even in the hands of the best animators, doesn’t always look all that realistic (as far as that’s possible for the kind of drawings you animate in the product). Project Sweet Talk is far smarter. It analyzes the voice-over and then uses its AI smarts to realistically animate the character’s mouth and head.