Dear actresses, dear actors, deepfakes will change your life

James Dean, who died in a tragic car accident in 1955, will soon play the role of Rogan in “Finding Jack”, a film adapted from Gareth Crocker’s book published in 2008. For the duration of the film, synthetic media will bring back the legend of “Rebel Without a Cause” from the dead. More than an event, it is a true revolution that the world of cinema and the community of actresses and actors are about to experience

❋

After a one-year hiatus, the Cannes Film Festival 2021 will inaugurate its first ascent of the steps on July 6. A must-see event for the film industry worldwide, actresses and actors are naturally the most expected on the red carpet of the Palais des Festivals. However, many of them are unaware that this small world will probably undergo a profound upheaval in the years to come.

The massive adoption of motion capture, virtual sets, and CGI (Motion capture: a technique for capturing the movement of actors by detecting tracers in a virtual environment in three dimensions (The virtual set) – CGI: Computer Generated Images or special effects) that have been used to make all the greatest science fiction films of the last 20 years, from James Cameron’s Avatar to the Avengers of the Marvel Universe or Star Wars and many others has already profoundly changed the way we act in the United States on some Hollywood productions.

For some time now, synthetic media have interested special effects studios and deepfakes (definition) in particular are making their appearance in some productions or mixed with more traditional digital creation techniques. The avowed objective of specialized studios is to reproduce as faithfully as possible the performances of their actors (and even of our furry friends who accompany them) to push back even further the limits of creativity.

Tools are being developed, notably in the video game industry, to generate synthetic humans ¹ Metahuman, a new artificial face by UnReal, journalism. design, March 4, 2021 which generates versions that are certainly similar, but not completely bluffing. On the other hand, attempts are multiplying all over the world to generate synthetic humans capable of substituting themselves for real individuals. The debut of the synthetic presenter took place first in China, in 2018 where the Xinhua news agency propelled an English-speaking presenter to the forefront ² China’s Xinhua agency unveils AI news presenter, BBC, November 8, 2018 .

In early 2020, British news agency Reuters, for example, partnered with Synthesia, a London-based company that specializes in making synthetic humans, to create a fully artificial presenter ³ Reuters and Synthesia unveil AI prototype for automated video reports, Reuters, 7 February 2020. In Korea, the cable channel MBN has also developed its own AI-synthetic anchor Kim ((South Korean News Channel Introduces An AI-Powered News Anchor, ubergizmo.com, 16 November 2020)).

Hollywood seeks its synthetic double

From the 8 o’clock news to the movie sets, there is only one step. In the next few years, the synthetic version of the 7th art will surely be enriched by creation techniques assisted by the same tools that produce deepfakes today. The quest for the perfect digital lookalike is stirring up the whole of Hollywood and giving rise to a few blockbusters that are still a little hesitant. In 2019, Gemini Man features Will Smith in the grip of his doppelganger 20 years his junior. The young Will Smith looks like Fresh Prince of Bel-Air on steroids. The resemblance is sometimes bluffing, sometimes very far from what we expect, but in the long run, the effect hits the mark. No more botox, anti-wrinkle, vitamin cures or beta-carotene, synthetic rejuvenation becomes art in which Weta Digital, the New Zealand leader in special effects, excels. ⁴ Weta Digital made the special effects for Peter Jackson’s Lord of the Rings | Body of Work : Weta Digital on the art of face replacement, Before and After, June 26, 2020. .

More recently, Matt Stone and Trey Parker opened their Deep Voodoo studio and brought together the biggest names in deepfakes creation. The production of their new series Sassy Justice features a larger-than-life synthetic double of Donald Trump. One of the studio’s creators, Chris Umé, also made news on the networks by creating @deepTomCruise ⁵ Chris Umé: the deepfake @deeptomcruise is made to alert the public, journalism. design, March 5, 2021 , a synthetic double brought to the screen by a certain Miles Fisher, an actor in his own time and an official impersonator of the real Tom Cruise. The Disney studios have been developing for a few years, in their Disney Research Hub in Zurich, Switzerland ⁶ the Disney Research Studio website , algorithms capable of making perfect faceswaps, that is to say, of interchanging the face of an actor with another one without any other retouching in post-production. Other research in artificial intelligence or machine learning reinforces this work on computer vision. Disney’s interest is clear: to model humans perfectly in order to make the best possible attractions.

At the center of all these digital manipulations are the actors and actresses who serve as material for the GANs’ algorithms. Deepfakes are about to upset the profession and its balance and we can hardly imagine the collateral effects of these new techniques.

Robin Wright eternal

In 2013, Robin Wright lived her last moments as an actress in The Congress ⁷A film I highly recommend, The Congress, 2013, Ari Folman. Faced with a disgustingly cynical Hollywood producer, the actress, who played herself, was offered the impossible: to sell the studio that employed her until then, Miramount Pictures, the eternal rights to her scanned image. In return, the human actress had to disappear forever so that her synthetic double could live. “We want to have this thing called Robin Wright,” concludes the Harvey Weinstein facsimile.

The questions raised by the first part of the film resonate terribly with our times and the issues that arise from the arrival of synthetic media in the film industry. It is now possible to bring an actress or actor (or any other individual for that matter) back to life ⁸ VFX Artists React to Resurrected Actors Bad and Great CGi [Coridor Crew] | Snoop Dogg Had us Bring 2pac Back to Life! [Coridor Crew] We can also age or rejuvenate an actor whether he is alive or dead ⁹ De-Aging Robert Deniro in The Irishman [Sham00k] | David Hasselhof deepfake de-aging turning back the time [Chris Umé] | De-Aging With Deepfakes (Machine Learning) [DeepCaked]. We can create from scratch a dialogue that never happened ¹⁰This AI Makes “Audio Deepfakes” [Two Minutes Paper] | RealTalk : We Recreated Joe Rogan’s Voice Using Artificial Intelligence [Dessa] . Lip movements can be synchronized to any speech, in the individual’s native language or in another language ¹¹ Flawless Ai demo [Flawless] | Google’s LipSync3D Offers Improved “Deepfaked” Mouth Movement Synchronization, Unite.AI, 9 June 2021. Or replace an actor at the drop of a hat with another ¹² [INÉDIT] Un “deepfake” dans Plus belle la vie !, France Télévision, 2020 . All these actions are already possible and will change the way actors live their art, their job.

To begin with, let’s take up the story of Robin Wright and the initial promise made to her in the film. Eternity. A fake eternity of course, only her image will be eternal. But eternity, all the same, an eternity that can seduce tormented egos. What are we willing to do to become eternal, even a little. What negotiating leverage does the one who has the technique of immortality have? To be young for as long as the series of 1’s and 0’s that make up one’s synthetic double allows, is certainly the dream of many actors and actresses. It is also the dream of the producers of Star Wars who brought Peter Cushing back to life for some scenes in Rogue One, or Carrie Fisher, young. Both versions were first created using traditional special effects and then improved by Sham00k using his algorithms ¹³Starwars is told in deepfakes, journalism. design, 2020.

Bringing back famous people from our history to include them in a cultural or commercial scenario, like Salvador Dalí welcoming visitors to his Museum in Florida ¹⁴ Museum creates deepfake Salvador Dalí to greet visitors, Dali Museum, raises some ethical questions. Who is able to accept the use of the image of the deceased? Who has the right to the image of a deceased person? The management of the image is one thing, but what to do when one adds the word to it? Can the purpose of the document created raise exceptions (education, information vs. commercial exploitation?).

Making a living person younger or older also raises a whole host of questions. To draw a parallel with The Congress, an actor must first give his consent to be fully scanned. Then he must decide how long to keep this data, by whom? By whom? How? Are synthetic data ¹⁵ digital data that make up the 3D scan of the person | How synthetic data could save AI, Venturebeat, 20 March 2021 transferable and to whom and under what regime? Do we give up the full rights to our image? Do we keep the ability to perform or, like Robin Wright, will an actor be condemned to disappear?

This new situation also raises market questions. Famous actors will be able to capitalize on their fame by giving up their faces for films on which they will not be physically present. We can imagine at this stage two particular cases. In the first case, the actor controls his synthetic double from a distance, the interpretation is his own, the fee remains unchanged. In the second case, another actor interprets the role according to the specifications of a ghostwriter, in this case, the fee is less important. What about the opportunity to bring out new talents? Will the opportunities still be as numerous? Don’t we risk seeing two classes of actors appear? The puppeteers, capable of breathing life into synthetic doubles, but deprived of public recognition, and the real stars, these actors who will appear everywhere in a perfect ubiquity.

Then the voice. Do deepfakes announce the end of dubbing? The ability of algorithms to reproduce a voice, its timbre, its flow is improving every day ( [AI Voice] Kylo Explains… Return Of The Jedi, Speaking of AI, 2021)). Some algorithms need only 5 seconds of recording to get a relatively correct result ¹⁶ This AI Clones Your Voice After Listening for 5 Seconds, Two Minutes Papers , others allow celebrities or influencers to record commercials without being present ((Generator for Celebrities and Influencers, Voicebot.ai )).

The digital capture of voice, image, expressions, body movements, gait represents a considerable volume of synthetic data, but also a very important commercial stake. We can easily imagine databases where each actress and each actor will be listed and classified according to a new typology: deepfake double or IRL actor. The development of such services, if they are successful, will quickly spread to the whole population who will not hesitate to pay a few euros to give up the photos of their Facebook album in exchange for a credible synthetic double. It is a question of opening the debate so that each one can understand the stakes which are profiled in a world populated by synthetic media.