(Headline USA) A new artificial intelligence tool is already being used to create deepfakes, or fake videos, of celebrities using racial and inappropriate slurs, according to a report.
Eleven Labs, a British AI firm, said on Monday that it would be putting additional “safeguards” on its platform after finding “an increasing number of voice cloning misuse cases.”
Crazy weekend – thank you to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter community for thoughts and feedback!
— ElevenLabs (@elevenlabsio) January 30, 2023
The firm did not elaborate on what it meant by “misuse cases,” but other reports found posts on 4chan of clips featuring generated voices that sound like celebrities. In one clip, for example, a voice that sounds like actress Emma Watson’s is heard reading a part of Mein Kampf. In another, a voice very similar to Ben Sharpio makes racist remarks about Rep. Alexandria Ocasio-Cortez, D-N.Y. In a third, someone saying ‘trans rights are human rights’ sounds like he’s being strangled.
Eleven Labs said one of the new security features it is considering implementing is an added layer to account verification, such as requiring users to enter payment info or a legal ID. It is also considering having users verify copyright ownership of the voice they want to close, such as getting them to submit a sample with prompted text.
The company also said it’s considering dropping the AI voice tool completely.
Our current ideas:
(1) Additional account verifications to enable Voice Cloning: such as payment info or even full ID verification
(2) Verifying copyright to the voice by submitting sample with prompted text
(3) Drop Voice Lab altogether and manually verify each Cloning Request— ElevenLabs (@elevenlabsio) January 30, 2023
Not too long ago, Eleven Labs was touting its voice cloning technology as having “the potential to revolutionize the way content is produced, delivered and interacted with across a range of industries.” It even shared a video it created of actor Leonardo DiCaprio being voiced over by other celebrities.
