Trying to "solve" deepfakes with recognition is a dead end, as it's an arms-race between generators and detectors.
We stop treating video as if its some perfect representation of reality that's inherently trustworthy, and learn from confronting similar issues with other media earlier in history.
Focus on media literacy, critical thinking and digital literacy.
>> Focus on media literacy, critical thinking and digital literacy.
That is just another arms race. I think it naïve to assume that people can be educated out of this problem. Some people can be, but large numbers will always fall for a good hoax. What we need is laws and policies to address the problem. The good news is that this problem is nothing new. The laws are already on the books.
Imho this is something that the law can manage so long as the legal system is allowed to run its course. People publishing true deepfakes, not parodies but actual attempts to fool people, should face the same slander and defamation charges that previous fakes have for centuries. Even copyright law has tests to draw lines between legitimate parody and improper fakery. Hustler v. Falwell (the Larry Flint case) was about a literary deepfake, a sham interview in a porn magazine. Those who take deepfake too far, or republish such material, open themselves to legal attack. Let the lawyers do their thing.
Law has its place, but we also need detection techniques to prevent the quick consequences that rumors and fake media can bring - 'doxing', public shaming, mob lynching, etc.
What if we took the other approach. Currently fake media can bring all sorts of negative consequences because people are conditioned to trust media to a degree. "Most" of the media is true-ish, so a well crafted hoax will be believed by many.
But what if we saturated the media with convincing deep fakes. I feel like there's an inflection point, where people are exposed to so much believable but contradictory fake news, that they might develop a natural distrust of anything presented to them. Believing that the media is for the most part true, sets you up to be manipulated or misled when the media gets it wrong.
What's the result of this? That would mean I basically can no longer believe anything at all which is reported anywhere. Which means my knowledge of events shrinks down to what I can verify personally - which is a regression to pre-print times.
This is also a propaganda technique made famous by Putin's Russia - the Firehose of Falsehood. Make people so unsure of everything they don't know what to believe, which makes them easier to manipulate.
You would probably get something that is worse than the status quo, in my opinion. When people feel like they cannot trust anything generally that's quite bad.
I agree on the regulation-part, but I still think many more are able to be educated (at least partway) out of the problem than are given an opportunity.
And laws have retroactive impact via e.g. police and courts. Education would have a preventive impact.
Deepfake laws won't do anything. You're not going to prosecute a disinformation campaign from a foreign power. The only way that would work is if you were to lock down and censor the entire information distributed to your population. Which obviously has its own problems.
Indeed, most politicians don't really want you to focus on critical thinking. It would solve some of their problems, and create lots more problems for them.
But for yourself, it is becoming more and more important to really think through what is being presented to you in whatever media form and analyze whether it is credible.
The print form of a deep fake is plain old well constructed word-lies, which is why we have had the phrase "don't believe everything you read on the internet" for awhile. There are people who do, but we live better lives if we don't.
Deep fakes aren't really a new problem, they are just a different form of an old problem.
Actually once the politicians speeches are being deepfaked they will BEG the public to make sure there is a digital watermark on the video before trusting it.
And then of course the keys to the watermarking will be leaked :)
"Don't believe anything without sitting down, investigating and evaluating" holds true for most cases, including your own impressions/emotions, what you see, hear, read, etc.
> We stop treating video as if its some perfect representation of reality that's inherently trustworthy, and learn from confronting similar issues with other media earlier in history.
Deep fakes won’t be any worse for the world than photoshop ever was. Until we have any reason to believe that deep fakes are actively misleading anyone in ways that actually matter I think it’s fine to ignore the problem.
We should measure harm and then fine the businesses that spread the harm.
Maybe, but this has nothing to do with deepfakes, and can be generalized.
Societies should teach people to recognize and deal with mis- and disinformation, rather than provide them faulty assumptions upon which to base their trust.
That requires people to have some trusted sources they can use as a baseline. It’s all well and good to assume objective truth is available, but really it’s the most blatant examples people catch not the more subtle.
Add in biases and people are all to willing to discount the truth they dislike as propaganda etc.
It requires teaching people the tools to evaluate arguments, statements and media.
I don't claim anything wrt objective truth (though I tend to think there is one, but whether we're able to directly observe it and/or agree on it is another matter) – I'm claiming we have tools (knowledge, cognitive ability) to weed out obviously incoherent, illogical, biased, untrustworthy or otherwise bad thought.
My point was detecting obviously untrustworthy isn’t enough. If a broadcast is for example inconsistent in what it’s reporting aka mentioning 7 dead then 9, then 3, etc sure you can pay attention and notice the issue. Similarly, poor logical arguments are detectable etc. However, bias can show up in what’s not reported just as easily and in that case their is nothing to detect.
Essentially, you can train people to deal with poor execution, but that’s not enough when the subtle stuff is equally if not more important.
However, bias can show up in what’s not reported just as easily and in that case their is nothing to detect.
You're right about not knowing what you don't know.
But that doesn't mean that you should blindly trust some discriminator system (which we know can't make any technical guarantees, and comes with additional risks) instead of learning to reason about these things.
I agree with the first part. They have just created a new adversary to train on. (Not saying that it's not an interesting result, but not a solution.)
Regarding media literacy, while it's important in general I don't think it will help much in the short run. (Because it takes a long time and becuase it needs the cooperation of those who are actually sometimes willingly deceive themselves.)
I expect digital signature and timestamped fingerprinting to appear in videos that want to prove that they are legit and from that point on you just don't believe anything that's not treated that way. This will eliminate all deep fakes other than the ones that claim to be a leak/recorded without the consent of the faked person.
Deep fakes will in the end make (real) hidden camera and leaked recordings lose a lot of their credibility and power, which is sad.
I agree media literacy (and critical thinking, for that matter) is a difficult problem, which is why we should get going already.
We've had some of the tools needed to distinguish good from bad thinking for millennia, so I find it kind of weird there hasn't been more focus on this in basic education earlier.
Digital signature and timestamped fingerprinting will also have errors, and they don't solve the social side of this issue.
It still leads us to a faulty thought-process;
We cannot inherently trust content on the basis of its medium.
It is in fact weird that there isn't more focus on logic and critical thinking in basic education. I wonder how much of that has to do with government control of education: that is, the people who run the schools are beholden to those who benefit most from a population that doesn't think too critically about what they're told.
I keep hearing this, but it's also a simple and convenient logical fallacy ;). If something doesn't go well, just find a perpetrator who you can blame. I'd stick with Hanlon's razor as usual: https://en.wikipedia.org/wiki/Hanlon's_razor .
The society is a complex system, politics is complex (you have to calculate what all those pesky voters say they want, think they want and actually want) and there are a lot of issues. No one in their right mind can be serious about trying to manipulate future elections through carefully crafting education for that purpose. Dictators do that, for sure. They can stay around long enough to benefit, but they rarely participate in fair elections.
What you're arguing against is the straw-man form of my point, but I didn't express it with much detail, so you can't be blamed for that. I'm not saying that leaving out critical thinking from elementary education is an intentional act by specific people; I'm saying it results from the internal culture of government, which is power-hungry, self-preserving, and long-term, like your dictator. If it controls education, the culture of government will ensure people are being educated in what reinforces the government: what increases revenue, ensures that we have a strong economy, a powerful and unified defense, and a law-abiding and biddable population. Critical thinking has not historically benefited government, so encouragement of it has not become a prominent part of the culture of government.
There is an obvious bias in teaching a version of history that casts our own country in a good light but the impetus against science and logic comes not from a perverse desire of government to maintain control over the population but from a perverse desire for religion to maintain control over society.
It might not be systematically biased toward religion as such, but I know of few other developed countries where creationism is taken seriously and taught as an "alternative" in school [0].
Or where books are stricken from reading lists or curriculums for fear they make kids question authority [1] – where or Harry Potter's inclusion in school libraries is challenged for their focus on magic (often based on claims that the novels contain occult or Satanic subtexts) [2].
We don't completely disagree, I'm just saying that cryptography can help quicker and can solve much of the problem for authenticity where authenticity is can be expected.
And we're talking about a very specific issue here, not simply whether we trust information in general based solely on the medium. Were talking about deep fakes. Whether politician X Y said a specific thing on TV or not. (And not e.g. whether a specific general claim is true or not because that person said it.)
Yes with adequate literacy you can sometimes deduct that you are seeing a deep fake because e.g. Barack Obama or Tom Cruise could have not said it in front of the cameras, but it can be solved with technology and sometimes you will be wrong anyway. (And lets not forget that all of the information you can acquire comes through some media source and in the end you do trust some of them. This includes your teachers. It's just that if you have solid foundations then you are harder to attack. E.g. because no one can go back in time and hack your teachers and school books. But in theory it's still possible. Just think hod dictators do it over the long run.)
In this case, authenticity cannot necessarily be expected, though (hackable vendors, supply chain, implementation errors, inevitable vulnerabilities, corruption, ...) – not to mention the ever-fluctuating assessment accuracy because of the arms race.
I'd rather not teach people to trust AUTHENTIC VIDEO™ (that can't guarantee authenticity), but to learn to deal with the fact that there's Photoshop for video now.
It all boils down to efficiency and probabilities. It's like saying the different platform security solutions (where you can only deploy signed apps) doesn't make sense because of the same reasons you have listed. Yep, sometimes companies will be hacked, and the ones that get hacked frequently will be deemed less reliable (the browser may even display a warning), but it solves 99+% of the issue.
Besides this, you people should be aware that technically any video could be faked. But you only have so much time and attention during a single day and if you can offload some of the checks to machines, which can do it reasonably well, then why don't? (Esp. taking into account that however I'd like too, you can't educate everyone. It will just not stick with a lot of them.)
> Trying to "solve" deepfakes with recognition is a dead end, as it's an arms-race between generators and detectors.
The fact that there is an arms-race doesn't mean that it's a dead end; it's possible that one side has a fundamental mathematical advantage. Consider discrete log problems: as computational power increases, it's possible to brute force smaller key spaces, but the same CPU will always be far, far better at the efficient solution than the brute one.
Similarly, it's plausible that deepfakes will always be far easier for computers to detect than to generate.
Why not learn to how to deal with the fact that we can't trust any information at face value, rather than keep making cognitive shortcuts based on assumptions?
Sure, but no tech can make that guarantee.
As the balance in this arms-race will always be in fluctuation, you can't rely on tech to tell them apart in one moment.
> Focus on media literacy, critical thinking and digital literacy
You're assuming that people want to spot the fake, but it's pretty clear by now that they don't.
This is why we have the American College of Pediatricians and the American Academy of Pediatrics.
ACP is a fringe group who spend time attacking LGTB+ rights. AAP is a mainstream respected group of pediatricians.
If you're a news editor and you want science you'd go to AAP, but editors often don't want science. They want a sound bite, and they'll get that from ACP.
I agree with your suggested approach but is it really realistic?
A non insignificant portion of the USA/World still sees doctored photographs in tabloids and takes them as fact. It feels easy to write them all off but that is a lot of people. Take the average intelligence. Half of the people in the world are less intelligence than that.
So is it really realistic to train everyone on the areas you mentioned?
I'd also suggest sufficient damage could be done before any reasonable corrections could be made, specifically if the faked individual "said" or "did" things that meet people's preconceptions about the individual.
I could imagine fake footage that would cause market impacts, riots, etc. before anyone could get in front of it, and even after official corrections come from Upon High many would believe the explanations themselves are the lies (not the faked footage). This would not be assisted by media organizations saying "allegedly", "can't confirm", and of course the words of anonymous sources that totally would never lie, right?
proof-of-work based blockchains are technically also an arms race between good and bad actors. It still works because extending the chain costs compute, so as long as the good actors have the majority of compute, all is well.
Training a GAN can be thought about as a proof-of-work type job, where you need to recover the best Generator from the SOTA Discriminator, or vice versa.
So the proof of work chain looks like D-G-D-G-D-G. As long as the good actors continue to dedicate more compute to tackling deepfakes than bad actors, I think it should help mitigate the issue.
But why should we ever go with this as a "solution" – with its computational cost, hackable vendors, inevitable implementation errors and vulnerabilities, vulnerable supply chain, inability to make any technical guarantees wrt to the accuracy of its assessment because of an arms race power balance in perpetual fluctuation – instead of learning to reason about these things?
Just because something might be impossible to solve fully, doesn’t mean we shouldn’t still try? For example cryptography is an arms race against code breakers, but that’s not a reason to just give up on encryption?
Also a multi-front defence seems sensible. Agree with points below that the law needs to catch up on this front, but is not mutually exclusive to efforts in improving detection.
The social side is better served with a signature from the device manufacturer stored in the metadata (or as a watermark) of the original file verifying that a photo/video was taken with an Apple etc lens. Make two copies of the file on capture one read only with the signature and one for editing.
No it's not, because there can be no technical guarantee wrt to the accuracy of its assessment because of an arms race power balance in perpetual fluctuation.
There's also computational cost, hackable vendors, inevitable implementation errors and vulnerabilities, vulnerable supply chain, corruption, etc.
I'd rather not teach people to trust AUTHENTIC VIDEO™ (that can't guarantee authenticity), but to learn to deal with the fact that there's Photoshop for video now.
"Despite this promising figure, the team notes there are still several limitations to the approach. Among them is the fact that these deviations could be fixed with editing software and that the image must present a clear view of the eye for the technique to work"
So interesting idea but really easy for the "deepfakers creators" to fix it
> these deviations could be fixed with editing software
Or even yet, couldn’t this also be part of the training of the deepfake models? So that when it’s learning it’ll try to achieve a high realism score as judged by the algorithm that they use in this other tool?
That is always the problem with this fight, yes. In a sense it's a meta-GAN, with other teams building better discriminators that can then be used to further train the generator. This works 'out of the box' if the discriminator is differentiable, but even if that's not the case you could use Reinforcement Learning to achieve the same, I suppose. If you have the man power, you could even train humans to spot deepfakes and then use them as discriminators to improve the fakes in turn... yay!
Be vary once reCAPTCHA asks you to 'spot the fake'.
Indeed. It’s similar to paper money counterfeiting.
There, they use two strategies to prevent that:
1) Do not tell the counterfeiters how you discriminate their products from the real ones.
2) Keep the technology you use out of the hands of the counterfeiters.
For this problem, for 2), I think the cat is out of the bag. A with money, keeping tech out of the hands of government-size entities is impossible, but keeping it out of the hands of small players is doable. Here, doing that that would require locking down all computers in the hands of small players to the level iOS does it. Not impossible, but won’t happen overnight, and I don’t think we want to go there.
That leaves 1). That’s a bit at odds with “publish or perish”, but I guess the likes of NSA personnel aren’t in that rat race, so it might be possible for the NSA to provide an online “fake or real?” tool. Of course, that would mean the likes of the NSA could lie when asked whether their fake was real and when asked whether a real image they don’t want to be known as real was real, so we would need multiple, independent such parties.
In the long run, I wonder if all video will be signed via blockchain-or-other-trusted-3rd-party. You'd need everything from the camera to the the editing software to support it though.
Unless somebody builds a kind of 'pseudo-random-generator' chain where the internal state is huge, changes substantially each tick and is very expensive to compute, so that it becomes infeasible to store or recompute (a large number of) previous states for anyone but the most powerful players.
You could then use that internal state as a private key to sign what you want and only keep the corresponding public key and a signed date+time to proof the veracity. The public keys themselves would need to form a blockchain in the traditional sense, so you can verify their integrity without the previous internal states.
If you wanted to go all-in with the verification, you'd probably also want to show something containing the date+time in the video, together with some complicated physical effects that are hard to fake.
Well, just thinking out loud here, but it seems like it might work :D
I vaguely recall reading somewhere several years ago about some cameras which signed the photos they produced, so one could prove they came directly from the camera and weren't edited. IIRC, it was in an article about these signatures being broken, though I don't recall the details (perhaps it was by dumping the signing key from the camera firmware?).
Early digital cameras could be fingerprinted by looking at dead pixels (there are over a hundred million ways to have three dead pixels in a one megapixel camera, so if these are uniformly distributed, that, in the early days, when few cameras existed, have almost guaranteed uniqueness). It wouldn’t surprise me if that still is possible by looking at minor variations in light sensitivity across pixels (that might require having access to lots of photos by a single camera, though)
Why would the blockchain help here? Basically you need public private key pairs where the private key exists only in silicon and the public key is associated with a serial number. Normally the serial number is trivially associated with the customer who ultimately purchases the hardware especially if the camera is part of a phone with an active plan.
It seems like this trivially works better with a centralized database especially since you don't necessarily want to make all of this data public. The general public needs to know that this is a true and accurate photo that hasn't been edited taken on such and such a date. The authorities with a warrant might well need to know that it was taken by a device owned by bob.
Yes going to definitely take an ensemble of detection signals to really build any confidence one way or the other.
Then ultimately it’s going to be left as a confidence value which humans are going to be free to override based on their own assessment of the content.
In the enterprise class FR software I work on, EVM is one of the methods used for "liveness detection" in live video - no pulse, or impossible-be-be-alive pulses get flagged.
Maybe a naive question: Would it be possible and useful to have some type of checksum system for videos? (Perhaps we already do).
This might not defend against all deep fakes, but at least it could in certain cases with more...official videos. I'm not sure how this could defend e.g. a person from a deep fake someone made of them without their knowledge. But I'd like to hear what people who are more knowledgable than me think.
I wonder if this style of profile picture will simply go out of fashion. When someone who's not obviously well known has this close-up nondescript professional shot on regular social media, I check it out to see if it's a fake. 95% of the time there are obvious artifacts. It's already so slanted to a point where people make themselves look fake by using this style rather than the other way around.
I mean I can't rememeber the last time a Uber/Lyft driver's profile pic didn't seem to also double as one of their Tinder pics. America is going to float away into the ether of mere appearances where everyone's constantly disappointed with... reality.
I’ve always wondered if physics can be useful in this context. Emulators will often approximate a physical system in such a way that it looks indistinguishable, but the emulator may grossly violate a fundamental physical law.
An example of this is training neutral networks to emulate the hydrodynamic equations of motion. The emulator temperature profile may look like the real thing, but it could also grossly violate total conservation of energy.
I’m curious if we can apply such “physics tests” to deep fakes to search for violations.
It seems like on some level GANs should already be immune to that kind of detection right? If there was that severe of a telltail sign, the adversary would pick up on it. It feels like these networks are optimizing for fooling us, but really they're optimizing for fooling each other, and they're good enough at detection that we get fooled too.
Clearly this had to happen, in due course, but these publicly-announced efforts only serve to teach deepfake makers to produce better (i.e. more believable) content.
The interesting place this could take us to is one within which video evidence of a crime is no longer sufficient evidence to convict, making charges like police brutality and domestic abuse even harder to consistently prove than they are now.
Can't we separate the hardware that runs the camera from the user accessible part of the device and have it cryptographically sign the created images/videos removing the need to spot the differences?
I once took one of these online-tests for spotting deep fakes.
By marking anything that has reflections in the eyes as fake I got only 1 answer wrong (the real guy there looked really strange).
This is like bug testing for deep fakes. Eventually all the bugs will be ironed out and we’ll see basically perfect deep fakes. Might take several years, but it’s coming.
You would have to realistically model the subject’s surroundings, which can be quite challenging if the camera only points at one direction the whole time.
We stop treating video as if its some perfect representation of reality that's inherently trustworthy, and learn from confronting similar issues with other media earlier in history.
Focus on media literacy, critical thinking and digital literacy.