Similarity Detection in Online Integrity
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61652 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Content (media)AlgorithmSimilarity (geometry)Data integrityComputer fontComputer-generated imageryVector spaceElectric currentDigital photographyVideoconferencingComputing platformRule of inferenceVolumeConsistencyPresentation of a groupMeta elementScale (map)Decision theoryGroup actionRotationPixelCryptographyMeasurementArchitectureEvent horizonCompact spaceRepresentation (politics)Price indexRevision controlKeyboard shortcutPopulation densityLibrary (computing)Vector graphicsComputer architecturePhase transitionExploit (computer security)Event horizonKeyboard shortcutMedical imagingInferenceContent (media)Subject indexingCASE <Informatik>Group actionVector spaceMatching (graph theory)Scaling (geometry)Similarity (geometry)Graphics processing unitSemiconductor memoryRepresentation (politics)Mathematical optimizationGoodness of fitCategory of beingFreewareLibrary (computing)FacebookPopulation densityINTEGRALWebsiteOrder (biology)Einbettung <Mathematik>Physical lawDependent and independent variablesVolume (thermodynamics)10 (number)Traffic reportingDecision theoryDatabasePresentation of a groupComputer fontDifferent (Kate Ryan album)PixelPosition operatorMassPairwise comparisonTerm (mathematics)Computing platformVideoconferencingHash functionMultiplication signMeta elementMathematicsPhysical systemMeasurementRevision controlDistanceSpacetimeComputer animation
09:13
PixelCryptographyRotationRevision controlKeyboard shortcutSimilarity (geometry)Visual systemComputer-generated imageryVector spaceMeasurementLibrary (computing)Vector graphicsPopulation densitySelf-organizationDigital photographyExploit (computer security)Content (media)FacebookMiniDiscTrigonometric functionsDiscrete groupSinguläres IntegralAlgorithmMetric systemHash functionMedianCategory of beingSpacetimeScale (map)Motion blurDigital filterDivision (mathematics)VideoconferencingSample (statistics)Kernel (computing)Temporal logicGeometric quantizationAverageElectronic signatureFrequencyAlgorithmAverageMedical imagingGroup actionVector spaceHash functionSimilarity (geometry)Exploit (computer security)Shared memoryCoordinate systemIntegerFrame problemElectronic signatureEinbettung <Mathematik>Virtual machineRepresentation (politics)Moment (mathematics)Self-organizationDistancePhysical systemWebsiteVideoconferencingGraph coloringLibrary (computing)CASE <Informatik>Right angleFingerprintFacebookProcedural programmingInformationTerm (mathematics)SummierbarkeitPixelMultiplication signOpen sourceTransformation (function)Overhead (computing)Metric systemDiscrete groupTrigonometric functionsCuboidTransformation (genetics)Form (programming)CoefficientCategory of beingPower (physics)Point (geometry)SpacetimeLengthDifferent (Kate Ryan album)2 (number)Level (video gaming)NumberMedianDemosceneNeighbourhood (graph theory)CurveUniqueness quantificationKernel (computing)Geometric quantizationFrequencyBit
18:20
Kernel (computing)Similarity (geometry)VideoconferencingAlgorithmTemporal logicHash functionGeometric quantizationFrequencyPairwise comparisonLevel (video gaming)Vector graphicsAverageElectronic signatureSample (statistics)Vector spaceCryptographyFacebookFrame problemQuery languageThresholding (image processing)Computing platformInternetworkingMeta elementGraph (mathematics)Client (computing)State of matterCodeEndliche ModelltheorieCNNAugmented realityoutputVideoconferencingTranscodierungNeuroinformatikAlgorithmPhysical systemRepository (publishing)Wave packetFacebookFreewareDialectGoodness of fitSoftwareSeries (mathematics)Maxima and minimaHash functionFrame problemContent (media)Category of beingMappingVector spaceComputing platformMeta elementMatching (graph theory)Subject indexingGraph (mathematics)Client (computing)AreaSimilarity (geometry)PixelTrigonometric functionsPairwise comparisonWordEndliche ModelltheorieCodeFinite-state machineGraph coloringUniformer RaumDifferent (Kate Ryan album)Degree (graph theory)InferenceSet (mathematics)DistanceLevel (video gaming)Form (programming)Vulnerability (computing)Insertion lossCASE <Informatik>Functional (mathematics)Medical imagingRepresentational state transferArtificial neural networkAverageMultiplication signPhysical lawShared memoryServer (computing)FrictionFingerprintAugmented realityRevision controloutputCore dumpDatabaseConnected spaceRepresentation (politics)SpacetimeWeb pageEinbettung <Mathematik>Computer animation
27:26
State of matterEndliche ModelltheorieCodeCNNoutputAugmented realityMeta elementVideoconferencingTrailVector spaceSimilarity (geometry)Task (computing)Latent heatData modelComputer-generated imageryQuery languageOpen sourceContent (media)Instance (computer science)Computing platformFlagVector potentialStandard deviationRule of inferenceGroup actionPhysical systemConfiguration spaceArchitectureComputing platformCompact spaceData integrityMedical imagingSimilarity (geometry)Physical systemMeta elementRepresentation (politics)FingerprintOpen sourceVideoconferencingVector spaceSubject indexingVertex (graph theory)Server (computing)Group actionFlow separationHash functionFront and back endsCollaborationismFacebookPhysical law2 (number)MereologyKeyboard shortcutContent (media)Computing platformInternetworkingShared memoryCompact spaceINTEGRALCASE <Informatik>Module (mathematics)Web 2.0AlgorithmCodeFrame problemSound effectInsertion lossQuery languageMatching (graph theory)Process (computing)Dimensional analysisComputer virusDigitizingOrder (biology)Procedural programming
36:33
Shared memoryComputing platformMultiplication signReverse engineeringMedical imagingNP-hardVideoconferencingMathematicsDependent and independent variablesGraph coloringProjective planeExtension (kinesiology)Presentation of a groupElectric generatorPixelMereologyProduct (business)AlgorithmArmArtificial neural networkDigital photographyHash functionException handlingComputer virusConfidence intervalFilter <Stochastik>ResultantBitPatch (Unix)Computer animation
40:33
State of matterCodeEndliche ModelltheorieCNNAugmented realityoutputTrigonometric functionsMaxima and minimaMenu (computing)Meta elementCodeProduct (business)SoftwareHash functionGraphics processing unitMathematicsStack (abstract data type)MiniDiscVacuumoutputPosition operatorMedical imagingPattern languageAlgorithmPhysical systemEinbettung <Mathematik>BlogRotationTransformation (function)Artificial neural networkPixelFilter <Stochastik>Software maintenanceExtension (kinesiology)ResultantBenchmarkInferenceContent (media)Different (Kate Ryan album)Multiplication signStapeldateiMatching (graph theory)Function (mathematics)Query languageEndliche ModelltheorieComputing platformInformationProcess (computing)Server (computing)CASE <Informatik>Presentation of a group2 (number)Computer hardwareComputer animation
48:43
Program flowchart
Transcript: English(auto-generated)
00:05
So, it's nice to see a nice crowd after two years of pandemic. You're beautiful. So, today we're going to talk about similarity detection and how we use it in integrity.
00:20
Integrity as a way to ensure that the website is a safe place, that people just maintain an integrity of place. The outline of the presentation is as follows. We're going to outline the problem, then how we use automation and similarity detection in order to achieve what we want.
00:45
The current technology that we use for images, which is the vector search. Then we're going to discuss in depth what is the actual technology the vector embedding makes possible to transform a picture into an element of search.
01:00
The current platform offering that meta puts disposal to allow other people to crowdsource all of their findings into a centralized place. And last but not least, what we have of open and free that you can install in your own you can deploy in your own site to benefit all these technological findings.
01:27
So, the problem is that any big platform bears the responsibility to ensure it's a safe place to serve. No matter what also law says that you have to make sure whatever the user posts, you are
01:41
ultimately responsible to make sure that everybody is just not exposed to things that will violate your community guidelines. Meta has almost 3 billion users. It's likely less than the world's population. And although the vast majority of our users follow rules, some fringe bad actors will always be present.
02:04
And at that scale, fringe means tens of millions of bad persons creating a lot of problems. And when I mean issues, problems, I mean child exploitation, imageries, nonconsensual, intimate imagery, which is a way to say revenge porn,
02:25
adult sex exploitation, people forced to perform sexual acts in front of camera against their will, terrorism, violence, whatever. And just to give you a couple of numbers, Meta publishes a transparency report quarterly about what we do to ensure that the platform stays safe.
02:49
And on the second quarter of 2022, we removed the 38 million of adult sexual exploitation pieces of content taken down. And it's just for this category, child exploitation is not so huge, thank God, but also there are other like violence, terrorism and stuff.
03:07
That accounted for the 0.04% of viewed content worldwide. And in case you were asking, 97% of this content was proactively taken off, even before people could even see it.
03:23
The remaining 2.8% is user reports, like, hey, I found this. And we take that down also and we also add to the databanks just to make sure that we are not forgetting about that. Sometimes there are false positives because it's just unavoidable.
03:40
And half a million was restored upon user appeal. And we restore accounts and mostly accounts and the pictures that were banned for. It goes by itself that the sheer volume of content, the huge scale, the problem we are facing, requires both automation and also human review to ensure either accuracy, both accuracy and also consistency.
04:06
Because it will be a problem if we had 1 million people clicking and making decisions and what is violating for one is not for the other and vice versa. And we cannot just also just employ automation because otherwise we will have this very powerful site decapitating everybody, also innocent users.
04:28
So, the role of automation and similarity detection, the tenet is that a lot of the things that happen online are things that are being repeated. So, there are things that have already occurred in the past, like people posting a
04:42
picture of some shooting, some mass shooting, for example, like the Buffalo or the Christ Church. It gets taken down and ten more accounts spawn and post the same things. So, it's much, it's very efficient to reason in terms of let's just redo the things that we already found that have worked.
05:06
We employ automation to scale, of course, handle the scale of the problem and to consistently repeat a decision that a human reviewer has already vetted in the past. So, we tie a content to a decision, a violating content to a decision, let's act upon this.
05:24
And we tie the decision to the actions. Let's just repeat this action every time we meet a piece of content that triggers this same decision. We do that for videos, for pictures, and also for text. Today, we'll be mostly talking about images because the techniques for video and pictures are somewhat very similar.
05:46
Text has a completely different array of techniques that will not be presented today. So, a way to, if you want to achieve similarity detection, you have to come up with a way to achieve similarity first.
06:02
So, how do we compare two pictures? Of course, we are not, we are not doing pixel by pixel comparison. We want to be much faster. A way to do that is just, okay, let's just MD5 hash all the pictures or SHA-1 all the pictures. And then we store them somewhere in an indexing system.
06:22
And whenever a new picture comes in, we just recreate the hash. And if it matches, we just ban, right? Well, that doesn't work very well because the cryptographic hashes are not resistant to resizing, rotation, one pixel alteration, all the hash changes altogether.
06:44
Instead, we can really benefit from local hashing because it allows for similarity measurement. Like, you change slightly one piece, one portion of the image, and the hash changes a little, but not completely. Then you can reason in terms of distance between two hashes.
07:06
So, you have to turn, you have to find a way to turn an image into a vector, and then you perform a vector search. Whenever two vectors are very, very close beyond a certain threshold, then it's probably a match. And just in case if you're asking, this is your base SD architecture.
07:24
You have more or less all the architectures share these four stages. Observation, an image has been generated, usually a push event like user upload something. Then you have the representation phase in which you hash the image to a compact representation.
07:42
If you're indexing, you store data into your own index. And instead, if you are at inference time, like an event or someone uploaded something, you search the index that you've built with representation. And in case you have a match, you action upon what you, you decide what to do with the match you got.
08:03
Usually, the idea is that this is very close to an image that I already seen in the past that was banned. Also, the account was taken down. Do the same to this user. So, first free piece of content. First, Facebook has released a library, which is FICE, the Facebook similarity search library.
08:27
It's a library to do similarity search over a vector of dense vectors or vector of floats or integers, for example. You can think about it like a C++ version of Lucene. So, you index stuff, puts that in a very big space, and you can search in this space very fast.
08:46
It supports CUDA, so you can use your GPUs to search. It's basically an index on steroids. And it's C++, but it has Python bindings available. And it scales almost linearly.
09:00
You can really index 100 millions of pieces on a single machine, and it just handles them really... It doesn't need to saturate all the memory. So, it has a very good optimization properties that makes it very good tool. And you can go and download that on GitHub.
09:21
Today, we are also mostly referring to with the perceptual hashing. This means that we are reasoning in terms of colors. Colors and images, shapes. We are not reasoning about what's happening inside the image. That is the semantic hashing, which we are not going to talk about this today.
09:43
Perceptual hashing just captures visual similarities. And it's very nice for our use case because it exactly does its job. So, you might think that we are all talking about machine learning systems that come up with very clever representations about our pictures.
10:03
And I'm asking, do we really need a ConvNet for that? Do we really need to employ GPUs? You already said that it's some CUDA, so perhaps that's a nice hint. But absolutely not. Most of this technology is like hashing technology. So, they just compute and represent a mathematical transformation over the image.
10:23
And it's really fast. And it's really cheap. And it can be executed almost everywhere. So, a little bit of history. The first very notable example. It comes from a source that nobody will have thought about. It's Microsoft.
10:40
In 2009, Microsoft invents PhotoDNA. PhotoDNA is the first algorithm employed in fight against exploitive images of children. So, pornography. Like. It transforms a picture into an ash of 144 unsigned integers on 8-bit representation.
11:04
It's proprietary. So, Microsoft licenses this to any nonprofit or any organization that wants to fight exploitive images of children. It gives you a license. You can use it for that and nothing else. But I cannot disclose the details of how that works.
11:23
It can be used only for that. But Microsoft donated the PhotoDNA to the National Center for the Missing and Exploited Children. It's this American nonprofit that basically acts as a coordination center in the global fight against this phenomenon.
11:42
And shares this library with anyone that wants to integrate. This, I cannot talk about how this works. This is the only moment in which I will say something like that. But we can talk about an open source counterpart that almost ten years later Facebook releases PDQ.
12:01
PDQ stands for Perceptual Algorithm Using Discrete Cuisine Transform. And gives a quality metric. It's a very, very bad acronym, but we need a three-letter acronym. So, it's that. It creates a 256-bit hash. Uses Hamming distance to compute the distance.
12:20
It's really fast. The compute overhead is negligible compared to discrete. Can tolerate some level of adversality. This means that you change the image because you want to fool the systems. In that this image is not something which is well-known. PDQ can resist a little to this manipulation, but not to all of them.
12:44
And it's used in StopNCII.org. It's a website where people, in case you have a fight with your ex-fiance and he's threatening to publish your intimate imagery, you go to StopNCII.org. You upload your intimate imageries.
13:02
Fingerprints get taken. Original images get deleted right away, of course. And these fingerprints are shared with partners that, okay, if I am going to see these fingerprints in my website, my platform, I'm going to take them down. So it's a crowdsourced effort and uses PDQ for images.
13:24
How does that work? So PDQ, hashing is optionally scaled down to a square image. Then you compute the luminance. Luminance is the idea that you take the pixel that contributes most in the RGB channel.
13:42
Instead of putting black and white, you use the luminance. It's just another procedure. And the idea is that luminance gives you better information about what was the channel that was contributing most to the color or to the light in that place. Then you downsample to 64 times 64 using a blur filter.
14:04
And the idea of the blur filter, a tint filter, is that it gets the most significant value in that region. Because if you keep convulating a pixel with your neighborhood, what you will have in the end will be the highest value. So you obtain a representation which is compact and retains the most significant information.
14:27
Then you divide the images in 16 times 16 boxes, each one 4 by 4 pixel. And you calculate a discrete cosine transform of each box. The discrete cosine transform, so the box is at the 4 bar color there.
14:40
You see that the grid with a lot of wobbly images, that is a discrete cosine transform. The idea is that any image, any signal, can be represented as a sum of cosines, the sum of cosine signals. You only take the signal, the most significant one. So it's a form of compression, actually. And you take the most significant coefficient for the biggest cosine you have.
15:08
And then you calculate if the median is above a certain value. Then it's 1, otherwise it's 0. So you get this 256 in an array of 010101 in case this pixel were a high luminance or a low luminance.
15:23
The DCT provides a spectral hashing property. It identifies what is the point in the images that contributes more or less. You have an hashing space which is 2 to the power of 1 to 28 because it's half the hashes. Because half is always 0, half is always 1.
15:41
To search, you just do a vector search against what you've just created. In case we want, we can use partially the same technology to do video hashing. And this is another, comes in almost the same paper. The TMK is the temporary matching kernel.
16:00
It's a way to use the PDQ creation to do a video similarity detection algorithm. It produces a fixed length video hash. So your hash stays the same length, which is like 256 kilobytes, if I'm not wrong.
16:20
Even if your video lasts for 3 hours or 30 seconds. It just produces a fixed length, so it's really nice. What you do is that you re-sample a video to 15 frames. Then you compute the PDQ without the 0-1 quantization. So you keep the float numbers. That's why it's called PDQF, PDQ float.
16:40
And then you compute the average of the old descriptors that you have within various periods of the cuisine and scene. Why we added the cuisine curves? Because a cuisine, or a scene, adds this wobbly movement that tells you whether a frame is before or later in the near surroundings, the near neighborhood of the frames.
17:06
So in case you have like 10 pictures, you add this cuisine signal. You know this picture is before this one because you see the cuisine curve which is going up and going down. And it's a nice uniqueness fingerprinting time signature algorithm to add a cuisine.
17:25
So you compute the average of all the frames, the PDQF for all the frames, with various periods, various scene and cuisine. And then you pack them all together and you have these like 5 or 6 averages. And that's your PDQF embedding.
17:42
Searching is just you compare first the vector 0, which is the average of all the frames, and doesn't retain a temporal signature. Then if there is a match, you compare also all the other vectors at different periods, which are the level 2 action as the time signature.
18:01
And so you can really be sure that the videos are really the same. Because if you find the same averages with the same periods, it must be the same video. It's nice that it's resistant to resembling because you always resemble. So in some way, if you vary the frame rate, the video will change. An MD5 hash will change, but this one is not full.
18:22
Hashing is really slow because you have to do a transcoding of all the videos first. And then you have to read all the frames and compute the PDQ for every frame. But search is actually very fast. Another nice hashing technique that we have is the video MD5. I said that we will not be using crypto hashes highlight.
18:43
We use crypto hashes, but just for videos. This is because if you take MD5 of video and find exact copies, it's really cheap in this way. A lot of actors just post unmodified, repost unmodified content. They're not going really through the hassle of do our encoding just to try to fool the systems.
19:05
They just try to repost again. So the MD5 actually works. And it can be done with vector search if we use the bytes for the MD5 algorithm. And it's used widely in stopNCI.org also.
19:21
In 2022, Facebook has released the video PDQ, which is a different algorithm from the former one. Hashing is we hash every frame to a PDQ hash, and we just pack the list. It's much bigger. It's not slower than the other one. But it has a nice property that we just have to search for individual frames.
19:46
So we treat the problem as a bag-of-words approach. So we just put all these frames inside an index library. Then we search, and we take all the candidates, and we do a pairwise comparison.
20:00
If the pairwise comparison is successful beyond a certain threshold, then it's a match. And also this you get for free. And it's released along with the PDQ, along with the TMK PDQF. All this is available inside the Facebook Research GitHub repository. What do you do once you have these hashes?
20:23
So your platform is computing the hashes. But it's the first time that you see this content. But perhaps older actors have already seen this content or two. Well, you upload them to the threat exchange platform. NACMEC shares the PDNA hashes, I told you, with all companies that are asking for them.
20:42
So can you please tell me where this picture that someone uploaded is a matching NACMEC? So I already know that this is something I should call the law enforcement. Meta does the equivalent, but for the PDQ. Because it's much less friction to adopt PDQ compared to the PDNA.
21:02
There's a team, the Internet Safety Engineering, that builds and operates all these servers where anyone can upload fingerprints. And so you can crowdsource a big graph of matches. Expose REST API to access and post new data. Has multi-language clients.
21:22
Uses PDQ. And users can also download the data. You are not forced to stay online, stay connected. You can just request for a dump of the database and you can search it. And you find all the data and all the APIs on the GitHub page. In 2020, Facebook also has released its most advanced algorithm
21:46
to spot similar images, the SimSearchNet++. This is an ARIN network. And it is capable of facing adversarial manipulation
22:01
that the other embeddings just are not able to. Unfortunately, SimSearchNet is proprietary. So I cannot really talk about that. But we have a cousin product, SSCD, the SimSearchCopyDetection,
22:21
or similarity search copy detection, which is open source and free. So I can really talk about that. They are somewhat related in some technological principles. So I can really talk about this. So this is a PyTorch-based model. So the problem that this, which is a state-of-the-art product,
22:43
is trying to solve is what happens if I take a picture and I put a caption on it, alternating so many pixels everywhere. A PDQ or a PDNA hash will be altered dramatically.
23:00
But is there anything we can do to teach a computer to just ignore all the captions, all the rotations, all the jitters, all the cropping of the image? Yes, there is. A person is able to do that. So we can teach a computer to do that, too. So models and code are available. What is now available is the training data
23:21
that we use to create a model, of course. For those which are into the deep learning, it's a ResNet-50 convulsive neural network. And the novelty of the approach is that it's based on our MAC vocabularies. A regional MAC, for those, how many of you know how a convulsive network works?
23:44
Raise your hand. Okay, fine. Very good. So it's a network for the others that looks at the image, looks at portions of the image. Each neuron looks at a different portion, and then they pass what they have understood to a higher level series of neurons,
24:03
the higher and the higher and the higher, until the last layer of neurons has a very wide overview of the whole picture. In this case, we are using the maximum activation of all the channels that we have. So we take note which are the regions of our Karnaugh maps for every different channel,
24:25
which across all channels have the maximum activation. If you have ten channels, and that region across all the different channels, all of them you have a maximum activation, that means that that area is an area of interest. So we use these areas of interest as a word in a vocabulary.
24:46
So exactly when you do the COSIN similarity search for documents, you take all the words, you index all the words, you say these documents as these words, so it's like a vector of words, and then we try to see which are the vectors that have the most words in common,
25:04
and put in the same place. We do the same things, but for portions of the image. So we use the Rmax. The idea is that it's a self-supervised system also. So it means that it's trained to recognize augmented input,
25:21
and it's trained to match an input to its augmented version. So what we do is that we take the training set, we repeat a lot of augmentation, we add the captions, we rotate, we flip, we alter the colors. For example, if you do one degree of whitening,
25:45
you make the image brighter, which is you add plus one to all the pixels in the image, you are altering all the pixels. But in this case, a PDQ hash is capable of understanding the difference. That's a very weak form of adversarial attack, because the PDQ just computes the difference between regions,
26:02
so it's not going to be fooled. But you can be much more violent and put just a spot color somewhere, and PDQ is going to be fooled by that. Then you do it through the CNN. You do a thing called gempool, which means that you do a generative mean pooling. It's a generalization of the average pooling, in case you were wondering.
26:23
Then you go, and at the end, you use an entropy-oriented loss function. This means that we want to encourage the network to spread the representation of training data along all different places,
26:42
because we want to maximize the distance between all the training examples in the training set. So you get a nice, uniform search space. At inference time, you do the same with the CNN, and then you obtain a vector, which is a representation of an image. And the idea is that there is a distance that you can compute
27:05
between the data set of the reference images. Of course, you can subtract a background data set that was used generally to augment the images. But in this case, what you obtain in the end is that the score of the augmented image
27:21
is almost the same of the non-augmented version, because it just learns to ignore the places which are not organic in the image. And SSCD is freely available. You can download that and start playing. You find both code and models, as I already said, but not the training data.
27:42
And by the way, Facebook has also announced an image similarity challenge. You have to determine whether a query image is a modified copy of any image in a reference corpus of one million. This is very similar to the Netflix recommendation challenge, where you had to recommend the movies, and you had to beat the Netflix algorithm.
28:04
And this image similarity challenge, and also the meta IE video similarity challenge, which is two tracks. Generate a useful vector representation for a video, and also try to find a reference video into this very big corpus.
28:24
And you don't have to only find a video. You have to find a clip, so a sub-portion of a video, into a very big corpus. And last but not least, since the last part of a donor is the tastier one,
28:42
we have your turnkey open source solution that you can install in your own on-premise. The Hasher-Matcher-Actioner. HMA is an open source turnkey safety solution. So you just download it, install it, and it starts working right away.
29:02
What it does is that it scans the images that you want to push towards it. It has an index that is updated with all the hashes coming from ThreadExchange, but also from yours, and is able to, say, to bind banks' verticals of violations.
29:26
You might have a non-severe violation or very severe violation. You might decide that for non-severe violation, you just delete the content and send a warning, or for high severity violation, you just immediately read the content,
29:40
shut down the account of the poster, and you also signal it to the law enforcement. You can do that also. And you can configure actions in a backend that are tied to the content that you want to bank into your HMA platform. Ken pulled violating seeds from Facebook ThreadExchange API
30:01
and works on AWS only because we wanted to make a very easy-to-use thing and also something that doesn't really mix your bill higher. So we built it on AWS Lambda. So it doesn't cost anything until it runs. Then it runs, spawns Lambda instance, and then goes down,
30:23
and you only pay for the seconds that it actually runs. But it's very fast. And there's a Terraform module available, thanks to the lovely folks of Internet Safety Engineering. This is how you deploy data. You're Infra. You co-locate HMA to your platform.
30:43
For example, you might own a platform where people have a chat or people post pictures. Whenever new content comes, the web server asks the Azure, have you seen this? Then the Azure goes to Matcher. Matcher goes to the index and says, do I know this?
31:01
And in case there is a Matcher, the actioner module will just tell you have to define a callback API in your own platform, like whenever the actioner calls, you are killing this content in your own backend. And of course, you can fetch from external API new content
31:21
from effect exchange platform. So wrapping up, automation is necessary to be effective. But you will lose precision, of course, because automation doesn't really think. It just does whatever you have configured blindly. Human support is always needed for appeals
31:42
and also to establish the ground truth. So what is actually violating what is not? Do expect false positive, because they will happen. You should put in place an appeal process to allow your users to restore the content. PDQ, video PDQ, MT5 and SSCD
32:03
will provide you with a way to obtain compact representation of high dimension ID content like pictures and videos. HMA provides you with a turnkey solution you can install on your premise and search and enforce your integrity policies at your platform.
32:24
And third exchange provides you with a platform for exchanging representation with other big actors like META itself, for example. That was all from me. Thank you very much for listening.
32:46
Any question? You mentioned for the challenge, I think. Oh, louder. So you mentioned for the challenge finding a clip of a video.
33:05
Can PDQ do that actually? You can hear me. So can PDQ find clips of videos? Sorry? That's my question actually.
33:21
So you should, you say, perhaps I heard about YouTube, whether it is something that already does. Like if the challenge is to find the clips of videos. Yeah. In general, it's possible, of course.
33:43
The video PDQ algorithms will hash every frame. So in case you send a very small sub-portion of a video, you will have like 100 frames, for example. Then these 100 frames will be treated as a bag of work. You search the index. You find the video that contained all of these words.
34:04
So you have a match of all your query frames inside the index at the very long video that has it. And so it's a match. That's how we do. Of course, there are more clever ways to do that.
34:21
Thanks. Hello. Not a technical question, but let's see. I was thinking that if you're using such a system to try to prevent digital crimes and such things like that, from an ethical perspective, I was just wondering that you,
34:43
I suppose you have such images to compare them. And how do you process those? How do you make the decisions? So I repeat the question. From the ethical perspective, the idea is that, of course, we have to see the images
35:02
in order to be able to know what's happening, right? Yeah, see and, of course, you have to save them and, I don't know, process them. How do you handle this? So this is not the kind of question that I really can answer because it is related to internal procedures.
35:23
But if we have to compute the fingerprint of an image, there must be one second in which the image is on our servers. Since the agents is like neck mac, they share ashes,
35:42
so you might have an ash for which you don't have a picture and you have to trust that these ashes coming from a trusted source that has already vetted whether this ash is nasty stuff or not. That's how we actually avoid sanctioning heavily innocent people.
36:01
So there is a collaboration with trusted entities for this. When you receive those from an external agent, if those images are on your platform, you already know what you've seen. Thank you. Can you hear me despite the mask? Can you hear me?
36:20
Yeah, thank you. So I have a question, but first I have a thanks because I have worked in this kind of thing and neck mac doesn't share any useful data. IWF doesn't share any useful data. Pharoah doesn't share any useful data. So I will definitely take a look at the threat exchange platform
36:42
and hope that it's much more useful. And thanks for that. Now I have a question anyway. If I was an attacker, I could download data from the threat exchange platform and try and run as many filters automatically until I find something that is not matched by PDQ, video PDQ, etc.
37:05
What's the way to counter that? Oh, you're asking whether adversarial attacks are possible on PDQ. Yeah, of course. PDQ is a very naive algorithm that just detects the patches of colors. It is actually possible to create adversarial attacks.
37:24
Just if you think that you alter many pixels in the image and perceptually, for us, it doesn't change anything, but you might end up changing the most relevant pictures for the DCT algorithm
37:43
and will create a completely different hashing in the end. Also, someone has demonstrated an attack, a reverse engineering attack on photo DNA, like from the project that's called Ribosome,
38:05
and it's a neural network that from a hash reconstructs a very blurry picture. So, it is actually possible to do that, but PDQ is a very simple and fast algorithm.
38:21
If you really want to combat seriously, adversarially engineer the things, you need neural networks like SSED because it contains so many relations to different parts of the images and it's much harder to fool. I'm not saying it's not impossible because, of course, it's possible.
38:41
So, sooner or later someone will find a way, but it's the usual arms race between attackers and defenders and it's no exception. Thank you for your question. Hello. First, thank you for the presentation. I think it's a very interesting topic.
39:00
Where are you? I wanted to link it because there's been a bit of a buzz the past few weeks, the generative AI, especially chat-gpt. I was wondering if when you use that kind of algorithm and you scan an image, detect something, is there a level of confidence attached to the result and can you detect when an image is potentially a fake?
39:23
I had a very hard time because there's an echo, so I cannot really… Can you do it louder, please? It's hard to understand from here. Hello? Okay. Is it better? Okay. So, I wanted to link to generative AI
39:41
and as I was asking, so when you run that kind of algorithm to detect violence or child abuse or anything else, can you also attach a level of confidence in the response to explain whether it's… well, to define whether it's a potentially fake picture or is there an extension to the algorithm
40:00
where you can link with the generative AI? I'm not sure about the answer. Sorry, we can go for a beer and I can explain more details. Let's see. Yeah, you have a question.
40:21
Hi, thank you for the talk. It was very interesting. One more question also. Do you run SSCD in production as well, the deep learning network? If we're using SSCD in production, can't I reply to this question? We use SimSearch Net++.
40:41
Yes. We use this other one because we have written a blog post about this, so I can confirm that we use SimSearch Net++. I cannot nor confirm or deny about SSCD, but those are related technologies, so I can… That's okay. What does the production stack for SimSearch Net++ look like?
41:02
How do you serve it? It must be pretty hard to deal with the GPUs. This is not a question that unfortunately… Okay, I'm sorry. I cannot talk about the production setups. I'm sorry. Okay. Any question nearby? Thank you. But of course, you can imagine that we do not operate in the vacuum.
41:22
So if you can think about how we serve results from a neural network, it is something perhaps similar to what would you do if you would have to put behind an API a model.
41:42
So I kind of have two questions. The first question is to what extent do you… So I think there are potentially two problems. Intentional mismatches and unintentional mismatches. So situations where perhaps an image has been recompressed
42:03
or has been cropped or is perhaps another image of the same situation versus situations where people have deliberately deformed the image to try and get around these kind of systems. Do you have any idea of how performant it is against the two scenarios of either accidental or unintentional mismatches
42:23
versus intentionally trying to avoid it? So it is of course possible to have unintentional mismatches and I've seen images that were adversarial engineered to give the same embedding.
42:42
Those are absolutely possible. Again, in PDQ, PDNA and all the perceptual hashing which is just a mathematical transformation. You just have to find a way where the input seems the same to the algorithm. For the neural network things, it depends.
43:02
You can study the code. You can study how it's done if you can. It is absolutely possible sooner or later because that Versailles attacker on ConvNet are a reality. So it's absolutely possible. I've seen some mismatches but usually to perceptual hashes.
43:26
Usually, the more finer technique, the harder it is to attack, of course. Otherwise, we just stay with MD5 because it will be enough. Crops. PDQ is resistant to crops.
43:41
SSED is very resistant to crops. If you have rotations, I believe also PDQ is resistant to rotations like flips but you cannot ask much more than that. Other questions? Do you have any information about the speed difference between SSED and PDQ?
44:07
Yeah. So the question is whether I have some speed benchmarks for the difference of performance between PDQ and SSED at inference time.
44:23
PDQ is faster than your time to read the image from disk. So it's negligible. It will just compute. It's a mathematical transformation on the pixel. The neural network requires dedicated hardware. If you do that on CPU, it will take seconds
44:41
also because the model, I think, is big enough. It's not as big as GPT but it's a 50-level CNET. So it's, of course, slower and requires dedicated hardware. But it's more precise. It just finds, SSED finds anything that PDQ is able to find and much more.
45:03
So in case, if you are very curious about, sorry, if you are very conscious about I have to scan this stuff just to make sure that it doesn't come from an ill source, you might want to set up an async process that will take more
45:21
but will just batch process all your stuff. If you need a super fast thing, PDQ will not really wait over your server. Thank you. Any other question? Hi. First of all, great question from my former colleague David, I think, down there.
45:44
Not even looking this way. But what happens if you get a false positive... Louder please. Sorry, what if you get a false positive match? How do you disregard that in the future without potentially disregarding a real match? So if we get a false positive match, how do we do to restore?
46:05
Yeah, how do you restore it and keep it there? Or in meta? Well, just anywhere, like as a concept. In meta, I cannot really say. With the Azure Match Reactioner, you should provide the capability to your own platform
46:22
for which you are soft deleting the image because you have to provide a way, an API in your platform that HMA will call on where you say, soft delete this picture, so make it unavailable but do not really delete it in case you want to appeal. So you need to provide like an undelete and an soft delete and soft delete capabilities.
46:48
This is the simplest and most effective way to deal with false positive in case, whoops, I did a mistake, I want to restore the content. Sure, but if you have an image that someone wants to upload
47:01
say it's a popular image that a lot of people are going to upload but it matches a pattern of another bad image, is there a good way to make a more precise hash and exclude that and say this one is a false positive, it doesn't match what you think it does? So you don't have to keep undoing.
47:21
Okay, if I understand correctly, if the image is popular, so we have many examples of an image which is not bad then comes a bad image. Whether we can use the fact that it's very widespread to augment our position. Is this the question? Okay.
47:41
Well, really there's nothing in this presentation that says this because once you train, the network is trained, you start serving and the network will give you the same answers to the same question, to the same query. PDQ or other mathematical algorithm, perceptual algorithm,
48:01
is just a mathematical function so it will not change, there's nothing to train. So to change a deficiency of your model, you have to retrain. And you can do a better retraining and sometimes model are retrained as anything which is still under maintenance.
48:21
For example, we get new data, for example, and we might want to retrain as any other model for the spam filters. It's the same. Do we have more room for questions? I think it's done. Thank you so much, you'll be a wonderful audience.