Logo TIB AV-Portal Logo TIB AV-Portal

Computer Vision, Surveillance, and Camouflage

Video in TIB AV-Portal: Computer Vision, Surveillance, and Camouflage

Formal Metadata

Computer Vision, Surveillance, and Camouflage
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
This talk will provide an overview of how we are seen by computer vision, what it means to be analyzed, and explore creative countermeasures for modulating visibility in a machine readable world.
vision vision means hypermedia Computer animation computation workgroup vision computational
flow track recognition statistics response information studies time views capture storage analytics part total Computer animation website addresses systems
Computer animation Lecture/Conference phase edge detection computational
track Facebook's image algorithm Universal lips shape heads number
bottom recognition information neural network states aliases analysis bits analysis lines open subsets Computing number computational Indexable processes Software Right systems management
purchasing man email track recognition heck gaps maximal knot analysis scans Rothe-Verfahren product MACH data management newsletters mathematics Carrying Computer animation green Software Iris implementation
recognition track Computer animation Open Source Meeting/Interview Gender smart tractability vision computational form systems
man image algorithm photos Actions images orders analytics Demo
man pixel building algorithm recognition link information neural network concentrations Guide vision product computational image Facebook's photos hypermedia profiles level Gates
point man Reporting Lecture/Conference phase effects Payload Databases fields
profiles words Actions processes profiles expression unit analysis drop sort Drums
Context algorithm DoS maximal perspective Sans Types angles Meeting/Interview case different Video man
tablets recognition wave Lecture/Conference Blocks communication PIT projects scans Mass pattern
algorithm key projects computation Continuation lines part vision computational vision image circle pattern data types
man Observation algorithm mapping information views Continuation part vision computational image processes pattern genetics objects reverse
area algorithm profiles bridges machine contrast asymmetry
area comparison algorithm Meeting/Interview bits
area image means Blocks profiles testing applications rectangles
man algorithm analysis threshold vision computational
man control Blocks AST scans Arm vision Demo Computing computational vision Types website Right
man shift report Broadcast Meeting/Interview Blocks projects Right ftp server
image Mutual bottom Lecture/Conference Forest radiation series Privacy part thermalization Payload signatures
image Types projectors Lecture/Conference differential Mass errors
man Lecture/Conference projects several man Compression God
point area man Context email projects Mass Mass several Compression Twitter words several Lecture/Conference Office Security man
area man response email projects bits open subsets part Privacy thermalization van different box Security
man response Lecture/Conference different green maximal Rolling pattern family thermalization spacetime
software engineers response real images law division similar parallelization Privacy powerful sign sign period Computer animation strategy terms information Alexander form form
man CAP Lecture/Conference gute Sans
the only means
thank you very much for coming this talk is going to be about and 3 general topics computer vision
surveillance and camouflage I'm if you ban to Gatwick Airport recently you may have seen 1 of these from what it is it is a facial recognition system that's designed to improve the flow of traffic around airports according to their website the system is designed to capture passengers facial features as they enter the airport and tracks them throughout the journey and it claims 10 times the capture of device tracking solutions such as tracking the MAC address of your Wi-Fi if you leave that on at the airport and the passenger of course never needs to switch on the technology because your face they say is always on but the company the company is interesting but not very unique they're part of a larger rapidly evolving industry of human analytics and on the website you'll see this statistic where they mention that 89 per cent of people are willing to give up the biometric information as they travel through international borders but this is a statistic that you would likely see on a biometric industry
website however this is a statistic that you would not likely see and this is view from a study by Accenture that says 75 per cent of respondents from would not even shop in a store that uses facial recognition for marketing purposes and while all of these cameras certainly look at new and innovative face detection and face recognition as certainly not anything new In fact the
history of face detection begins around 1969 when 3 Japanese researchers published of a research paper and proving that a computer could be used to detect the human face and it looks something like this
from what you see is basically edge detection being used to trace the outline of human so being able to match that unique silhouette to the silhouette of a human was what led to the 1st phase being detected the following that
breakthrough 1969 in the early seventies the researcher at Stanford I followed by another researcher at a Japanese university published the 1st research that showed how to do facial landmark tracking being able to detect the eyes the nose the mouth and the shape of the head and so this facial landmarking is what's used today by Facebook for other companies to do high-performance facial recognition and
you can see in this image how all this algorithm in 1973 was able to segment the i the lips and nose and if we compare this number from 1973 about 75 per cent accuracy to the numbers in 2000
16 and it's interesting to note that computers we are now at around 98 . 5 per cent from success in recognizing and classifying humans but meanwhile humans the stuck at 97 . 5 3 and in 2017 that number for humans is going to stay the same and the number for computers on the bottom is going to keep increasing so certainly
some heading in the 2016 that's an interesting year and exciting for facial recognition for that really depends on where you're standing that depends on whether you're someone using facial recognition to collect a process information or if you're somebody that's being watched by it out 1 note to the toolkits have been recently released them both called open phase but they're a little bit different the 1 on the left is a deep neural network that uses some I think it gets between about 97 98 per cent success of classifying faces it's completely free the 1 on the right is used for behavior analysis so all the 1 on the left is used to read the outside if you to use you as an index to your identity the 1 on the right reads the inside of you then it reads the emotional states both of these could be deployed for the costs of a raspberry pi so around 50 dollars to implement a very high-performance facial recognition system in 2016 but this line actually comes from the marketing
newsletters from marketing land outcome there are also very excited about face recognition because in the marketing business 1 of the problems is that people like to pay in cash and I notice here in in Berlin especially a lot of things are a cash business the track said
facial recognition changes that than what marketers are doing right now is using facial recognition to make it possible to track cash purchases so so
when you think about the history of facial recognition how was originally funded by the Department of Defense for military purposes specifically located in the combatants and criminals that same technology is now being used to identify and track consumers who did not pay with a tractable form of payment by credit card and that's not great no I wanted to provide a quick overview of some of the capabilities in computer vision and they become very
easy to deployed on anything as simple as a raspberry pi surveillance system the 1st 1 is using the gender and age detection although you can see that it's not 100 per cent accurate and the it's open source and
accessible to anyone another 1 is called gaze detection and gaze detection looks at what you're looking at so is a photo of you this algorithm able to tell what you find interesting in that photo they can drive analytics and even understand the scene from doing that and building off gaze
detection it do something called um finding important people in images so when you know who's looking at who the started determine the social order of a group of people and photos
this is a image from a company called jet pack and what they're doing is using every public pixel of Instagram to build tour guides for cities that we see here is that they're they're looking at cues like facial hair hipster moustaches and lipstick and they use this visual information to Our Savior visiting a city die due to a place where there is no at a higher concentration of hipsters with facial hair for a place where a lot of the girls are hanging out with lipstick and so when you post photos online in social media and these these social media posts are really the superfood of artificial intelligence feeding the algorithms that are being trained to then for example sell products back to you the or
provided and kind of gated levels of access with facial recognition technologies this is 1 that I came across on and read it and think and somebody was bragging about how they use computer vision and which is the base can Facebook profile photos for overweight people store the ideas and then target as to them on Facebook I'll provide just another 1
and as a company called aspect either the claims to have the world's largest emotion database with
40 billion emotion data points ended nearly 4
million faces analyzed compare that to the NSA effectivities getting about 5 . 5 thousand phases per day whereas you look at the report by Laura Poitras and James in from this node in documents and has a comparatively is scanning 55 thousand faces a day so we see these um commercial surveillance companies and nearly on the same playing field as government
agencies and I said you can now use this facial expression analysis to read for the inside to read the thoughts of somebody to know what they're thinking word a profile the emotions and is read a set of
things called facial action units 1 of them is called that sort or job drop or blank every facial movement can be red and analyzed so I wondered what would happen
if horrendous under video just 1 man Ramchurn and I thank you in the Senate Intelligence Lebanon's they want from and I hope we can do this you just yes or no answer at the end of the month and wanted 1 last summer the NSA director was at a conference and he was asked a question about the NSA surveillance amount you will find and I quote here the story that we have millions of hundreds of millions of DOS yeas on people is completely stops rely Matthew question is having served on the committee now after a dozen years I don't really know what it does here is in this context so what I want to see is if you could give me yes or no answer to the question does the NSA collect any type of data at all on millions or hundreds of millions of Americans most of it does not not moving there cases it could inadvertently so a
lot of these algorithms are about 92 98 % accurate but I'm pretty convinced that this 1 is just about perfect however reading it from another angle another perspective gives you a different answer because from joy to discuss today a lot of bring attention
to as I mentioned um comparing commercial surveillance a government surveillance the really not that different kind of they actually are the same thing sometimes this highlighted text this from a company called pit pad and who is has offered a blanket license to United States government that patterns notable because that's the facial recognition company that was sold to Google and that's the 1 that the NSA is using the scan um facial faces that are intercepted from communications so what I think is that a face is 1 of the most powerful tools for communication yet there are very few if any acceptable waves to protect it from dubious or invasive kinds of surveillance and 6 years ago they're working on a project Gold CV dazzle has my thesis at
NYU and made this sketch to see if it was possible to make some kind of hair and makeup from appearance that would block face detection any came out looking like
this you can see um it what is unique but it looks works in the same way to block in the face from being detected let the faces in these images are apparent to computer vision algorithms yet to human it's very obvious that there's a person here and that's that's the key to this project is to explore this very fine line between what can a computer C what can I human C and of course what socially acceptable and I think there's is very this very fine line in between all those overlapping circles for the right occasion of course so really the
definition is that CD does a kind of camouflage from computer vision and it uses these bold pattern and then hairstyles to you had a break apart the most important facial features that are targeted by face detection algorithms the dazzle part of it comes from this kind of
camouflage that was 1st introduced in world war 1 on ships call dazzle which interestingly was really inspired by Picasso's cubism in a way that it worked is if you're launching ammunition at this ship it's difficult to tell sometimes whether it's 1 ship to ships weather's going left to right and the idea is to confuse the observer by using this bold patterning to kind of break apart the gestalt of the object so
not similarly with these computer vision algorithms and they can be reverse-engineered to see what they're thinking about and this is a and genetic algorithm reverse engineering process of open the views from ah cascade algorithm was abhorrent is that you can see that in the heat map there some parts of the face that are more important than others and with this information then you can begin to design the camouflage pattern that would target those those most salient and parts of the image if we look
closer these are the faces that are hidden inside some of the face detection algorithms that have a ghoulish looking ghost in the machine and that's the frontal face profile and that's the profile face so I can give you
some tips on what works and what doesn't work but what does work is creating a asymmetry on the face of using the dark hair against light skin were light hair against dark skin from altering the contrast of the darkness of the cheekbone area and a very important area is the nose bridge for the area between the eyes and the centroid of the face
in at the 2014 The New York Times commissioned at new looks and these were um developed to be a little bit of better performing adapted to some the new algorithms and I'll show you kind of how these are measured but I think 1st and he to show you a
ground truth to show what a face looks like when it's detected and how easy it is to detect so the red areas of show an area that is detected by the face detection algorithm and for comparison if you're doing military
camouflage and obtained all over your face the doesn't mean that you in a block face detection because it's a specific application to those targeted come highly sensitive areas of the face
so when we run that same saliency tests on the images Commission for the New York Times you see that there are no positive detections and that the different rectangles represent each of the 5 face detection profiles so this 1 it's
also it's working well but you can see that 1 of them was of the 5 detectors dossier it's important to consider camouflage not as something that makes you invisible I think that's the way it's often written about in headlines but camouflage is really the idea that your understanding the threshold of detection and then creating some way of appearing that lowers you just 1 step below the threshold of detection and that's
done by an analysis of with computer vision the specific algorithm that you're targeting but since 2010 which is a while ago now that's some interesting things that happened putting it online
there's been a few articles somebody said that they saw a young woman was at a computer vision and a surveillance make up for citing in the wild for him and I surveillance and his poetry Harriet make-up party that the bad Big Data use it for the promos there than art and science festivals like on the bottom right that have done it in everyone interpreted as some kind of in their own way and just using the tips on the website are able to construct some unique camouflage designs that actually block you know millions of dollars worth of face detection i and sometimes it's just beyond my control what happens this is how appeared in as a
TV show called elementary and in 1 of the scenes this you know the criminal type character was trying to evade face detection and so they created let's look at some which is which is quite colorful and this interesting because it does function it does work and in other
popular culture references the TV show by daughter poured not to move the imagined that in the future I think it's the year 2065 people would wear this ironically and it is a throwback to the past when it was so easy to block face detection that all you had to do was use make up and so this becomes kind of like a tattoo that maybe you regret when it's 20 60 per cent because it only functioned of 40 50 years ago some examples that found would probably be more suitable for going out at nite to a club but it's worth pointing out that it can
look pretty good this is not something that I was involved with this is just an example of a hairstyle that happens to work well so with the right hair stylist and makeup artist you can go places with it I when a shift down to a more recent project that it worked on starting in 2012 and this project is about drone surveillance but in particular military drones and the capability of thermal imaging systems like like
this which is a um camera payload on the bottom of the drone with multispectral imaging capabilities 1 of those imaging capabilities is seeing in the dark we're seeing the radiation uh the thermal radiation that's emitted from your body and it's quite hard to I think of yourself as always visible even if you're hiding say in a forest undercover are you could still be detected and I thought and opposes a future threat to privacy and this is quite a powerful technology I wonder if it is really uh unstoppable and researched ways that maybe the thermal energy could be blocked it and in
2013 release uh series of garments called self where is odd Gómez designed with a metal-plated fabric and the metal is typically silver was highly flexible and you can see in these images that the metal-plated fabrics and reduce the thermal signature or entirely blackout that part of the body here
is an example of somebody wearing it and you can see that there are 4 people but there actually 5 and the projector makes it hard to see and once the individual starts moving it will become very apparent but in a still image um somebody is blending in perfectly to the background in the winter which is even more difficult because you have a higher temperature differential summer played and is going to be very obvious that that the intent is not to provide a full military solution but to you illustrate the
possibility for a new type of fashion as may be more appropriate for an error of mass surveillance the and so there's
a hoodie and there's a burqa and there's a user out and these items you are of
course inspired by Islamic dress and the idea with this project is whereas a religious dress is traditionally and In some ways thought of as providing a separation between man and God and this collection
of items and stealth where re imagines religious dress in the context of mass surveillance as providing a separation between man and drones some of the other more
interesting things and slightly concerning word and these e-mails and some articles I came out so of course this is projects had but up against national security interests which it makes me as an artist from sometimes quite no aware of the possibilities for that to go wrong in 2013 received an e-mail for a request for publication a classified and intelligence document which is strange because Americans see that do you really need my permission most news publications don't even ask for it and the other 1 that was the Washington Post put a request for comment on the Office of the Director of National Intelligence at desk and for the NSA had I tried to you create work that doesn't put me possibly at rest for if who knows the area of national security is sometimes black-and-white and it's not a it's not a very open area to you are that's exactly why I'm doing his projects that I do the other 1 0 point out is a tweet from the Air Force general counsel at the Pentagon just making note that they are aware of the idea of self where to hide from military
drones so I see this gray area at were artist can operate as something that's very essential being able to not make some projects that are threatening to to any person in particular by kind of gives some room for us to think about the future that were creating and as a response to some of these anxiety-provoking e-mails that I received the next project I created was the privacy gift shop which is a way to make this sometimes of threatening ideas about national security and a little bit more friendly this gift shops friendly open accessible and safe places they differences that this is a gift shop for counter-surveillance part and privacy accessories so I
sell the items that can directly and the anti-drug very did quite well and sold out and it is expensive and I am a terrible businessman because emit tell you that you can also do that for less than 1 dollar and the way that thermal radiation is boxes by any kind of GoMiner fabric that's thermally reflective for thermally insulating and so the most obvious example would be a
space and you can get a pack for your whole family for less than 10 dollars and while that has the same of functional appeal to block thermal radiation the main difference is in the psychology of camouflage that is so great
that this is the status of my or blankets and mats and how it works so to wrap things up in the summary that there are functional solutions to hiding in blocking surveillance and there are more I'm artistic or psychological responses from more thought out kind of camouflage I think that when when we when we think of camouflage it's a woodland Vietnam era 81 pattern blotchy browns and greens and blacks but really camouflage didn't even really exist the way it does that before the 20th century and in the early 20th century
and camouflage was actually term for criminals hiding from the police the you can drop out a similarity parallel between the way that camouflage was thought of in the early 20th century to where that privacy was thought of as early in this century and some other quotes from a fantastic book about camouflage its impacts on Australia Theodore Roosevelt considered camouflage is a form of effeminate cowardice a mere defensive strategy but then you'll see as world war 2 as law 1 in World War 2 unfolded people came to realize the
significance in the power the strategic advantage and most importantly that camouflage became a sign of humanities increasing intelligence so I think that privacy can be thought of in the same way not as a defensive for cowardice response the privacy is also something that shows humanities increasing intelligence
and in closing we think maybe these solutions are a little bit eccentric but if you look at back in history the year 1918 in New York City everybody is learnable OK if you look around in the audience after this 1 person that's not with you look rather the audience nobody's wearing a bullet cap so it's very 3 possible for something that's completely normal to become abnormal and likewise possible for something that's very abnormal or eccentric now to become completely normal in the future thank you so
thank you and my amendment an