Computer Vision, Surveillance, and Camouflage

Video thumbnail (Frame 0) Video thumbnail (Frame 624) Video thumbnail (Frame 2832) Video thumbnail (Frame 3668) Video thumbnail (Frame 4722) Video thumbnail (Frame 5572) Video thumbnail (Frame 6810) Video thumbnail (Frame 8910) Video thumbnail (Frame 9812) Video thumbnail (Frame 10688) Video thumbnail (Frame 11803) Video thumbnail (Frame 13733) Video thumbnail (Frame 14382) Video thumbnail (Frame 15469) Video thumbnail (Frame 16115) Video thumbnail (Frame 17798) Video thumbnail (Frame 19879) Video thumbnail (Frame 21401) Video thumbnail (Frame 22824) Video thumbnail (Frame 23900) Video thumbnail (Frame 25112) Video thumbnail (Frame 26087) Video thumbnail (Frame 26924) Video thumbnail (Frame 27853) Video thumbnail (Frame 29465) Video thumbnail (Frame 30074) Video thumbnail (Frame 31183) Video thumbnail (Frame 32089) Video thumbnail (Frame 33372) Video thumbnail (Frame 34068) Video thumbnail (Frame 35226) Video thumbnail (Frame 36109) Video thumbnail (Frame 38920) Video thumbnail (Frame 40229) Video thumbnail (Frame 41090) Video thumbnail (Frame 42756) Video thumbnail (Frame 43776) Video thumbnail (Frame 44538) Video thumbnail (Frame 45769)
Video in TIB AV-Portal: Computer Vision, Surveillance, and Camouflage

Formal Metadata

Computer Vision, Surveillance, and Camouflage
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
This talk will provide an overview of how we are seen by computer vision, what it means to be analyzed, and explore creative countermeasures for modulating visibility in a machine readable world.
Machine vision Arithmetic mean Hypermedia Computer animation Computer Computer network Computational visualistics Machine vision
Trail Dataflow Pattern recognition Statistics Dependent and independent variables Information Observational study Multiplication sign View (database) Motion capture Data storage device Analytic set Mereology Total S.A. Computer animation Website Address space Physical system
Computer animation Lecture/Conference Phase transition Konturfindung Computational visualistics
Facebook Trail Medical imaging Algorithm Universe (mathematics) Lipschitz-Stetigkeit Shape (magazine) Disk read-and-write head Number
Greatest element Pattern recognition Information Artificial neural network State of matter Aliasing Mathematical analysis Bit Mathematical analysis Line (geometry) Open set Computer Number Subject indexing Process (computing) Software Right angle Computational visualistics Identity management Physical system
Purchasing Metropolitan area network Email Trail Pattern recognition Hecke operator Mountain pass Maxima and minima Knot Mathematical analysis Port scanner Rothe-Verfahren Product (business) Mach's principle Data management Newsletter Mathematics Carry (arithmetic) Computer animation Green's function Software IRIS-T Implementation
Pattern recognition Trail Computer animation Open source Meeting/Interview Gender Plastikkarte P (complexity) Computational visualistics Machine vision Form (programming) Physical system
Metropolitan area network Medical imaging Algorithm Digital photography Group action Computer-generated imagery Order (biology) Analytic set Demoscene
Metropolitan area network Building Algorithm Pixel Pattern recognition Link (knot theory) Information Artificial neural network Concentric Electronic program guide Machine vision Product (business) Medical imaging Facebook Digital photography Hypermedia Profil (magazine) Energy level Computational visualistics Logic gate
Point (geometry) Metropolitan area network Business reporting Lecture/Conference Phase transition Sound effect Computer worm Database Field (computer science)
User profile Word Group action Process (computing) Profil (magazine) Expression Execution unit Mathematical analysis Quicksort Drop (liquid) Drum memory
Context awareness Algorithm Density of states Maxima and minima Perspective (visual) Storage area network Type theory Angle Meeting/Interview Personal digital assistant Different (Kate Ryan album) Videoconferencing Metropolitan area network
Graphics tablet Pattern recognition Wave Lecture/Conference Block (periodic table) Telecommunication Radio-frequency identification Projective plane Port scanner Mass Pattern language
Machine vision Algorithm Key (cryptography) Projective plane Computer Control flow Line (geometry) Mereology Machine vision Medical imaging Circle Pattern language Computational visualistics Data type
Metropolitan area network State observer Algorithm Mapping Information View (database) Control flow Mereology Machine vision Medical imaging Process (computing) Pattern language Natural language Object (grammar) Computational visualistics Reverse engineering
Area Algorithm Profil (magazine) Bridging (networking) Virtual machine Contrast (vision) Asymmetry
Area Pairwise comparison Algorithm Meeting/Interview Bit
Area Medical imaging Arithmetic mean Block (periodic table) Profil (magazine) Software testing Cartesian coordinate system Rectangle
Metropolitan area network Algorithm Mathematical analysis Computational visualistics Thresholding (image processing) Machine vision
Machine vision Metropolitan area network Game controller Block (periodic table) Abstract syntax tree Port scanner Arm Machine vision Demoscene Computer Type theory Website Right angle Computational visualistics
Metropolitan area network Shift operator Explosion Broadcasting (networking) Meeting/Interview Block (periodic table) Projective plane Right angle File Transfer Protocol
Medical imaging Wechselseitige Information Greatest element Lecture/Conference Thermal radiation Forest Series (mathematics) Information privacy Mereology Wärmestrahlung Computer worm Electronic signature
Medical imaging Type theory Video projector Lecture/Conference Differential (mechanical device) Mass Error message
Metropolitan area network Lecture/Conference Projective plane Flow separation Dressing (medical) Metropolitan area network God
Point (geometry) Area Metropolitan area network Context awareness Email Projective plane Mass Mass Flow separation Dressing (medical) Twitter Word Flow separation Lecture/Conference Office suite Information security Metropolitan area network
Area Metropolitan area network Email Dependent and independent variables Projective plane Bit Open set Information privacy Mereology Wärmestrahlung Value-added network Different (Kate Ryan album) Cuboid Information security
Metropolitan area network Dependent and independent variables Lecture/Conference Different (Kate Ryan album) Green's function Maxima and minima Moving average Pattern language Family Wärmestrahlung Spacetime
Software engineering Dependent and independent variables Real number Computer-generated imagery Physical law Division (mathematics) Parallel port Similarity (geometry) Information privacy Power (physics) Sign (mathematics) Sign (mathematics) Frequency Computer animation Strategy game Term (mathematics) Information James Waddell Alexander II Form (programming) Form (programming)
Metropolitan area network Spherical cap Lecture/Conference Grand Unified Theory Storage area network
the only means
thank you very much for coming this talk is going to be about and 3 general topics computer vision
surveillance and camouflage I'm if you ban to Gatwick Airport recently you may have seen 1 of these from what it is it is a facial recognition system that's designed to improve the flow of traffic around airports according to their website the system is designed to capture passengers facial features as they enter the airport and tracks them throughout the journey and it claims 10 times the capture of device tracking solutions such as tracking the MAC address of your Wi-Fi if you leave that on at the airport and the passenger of course never needs to switch on the technology because your face they say is always on but the company the company is interesting but not very unique they're part of a larger rapidly evolving industry of human analytics and on the website you'll see this statistic where they mention that 89 per cent of people are willing to give up the biometric information as they travel through international borders but this is a statistic that you would likely see on a biometric industry
website however this is a statistic that you would not likely see and this is view from a study by Accenture that says 75 per cent of respondents from would not even shop in a store that uses facial recognition for marketing purposes and while all of these cameras certainly look at new and innovative face detection and face recognition as certainly not anything new In fact the
history of face detection begins around 1969 when 3 Japanese researchers published of a research paper and proving that a computer could be used to detect the human face and it looks something like this
from what you see is basically edge detection being used to trace the outline of human so being able to match that unique silhouette to the silhouette of a human was what led to the 1st phase being detected the following that
breakthrough 1969 in the early seventies the researcher at Stanford I followed by another researcher at a Japanese university published the 1st research that showed how to do facial landmark tracking being able to detect the eyes the nose the mouth and the shape of the head and so this facial landmarking is what's used today by Facebook for other companies to do high-performance facial recognition and
you can see in this image how all this algorithm in 1973 was able to segment the i the lips and nose and if we compare this number from 1973 about 75 per cent accuracy to the numbers in 2000
16 and it's interesting to note that computers we are now at around 98 . 5 per cent from success in recognizing and classifying humans but meanwhile humans the stuck at 97 . 5 3 and in 2017 that number for humans is going to stay the same and the number for computers on the bottom is going to keep increasing so certainly
some heading in the 2016 that's an interesting year and exciting for facial recognition for that really depends on where you're standing that depends on whether you're someone using facial recognition to collect a process information or if you're somebody that's being watched by it out 1 note to the toolkits have been recently released them both called open phase but they're a little bit different the 1 on the left is a deep neural network that uses some I think it gets between about 97 98 per cent success of classifying faces it's completely free the 1 on the right is used for behavior analysis so all the 1 on the left is used to read the outside if you to use you as an index to your identity the 1 on the right reads the inside of you then it reads the emotional states both of these could be deployed for the costs of a raspberry pi so around 50 dollars to implement a very high-performance facial recognition system in 2016 but this line actually comes from the marketing
newsletters from marketing land outcome there are also very excited about face recognition because in the marketing business 1 of the problems is that people like to pay in cash and I notice here in in Berlin especially a lot of things are a cash business the track said
facial recognition changes that than what marketers are doing right now is using facial recognition to make it possible to track cash purchases so so
when you think about the history of facial recognition how was originally funded by the Department of Defense for military purposes specifically located in the combatants and criminals that same technology is now being used to identify and track consumers who did not pay with a tractable form of payment by credit card and that's not great no I wanted to provide a quick overview of some of the capabilities in computer vision and they become very
easy to deployed on anything as simple as a raspberry pi surveillance system the 1st 1 is using the gender and age detection although you can see that it's not 100 per cent accurate and the it's open source and
accessible to anyone another 1 is called gaze detection and gaze detection looks at what you're looking at so is a photo of you this algorithm able to tell what you find interesting in that photo they can drive analytics and even understand the scene from doing that and building off gaze
detection it do something called um finding important people in images so when you know who's looking at who the started determine the social order of a group of people and photos
this is a image from a company called jet pack and what they're doing is using every public pixel of Instagram to build tour guides for cities that we see here is that they're they're looking at cues like facial hair hipster moustaches and lipstick and they use this visual information to Our Savior visiting a city die due to a place where there is no at a higher concentration of hipsters with facial hair for a place where a lot of the girls are hanging out with lipstick and so when you post photos online in social media and these these social media posts are really the superfood of artificial intelligence feeding the algorithms that are being trained to then for example sell products back to you the or
provided and kind of gated levels of access with facial recognition technologies this is 1 that I came across on and read it and think and somebody was bragging about how they use computer vision and which is the base can Facebook profile photos for overweight people store the ideas and then target as to them on Facebook I'll provide just another 1
and as a company called aspect either the claims to have the world's largest emotion database with
40 billion emotion data points ended nearly 4
million faces analyzed compare that to the NSA effectivities getting about 5 . 5 thousand phases per day whereas you look at the report by Laura Poitras and James in from this node in documents and has a comparatively is scanning 55 thousand faces a day so we see these um commercial surveillance companies and nearly on the same playing field as government
agencies and I said you can now use this facial expression analysis to read for the inside to read the thoughts of somebody to know what they're thinking word a profile the emotions and is read a set of
things called facial action units 1 of them is called that sort or job drop or blank every facial movement can be red and analyzed so I wondered what would happen
if horrendous under video just 1 man Ramchurn and I thank you in the Senate Intelligence Lebanon's they want from and I hope we can do this you just yes or no answer at the end of the month and wanted 1 last summer the NSA director was at a conference and he was asked a question about the NSA surveillance amount you will find and I quote here the story that we have millions of hundreds of millions of DOS yeas on people is completely stops rely Matthew question is having served on the committee now after a dozen years I don't really know what it does here is in this context so what I want to see is if you could give me yes or no answer to the question does the NSA collect any type of data at all on millions or hundreds of millions of Americans most of it does not not moving there cases it could inadvertently so a
lot of these algorithms are about 92 98 % accurate but I'm pretty convinced that this 1 is just about perfect however reading it from another angle another perspective gives you a different answer because from joy to discuss today a lot of bring attention
to as I mentioned um comparing commercial surveillance a government surveillance the really not that different kind of they actually are the same thing sometimes this highlighted text this from a company called pit pad and who is has offered a blanket license to United States government that patterns notable because that's the facial recognition company that was sold to Google and that's the 1 that the NSA is using the scan um facial faces that are intercepted from communications so what I think is that a face is 1 of the most powerful tools for communication yet there are very few if any acceptable waves to protect it from dubious or invasive kinds of surveillance and 6 years ago they're working on a project Gold CV dazzle has my thesis at
NYU and made this sketch to see if it was possible to make some kind of hair and makeup from appearance that would block face detection any came out looking like
this you can see um it what is unique but it looks works in the same way to block in the face from being detected let the faces in these images are apparent to computer vision algorithms yet to human it's very obvious that there's a person here and that's that's the key to this project is to explore this very fine line between what can a computer C what can I human C and of course what socially acceptable and I think there's is very this very fine line in between all those overlapping circles for the right occasion of course so really the
definition is that CD does a kind of camouflage from computer vision and it uses these bold pattern and then hairstyles to you had a break apart the most important facial features that are targeted by face detection algorithms the dazzle part of it comes from this kind of
camouflage that was 1st introduced in world war 1 on ships call dazzle which interestingly was really inspired by Picasso's cubism in a way that it worked is if you're launching ammunition at this ship it's difficult to tell sometimes whether it's 1 ship to ships weather's going left to right and the idea is to confuse the observer by using this bold patterning to kind of break apart the gestalt of the object so
not similarly with these computer vision algorithms and they can be reverse-engineered to see what they're thinking about and this is a and genetic algorithm reverse engineering process of open the views from ah cascade algorithm was abhorrent is that you can see that in the heat map there some parts of the face that are more important than others and with this information then you can begin to design the camouflage pattern that would target those those most salient and parts of the image if we look
closer these are the faces that are hidden inside some of the face detection algorithms that have a ghoulish looking ghost in the machine and that's the frontal face profile and that's the profile face so I can give you
some tips on what works and what doesn't work but what does work is creating a asymmetry on the face of using the dark hair against light skin were light hair against dark skin from altering the contrast of the darkness of the cheekbone area and a very important area is the nose bridge for the area between the eyes and the centroid of the face
in at the 2014 The New York Times commissioned at new looks and these were um developed to be a little bit of better performing adapted to some the new algorithms and I'll show you kind of how these are measured but I think 1st and he to show you a
ground truth to show what a face looks like when it's detected and how easy it is to detect so the red areas of show an area that is detected by the face detection algorithm and for comparison if you're doing military
camouflage and obtained all over your face the doesn't mean that you in a block face detection because it's a specific application to those targeted come highly sensitive areas of the face
so when we run that same saliency tests on the images Commission for the New York Times you see that there are no positive detections and that the different rectangles represent each of the 5 face detection profiles so this 1 it's
also it's working well but you can see that 1 of them was of the 5 detectors dossier it's important to consider camouflage not as something that makes you invisible I think that's the way it's often written about in headlines but camouflage is really the idea that your understanding the threshold of detection and then creating some way of appearing that lowers you just 1 step below the threshold of detection and that's
done by an analysis of with computer vision the specific algorithm that you're targeting but since 2010 which is a while ago now that's some interesting things that happened putting it online
there's been a few articles somebody said that they saw a young woman was at a computer vision and a surveillance make up for citing in the wild for him and I surveillance and his poetry Harriet make-up party that the bad Big Data use it for the promos there than art and science festivals like on the bottom right that have done it in everyone interpreted as some kind of in their own way and just using the tips on the website are able to construct some unique camouflage designs that actually block you know millions of dollars worth of face detection i and sometimes it's just beyond my control what happens this is how appeared in as a
TV show called elementary and in 1 of the scenes this you know the criminal type character was trying to evade face detection and so they created let's look at some which is which is quite colorful and this interesting because it does function it does work and in other
popular culture references the TV show by daughter poured not to move the imagined that in the future I think it's the year 2065 people would wear this ironically and it is a throwback to the past when it was so easy to block face detection that all you had to do was use make up and so this becomes kind of like a tattoo that maybe you regret when it's 20 60 per cent because it only functioned of 40 50 years ago some examples that found would probably be more suitable for going out at nite to a club but it's worth pointing out that it can
look pretty good this is not something that I was involved with this is just an example of a hairstyle that happens to work well so with the right hair stylist and makeup artist you can go places with it I when a shift down to a more recent project that it worked on starting in 2012 and this project is about drone surveillance but in particular military drones and the capability of thermal imaging systems like like
this which is a um camera payload on the bottom of the drone with multispectral imaging capabilities 1 of those imaging capabilities is seeing in the dark we're seeing the radiation uh the thermal radiation that's emitted from your body and it's quite hard to I think of yourself as always visible even if you're hiding say in a forest undercover are you could still be detected and I thought and opposes a future threat to privacy and this is quite a powerful technology I wonder if it is really uh unstoppable and researched ways that maybe the thermal energy could be blocked it and in
2013 release uh series of garments called self where is odd Gómez designed with a metal-plated fabric and the metal is typically silver was highly flexible and you can see in these images that the metal-plated fabrics and reduce the thermal signature or entirely blackout that part of the body here
is an example of somebody wearing it and you can see that there are 4 people but there actually 5 and the projector makes it hard to see and once the individual starts moving it will become very apparent but in a still image um somebody is blending in perfectly to the background in the winter which is even more difficult because you have a higher temperature differential summer played and is going to be very obvious that that the intent is not to provide a full military solution but to you illustrate the
possibility for a new type of fashion as may be more appropriate for an error of mass surveillance the and so there's
a hoodie and there's a burqa and there's a user out and these items you are of
course inspired by Islamic dress and the idea with this project is whereas a religious dress is traditionally and In some ways thought of as providing a separation between man and God and this collection
of items and stealth where re imagines religious dress in the context of mass surveillance as providing a separation between man and drones some of the other more
interesting things and slightly concerning word and these e-mails and some articles I came out so of course this is projects had but up against national security interests which it makes me as an artist from sometimes quite no aware of the possibilities for that to go wrong in 2013 received an e-mail for a request for publication a classified and intelligence document which is strange because Americans see that do you really need my permission most news publications don't even ask for it and the other 1 that was the Washington Post put a request for comment on the Office of the Director of National Intelligence at desk and for the NSA had I tried to you create work that doesn't put me possibly at rest for if who knows the area of national security is sometimes black-and-white and it's not a it's not a very open area to you are that's exactly why I'm doing his projects that I do the other 1 0 point out is a tweet from the Air Force general counsel at the Pentagon just making note that they are aware of the idea of self where to hide from military
drones so I see this gray area at were artist can operate as something that's very essential being able to not make some projects that are threatening to to any person in particular by kind of gives some room for us to think about the future that were creating and as a response to some of these anxiety-provoking e-mails that I received the next project I created was the privacy gift shop which is a way to make this sometimes of threatening ideas about national security and a little bit more friendly this gift shops friendly open accessible and safe places they differences that this is a gift shop for counter-surveillance part and privacy accessories so I
sell the items that can directly and the anti-drug very did quite well and sold out and it is expensive and I am a terrible businessman because emit tell you that you can also do that for less than 1 dollar and the way that thermal radiation is boxes by any kind of GoMiner fabric that's thermally reflective for thermally insulating and so the most obvious example would be a
space and you can get a pack for your whole family for less than 10 dollars and while that has the same of functional appeal to block thermal radiation the main difference is in the psychology of camouflage that is so great
that this is the status of my or blankets and mats and how it works so to wrap things up in the summary that there are functional solutions to hiding in blocking surveillance and there are more I'm artistic or psychological responses from more thought out kind of camouflage I think that when when we when we think of camouflage it's a woodland Vietnam era 81 pattern blotchy browns and greens and blacks but really camouflage didn't even really exist the way it does that before the 20th century and in the early 20th century
and camouflage was actually term for criminals hiding from the police the you can drop out a similarity parallel between the way that camouflage was thought of in the early 20th century to where that privacy was thought of as early in this century and some other quotes from a fantastic book about camouflage its impacts on Australia Theodore Roosevelt considered camouflage is a form of effeminate cowardice a mere defensive strategy but then you'll see as world war 2 as law 1 in World War 2 unfolded people came to realize the
significance in the power the strategic advantage and most importantly that camouflage became a sign of humanities increasing intelligence so I think that privacy can be thought of in the same way not as a defensive for cowardice response the privacy is also something that shows humanities increasing intelligence
and in closing we think maybe these solutions are a little bit eccentric but if you look at back in history the year 1918 in New York City everybody is learnable OK if you look around in the audience after this 1 person that's not with you look rather the audience nobody's wearing a bullet cap so it's very 3 possible for something that's completely normal to become abnormal and likewise possible for something that's very abnormal or eccentric now to become completely normal in the future thank you so
thank you and my amendment an