We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Fast Global Internet Scanning - Challenges and new Approaches

Formal Metadata

Title
Fast Global Internet Scanning - Challenges and new Approaches
Subtitle
Or how to become your own ISP
Title of Series
Number of Parts
102
Author
License
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Current search engines such as censys or shodan give everyone an insight into the global Internet. Unfortunately, they don't provide a comprehensive view of the Internet because you can't access the raw data. Consequently, you have to scan the Internet yourself. Anyone can perform a one-shot scan via Mass-Scan & Co. However, how to build an infrastructure for regular Internet scans that is not blocked after a short time by Intrussion Detection Systems and Spam/Blacklists is not easy. First we will talk about the right scan setup, infrastructure, scan strategies, and data enrichment. We will then take a look at the data and gain common and interesting insights into the structure of the Internet. Current search engines such as censys or shodan give everyone an insight into the global Internet. Unfortunately, they don't provide a comprehensive view of the Internet because you can't access the raw data. Consequently, you have to scan the Internet yourself. Anyone can perform a one-shot scan via Mass-Scan & Co. However, how to build an infrastructure for regular Internet scans that is not blocked after a short time by Intrussion Detection System and Spam/Blacklists is not easy. The following questions must be answered: Which scanning algorithms are used (centralized, distributed, BGP prefix hit lists)? How could you reduce scan traffic? How do I process the data in the long term (up to 600GB / scan)? With which further data do I enrich the scans for further analyses (BGP prefixes, Inetnum objects) ? How do I build the right server without a bottleneck and how do I connect it to the internet (rent a server or become a RIPE-Member/ your own ISP with a /22 IPv4 /32 IPv6 Block)? In the first half of the talk we will deal with these questions. In the second half of the lecture we will discuss real scan data. We will concentrate on the analysis of the network topology and distribution of BGP prefixes, whois blocks and network services of well-known autonomous systems on the Internet. As a further example, we will look at the network structure of a large well-known German hoster, which gives us a good overview of its internal organization of data centers and other services. Finally, we will look at some data and analysis from a security perspective.