WIRELESS VILLAGE - Capture & Analye Like a Bawss

Video thumbnail (Frame 0) Video thumbnail (Frame 3268) Video thumbnail (Frame 4864) Video thumbnail (Frame 6292) Video thumbnail (Frame 19416) Video thumbnail (Frame 21345) Video thumbnail (Frame 22894) Video thumbnail (Frame 24247) Video thumbnail (Frame 25567) Video thumbnail (Frame 27519) Video thumbnail (Frame 28665) Video thumbnail (Frame 30590) Video thumbnail (Frame 34474) Video thumbnail (Frame 35564) Video thumbnail (Frame 49485) Video thumbnail (Frame 55550) Video thumbnail (Frame 57382) Video thumbnail (Frame 62123) Video thumbnail (Frame 63284) Video thumbnail (Frame 65181) Video thumbnail (Frame 66933) Video thumbnail (Frame 68325) Video thumbnail (Frame 69608) Video thumbnail (Frame 70796) Video thumbnail (Frame 71928) Video thumbnail (Frame 73256) Video thumbnail (Frame 74744)
Video in TIB AV-Portal: WIRELESS VILLAGE - Capture & Analye Like a Bawss

Formal Metadata

Title
WIRELESS VILLAGE - Capture & Analye Like a Bawss
Subtitle
kitchen sink and baz fft
Alternative Title
SDR Basics Class
Title of Series
Author
License
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
2018
Language
English

Content Metadata

Subject Area
Laptop Boss Corporation Goodness of fit Word Software Multiplication sign Motion capture Metadata Number
Web page Slide rule Open source Software repository Personal digital assistant Source code Computer hardware Bit Quicksort Directed graph
Module (mathematics) Module (mathematics) Installation art Computer file Software-defined radio Cartesian coordinate system Type theory Hooking Software Different (Kate Ryan album) Single-precision floating-point format Computer hardware Utility software Utility software Quicksort Directed graph
Greatest element Length Multiplication sign Direction (geometry) Zoom lens 1 (number) Digital signal Parameter (computer programming) Mereology Computer programming Usability Neuroinformatik Bit rate Computer configuration Synchronization Different (Kate Ryan album) Physical system File format Sampling (statistics) Data storage device Electronic mailing list Bit Motion capture Band matrix Befehlsprozessor Buffer solution MiniDisc Configuration space Right angle Quicksort Arithmetic progression Spacetime Directed graph Row (database) Point (geometry) Game controller Mobile app Finitismus Overhead (computing) Computer file Data recovery Motion capture Letterpress printing Control flow Streaming media Software-defined radio Event horizon Number Frequency Latent heat Programmschleife Term (mathematics) Computer hardware Software testing Analytic continuation Loop (music) Demo (music) Counting Limit (category theory) Cursor (computers) Timestamp System call Frame problem Loop (music) Personal digital assistant Function (mathematics) Game theory Buffer overflow
Overlay-Netz Mobile app Email Identifiability Computer file Code Plotter Execution unit Motion capture Line (geometry) Rectangle Fast Fourier transform System call Timestamp Data transmission Frame problem Frequency Type theory Medical imaging Different (Kate Ryan album)
CAN bus Computer file Demo (music) Motion capture Directed graph
Frequency Execution unit Computer file Motion capture Electronic mailing list Function (mathematics) Line (geometry) Event horizon Computer programming
Area Computer file Right angle Line (geometry) Quicksort Food energy
Band matrix Frequency Game controller Process (computing) Motion capture Electronic mailing list Bit Coprocessor
Medical imaging Game controller Computer file Software Block (periodic table) Function (mathematics) Series (mathematics) Graph coloring Data transmission Frame problem Asynchronous Transfer Mode
Dataflow Greatest element Game controller Link (knot theory) Computer file Multiplication sign 1 (number) Motion capture Streaming media Function (mathematics) Event horizon Data transmission Template (C++) Revision control Frequency Social class Addition Graph (mathematics) Touchscreen Inheritance (object-oriented programming) Block (periodic table) Graph (mathematics) Electronic mailing list Instance (computer science) Frame problem Symbol table Wave Message passing Personal digital assistant output Right angle Quicksort
Scripting language Message passing Greatest element Computer file Cuboid Timestamp Data transmission Power (physics)
Satellite Pulse (signal processing) Existential quantification Length Multiplication sign Shape (magazine) Mereology Mechanism design Bit rate Synchronization Different (Kate Ryan album) Position operator Physical system Area Sampling (statistics) Shared memory Bit Oscillation Time domain Band matrix Arithmetic mean Wave Tower Website Right angle Quicksort Whiteboard Directed graph Point (geometry) Sine Computer file Data recovery Characteristic polynomial Motion capture Software-defined radio 2 (number) Element (mathematics) Frequency Broadcasting (networking) Propagator Computer hardware Tunis Optical character recognition Information Cellular automaton Uniform resource locator Word Wiki Personal digital assistant Synchronization Transmissionskoeffizient
Scripting language Email Addition Default (computer science) Statistics Information Multiplication sign Source code Sampling (statistics) Motion capture Letterpress printing Counting Menu (computing) Bit Software-defined radio Oscillation Number 2 (number) Inclusion map Differenz <Mathematik> Bit rate Different (Kate Ryan album) Right angle Maize
Point (geometry) Computer file Divisor Multiplication sign Motion capture Sampling (statistics) Real-time operating system Function (mathematics) 2 (number) Neuroinformatik Frequency Uniform resource locator Sample (statistics) Frequency Software Digitale Videotechnik Bus (computing) Row (database) Absolute value Summierbarkeit Directed graph
Point (geometry) Dataflow Overlay-Netz Computer file Multiplication sign Source code Software-defined radio Function (mathematics) Streaming media Event horizon Neuroinformatik Heegaard splitting Optical disc drive Frequency Roundness (object) Case modding Synchronization Analytic continuation Physical system Graphics tablet Stapeldatei Meta element Email Graph (mathematics) File format Block (periodic table) Structural load Sampling (statistics) Data storage device Interactive television Bit Line (geometry) Cartesian coordinate system Sequence Process (computing) Event horizon MiniDisc Right angle Buffer overflow Row (database) Asynchronous Transfer Mode
Revision control Default (computer science) Computer file Structural load Motion capture Software-defined radio Fast Fourier transform Window Resultant Probability density function 10 (number) Neuroinformatik
Dataflow Execution unit Decision tree learning Graph (mathematics) Computer file Multiplication sign Plotter Motion capture Electronic mailing list Software-defined radio Fast Fourier transform Sequence Heegaard splitting Goodness of fit Intrusion detection system Hard disk drive Address space Chi-squared distribution
Frequency Touchscreen Computer file Personal digital assistant Motion capture Line (geometry) Regular graph
Existence Computer file Block (periodic table) String (computer science) Source code 1 (number) Timestamp Metadata Disk read-and-write head
Band matrix Cache (computing) Frequency Mechanism design Computer file Semiconductor memory MiniDisc Cartesian coordinate system
Computer file Multiplication sign Line (geometry) Cursor (computers) Cartesian coordinate system Data transmission Symbol table
Point (geometry) Frequency Computer file Structural load Right angle Line (geometry) Position operator
Frequency Computer file Multiplication sign Zoom lens Motion capture Musical ensemble Quicksort
Context awareness Scheduling (computing) System call Multiplication sign 1 (number) Function (mathematics) Field programmable gate array Parameter (computer programming) Computer font Computer programming Response time (technology) MKS system of units Bit rate Computer configuration Feasibility study Software framework Series (mathematics) Physical system Flux Data storage device Bit Maxima and minima Regulärer Ausdruck <Textverarbeitung> Process (computing) Hard disk drive MiniDisc Quicksort Directed graph Row (database) Point (geometry) Game controller Divisor Motion capture Web browser Software-defined radio Login Metadata Goodness of fit Utility software Computer-assisted translation Computing platform Default (computer science) Information Projective plane Mathematical analysis Volume (thermodynamics) System call Visualization (computer graphics) Blog
and I'm done here everybody clapped for ballin no I didn't suppose Anders is here again I'd like to call him out every time every Def Con now Anders very kindly lent me his his prepaid cellular phone when he was visiting from overseas a couple of def cons ago we had word that I think we needed to switch on one of the science instruments on that space probe and I was here and so I tethered to his phone and then ssh into a laptop we left at the Arecibo radio telescope and they powered it up and then using some software that we wrote we sent the command while I was watching the talk because you know the talk was pretty good too so I didn't didn't want to leave that but anyway today thank you very much for coming and watching I hope this is going to be informative and you'll be able to learn some tips and tricks from this I just want to talk about some suggestions I might have and I'm going to be biased because it's just using some tools that I've been working on for quite some time but how do I basically use some tools to get good captures IQ captures in a manner that you can use to make sure that you record you know sufficient metadata under certain circumstances for if you're doing lots of captures and then once you have all that IQ some tools that you can use to actually review it and see where the signals that you're looking for or anything interesting may actually be in those captures so capture and analyze like a bowels and I just want to put it do a shout out to tonight and this night Nate temple dev knowing stick your hands up Nate that's balls number one where's balls number two where's niet Neil Neil Pandya where are you there he is hand out there this boss number two they've been very very helpful with all my efforts thank you guys so anyway the
first one is let me just get these slides up here there we go who's heard
of kitchen sink that's great then I have something to teach you kitchen sink is I'm probably the only one that uses in the entire world now but it also might be a little bit daunting because it has three pages of command line options so it's highly flexible but kitchen sink is a tool that I originally wrote when I was at a Surrey search and I've been sort of extending a little bit it's open source so I fork the repo and I've been adding features to it it's basically a Swiss Army knife for capturing and in this particular case also testing some of the more advanced features of of USRP so I've got one plugged in here and the thing about this is that I haven't tested it with kitchen sink but if you wanted to use this with other hardware like an RTL or any other SDI you can imagine what you can do actually is either modify the source code to use
another API or Josh Blum made soapy SDIO who's heard of soapy SDR yeah a couple of you so so BSD are supposed to be a totally vendor neutral API that allows you to access a whole lot of different types of software defined radio hardware it has been very comprehensive it's a very very wide support for hardware however when you check it out it's also really slick API and but one of the things he added to sophie was the ability to have soapy talked to usurps through UHD but then also he made a loadable module for UHD to access so P devices so you can install so P SDR and any application that uses UHD will then hook into so P and then you have every other SDR available to you to use as well so if you want to use kitchen sink with something else and stall so P install the UHD module and then you can access whatever you want so just yeah so
kitchen sink there's a command-line utility it's a single C++ file that you can compile and you basically can tell it to do all sorts of things I've mainly been using
it to do captures so you the command line ends up getting pretty long if you need to do more advanced stuff but let's break it down so here's an example you just run kitchen sink you tell it how many rx channels you want to capture on if your SDR supports one then obviously it's just going to be a single channel but if you're using an SDI that there's multiple channels like a B 210 or or what-have-you then you can have it capture both channels at the same time you tell it how fast you want it to capture the sample rate our x-ray progress interval is not and I'll show you this in a in a demo in progress intervals nice because you can either have it just sit there silently doing its thing or every second or however long you want it will print out the timestamp and how many samples is captured and so on that can be good for testing interestingly enough I found that if you're really pushing the limits of capture you know these things operate over USB 3 nowadays and you can do you know 56 megahertz worth of usable bandwidth you can stream that to ramdisk you can create a ram disk on on you know OS X or or your Mac or whatever you want and then stream it directly into RAM and then you can support that capture rate which is great if you want to do really higher bandwidth captures but I found interestingly that depending on upon your system configuration if you end up having the program print out the progress interval every however long there's you know maybe a system call that happens or there's a little bit of overhead there and sometimes you can drop a sample if you're really at the limit so sometimes you want to have that off obviously then you say the frequency that you want to capture at the antenna and then I'll talk about this in a minute but the CPU format that you want the samples in the gain the capture file and then comes the cool stuff which is about time synchronization so I found myself recently needing to do captures at specific times for a specific length it has some options depending upon your you know timing set up with your SDR about when you want to start because if you're going to ramdisk or you only have limited drive space and you're capturing a whole bunch of things to vol 10 easily you necessarily want a supervisor to let it go on forever another thing that I'll talk about is the timing file which is important so who's done captures where you've ended up dropping samples you try and capture a bunch of data and then your SDR is producing samples quicker than what your computer can record to disk this doesn't happen obviously when you're you know doing quite narrow captures slower rates but if you're doing you look really fast stuff like 50 mega samples per second you can overflow very quickly so what the timing file does is it actually stores in a sidecar file the hardware timestamp and the number of samples that have been received so that if you do in fact have any overflows you can go back and it's just a CSV file and see the time at which the next sample came in after you had a discontinuity which means that if you need to you can actually recreate a continuous stream because often what happens is when you take your capture and then you put it into some decoder right if you're if you're tracking a digital signal then if you have a discontinuity it might screw up your clock recovery your decoder will will you know lose lock on your signal and then you won't know what happened because it's assuming that the sample stream is continuously haven't dropped anything so there's something else that I'll show you you can use to that'll take in that timing file and then add the missing data and then you know whatever is consuming this won't have any problems another thing for example is you can add an rx start time down the bottom so if you have the ability to set the time if you have the ability to set the time in your device like on this one you can actually set you know tell it all right the time is now this or or you know I'll show you how else you can synchronize time but then you can actually say I want you to start streaming it at this time I'll come back to that in a minute and explain what's that's important but if you're doing multi-channel stuff this can be you know part of a MIMO experiment or if you if you go to different antennas pointing in two different directions and you want to capture at the same time or you want to look at two different frequencies but make sure that you record in a time aligned manner then you can actually tell to do multiple channels so you can give it you know a list of frequencies in a list of games and what's nice is that you can use this formatting option in the RX file so that instead of just spitting out one IQ file it will spit out two or if you wanted one but you want in to leave samples you can tell it to do that as well it depending upon you know whatever is consuming this would require and then some tips if anybody's using say a B to ten if you want to decrease the chance of overflows you can increase the number num receive frames and your UHD arguments just another thing that you add to to the command line options and people you know try out different values until a different values but I found that actually depends upon the hardware you're using and the controller and what-have-you so you need to you need to use an experiment with some different values there because some will just crash the air app some will crash the you know the the controller you'll get corrupt data so it's always good to experiment with those values and then there are the ones that that you know if you have a device like this it has an a side on the B side so if you want to use the B side as for the a side you can set that as well and then important thing about the CPU format when you capture IQ samples they're usually in one of two standard formats floating point or fixed point and floating point is you the FC 32 or CF 32 in soapy land and then you've got SC 16 so you got you know pairs of 16-bit short samples or pairs of floats and obviously if you have pairs of floats they're going to be twice the amount of data then to fix point you know short value so if you're trying to fill up your RAM disk at 50 mega samples per second obviously you want more bang for your buck so you can tell it to use the short samples and and what's interesting is that commonly when you actually stream from a device it actually ends up coming off the FPGA in a fixpoint format anyway so there's no point in having the host convert it to a floating-point format which is you know good when you're using it in various applications but if you're just trying to store it you can get twice as much in there and the other feature that I added recently was this rx file loop size and that's important where you have a finite amount of storage let's say Ram disk let's say you know you got 32 gigs there you you leave like 500 megabytes for your OS and if you want the system to just keep capturing and then you stop it when you want to once you've seen some other external event occur maybe you're looking at another monitor or maybe you're listening for something or looking at something then and you manually stop the capture it means that you it will store the last you know 31 and a half gigabytes worth of data so in this case it'll just keep basically recording into a circular buffer in RAM and if you specify the loop size it'll it'll tell you how big you want that circular buffer to be and then when you do that it'll create a loop file as well which is another sidecar file that will store the the file cursor and the sample count every time it loops and then at the end so that once you end up with your final capture file if you think in terms of a circular buffer the beginning of the file will be somewhere the you know the beginning of your your last 31 and a half gigs is not at the beginning of the file it's somewhere inside the file so it stores they'll offset there and then you can you can loop that through to recover your continuous recording so what's cool is once you do these big recordings you
can capture a lot of data this is a waterfall here captured around you know I think 437 megahertz and you can see there are there are a lot of different signals in there and it's a very very large bandwidth so what lots to wants to see and explore but the thing is if you want to actually go in and analyze these signals there's a lot of them and so it'd be nice to have some tools to sort of zoom in and look around and look a little bit closer so I'll show you some
interactive stuff in just a second this is an example where you can zoom in with this little tool called Baz FFT it can read in IQ and it'll plot it you can have a text file that lists frequencies and you can give it names and then there's this different app that users can do ready call the multi-channel runner that I'll show you and that will take in this massive IQ on all these frequencies that you specify and then extract the various channels and then decode them in in whatever way you wish and then it'll take that metadata out and then overlay it back on your waterfall so you can see these are the transmissions and it's got those dotted lines around the transmissions to indicate that's when the squelch opened if you're dealing with p25 transmissions
then you can do a big capture analyze that put it through the multi-channel p25 decoder and then who's familiar with the different frame types in p25 where you got your your header data unit and the tail and the the two voice code you know packets so when you transmit p25 there are different frame types that transmits there's a header and then when it's transmitting voice traffic it actually alternates between these two voice packets voice frames and as you can see there you can identify voice traffic by the cyan and darker blue alternating rectangles if you you know look at each of these channels and the waterfall so I'll show you that in a sec - but it's basically the vo p25 Dakota with a modification to spit out the timestamp at which each of the these frame types are decoded by the you know in an image channel so let's take a quick look at that I wonder whether I
can
where did that file now here it is
so I'll do some command line demos of
kitchen sink in just a second but these are just some captures that I did with
it and why is that not okay here we go so let's have a look at
the whole capture so when you run the
program it looks like like this so sorry
it's difficult for me to see you run you
run the program you give it your you'll capture file you give it a frequency list and then you give it the the event list which basically contains the output from once you've run it through them the multi-channel decoder and then it shows
you all of the channels that you've put into your text file it's just a list of frequencies and then it'll it'll draw the lines to show the channels and then once it's gone through and analyzed all those channels then you can just zoom in
and you can see that you have the IQ that's been representing the waterfall and you can see the the energy there the you know the red there but then because it's gone through a narrowband FM decoder and with the squelch in front of it it's identified with that white line around it saying oh I actually found something here and when it does that then it actually demodulates using narrowband FM makes a WAV file and then you end up with all these WAV files here
so you can you know click on it there in
your oh s and listen to them or you can also just double click them in the waterfall like this right and then it'll it'll play it back so anything that you like to you know the of interest to you you can just click the button here click it inside any the the area of any of these signals that it's found and then you know review it so that can be quite a powerful tool if you're sort of hunting around searching for a signal
that's a that's a trunking control channel where or maybe that's tetra so
this is kind of nice because you can record you know very very wide bandwidth
captures and then run it through this
multi-channel processor it takes a little bit of while to you know process all the channels per your channel frequency list but once it does that
then it'll spit everything out and then you can you can use that tool to analyze it and then let's see yeah so you can
also do it for p25 and similarly I won't show you the actual interactive view but with that besser 50 you can also have an output let me find a good example here also have an output just static images
if you want to render it to an image
file like a PNG so this is a the same
example instead of you know being that
interactive mode spits it out to a picture and you can zoom right in and again we can see those individual transmissions with the data individual data frames color coded there so in a way there's also a good diagnostic tool to determine how well your decoder is actually working so this is found in voice traffic here and then on the side this these series of pink blocks are actually try control blocks on the trunking control channel that's used to coordinate traffic on this particular network so you can see all the individual p25 packets and then if this was in the interactive mode if you ran it through
the multi-channel decoder with p25 then you would get out all the the p25 traffic so just as an example of what that looks like here I've talked about
this in the past but I I have this notion of the multi-channel decoder block in GUI radio and the way that works and and I should say that it's it's not optimal because you can use polyphase filtering and all sorts of advanced DSP techniques to to split off multiple channels in an efficient manner and pass them through some decoder but in this case fundamentally you just give it a list of frequencies and you have a template flow graph and it instantiates a bunch of those flow graphs and links the original capture file into each of them at the frequency so they all just operate in parallel so the way it works is that you make these templates and in this flow graph this is the opie 25 channel template and you you have a an input here which gets linked up to the original file at a particular channel offset and then it goes through a bunch of blocks squelch and what-have-you but the important ones down here in the bottom right where it can't quite quite read what's going on there I think this is I'm not showing my screen so I can't can't see the the baseband signal gets pumped through the opie 25 decoder just down the bottom here and then it produces the symbol stream for the channel and that goes into the audio decoder the first port here outputs the decoded audio which will then get put into a wave file and in addition it also outputs messages oops I didn't want it to like that it also outputs messages that get output from this instance and then aggregated in the in the controller class in the parent class and that all gets output to a file so every single time the decoder sees a a new frame in a transmission it'll output what that frame is and the time to produce an event list and that's the event list that gets pulled in and you know rendered out in in this tool that I was slinging so this one is the is the conventional FM version so if we were to
not look at the opie 25-1 but look at just the narrowband FM one then similarly this outputs the FM d modulated audio down the bottom here but also you get this message from the power scrolls coming out that the timestamp of when the squelch opened and closed gets logged into a central file for all of the channels and then you know the
script is able to know that in actually there was a transmission here and then it it it indicates that with the with the box so you can click on that again and then you know listen to whatever transmission is going on there so it's
it's a good way to review this sort of stuff offline and by the way if
anybody's got any questions then please please ask so let's talk a little bit
about time synchronization this is
important where you're trying to capture things either the same location but using different devices or capture things at different locations at the same time or capture things in a very very precise manner where the signal that you're capturing is also synchronized and you want to be perfectly locked to it so usually you know when you do your average old capture just say alright give me some I'm going to store them to a file but sometimes you need to make sure that for example the time that you end up sitting in your device is accurate and in the same time domain as say GPS or some golden reference time so you could have as an example on this board there's a GPS do so you plug in a GPS antenna this GPS do would synchronize to the GPS constellation and then give you a very accurate time base for this radio you can have other radios where you have a house sink where 10 megahertz and one PPS is distributed through some some location and you just plug it in there and then you get your time your time information so time this time information is usually split up into two elements one is pulse per second and the other one is a 10 megahertz reference clock reference frequency the PPS is important because it demarcates exactly when a new ii starts so commonly with the gps receiver you might see a little light blinking who's seen a light blinking on a gps geo GPS receiver for that light blinking indicates the precise time when a new ii has started and then the other element is this 10 megahertz sine wave 10 megahertz is just a commonly you know use de facto frequency and it basically gives you the clock reference for how quickly time should be advancing so commonly the way it works is with a SDR you give it both signals and you say on the next PPS edge the time is going to be you know however many seconds pass the epoch and then when the software-defined radio sees that pulse it latches that time and makes it active so just wait sits there waiting for the pulse and then as soon as it comes it knows this is the precise time at that instant but then that only usually happens at the beginning before you start capturing so what happens after it sets the time and then gets going well these STRs have their own internal oscillator right they have their own internal clock and they're all slightly different if you're talking about very very large bandwidth captures or high bandwidth signals you will very quickly start to see a difference in the timing of the signal that you captured and what the actual transmitter is is using and commonly that's why you need to use clock recovery mechanisms in a decoder because your oscillator and here will not be synchronized in the oscillator at the other at the transmitter and so a clock recovery will take into account these slight frequency differences that are produced by manufacturing differences in in the crystal so when you when you buy crystals actually come with a tolerance to tell you oh it's you know this frequency plus or minus whether ppm parts per million and you can then calculate out at a particular frequency how far off that frequency you might be within the tolerance of the crystal however if you use an external reference like 10 megahertz and plug that in and share that amongst devices you can use that 10 megahertz to discipline the oscillator on the SDR so the way that works on this for example is you feed the 10 megahertz in to a special chip a PLL chip and that 10 megahertz will then be used to discipline and synchronize the you know whatever oscillator or Hardware clock is running on here so it's basically saying this is the reference this is how fast 10 megahertz is really going you need to sort of tune and discipline your internal oscillator on there and and sync with it and once you feed at this 10 megahertz you can then you know capture or and and and capture any rate tuned to any frequency and that will be you know precise with respect to this reference so then going back when you get the PPS you set the time at that point and latch it and then once you've got the 10 megahertz then from that point on your radio will be capturing at exactly the rate that you want so let's let's keep it easy and say we want to capture the rate of 10 mega samples per second every second we expect to get 10,000,000 IQ points per second once we have the 10 megahertz disciplining that we'll get exactly that and then we don't need to worry about anything else and we'll be perfectly synchronized so let's say we have a couple of these at different locations with GPS do s they can all be synced up with a GPS antenna they all listen to the satellite constellation and if we need them to start recording at precisely the same time we tell to use the GPS do which produces a PPS and 10 megahertz and then they can all be locked on and running at the same rate and this is also important because a lot of systems out there that actually transmit use GPS as a means to provide an accurate reference so who can who can name some of them give me some examples FM broadcasts LTE digital TV they all use GPS as a reference if you look at a cell tower you can usually see a little sort of a white up word it's called you know bullet shape GPS antenna and that basically sits there and provides a very or hopefully stable and accurate reference and I won't go down the rabbit hole but it's really interesting if you consider how you know potentially vulnerable systems are if you're able to somehow spoof or disable GPS because once you do that in those cases you don't care about position it's purely used for time and they're all you know advanced systems are able to detect that and warn against that but still it's kind of a problem when you have that as a reference yes no no please ask questions in it don't be afraid yeah so that's a very good question what's providing the ten megahertz reference so with this GPS do actually there is a 10 megahertz oscillator on the GPS area which is much higher quality than the one you know the the whatever whatever reference is originating the board the GPS is providing both 10 Meg and PBS yep so you know if you're not using GPS but using a house sink you know just as an example when we were at Arecibo we needed to have very accurate time information on when we recorded everything so there they had a I think was a hydrogen maser clock that was very accurate and local on site and it provided PPS and 10 megahertz but they had that routed through the entire facility so you could just go anywhere plug your 10 megahertz and PPS into the wall and you would get super accurate time and that's what we did with the GPS is there and what it meant was when you set the time and I'll show you that in kitchen sink in a minute when the samples come back from the the SDR each sample then you can calculate the precise time at which the you know sample entered the the antenna port if you I mean there's a bit of latency there through the hardware but ignoring that you know very accurately when your samples were recorded question yeah so the question is how does the coax affect sample delay and what have you it does affect it so if you're using multiple channels in a system you need to make sure that your coax is all the same length you know if it these signals travel at a finite you know propagation rate through through that kind of medium so if you need that kind of accuracy and it need to measure off but and also you need to consider you know the bandwidth of your signal and and and that sort of thing maybe it doesn't matter too much it depends upon you know how precise you need to be in the overall characteristics of the system any other questions I'm glad that there there have been some keep them coming if something's not clear then you know let's keep it interactive and I can I can try and clean it up for you so just to demonstrate this a while back I wrote
this script called PBS diff and what it does is it it sets the the PBS and sets the reference information on the SDR and then it checks whether the samples that you're getting are actually at a rate that you expect so let me demonstrate that to you here
so if you look I'm gonna can everybody read that is that isn't alright so what I'm doing is I'm starting up as PBS diff and I'm telling it to use the GPS do so this only works if you have a reference that it can lock to and a PPS then it can detect if it can't detect the PBS it'll say you know there's something wrong with your PBS so it's a good way to diagnose if everything's connected properly and the way it does that is that basically you HD has an API where you can say get the time the last time you saw a PBS edge and it just keeps asking for it and if more than a second has elapsed or you know you can figure it more than two seconds or whatever I remember has elapsed on the host if it hasn't seen two different times at the last PPS then obviously it's not getting PBS so you can use that to diagnose what's going on so it looks for that it checks it asks the SDR the use of whether the PLL that I was telling you about on there has actually locked to the ten megahertz that you're providing it because sometimes if you give it a two-week a signal it won't lock and then you might think it it might be capturing and locked your your external reference but it won't be so it's always important to check that it's locked and once it's it's got a PPS and it's locked then it starts printing this this out every second and what I want you to look at here is on the very right hand side it says text if so tix is another way of thinking about samples I think here the default sample rate is is 16 megahertz and so what you expect is if you have a precisely tuned ten Meg reference coming in and the use of synchronized to it every second we expect exactly 16 million samples to have come out of the user every second so this is happening at you know printing printing out every second and it's detecting when the PBS edge comes in a minute I think it uses that to calculate stuff and then print this out but it's basically saying from the time at the last PPS to the next one count the number of samples and then you can check that it's correct so watch what happens now when I trick it by saying I want to use the the ten megahertz reference is now going to be the internal clock so don't use the ten mate coming out of the GPS but use the PPS coming out of GPS just to trick it and then we don't care about the lock so if I run that then the PPS is still coming out of the GPS right at its at its own you know more correct rate because it's high quality even though we're not synchronized in GPS here the PPS and ten make is still running and you can see now that the text if is not quite 16 million it's a little bit less because the oscillator on them on the SDR is not being disciplined by the same source as what's in the GPS that's providing that the PPS so you can see how there you can get a slightly different capture rate and if you need very very precise you know timing and your captures that's obviously a problem and then you need to work out why that's happening so in practice if everything's wired up and working correctly we would see 16 million all the time and then this also can be used to dump out some additional timing stats and why have you so that's
PPS DIF handy little tool there now what
about the timing file the timing file is important because say if you want to record something and you need to know when the samples are coming in relative to to real time the kitchen sink and output this timing file and you can go back and say oh you know it was exactly you know 333 p.m. and six seconds and microseconds and then a seconds all the way down so you can very very precisely identify the point at which you're you're looking at in a capture file the you know obvious way to do that ignoring all this is you just you know note the time on the wall clock on the computer when you started recording but there's going to be some latency between what the wall clock on the computer is and you know when the samples actually arrived there because you've got the bus latency you know Network USB what have you and other factors so if if you're willing to live with that that's fine but if you need very very accurate timing then you know you need to have these these references there and as I said you can use this to sing cries captures across frequencies across devices across locations now once you
have actually recorded your IQ and you have it an ax timing file then in job as I added this block which is the the Baz file source and it lets you open up an IQ file as you did before but it also lets you tell it about timing information as kitchen-sink wrote it to that timing file and that's important because you can either have well it will output time tags if you're familiar with canoe radio you can send metadata along with your samples so when the time takes a jump or start streaming you know if there's an overflow it jumps it'll produce time tags and send them along with your sample stream there is already a block that does this I think it's the with the meta meta file sync and meta file source and that works really nicely but they have a bit of a an odd file format that it saves everything in and this is just nice and simple you know line delimited comma delimited text file and the other nice thing is that it can automatically load a whole sequence of files so for example whose use HD SD are here or yeah a couple of you so HD SDR is a windows-based application makes it very simple to use your SD on receive signals and look at a waterfall and and D mod in your am/fm single sideband when you can't record there it actually splits files every you know 4 gigs if you recall with fat32 you can't store more than 4 gigs in a file so it'll automatically split them and if you end up using that to record data for a very very long time you'll end up with a lot of files and if you want to process them all as one big batch in canoe radio then it's going to be pain to have to run your flow graph load up the next file run it again and so on so with this this will automatically detect a sequence and then load them all at the same time so you just basically treat all of your split files as one big one and if you need to seek around in there you know this has a Python API so you can call and it will load the right file and sync to the right point in that in that file I'll show you that in a minute and it could automatically get the sample rate out of the file depending upon if it's in the timing file or if it's a wav file and the header and so on so it's it's nice if you've got stuff downstream of that and I'll show you that as well now regarding the overflows remember I was telling you that an overflow is a discontinuity so you have lots of continuous samples your computer can't write to disk fast enough it'll drop some samples and then it will start going again so with the timing file you know the time at which it started streaming in continuously but with this block you can change the the pad mode so you can turn padding to on and it'll actually produce you know fill of samples zero samples to pad out all those missing blocks so if you've got something downstream that's assuming that your sample stream is continuous with it out and you discontinuity this can fill those holes so I showed you
some of the things that beta 50 can do it'll produce a nice waterfall it'll annotate it with events that are produced by Dakota or it'll draw lines through the frequencies that you're interested in but now I want to show you the interactive playback mode and then I'll round out the talk I think the interactive playback mode is a system
where you can load up a bunch of capture
files it'll load them all automatically
and then you can interactively review don't have it loaded already now I do you can interactively review everything in the file so I'm just going to stop it again one thing there are some
features to this that I won't discuss now but there's FFT has another feature I showed you know I showed you where instead of just going to a a matplotlib a window you can have it dump to a PDF or to a PNG it also has one where it'll compute all the FFT data because when you run it by default it needs to go through read all the IQ compute the F of T zavoral emit cetera if you don't want to do that every time because you're dealing with you know tens and hundreds of gigabytes worth of data you can have it computer and then store the result to a separate file which is like a cached version of all the the FFT data and that's usually much smaller and much much faster to load if you you know need to reload it later so I have you know this HD SDR file just to illustrate what I'm when I'm talking about I'm going to copy that so I don't lose it and then you'll see here after wait for my drive to spin up maybe hello there we go
so there are a lot of capture files here right so these were recorded with HD SDR they've been split at you know whatever the split I think it maybe might have been two gigs actually and there are a lot of files so you don't want to have to open all of these up individually it'll automatically load them you don't have to recompute the FFT every time you want to view something so it'll do all that and then store it as this
intermediate file so if we load that up it will open that up and then you see here I've also added this XML address this is going to use XML RPC to talk to a good new radio flow graph and then we can interactively that's just being read from my hard drive if you if you do this make sure you use SSD because using mechanical drives is much much slower so as you can see here that's actually a list of all of the original files that we used to build this composite there are a whole list of them that were automatically detected in sequence when I ran it before you know it's rendering the plot out here we go so if you can
see there you've got these horizontal lines that run across the screen at regular intervals that demarcates which capture file was was used at that particular point and if you look over on the left hand side you can actually see each of the capture files that were loaded to produce some data and then when you ran out of data in that file it moved on to the next one and then again we have a lot of signals in here so you can you can zoom in to them and look
around in this case I didn't run it through the multi-channel decoder that I was describing before so it hasn't gone through I haven't given it the frequencies to look at and decode and separate but now we can do that interactively and I'm going to run not
that one
I'm going to run this so this is the the
playback and what I do is I just give it the first file and that's the file that gets given to that gr bass file source block and then the file source block will automatically load everything else so when I run it if you watch you'll see it load the first one and then load up each of the other ones in succession and this is just uh nota matically because it the metadata and the HD STR file has a wall clock time stamp of when it ends and so it looks if there's a file in existence that would match that the timestamp you know that it should continue at so let's see what happens look you see it loaded them all up there and now you'll have to pardon the the
breaking in the audio because it's a very very high bandwidth file and it's reading off my mechanical Drive it's slow but it's also going to the disk
cache so when I load it up and replay in
a minute from memory it should be faster so this is this is the channel that
we're looking at but if you look
somewhere there is that is the frequency offset so there's some frequency offset that I've set in the file let me just change the squelch here so that we don't have this annoying there we go sound so this is all well and good we want to explore the file interactively so we can bring this up and now these two
applications are actually talking to one another so if you scroll here see the
white line that white line is the time cursor in the file that's actually being played back from the gurney radio application so the griot and radio applications loaded up all this stuff and this is showing us you know the play cursor for the file so let's say we want to actually listen to some of the these transmissions here it's I don't I can't remember what's in this file let's see
if there's anything interesting here we go this looks like some audio here so I can
have this here and then let's say I want to listen to clicking the the symbols putting the file you can just click
it'll load that file seek to that position and then select that that frequency in the in the channel Iser for the playback and then so again it's
because it's really off mechanical but
if we start it again yeah so and then you know maybe that's uninteresting so you click over here and then it'll seek to that point you know pick the right file stick to that point start replaying and demodulating that file so in that respect it's I think a pretty pretty powerful tool line to you
know review individual signals if you're looking for a needle in a haystack in a large capture file and you know if nothing's interesting there then you would just zoom out in time of frequency zoom out in time of frequency and then have a look at somewhere else in the
file and then you know if you're interested in you just click there [Music] who can tell me what that signal is [Music] yeah some sort of paging system so that's that's a good way to do it and then once you find your candidates
signal then you can isolate that channel pull it out and do continue or you're processing on it but this is kind of a neat kind of visualization search tool so the last thing that I'll show you
then is just a little bit of kitchen sink
so as I was saying it has quite a few commandlineoptions oops as you can see there quite a few options did I already surely there aren't that many no that's actually all of them yeah it looks it looks like it's more because I'm not used to seeing in such a large font so if you just run it with no arguments then it'll just start testing the streaming on TX and rx and then you know you can do things like rx then this will just do RX only and then in every second it will print out the rate that it's going at and it's it's on purpose quite verbose so it's telling you what it's doing what it's setting at every step so that if you need to you know say check your logs or copy and paste that your console output so you need to have a record or what have you then it's there you know other enhancements might be adding this to some other metadata that's that's output alongside all the timing information but again you know with all those other command-line options you can sort of tell it exactly what to do and when and store metadata so that you have all the information that you need later on to perform analysis with a really good notion of the the time that was used at that point so I think I'll wrap up their questions so the question is what kind of maximum sample rate can you expect with usb3 when using hard drives and SSD it really depends on a lot of factors it's sort of a default answer with I think with modern SSDs you can as long as your system is perform it you can I mean I think I can do 50 mega samples per second usually got a ram the problem is if you want the best performance you need an Intel chipset until USB controller and you need to run under Linux and you can play around with the kernel and if you want to eke out a little bit more performance I love OSX you know all this is running on OS X obviously but OS X as I think we all know is designed to give you really slick UI performance and good response time there which means it comes at the sacrifice of scheduling in like a capture program so when I when I cat if you wanna do on the OS X you need to reboot your system make sure nothing comes up and then capture like that and I'm actually done I have spent so much time trying to figure out how to optimize performance on here turns out it when you're on instruments and you look at the call stack and that what's the heaviest it's the Darwins system call that's submitting the the USB request so everything else is pretty lightweight so there's a neat program called Ram disk Creator and you can use that to make a ram disk it'll just pop up as a volume what I found funnily enough is if you use Disk Utility and then reformat reformat it as exFAT instead of just HFS+ which is a default you get a little bit of extra and then if I have my web browsers running I need to use kill and then run a regex to like stop every single thing to do with any web browser that I've got because they're just so heavy and cause the scheduler to context switch so in all these little things yeah the question is what's the feasibility of doing custom processing and decoding on the FPGAs that's a whole topic unto itself there are lots of possibilities if you want to talk to Neil or Nate they gave a talk yesterday about using RF knock on the FPGAs a lot of people have done some amazing custom stuff and there's all sorts of frameworks to do it in so a lot of options out there this is a USRP b200 but you know as i've mentioned if you use soapy SDR then all this stuff can apply to whatever radio you might have any other questions no all right yeah one more for like timing stuff and all that kind of jazz oh one of good places Guinea Radio dorg they've got lots of stuff there the RTL SDR blog has you know a long-running series of all sorts of really cool projects on all sorts of platforms and yeah I mean they're the two other main ones it has anybody else come across any other good resources lately yeah please yeah so yeah I think that they're too good ones other people might have some some other suggestions that they've they've used recently but anyway thank you for for coming and thanks for your attention if you had any other questions please find me afterward thanks a lot [Applause]
Feedback