Forban: a simple link-local opportunistic p2p free software
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 64 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45926 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201123 / 64
3
7
10
11
17
19
21
28
33
34
35
37
40
44
48
49
52
55
57
59
62
63
64
00:00
SoftwareLocal ringLink (knot theory)Shared memoryProjective planeFrustrationArithmetic meanInternetworkingGoodness of fitWordCopyright infringementComputer animationXMLLecture/Conference
00:47
Software frameworkInternetworkingStructural loadBlock (periodic table)InternetworkingText editorCohen's kappaSoftware frameworkPoint (geometry)Physical lawInformationUniformer RaumComputer animationLecture/Conference
01:52
RingnetzProcess (computing)Digital rights managementDigital signalLibrary (computing)Library (computing)Digital rights managementPoint (geometry)View (database)MereologyPhysicalismLimit (category theory)Text editorFunctional (mathematics)E-bookHeat transferChemical equationComputer animationLecture/Conference
03:12
Kolmogorov complexityCommunications protocolInformation securityDistanceInternetworkingFile formatWireless LANComputer networkProgrammer (hardware)Extension (kinesiology)Mathematical optimizationBand matrixData structureCodeShared memoryDefault (computer science)Computer fileVertex (graph theory)Subject indexingClient (computing)Web browserContent (media)PrototypeDirectory serviceRevision controlAsynchronous Transfer ModeSoftwarePoint (geometry)Core dumpComputer fileInterface (computing)Default (computer science)Band matrixData structureComputerProjective planeWeb browserCodeMathematical optimizationObject (grammar)Extension (kinesiology)Information securityUser interfaceWireless LANPersonal area networkCommunications protocolShared memoryClient (computing)Different (Kate Ryan album)Operator (mathematics)BitSession Initiation ProtocolModemInternetworkingSummierbarkeitPattern languageFilesharing-SystemFile format1 (number)Forcing (mathematics)Physical systemOrder (biology)System callComputer animationLecture/Conference
10:22
Communications protocolLine (geometry)Computer filePrice indexInterface (computing)UDP <Protokoll>Computer networkRevision controlMessage passing19 (number)Hacker (term)Directory serviceEmailContent (media)Asynchronous Transfer ModeDigital filterAlgorithmInterface (computing)Computer fileDefault (computer science)Subject indexingSoftwareDifferenz <Mathematik>Lattice (order)Codierung <Programmierung>ModemLocal ringCodeCommunications protocolMessage passingCASE <Informatik>InformationTwitterLimit (category theory)Link (knot theory)IP addressKey (cryptography)User interfacePole (complex analysis)Binary fileUniversal product codeComputerTotal S.A.Scripting languageHand fanDifferent (Kate Ryan album)Address spaceDressing (medical)Open sourceBroadcasting (networking)Web browserElectronic mailing listFile formatFrame problemLine (geometry)Filter <Stochastik>CuboidFormal languageGoodness of fitBookmark (World Wide Web)Library (computing)AlgorithmAsynchronous Transfer ModeHash functionMereologyShared memoryComplex (psychology)Web 2.0Core dumpEmail1 (number)Denial-of-service attackComputer animationLecture/Conference
15:32
XML
Transcript: English(auto-generated)
00:07
Thanks Fosdom because it's a great place for sharing and I think as we are doing technologies for sharing it's a good idea today to do sharing. So we'll talk about something called Fourbon, you can pronounce it with a French accent,
00:21
sorry for my French accent, but Fourbon is a play word between Fourbon like the pirates but Fourbon like for ban, meaning banning on the internet. I will explain all the background of the project in the 15 minutes I have, basically because the project came out from some frustration I had.
00:47
So the first thing that is very important is right now the legal framework, so the copyright law and so on, is basically limiting your ability to share information and to share with your neighbour or your friends is becoming very complex and very difficult.
01:06
And that's for us a very key point why we developed Fourbon is we want something that we can share with friend and neighbour without too much difficulty. Another problem that we have seen is basically editors, so the one publishing books or publisher
01:21
are basically trying to kill all the confusion in the society. So whatever you try to do for exchanging books and so on, they try to limit your capability of exchanging books. And worse, and I think Eben Moglen talked about it this morning, is internet starts to be controlled.
01:41
So more and more laws are limiting your behaviour on the internet. So for example if you start to share on the internet, now you receive a letter and even worse you get a block from the internet. And I wanted to take the example of the books because books are really a part of our society. From my personal point of view I can't live without books.
02:04
You can learn from it, you can learn science, you can learn engineering, you can escape from the world, it's a great thing. The thing is books start to be a bit useless with those bloody DRM, these kind of digital restriction management tools where basically the editor are pushing the limits and try
02:26
to avoid all the nice functionality we had with physical books and limit people buying books and limit their ability to share books. So that's why we were wondering because currently with physical books it's very easy to go
02:41
to a traditional library to get a book, to rent a book, to lend a book, it's very easy. Now with digital books is it easy, is it more difficult? It starts to be very strange because basically the library is not able anymore to share books. So I think we have to counterbalance this, that we have to maybe play the role of the
03:06
liberalians and transport by yourself all your books. And that was the crazy idea we had. So basically it started as a discussion, you know one of those crazy discussions between geeks and the major question we had was the following.
03:25
The first one was can we do file sharing with all the internet? And I think today it's very easy. Computers are everywhere, each phone is basically a full blown computer. So sharing starts to be very easy, setting up a wireless network, a personal area network
03:44
is very easy. Another thing that we discovered, it's basically more easy to exchange or share books with strangers. Why? You go to Usenet or you start your peer-to-peer client and you start to look for books. But basically the guys who are sharing books or girls who are sharing the books with
04:02
you are basically strangers. Why is it not easier with your friends? Another thing we were wondering, if we start a new project, we want to start something very small and simple and we don't want to start by a very complex project. Another thing we had in mind is security. Do we really care about security when we do file sharing?
04:23
And it's something I really should not say about it because I'm working in security. We completely forget about security because it's for us a pain in the ass to handle and manage, so that's why we started to work on this project. And we wanted to make a very simple protocol when we have been very kind of proximity,
04:42
so that means when you are close to a friend, how do you do sharing? Right now when you are in a bookshelf or next to a bookshelf, you just share with your friends which kind of book you have. So you look at book cover and you exchange books, it's very straightforward. So that means the technology needs to be as simple. So that's why we created Formon with some basic objective and some basic technical recommendations.
05:05
First, we wanted to basically rely on the HTTP protocol without going into very complex stuff. We just had to add minimal extensions to do the announce, but basically it's full-blown HTTP. So you can use any HTTP client out there to access some Formon nodes.
05:23
Something else that we didn't care too much about in the initial prototype is bandwidth optimizations and so on, so that means if you have a Formon node, they will use all the bandwidths and they will do best effort. So for the first version, it might be sometimes unstable, but it's quite interesting for designing a protocol.
05:43
Something that we discussed too is we wanted to have something that is really independent from files, format, and file structure used. We want something that people are just putting files and it's automatically replicated. Something that is quite important, a lot of networks are basically unstable. That means they are going up and down very often.
06:02
Wireless networks are used to be unstable. Maybe you see here at Fosdam. It's very easy if you're not very close to the access point. You might not have wireless access, so it's basically unstable. So you have to take into consideration such kind of thing.
06:20
The idea is basically to have a prototype that can be implemented and even remixes. That's very important for us. It's an early experiment, so don't expect everything to work perfectly. But basically we wanted to have software, so we just implemented the idea,
06:42
and afterwards we hope that other people will create other versions and other ideas based on this idea. As always I say, you can't blame us for what we are doing, but we are doing something. So if you see a problem or if you want to improve the protocol, it's fine, it's great. But show us the code.
07:03
The core concept between 4.1 is different mode of operations, but the main mode is called the opportunistic mode. What is it all about? The opportunistic mode really has the name stands, it's an opportunist. So that means it's sharing everything and it's copying everything.
07:21
In the academic literature, you can see it as a well-known protocol called epidemic protocol or gossip protocol. So that's basically copying everything. There is another mode, the shared mode, where you can share everything. And you have another mode where you can basically get any file you want.
08:10
So all it looks like, it's very easy. What do you see? It's all the nodes available on the networks. Here you see one called elliptic, and you see another one called Clarisse-McLeans.
08:24
Each node has files that are basically books or whatever you want to share. There are seen IPv4 and IPv6. You can see basically the files missing between the nodes. So this one is Clarisse-McLeans and she's missing 567 files.
08:43
That's how it works on the interface. Obviously, we are not user interface designers, but the idea is every node sees each other. So to go maybe deeper into the systems, it's how it works. It's basically you are in a wireless network.
09:03
You have two four-band nodes. One is using the opportunistic mode, that's the default mode. And then you're using the shared mode, where it's basically sharing files but without getting the files. So how does it work?
09:20
That's very easy. The opportunistic mode is getting all the files from the shared one and stacking all the files. So that's basically a simple fetch and merge all the files. Imagine a third node is coming into the wireless, so basically it's entering the wireless networks. You see you have still the opportunistic ones, you have the shared mode one, and you have a node without any files.
09:45
This one is going into the network and you will basically do the same thing as the previous one. You will see the two nodes, get all the content, and that's it. So that means each node is basically replicating all the content. That's very brute force, but it works.
10:04
So you see now the third node has all the files. It works even with a web browser. If you know where the four bar is, you can get and browse any directory you want. How you browse it is very straightforward.
10:22
You basically go to the interface and you have here a small button browse, and you can browse any files. So that means any web browser should work out of the box. How does it work? We have technical people over here.
10:40
It's very simple. We have an announce protocol to just do the announce. I'm pretty sure we'll get questions about this. Why don't we use rendezvous, DNSSD and so on? It's basically because we wanted one single message containing all the information. So that means if the index is new, the source IP and so on.
11:01
Doing this with DNSSD or SSDP or any of those protocols, it's very complex. So that's basically a flooding protocol with just regularly flooding the network with a UDP frame in v4 and v6. Each four bar is named by a unique ID. This unique ID is randomly generated.
11:21
It's flooding the networks, and each four bar is maintaining a list of all the ones you discover. That's basically it. Afterwards, it knows how to get the index files. So it's basically using the HTTP protocol to get each file. So how does the format look like?
11:42
The format is basically no binary format. It's a full text format. So you have a source interface. That's basically where the announce is coming from. In the destination, you see we are using broadcasts or multicast link local addresses for the announce. What's inside? Basically, your header is called four bar, and you have a key value entry with none of the four bar, the unique ID, and the ASHMAQ.
12:05
The ASHMAQ is basically the index hash of this format. If the index changes, it will be changed. Right now, the ASHMAQ is outcoded to one key, but it can't be elaborated after to a private format with a specific network.
12:22
Why we are doing this? It's basically a simple protocol. So that means you can have, in one-liner, using TCP dump or Wireshark, whatever you want. Doing scripting around the protocol is very easy. What you can do is, you can filter on listening on an interface, get the payload, filter on the default port that we are using is 12555,
12:46
and look for the beginning of the UDP frame for four bar. That's it. And like that, what you have is a one-liner containing the announce. Easy. Afterwards, when you know how to reach a four bar, you can get the index.
13:01
Or you get the index, you basically use the source IP addresses from the announce, use the default destination port that you get, too, and you download the index. What is the index? It can't be more simple than that. It's basically a file path with a file name and the size of the files.
13:22
That's it. What is the index? It's basically a text file containing recursively all the files available in the share directory, the total size, and everyone can fetch this file, and the file is a text file without any limitations. It's a raw file, so it's not UTF encoded whatsoever, it's depending on what each node is doing.
13:45
So that might be a problem in some cases. How to get a file? Basically the same stuff. You get the files, it's a get method. You still use the information that you get from the source interface. You encode the file name from the index as a base64 URL save encoder,
14:03
and that's it, and you get the files. From the web interface, it works from the same way. You just have content dispositions to make everyone happy. All the opportunistic mode. This one is very simple, too. It's basically comparing all the index, doing a diff of each index,
14:24
and merging into a single file and downloading the files that are missing. That's basically it. Nothing very complex. You can build filters to extract some files, but the algorithm is not that much complex. If you have the code, I think this part is around 25 lines of Python.
14:42
So if you want to do it in Bash, Lua, or your favorite language, or your favorite scripting language, you can do it. That's pretty straightforward. There is something quite interesting that I think is worth mentioning. Basically, as you see, the index is relying on file names.
15:02
We don't look at the hash of the files, so that means, and that was a good question from Thias, if you have files with basically the same file name, always you will keep only the largest file. And that's it. So, do you want to share at first them?
15:22
So I have my library with me. I have around 4 gigs of books. And that's it. So you have the information.