Mongrel2 experiences adding IPv6
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Alternative Title |
| |
| Title of Series | ||
| Number of Parts | 64 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45938 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201111 / 64
3
7
10
11
17
19
21
28
33
34
35
37
40
44
48
49
52
55
57
59
62
63
64
00:00
SoftwareCommunications protocolCartesian coordinate systemSoftware developerSlide ruleComputer animationXMLLecture/Conference
00:35
Fluid staticsComputer fileProxy serverFront and back endsFormal languageLogicFlow separationFlash memorySocket-SchnittstelleUniform resource locatorDeterministic finite automatonHost Identity ProtocolParsingFormal grammarChemical equationSingle-precision floating-point formatParsingProxy serverBuildingFlash memoryConfiguration spaceLibrary (computing)Web 2.0Multiplication signWeb applicationComputer fileCommunications protocolNetwork socketProgramming languageCodeClient (computing)Block (periodic table)Operating systemDatabaseFront and back endsGastropod shellDebuggerReverse engineeringData managementNormal (geometry)NumberParsingCartesian coordinate systemCASE <Informatik>Electric generatorSkeleton (computer programming)Message passingVolume (thermodynamics)Operator (mathematics)Standard deviationResultantDescriptive statisticsOnline chatSocket-SchnittstelleSoftware bugToken ringMathematicsComputer animationLecture/Conference
03:59
Interface (computing)CodeTask (computing)Server (computing)AdditionKernel (computing)Cache (computing)Proxy serverDirect numerical simulationDefault (computer science)CodeSoftwareKernel (computing)MereologySlide ruleLibrary (computing)Logische ProgrammierspracheLevel (video gaming)Task (computing)Connected spaceAbstractionWeb 2.0Front and back endsServer (computing)Network socketKeyboard shortcutFamilyConfiguration spaceDirect numerical simulationBitInformationProxy serverMathematicsPhysical systemAddress spaceData structureCache (computing)Socket-SchnittstelleInterface (computing)Reverse engineeringImage resolutionPoint (geometry)Client (computing)Primitive (album)EmailLatin squarePlastikkarteScaling (geometry)Multiplication signComputer animationLecture/Conference
07:53
CodeServer (computing)Address spaceLetterpress printingAnalog-to-digital converterFamilyOperator (mathematics)CodeNetwork socketData structureComputer programmingAddress spaceBitSocket-SchnittstelleComputer animationLecture/Conference
08:30
Network socketDefault (computer science)Single-precision floating-point formatMobile appProblemorientierte ProgrammierspracheAddress spaceImage resolutionPoint (geometry)Address spaceDependent and independent variablesDigital electronicsImplementationNetwork socketCartesian coordinate systemSpacetimeSelectivity (electronic)Real-time operating systemMultiplication signDefault (computer science)BlogClient (computing)Front and back endsSet (mathematics)CASE <Informatik>Connected spaceProblemorientierte ProgrammierspracheView (database)AlgorithmServer (computing)Keyboard shortcutInternetworkingIsing-ModellCodeFamilyComputer animation
11:00
CodeCartesian coordinate systemSoftwareTerm (mathematics)BenchmarkOpen sourceState of matterDirect numerical simulationMereologyStatisticsCodePresentation of a groupLibrary (computing)1 (number)Fluid staticsDifferent (Kate Ryan album)Projective planeProcess (computing)Multiplication signOverhead (computing)Computer fileThread (computing)Network socketCommunications protocolProblemorientierte ProgrammierspracheResolvent formalismDemo (music)Computer animationLecture/Conference
14:19
XML
Transcript: English(auto-generated)
00:07
I'm one of the network guys as you can guess by the t-shirt so hopefully the network was working okay and how a networking guy can talk about the networking protocol at an application developers conference. Well add this
00:24
protocol into a cool application and then talk about it and talk about both. So we will start talking about the mongrel tool if the slides go and change. Okay so mongrel tool first of all is a really a Lego building block
00:41
for building your web applications. So you have clients coming in on the front end connecting by normal HTTP requests or if you do the things like web chats you can do long polling or you can have the flash applications on the client side connecting to the mongrel tool using the flash sockets and what
01:03
mongrel tool does is that it converts those requests on the back end into the zero MQ messages and you connect your backend handlers via zero MQ to the mongrel front end. So what this gives you is very much flexibility and
01:21
language agnosticism so you don't care which language your backend is written in. So the mongrel is very much language agnostic in this case. Now it obviously also handles the static files and the reverse proxy. Manageability of mongrel tool is also very interesting thing. It has its
01:43
configuration embedded into the SQL I3 database. So why that? That means that you can not only use the built-in m2sh so mongrel shell that is used to control the operation of mongrel tool to convert the standard text configs
02:02
into SQL I3 but you can also use your own configuration database and then you can integrate SQL I3 that mongrel tool uses together with your configuration backend. Another interesting thing is that you can use the rails like routing so you specify which handle attaches to which URL subtree. So think
02:24
of it like mounting of the volumes under any of the UNIX operating systems. Efficiency. So how many people in the room worked with the mongrel, with the previous generation thing? Okay, a few. What mongrel and mongrel 2 have in
02:45
common is the skeleton. So the skeleton is the ragel made parser for the HTTP protocol and it's formally generated code from the description of the HTTP protocol taken from the RFC. So as a result the performance of this
03:05
parser is quite good and also the number of bugs that you create when you make the code manually is also much lower. So you have automatically generated parser and this gives both the performance and the strictness of the parsing. On the backend at the same time we have zeroMQ. So zeroMQ is
03:24
very lightweight messaging library and using this library again allows to abstract from any language, any kind of protocol that is used to signal with the backend. Because with zeroMQ if you were, I think there is another lightning
03:44
token zeroMQ library itself and you can see there that the number of languages that the library supports is astounding. So there is probably 15 to 20 languages that it supports. So that's a really good thing. Now for me I wanted to
04:03
have a great web server to use that would support IPv6. So mongrel 2 was perfect candidate with one detail that it didn't support IPv6. So I started digging into the code and looking how the networking part is implemented. And the networking part is implemented as part of leap task which is really a
04:25
task library by Ruth Cox and the abstractions that are in that library are really pleasant to work with. So effectively you don't need to work with this one-time use structures where you take the name then you convert it into the structure and then you just use that structure once. So all of that is
04:43
abstracted inside the library. So effectively there are three primitives that are there that you need to work with. It's announce the socket then start connecting to a client or a server and accept the incoming connections. So this provides an abstraction layer from the rest of the code. And how to add
05:06
IPv6? There's plenty of documentation on the web so I just picked one of them which shows the example of code before porting to IPv6 and after porting IPv6 support. So I basically just went through the same logic as was there.
05:24
Obviously what I showed in the previous slide this very nice level of abstraction also gave very few changes in other places in the system. So there was only one place to change in the remaining part of the code of mongrel 2 which is kind of giving an impression of the niceness of the quality of the code base.
05:47
Another assumption that I made was that bind v6 only variable is set to zero. So we'll talk a little bit about that variable later on. Now after I made this code then we shaped the mongrel 2 with the support and there came
06:03
some interesting problems. So first of all get address info is not cached. So when we use it in a reverse proxy and the configuration specifies the target by name then per connection attempt to the backend server we have at least
06:21
one DNS request. Well in reality it's two packets the IPv4 and IPv6 resolution attempt. So that's not very performant if you have DNS server a couple of hops away. So you would either put the DNS server closer or maybe in the future we will change the logic a little bit by adding the local cache. Now another
06:43
interesting point that I heard is that some people at this they don't have the kernels with IPv6 enabled at all. So my optimistic assumption that hey well if I need six sockets will be just created it failed. And finally which is not specific to IPv6 was the increased latency in serving the
07:06
request because there is no non-blocking name resolution interface in the socket library. So for the users who don't have IPv6 in the kernel basically that
07:23
came up with an interesting hack where you explicitly specify the bind address. So if you specify the v4 wildcard bind address then you bind only to v4 address family and then you work around this fact that some people don't have the v6 support in their kernel. Or if you if you're ready when you're ready to
07:42
deploy v6 then you can just change that variable to colon colon which is the wildcard for IPv6 address and then you can bind your server to IPv6 socket. Now a little bit of rant about having to deal with two address families is that inet N2P is not really suited for printing out the
08:05
socket addresses because it deals with the address as itself whereas in the code you mainly deal with the socket structure where you have the address and the port. So then you need to have a little bit of this kind of slightly spaghetti code that has if families 4 then do this or a family 6
08:24
then do that which is a bit of inconvenience because I would expect every program would need to have this kind of operation. Now remember I mentioned the single socket so the bind v6 only. It's a very interesting feature where you are able to port your v4 applications in my opinion
08:44
much easier because you had one socket before and you still have only one socket afterwards. It's just that this socket can handle both v4 and v6 addresses kind of magically. So your previous v4 space gets mapped into v4
09:01
mapped addresses on this socket. So on the other hand there are different opinions. So this was my opinion from the implementation point of view. There are other implementers who may have other opinions and this setting had a interesting history towards first on then off and finally now it's again on
09:22
in the Linux by default but in VSD it's off by default. Another interesting problem is address family selection. So it was not really a big problem I think for the mongrel 2 code per se because normally the scenario that where
09:43
we use the name resolution as a client imply that we have both the backend server and mongrel 2 as a client. So in this case both hosts are in the same administrative domain so you can fix if something is broken but if you are developing applications that are only client-side and if you have
10:04
client-side applications that require the real-time user response then you probably would need to deal with it somehow less relaxed manner I would say than RFC 3484 specifies because 3484 just says okay you first select v6 and
10:22
then you start going by the addresses you get from get address and for. So for that me and one of my colleagues we had a small internet draft on the idea of how to do the address selection and also very recently Mark Andrews from ISE he published a blog post where he specifies a different algorithm but
10:43
probably more efficient than ours where he bounds the connection time by fixed interval. Now obviously all of these cool things for user experience are not to replace fixing the brokenness. Finally a few conclusions
11:02
that I've taken for myself from this mongrel 2 IPv6 exercise is that well first mongrel 2 is really awesome application so if you don't hack on anything open source I would really invite you to start playing with it and use it in your project. By the way the stats for the network so if you
11:23
looked at the stats for the network at FOSDEM.stdio.b that's my domain they were served from mongrel 2 so I'm not gonna demo it because we were demoing it for the past one and a half days. And really adding v6 in itself is not that much of a big deal but tuning it and making sure that it works
11:44
correctly for all the users that's something that needs some time so really if you have your application that doesn't support v6 then you would really want to start now and start to exchange opinions on how things work better. So with this I would like to thank you and open for questions if any.
12:24
Yes there are thoughts to add the non-blocking DNS but looking so the question was are there thoughts to add the non-blocking DNS. There are thoughts and we've been trying very hard to find the asynchronous DNS resolver library that well to state it politically correct that that would work because the
12:47
existing ones are interfering very heavily with the green threads that are used in the mongrel 2 because they they basically either start threads or they fork a process or they do some other weird things from from just underneath your back. So I have some very very initial code but it's not
13:04
yet the asynch resolver as such. So performance benchmarks the get-other-in-four
13:20
is something that is really hurting quite a lot. So in terms of pure static files I think I don't have the exact data but I think they should be pretty much the same. So protocol wise there is not a whole lot of overhead and code wise the paths as I said they are very very similar so basically we use
13:41
the exact same socket code for v4 and v6 with the only difference that the well this this part is one of the few places where it's different. So otherwise it's exactly the same code.