Centrally managing an ISP with NetBSD and PostgreSQL
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 31 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45253 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
1
3
5
6
7
9
10
11
13
14
15
16
18
19
21
22
26
27
28
29
30
00:00
Internet service providerView (database)Computer programGastropod shellAssembly languageSoftwareSoftware developerSoftware maintenanceCorrelation and dependenceOpen sourceSoftware developerSoftwareProduct (business)Hand fanBitProgramming languageCodeQuicksortComputer programmingCodeNumberPlastikkarteBelegleserString (computer science)NeuroinformatikLetterpress printingNetwork topologyClefXMLComputer animation
03:21
Operations researchIndependence (probability theory)InternetworkingScripting languageInternet service providerFingerprintDatabaseControl flowModul <Datentyp>ConsistencyComponent-based software engineeringOpen sourceComputer hardwareSystem administratorComputer configurationPhysical systemTrailGroup actionOpen sourceComputer programmingOperator (mathematics)Configuration spaceDatabaseSystem administratorComputer hardwareComputer fileProgrammer (hardware)Programming paradigmMainframe computerConnectivity (graph theory)Projective planeSystem programmingScripting languageMultiplication signAreaInternetworking1 (number)CurvatureLatent heatGame controllerComputer animation
07:18
SynchronizationComponent-based software engineeringInternet service providerComputer programmingSystem programmingPhysical systemSoftware maintenanceComputer fileSource codeSynchronizationWeb serviceCodeTrailServer (computing)Multiplication signProgramming languageDirected graphComputer animation
08:29
Software developerPhysical systemCodeFingerprintRight angleNumberSoftware developerPhysical systemHand fanCodeComputer animation
09:20
FingerprintOpen setEnterprise architectureOpen sourceFormal languageLevel (video gaming)Enterprise architectureACIDQuicksortMereologyWindowType theoryOpen source
10:28
Object (grammar)CodeData managementInternet service providerSystem programmingComponent-based software engineeringMereologyComputer programmingBlock (periodic table)Module (mathematics)Type theoryComputer-assisted translationDifferent (Kate Ryan album)Connectivity (graph theory)Physical systemInternet service providerDigital rights management.NET FrameworkSystem programmingComputer animation
11:25
Physical systemWeb serviceGroup actionService (economics)Multiplication signClient (computing)TrailWeb pageDatabase transactionEnterprise architectureDiagramProgram flowchart
12:40
Web serviceType theoryGroup actionDatabase transactionFingerprintSatelliteProduct (business)outputFile formatWebsiteMatching (graph theory)Descriptive statisticsMereologyTouchscreenDatabase transactionWeb serviceEmailoutputNumberSlide ruleTask (computing)VirtualizationReal number.NET FrameworkWeb 2.0Field (computer science)Instance (computer science)System administratorRight angleGroup actionTable (information)Vertex (graph theory)1 (number)Domain nameAddress spaceType theoryData storage deviceFile formatProduct (business)Computer animation
15:36
Similarity (geometry)Computer musicExecution unitTerm (mathematics)FrequencyException handlingComponent-based software engineeringType theoryConnectivity (graph theory)Web 2.0PasswordData storage deviceGroup actionLoginWeb serviceDescriptive statisticsExecution unitType theoryField (computer science)TouchscreenArithmetic meanTerm (mathematics)Connected spaceSystem administratorLine (geometry)CodeDefault (computer science)outputSpacetimeTrailCASE <Informatik>Exception handlingEmailRouter (computing)CuboidServer (computing)Different (Kate Ryan album)Video gamePhysical systemWeb pageProgram flowchart
18:28
Internet service providerProduct (business)Web serviceDefault (computer science)Type theoryDrop (liquid)Musical ensembleFile formatDomain nameNumberCASE <Informatik>Internet service providerFrequencyBitCodeDatabaseHidden Markov modelField (computer science)Inclusion mapGastropod shellMultilaterationArithmetic meanVirtualizationProgram flowchart
20:16
Client (computing)TrailInclusion mapService (economics)InformationGroup actionWeb serviceInternet service providerTable (information)Electronic visual display1 (number)Type theoryDefault (computer science)Web serviceAbsolute valueComputer animationProgram flowchart
21:59
Client (computing)TrailService (economics)Inclusion mapInformationWeb serviceGroup actionRepeating decimalFingerprintWeb pageGUI widgetInternet service providerGroup actionElectronic visual displayDescriptive statisticsInformationWeb pagePoint (geometry)Web serviceSet (mathematics)Instance (computer science)Physical systemCommodore VIC-20VirtualizationXMLProgram flowchartComputer animation
23:27
Local GroupContinuous trackWeb serviceClient (computing)Service (economics)Database transactionModule (mathematics)Internet service providerData managementMathematicsDatabase transactionPhysical systemClient (computing)Multiplication signWeb serviceConfiguration spaceTable (information)Group actionService (economics)Digital rights managementField (computer science)Web pageDatabaseServer (computing)Internet service providerDomain nameModemComputer programmingOnline helpData storage deviceEmailAdditionGenderWeb 2.0Moment (mathematics)Slide ruleRow (database)Computer fileModule (mathematics)CodeInstance (computer science)Radio-frequency identificationVirtualizationEnterprise architecture2 (number)Natural numberComputer animation
30:42
Module (mathematics)Service (economics)Internet service providerData managementPhysical systemObject (grammar)Server (computing)Web pageDifferent (Kate Ryan album)System callEvent horizonPlastikkarteEmailConfiguration spaceAddress spacePoint (geometry)Hand fanVirtual machineSoftwareMultiplication signPhysical systemPhysical lawTrailRow (database)Key (cryptography)DatabaseCASE <Informatik>Server (computing)Observational study1 (number)System administratorObject (grammar)Web serviceWeb 2.0InformationCountingTable (information)Data storage deviceSystem programmingExtension (kinesiology)BootingScripting languageProgrammer (hardware)Form (programming)Hidden Markov modelCuboidMathematicsReal numberEncryptionBell and HowellComputer animation
37:56
Object (grammar)Physical systemServer (computing)Source codeControl flowDatabaseData managementMathematicsMessage passingSystem administratorGame controllerOffice suiteWeb serviceClient (computing)Group actionWeb applicationEmailCASE <Informatik>Web 2.0Open setDatabaseRemote procedure callPhysical systemElectronic mailing list.NET FrameworkServer (computing)Open sourcePasswordProjective planeDigital rights managementComputer animation
39:51
FingerprintComputer networkDew pointInternet service providerRight angleMultiplication signCuboidMathematicsDatabaseFile formatWeb serviceConfiguration spaceSystem programmingSystem administratorData structureHydraulic jumpDifferent (Kate Ryan album)Operator (mathematics)Moving averageGroup actionFitness functionIterationServer (computing)FlagWindowWeb pageSlide ruleWeb 2.0Digital rights managementBit.NET FrameworkOpen source3 (number)State of matterScheduling (computing)Tablet computerDampingSoftwareTable (information)Computer animation
44:06
GUI widgetWeb pageClient (computing)Inclusion mapService (economics)InformationTrailWeb serviceGroup actionInternet service providerExecution unitSlide ruleMusical ensembleClient (computing)Group actionGame controllerWeb serviceFlagMultiplicationScaling (geometry)HierarchyRight angleComputer animationXMLProgram flowchart
45:33
FingerprintInternet service providerDew pointMereologyEndliche ModelltheorieVirtual machineSound effectWeb serviceField (computer science)Server (computing)Web 2.0DataflowSystem administratorDatabaseLoop (music)Existential quantificationConfiguration spaceEmailEvent-driven programmingTable (information)Connectivity (graph theory)DampingMathematicsMessage passingService (economics)Relational databasePhysical systemAlgebraWeb pageFlowchartClient (computing)Musical ensembleMetropolitan area networkFunctional (mathematics)Object (grammar)DiagramProgram flowchart
49:27
Web pageGUI widgetCountingInternet service providerData managementSource codeControl flowDatabaseComputer networkFingerprintPlastikkarteVirtual machineBlock (periodic table)Interface (computing)Computer fileNumberWindowPhysical systemIP address1 (number)System programmingServer (computing)Software maintenanceLocal ringStaff (military).NET FrameworkPoint (geometry)Latent heatSecant methodDevice driverSource codeTable (information)BitComputer animation
Transcript: English(auto-generated)
00:06
Hi. Can everybody hear me OK in the back there? Wonderful. There's me. I'm also a bass player, so I thought I'd throw up a different picture there. I work at Vibe Networks, and I am a NetBSD developer.
00:24
So, sorry? The fifth string is a low B. Yes. I'm not really a big fan of higher strings on the bass.
00:41
I mean, it's a bass. You know, we don't want no treble around here. Come on. It's like when you play a guitar player. Exactly. I also play guitar. You know, when I want to play up there, I'll pick up the last ball. You know, come on. OK, so a little bit about me. My first program was in 1968.
01:01
It was actually in high school, a Fortran 4, coding sheets that went down into the basement. I always imagined there was a door there, and somebody in the hood sort of opened it this much, and took the coding sheets, and magic happened. And they gave you back a deck of cards and the printouts.
01:20
And I didn't do much after that for a while. But eventually, I got back. I got into computers, and I worked with a number of technology companies, Monarch Marketing. I did a lot of work programming barcode scanners and readers and setting up companies to basically use our printers. Because our primary product was printers.
01:43
Not true, actually. The primary product was printer stock. We sold our printers so that we could keep selling stock. I program primarily in C and Python, mostly Python now, but a little bit of C as well. The usual every programming language I ever smelled
02:04
on there. And I'm published by Academic Press in this book here, which is probably out of print by now, Software Solutions in C. I'm an FBSD developer, as I said. I'm a contributor to PostgreSQL.
02:22
I used to contribute a lot of code. I just found out recently that I'm technically still a developer. I still have access to the tree, but I haven't done anything for many years. And anybody use PostgreSQL here? You ever use PostgreSQL here?
04:34
So keeping track of who's actually on the system so we can send them a bill, very difficult. Every time a new feature was added,
04:42
the system just got worse. Also, because it was in the 90s, it was a new thing. And every day somebody said, you know what it should do? And I'm sure you've all been there. We said, no, you can't do that. And then go in the back with the other programmers
05:00
for five minutes and come back and say, OK, we figured out how to do it. Took over internex online. Any ex-internex online people here? This is a bigger group than us. I mean, we had something. They actually had some financial problems. We wound up taking them over and merging them in with us. So it increased our base.
05:21
And they had their own system, many disconnected Perl scripts, flat file database. It worked. It was not bad. But it still required the personal involvement in the day-to-day of the chief programmer
05:42
to keep everything running well. So out of all that, I came up with these requirements. Control everything from one database, as I said. I want billing to be incorporated into the operations, but still be able to handle some ad hoc billing, both in recurring billing, in the billing
06:00
that didn't really fit the billing operations paradigm. And one of. We just sold this guy some software programming. We got to charge him $300. We needed our system. Wanted to be modular.
06:22
So as I grew and found other people to use this, I could see going into some place and they say, yeah, we love it all. And that's kind of an OK GL you have. But we have this GL in our mainframe that we paid like $6 million for, and we don't want to throw it out. I want to be able to merge them.
06:43
Even though it's modular, I wanted all the components to be consistent. So once you worked in one area, it didn't look like you were working on different systems. They were all basically the same system. Robust, of course. Well, every project pretty much has those requirements, all those bottom ones.
07:01
Zero bust, open source components, of course, because I'm an open source guy. Commodity hardware, because I didn't want to be locked into a specific supplier. User, i.e. Administrator friendly. And configurable. So I put together, and this wasn't overnight, obviously,
07:23
but eventually I came up with this. So I'm using NetBSD and package source. Anybody know about that? Package source is similar to ports. I think it's better. Actually, the last time I gave a talk here was on package source and how it's better than ports.
07:42
It's very good for helping me organize all the programs that I have and keeping them up to date and everything else. The same thing can happen on FreeBSD, of course. And in fact, my system used to run on FreeBSD. I actually started out on NetBSD, moved to FreeBSD when someone took it over for a few years, and then I moved it back to NetBSD afterwards.
08:02
Using Python for the main programming language, using PostgreSQL and PyGrescl, which I'm the chief maintainer for. Apache, Asterisk for the phone service. Postfix and Dovecot for SMTP and IMAP. Subversion and rsync to keep track of all the code
08:22
and the necessary files. And to keep systems in sync across all the various servers that are running this. Why NetBSD? Well, first of all, I'm a fan of BSD in general. That's why I'm here. Yay, BSD. I'm a NetBSD developer.
08:41
I say, eat your own breakfast. I like the package system. I think it's a really clean code base. A lot of people say, well, you don't have as many developers as FreeBSD. I don't see that as necessarily a bad thing. I can see that as could be a feature, right size, developer base, I call it. And the developers are really, really anal about correctness.
09:02
I mean, I've had a number of things that was functional that somebody said, no, no. Not until you fix it, until it's perfectly correct. Here's the right way to do it. And I had to back it out and put it back in again the right way. So I like the NetBSD community, and I like the general BSD community
09:21
that it's part of. PostgreSQL, it's open source. Dr. Michael Stonebraker created Ingress way back in the, I guess it was the 70s, 80s. Andrew Yu and Joly Chen took it over when Ingress went to,
09:42
it was picked up by, I can't remember the name of the company now, but it went commercial. Changed the name to Postgres. Actually, it was Postgres before they took it over. They changed the Postgres 95 as sort of a take on Windows 95 for some reason.
10:01
It's got a BSD-type license, so yay, BSD. And it's an enterprise level. It's fully ACID compliant and just very strong. The name eventually changed to PostgreSQL because the original language was not SQL. It was, Dr. Stonebraker invented his own language,
10:22
but due to the popularity of SQL, they just changed the language on top of it. The engine stayed the same, very powerful. Python, I like that. It's easy to learn. It's interpreted, but it's very fast. It's comprehensive, got lots of packages and modules that you can either come with it or you can download.
10:41
Object-oriented, clean style, executable pseudocode. I mean, it's what, you know, you write something out, you know, say, well, listen, the program should do something like this. And you write it out and you indent the stuff that's part of a block and, you know, and when you're done, it almost looks like Python. And again, BSD-type license, yay, BSD.
11:04
System components, a general ledger, accounts receivable. My wife thinks it should be a cat, by the way, but that's what I found on the net. Accounts payable, billing, ISP management. And most of the top stuff, you know, pretty standard stuff. The billing is a little bit different
11:21
than the ISP management. Of course, it is completely custom. This is a basic idea of how the system is set up. I just have to remember to keep track of the time. I never got a chance to time my talk. So as I get close to the end,
11:41
I'm either going to really talk really, really fast, which means that it's longer than it should have been, or I'm going to be begging for questions, which means it's shorter than it should have been. So we have this, you know, clients have transactions which go onto invoices.
12:00
They have service groups, which is the stuff that they buy. They have recurring billing. Service groups are based on service definitions. Service group has a billing ID, which is basically the group in UNIX, and account, which is basically the user account in UNIX.
12:23
That's where the mailboxes are and other web pages and other services that we offer. But we can have an enterprise, could have one service group and then all their employees could be users and they manage that separately because they have access to that one there.
12:43
Billing, as I said, we have service definitions and pricing. By the way, there's more detail on this in the paper, which I'm going to make available on the site. Some of you, if you're academics, you may not think it's a real paper because there's no footnotes in it, because I hate footnotes. I like to read, you know, if it's important enough to say,
13:01
put it in the damn text, you know, don't make it look like a vertical tennis match. Service definitions and pricing are defined in one place. Account types, talk about the various things you can buy, you know, virtual domain, basic web, yearly, monthly,
13:25
web with extra storage, you know, whatever you can describe in the tables goes in there. The service groups and accounts, as I explained, basically GID and UID, in fact,
13:43
that's the actual table field names for them. Service instances, you know, somebody buys a phone, they get a service instance of a phone and then they have, you know, it has, part of the definition is how much we charge for extra minutes and how many minutes are included
14:01
and if it's, you know, Canada or North America or the world, you know, whatever, has transactions which go on to invoices, as we saw in the previous slide right there. Service definitions have the description of the product, the base price and the extra charges,
14:22
so the base price may say, for instance, on a virtual, on a basic web, we may say you get, it includes one email address, mailbox, and if you want more, they're $3 a month more or whatever.
14:40
You know, some of the higher priced ones might say, you know, you get 30 mailboxes and you pay for extra mailboxes above the 30. Input formats, so that when you're defining it, again, we don't know what people are going to sell tomorrow because we deal with salesmen. We're usually selling our product before we've created it.
15:00
So, we want to be able to describe the product in such a way that the input screen is also defined. So, like for a phone service, you need the phone number and for, you know, some other stuff, we have a net task or product, we need to know which instance it is,
15:22
you know, so we allow them to put in whatever fields they want in the description and when the administrator goes to add that, it simply comes up with that definition. I think I have some examples going forward. Yeah, here's the service, added service.
15:40
So, this is a residential 50 megabit ADSL service. ADSL residential light speed 50 megabyte is the description that goes on the invoice. It's a group service. We apply tax, well, we actually have a different way of doing that, so that field's going to go away.
16:01
It can be edited by the group leader and we can have multiple. So, you know, somebody wants 250 megabit service on the same service, no problem, we can do that. The base unit is each, could be megabits, could be 100 megabits, could be, you know, megabytes, storage, gigabytes, whatever.
16:21
We just put there what the actual unit is, again, for the description on the invoices and on the administrator screen. Which GL account goes in, because we have a full accounting system, so when we charge something, this service, we charge it to connectivity revenue. And commission code means if we're giving commissions
16:43
to someone, they get the connectivity one, which might be less because we don't make as much money so we may not give as high a commission on that as we do on something like, you know, web hosting, which, you know, not free, but certainly not, we don't have, you know, the next customer
17:00
doesn't cost us the whole amount again. And this is where we define, you know, here we need a login and password, so we can keep track of their login and password in case they call us up and say, I can't connect, and say, well, I'll go into your router, here's your login, here's your password. And the idea of the blank space in between means that it goes on a different line.
17:20
If there was no blank space, that the input screen would have login password. With the space there, it's just login password. So we have some control over that. We can put, you know, so we can make the screen look more or less like we want.
17:42
Account types, we define the package, talk about the term and the billing, most of the terms are one or 12, you know, monthly or yearly, it allows for others, we never use them. Exceptions of default pricing. So we have all the default pricing for the storage,
18:01
the mailboxes, you know, how much is included, everything, but we can say for this particular one, instead of including the default of, say, five gigabytes of storage, we can say this one includes 15 gigabytes of storage. Obviously the price would be higher.
18:20
And the basic components that are included, do they have a webpage, do they have email, do they have check, check, check, just check boxes, which is, for example, here's account type 101, which is our virtual domain SOHO monthly, small office, home office, for those who don't know what SOHO means.
18:42
It's commission code, in this case it's ISP services. There's our base price, we don't charge anything for setup. If we did, that would go under the setup GL account, but there's the ISP services GL account for when we sell this product.
19:01
Period is monthly. As I say, we used to have a feel there, we just put in the number of months of the term, but since it was only ever one or 12 months, I just changed the dropdown to be monthly or yearly, and it puts a one or a 12 into the database.
19:24
Included, I don't know what's included. I've never really noticed that before. I put it in there, but I don't remember. And what shell they get by default, the bash as a default, we can put anything we want in there. And I'll get to why that's important a little bit later.
19:44
These are all the default amounts for the various things. So if they want the 10 megabit service, it's $43. If we had a special price for this particular account, this would be in a different format
20:01
and we would be able to put a number in there. Usually this number is the same in this case because it's $36 for one, while the second one is also $36. Am I going too fast, too slow? Anybody have any questions as a goal? Just jump in there,
20:21
because I think I'm going to be really short. So I'll be begging for questions. I'll ask a question now then. Okay, this UI? That one will add.
20:48
Yes, absolutely. Yes, we have many tables and this display comes from reading the account type table for the basic stuff
21:02
and then throwing in the service definitions, throwing in the specific, the ones that are special, that are not the default ones here. So I should have probably tried to find something that had a different one there, but it's further down.
21:25
Yes. Well, sure, I'm sure there's some stuff there. I'm trying to think offhand what they might be. I mean, everything that you need to administer the account
21:43
is certainly on here. I mean, there's things like last update and stuff like that, which don't show up here because they're used internally, but anything that you're going to need to create it is definitely going to be here.
22:04
Did I skip something? Accounts. I was looking for a picture of some user using account. I came across this one. I'll be cool. Some girl using her Commodore 64. I have no idea what year that is.
22:20
I'm sure that girl's probably older than me. I don't know how long ago this was. Maybe not, but certainly around my age. She's certainly no kid anymore. So accounts have individual mailbox. So each account is a mailbox. Each account is a webpage, which may be the virtual webpage if they're the main account on that service group.
22:45
It has the user feature. So the user can go in and set various things. For instance, they can pick a feature that says display my information on who's on our system page.
23:02
We don't put them there automatically. If you want to be on there, you go here. Here's the description that I want. And if you go on one of our pages, it says, who's using our system? All these people, which we check for rude stuff every once in a while.
23:20
And it holds and controls all the resources, which we count at some point. Have I already been on this slide? Service instance. Oh, no, service instance. So you can attach a service instance to a group.
23:43
So again, a phone, a modem, a DSL service, whatever. You attach it, you have things like details, like when is, it tracks when is it due,
24:01
how much is used, and then all of that's used at the end of the month. We suck that out, put it onto a transaction. The transaction becomes an invoice. Thanks to Carol every morning, who goes through and invoices everything, pretty much pressing a button. But so far, we're small enough that we like to eyeball it
24:23
before we send them out. So the transactions are created based on the accounts and the services. So we look and see how much storage that they use. Did they go over there, allocate the storage? We charge for that, we make a transaction.
24:41
Did they, obviously, for the basic service, you know, whatever it is for the monthly, they get a charge for that. They got additional services, DSL, we add that on. And then they get one invoice with all the details of what they use. And then it marks it as, you know, due the following month.
25:05
Seems pretty basic. I'm going to get into the cool stuff in a moment. So I'm expecting to slow down in a moment when I get to the, in a couple of slides. The services, the transactions, of course, and the invoices are attached to clients.
25:22
So we know who's going to be billing them. And they're added to the AR import table. Remember before I said everything is modular, we don't just take those invoices and plug them into the respective tables in the accounts receivable. We have one table that has all of the import records
25:41
for the accounts receivable. If we use our own accounts receivable, then that module goes and reads that table and plugs it into the proper tables in its own system. If they have their own accounts receivable system, then we just write some code that goes and reads our database
26:01
and exports it to their system with their help, obviously. What's that? Well, not the customer's AR. I'm suggesting if another enterprise
26:20
wanted to run an ISP using our system and they, so they took our ISP module and said, yeah, the billing module makes sense. They may want to use their own AR. They may want to use their own GL. I mean, obviously if they use their own AR and they're also using their own GL, but they may want to have it managed. Actually, I guess they could actually use their own AR and use our GL, but chances are if they're that big,
26:43
they're probably using their own GL too. Sorry? Oh, sorry, yes, yes. AR is accounts receivable, AP is accounts payable, GL is general ledger.
27:01
Sorry, I'm not an accounting group out here. I just had to learn all these things. They're second nature to me now because I had to program them all. So the ISP management, now here's where we fix a lot of the problems I talked about at the beginning.
27:20
So we want to have webpages. So who gets a webpage? Well, people who are paying for them. Actually, not necessarily. We may have complementary accounts, but they're in our system and we're tracking them and we know that we have complementary accounts. We can take a look at all the complementary accounts and see if we're giving too much away, but if they're not in our system,
27:41
they don't have a webpage. And that's because the web server, which is a completely separate server, goes into the database and says, has anything changed in the tables that I care about? As I say, one of the fields I was talking about in the database tables has is the last updated.
28:00
So automatically when you update a table, a field changes and we have a way of looking to see has something changed since the last time I generated the webpage configuration? Yes, suck down everything I need, create the Apache config, whatever other files have to be created,
28:22
restart the Apache web server, and now everybody who has paid for the service is now going to webpage. We have in the database, we have the ability to tell which is the main account and we can alternate and say, but we don't want the main account to be the virtual webpage
28:42
so the virtual account, the domain attached to the service group, not the account, can be serviced by any one of the accounts underneath it. We default to the first one and they can adjust that to whichever one they want, including on some server other than ours.
29:01
We can just point it to some other server if they want. It's pull technology, as I said. The mailboxes, we create basically user accounts on the mail server based on, well, everybody has a mailbox, so if they have an account in the system and it's active, then they get a mailbox.
29:21
If we turn off the account, the mailbox goes away. If we turn off the service group, all the mailboxes under that service group go away. If we turn off the client, all the service groups and all the, as you go down. So nobody gets something unless they're paying for it, or at least we know that they're not paying for it. Question, Jim. Yep. Do you have any kind of deletion
29:45
with Cascade, like you just described? You can back it up. Deletion, oh, no, yes, yes. We don't actually delete anything. We never delete. I'm a database person, we never delete anything. We just go, you're not active anymore, click.
30:03
So somebody's not paying their bill, we go to the client, go click, and everything disappears because it's all pull technology. You know, the web server doesn't care about what's going on in the database. It just says, I don't have an active account, so I'm not gonna create a webpage for this person. The guy comes running and says, I'm sorry, I'm sorry,
30:22
here's my check, here's my check, here's my mail. Okay, great, click it on. All the service groups, all the accounts are active again. They were always there, they just, we just wouldn't let them trickle down to the servers. Yes, things happen, we make transactions. We don't change, very seldom change a database table.
30:44
That's some basic stuff, yeah, like name, address, you know, we'll do that. And even then, we keep track of events. You know, we change this guy's, so-and-so changes this guy's address from this to this. So that's in our A-Trail, we call it our events. It's basically an A-Trail.
31:01
Configures are stored locally. So the web server, the mail server, the voice system, you know, like the phone system, they all pull their information, they create the configs and everything is running fine. And suddenly the administration machine goes down. Maybe we took it down to do an upgrade or whatever.
31:22
Doesn't matter. The servers say, oh, time to check to see if anything needs to be upgraded. Can't get to it, oh well, I'll just keep what I have. And it just keeps what it has in the configs until the admin machine comes back up and says, oh, okay, let's see if there's anything to do now. And it gets it updated. So everything is independent.
31:41
If the mail server goes down, the web pages still stay up. Nothing is dependent. No one machine takes down the whole system. Individual servers do the counting. So for example, the phone service.
32:01
Minutes are counted. Every time somebody makes a call, we track the minutes, it goes into a database table, CDR records, and at some point we go through the unchecked ones and put a price on them. So we say, oh, we charge one and a half cents for Canada.
32:22
We do all that. We don't worry about what included minutes at that point. We just put each record how much it would cost if they were paying for that particular call. Monthly, we count all those, but we say, oh, but that account gets 1,000 free minutes. So don't even worry about the first 1,000 that are included.
32:44
You know, like if they make a call to Europe, no, that doesn't count in 1,000. They skip that one, and then at the end, they charge for the European one and any overages they have, and that all goes in. Storage, we just have little scripts to go daily
33:02
and check and see how much storage people are using in their database, in their email, their webpage, whatever we're charging storage for. And sometimes you have to check them manually or semi-manually. We cannot get our DSL distributor to give us something
33:23
in an electronic form that we can just apply and look at our database and see if it's right. So we get the invoice once a month. You know, it's got all the different accounts and poor Carol here has to sit down and go through the database, you know, the records in the database and make sure
33:40
that we're charging for everything that's on there. So some of it is kind of manual. There ought to be a law. You know, I'm just not a fan of there ought to be a law automatically.
34:01
You know, so yeah, I mean, I'm more of a fan of there should not be a law, which would solve a lot of the problems that we're seeing, right? We don't need a law for everything. And yeah, we definitely don't need a law for that. Well, we'll deal with it. I mean, the marketplace will take care of it. If they lose business because they can't, you know, if we get too big
34:21
and Carol just can't handle it anymore, we just say, listen, we've got to go find another supplier because you can't give us what we want. And they'll either say, hold on, you know, get their programmers busy. And next thing you know, they're giving us what we need. Or we go someplace where, you know, they can handle it, the free market.
34:41
Well, I mean, yeah, I mean, there is competition. I mean, we're not with Bell. We're with a smaller supplier who uses Bell, obviously. But we, you know, we could go to like tech savvy or, you know, one of the others. It's not a, hmm, me. Oh, okay.
35:00
Me card, you never know. Okay. Here's a case study. This is a big phone service. This is the actual phone. Well, one of the actual phones that we use. Very nice, actually. It's self provision. So the reason we use this, I mean,
35:20
we looked at some of the other big name ones and none of them really self provisioned very well, unless you're big enough to buy, you know, tons of phones and have them chips made specifically for you. For you're buying one or two here and there, you know, you want to buy something off the shelf.
35:41
And what happens is we buy these, the MAC address is sent to the manufacturer who puts it into their database that we have bought this one. And when you plug it in, it calls home, finds out that, oh, Vibe Networks is the one
36:00
that's going to provision this. So it sends them a configuration that says you get your configuration from Vibe Networks. Nothing says fine, hangs up and then reconnects, reboots actually, comes up and says, oh, I have a configuration from here. Oh, it's different than the one I have, obviously. So it installs and what we do is we send
36:21
a short one that has a really long key in it that is used to encrypt any configs going to that particular phone. And they say, fine, they reboot, they come back up. Now the second time, because we know we've already sent them that key, we send them the real configuration encrypted.
36:42
They use the key to decrypt it and so now we've got a secure phone and you know, by the time it's done this dance and it's all, they just plug it in. They don't have to go in and set anything up, do anything, no, they just plug it in and it's ready within a few minutes, maybe 15 minutes, it's rebooted a few times and comes up and hello, they have a phone.
37:04
So we added objects to the system of a phone. They described the details of the service, a thousand minutes, North America, Canada, on that, whatever. The Astra server pulls the phone details out of the database, creates the Apache configs,
37:20
the extensions, the voicemail box information, you know, all the basic stuff that we've heard, you know, some headers and footers for everything. And the phones are out of provision as I described and then we use the CDR records. They just go into the PostgreSQL database
37:41
and just one record every time a call is made and then other systems, we don't try to do too much at the time of the call, we go in later on and run through it and see what has to be maintained. So the summary, based on open source,
38:03
very important to us. One control panel, so the admins go in, they can bring up clients and we got, you know, there's one control panel, handles everything in the system. They don't have to go in and configure a user on the mail server, the web server, phone, nothing like that. If it's in the database, it's in the administrative system and everything runs from there.
38:23
One control panel for users, so they can go in and change their password, add themselves to, you know, various features. We actually have two mailing lists for our users and we require that the primary user
38:40
on every service group has to be in at least one of them. So we put them in what we call our MOTD one, which is supposed to be a message of the day, it winds up being, you know, every few weeks, if we're lucky. And we just send out the message to that and they read it and they say, fine. And then we have another one which we call Vibrant,
39:01
which gets exactly the same messages, but it's an open mailing list. So they can actually discuss us on our mailing list. So we kind of have to be a little confident before we do that and not a lot of places will do that. But some people don't just get, don't want to be involved in the discussions. It's too technical for them. So, you know, they don't have to be on anyone,
39:23
any mailing list, unless they're the primary user, in which case we want to be able to say, hey, there's a change coming. Somebody in your office needs to know about it. Everything's tracked in the database. Easy remote management, because it's all web based. So I could bring it up here and, you know, I could disable David Maxwell, you know,
39:43
because I don't like the way he's looking at me right now. And that's it. It's a NetBSD project, obviously, as I talked about. And I'm going to be like right on time, if we didn't start late.
40:08
And there I am with my partner. We actually were VEXnet, which was myself. He had an operation called Vibe Networks, which was,
40:22
he's more of a marketing person. And also, he brings companies public, hopefully. So it was a good fit. So we got together and merged our stuff into it. I wound up being the majority partner, so I got to have my say in how things are run. And, you know, so we didn't push everything over
40:42
onto a Windows box. And there's our details. And that's it. So I need lots of questions, because we're running way ahead of schedule. Config management?
41:11
Yeah, but I mean, it's very structured. Every time I've looked at a tool that doesn't,
41:21
like, you know, like everybody says, use cPanel, it does everything. Well, it doesn't. I mean, I've looked at it. OK. A chat will probably alleviate a system administrator from having to do repeatable things or iterative
41:41
things. Right, but we don't need to worry about that, because we're not managing these pages individually. We're not going into the web server and setting up. I know, and if we create things that exist.
42:08
We've never found anything that does exactly what we want, as flexibly as we want. I know at open source, you can go in and change it yourself. But it's very iterative, or recursive.
42:33
I mean, you know, this is something I didn't build this overnight either. You know, it started off in the 90s, and it's progressed.
42:42
You know, and, you know. And if I had a clean slate, I mean, yes. If I was starting with a clean slate, I would set up the structures maybe a little bit differently. Not too much. I mean, because a lot of the ways that I did stuff,
43:02
because it's all in a MAT database that I control, and I know the format and the structures of it, when I've had to make major changes to the underlying structure of the database, there's always been a way of migrating from one to the other. Sometimes a little more painful than others,
43:20
but it's always been possible. Again, because it's like my stuff, right? I know exactly how to make that jump. I guess if I was starting from scratch, there's a few things I would do. One of the things is the features I talked about.
43:41
I want to put that into the service table. Again, it's something I can do over time. It's not that hard. Just, you know, it'd be nice if I had just started off that way to begin with, you know, instead of having two different systems for tracking adjustments to things, have one place, and just have a flag that says this is a group thing
44:01
or this is a user thing. In fact, I even have that if you look at that slide. I'm going to find that slide. Yeah. See, a group edit flag means that the group leader
44:26
can edit it.
44:42
Full scale or a result. So you don't have the same hierarchical relationship. It becomes very difficult very quickly to manage. Yeah, by group edit, I mean like the group that has maybe, you know, their employees are the accounts underneath it. That's what I mean by a group.
45:00
Right, right. So we have this concept of a client. We also have a client above that. So the client can have multiple service groups. So the client can have that control too. Yeah, yeah.
45:25
Yeah, you do struggle with some of those relationships. One of the things is, if you saw the, this is a very small, isolated part of our ER chart.
45:41
Everybody know what an ER chart is? For those who don't know, entity relationship, this is not a flow chart. This is a chart detailing the relationships between data objects. And that usually becomes the database.
46:04
And so, you know, we worked out the full chart. You know, every relationship, how it fits, we can see, oh, this doesn't really work there, you know, because we've got a loop, a data loop. You know, you can have a data loop as well as a flow loop. And we deal with all that stuff in the ER chart.
46:23
And then making the database becomes a simpler. Yeah, well, I mean, that's a relational database. That's what you do. Actually, relational databases doesn't mean that tables relate to each other. It actually refers to relational algebra.
46:48
I was wondering, so you just did a lot of thinking and actually brought up, do you look at event-driven models for making modifications to each one of the individual components for making this relationship?
47:03
Well, event-driven in a way. The database is managed as just a big control panel that you make changes to. There are various things that run on various servers that count things, check things,
47:22
make sure that something is not set up wrong. But if you make a change in the database through the admin page, automatically that updates the last update field in whatever table you've changed underneath. You may not even know what it is.
47:41
The system knows, oh, you're changing the client table. You're changing the service table. Whatever table you're changing, it updates automatically. The database handles that. There's not even any programming for that. The database updates that last update field. And then the various services that
48:00
need to create configs just use, we have a subroutine that says, I'm interested in this table, this table, this table, this table, and I am this function. Because the mail server is going to be interested in some of the same tables as the web server and the phone server. So you say, phone, interested in these tables.
48:24
And the subroutine just goes and says, nothing's changed. Oh, something's changed. Now you have to go and read. And then it just rebuilds everything. We don't try to figure out, well, OK, we have to update this table, but leave that one alone. We just build the whole thing from scratch. So it's kind of event-driven that way.
48:44
Exactly. Well, that's what we've done. Well, we don't do the message passing. Yes, yeah. Well, you don't unless you are on various independent machines.
49:09
As long as you can run it from the admin machine and, you know, I guess if the admin machine goes down, same effect.
49:20
OK. Any other questions? Yes. Sorry. Sorry?
49:44
No, I think it's pretty complete. I mean, you know, I run it on NetBSD. I don't know if there's that much that's really NetBSD-specific. I just happen to like NetBSD. As I say, it ran on FreeBSD before.
50:00
It could probably run on Linux. I hope it can't run on Windows. Yeah, I mean, as systems improve, you know, we can use more. I can't think of anything offhand that we would do differently if we had another feature.
50:21
Are you thinking of anything in particular? Yeah, yeah, I mean, obviously we care about drivers, but they're all, you know, if it's an ethernet card and it works, you know, not sure what else we need.
50:42
It'd be nice if every server we had had the same ethernet card. It would make things a little bit simpler, but we deal with doing that because we also distribute our, like, rc.conf is all, you know, mostly in one file with another file that
51:01
includes that that says, yeah, my IP address is this and my name is this. Now go and get the usual stuff. And then, oh, on this machine, I also want to start, you know, dovcot or the phone server, asterisk or whatever that machine is doing. Anything else?
51:27
We, I tried to do that. The problem is I didn't want, I wanted to have, I wanted to define the interface first before calling anything, even rc.conf, which
51:43
means I'd have to put something in rc.conf anyway. So basically what I have now is rc.conf is my machine name, my IP. Actually, my IP address is just one number because I know what the, because I have two interfaces and I know what the net block is for those interfaces.
52:05
And so the same last octet for both of them, one's an internal one. Then I say, you know, basically source, you know, my rc, you know, my system rc.conf. And at that point, because it's,
52:22
and that's all in subversion. So it's maintained. And then, so once I'm there, it's my file anyway. It's not like I'm using the system rc.conf. So I just add whatever I need to that one. I don't really need another rc.local.
52:43
Anything else? Well, thank you very much. And if anybody has any questions over, and if anybody has any gigs for me, you know, I'm always ready to play. Colosseum, you know, hey, okay, where are you?
53:13
Sorry, okay, it used to be at 151.
53:20
50 Richmond, I don't think so. Maybe you're thinking of somebody else? Are you thinking of IO? Yeah, 50 Richmond, yeah, okay.