Verisk Analytics Keynote
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 45 | |
Author | ||
License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
Identifiers | 10.5446/34596 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | |
Genre |
ChefConf 201712 / 45
1
4
5
9
12
16
18
20
21
22
24
25
27
28
35
37
39
42
00:00
Video gameBitOperator (mathematics)Lecture/Conference
00:25
Point cloudLecture/Conference
00:41
Data managementComputer-generated imageryPoint cloudGUI widgetInformation securityCore dumpEnterprise architectureService (economics)Point cloudInformation securityDivision (mathematics)Binary multiplierSocial classAreaExecution unitIntegrated development environmentComputer virusCartesian coordinate systemTerm (mathematics)TrailDialectNumberNeighbourhood (graph theory)Human migrationLengthPlastikkarteBitLecture/Conference
01:54
ArchitecturePoint cloudLambda calculusFile formatLastteilungVirtual machineSpacetimeCartesian coordinate systemParameter (computer programming)Goodness of fitArithmetic progressionWindowServer (computing)Revision controlMiddlewareMedical imagingShared memoryAxiom of choiceChemical equationGroup actionSubsetStructural loadRelational databaseCodeAddressing modePower (physics)Line (geometry)Endliche ModelltheorieIntegrated development environmentFigurate numberWorkloadMereologyFunctional (mathematics)Operating systemOrder (biology)Software frameworkGrass (card game)Cellular automatonDatabaseRepresentation (politics)Set (mathematics)Type theoryAnalytic continuationConnectivity (graph theory)Bound statePoint (geometry)Bootstrap aggregatingDisk read-and-write headData managementInstance (computer science)Point cloudDifferent (Kate Ryan album)Data storage deviceExecution unitSystem callDecision theoryService (economics)Client (computing)MetadataBuildingProduct (business)Enterprise architectureWeb 2.0Computer virusElectronic mailing listFunction (mathematics)Process (computing)DialectInformationOperator (mathematics)Plug-in (computing)Computer fileWordComputing platformLogicMobile appSource codePlanningConfiguration spaceMultitier architectureEncryptionInformation securityIP addressKey (cryptography)Wrapper (data mining)AutomationLecture/Conference
Transcript: English(auto-generated)
00:05
I'd like to introduce one of my colleagues, Mike Ryan. He's on my team. He spent the last year of his life really driving our centers of excellence around both DevOps and automation, and he's going to talk to you a little bit more about all the details that go into the HOV lane and everything that he's worked on.
00:22
Mike? Thank you, Eric. As he said, we're moving very aggressively to the cloud.
00:44
I joined the team last year because I knew Verisk was very interested in that migration, but I wasn't prepared for the breadth that Eric touched on in terms of the number of divisions we have within Verisk and the speed at which they were looking to move to the
01:02
cloud. To give you an understanding of the scope of what we're dealing with, by the end of the year, we're on track to have somewhere in the neighborhood of 60 to 80 accounts in Amazon across our business units. Each one of those has multiple virtual private clouds operating in multiple regions.
01:22
If you put those multipliers together, you very quickly realize we have a lot of environments we have to manage. How do we get there? Our business users are smart people. They're smart about cloud. They're smart about technology. What we had to do was leverage that smart, leverage their ability to develop applications,
01:49
and yet not compromise our security as we move things to the cloud. I'm going to walk a little bit through the details of how we achieve this with HOV.
02:01
Overall, essentially, it's a framework for automating everything. We decompose that into the orchestration piece, which is defining when you click a button and say, I want to deploy my workload or deploy my application, how many virtual machines, how many load balancers, are you building a database, are there other Amazon services
02:23
that you want to deploy? Those all get orchestrated and deployed as part of that one click. We very quickly realized as we started modeling our applications that the app models needed to be arbitrarily deep, arbitrarily complex, end-tier applications that we wanted to move
02:42
out to the cloud. As we did a few of them, we realized that even though they were very diverse workloads against very diverse datasets, there was a lot of commonality. We've arrived at about a dozen of these foundation components that I and my team manage
03:03
on behalf of the enterprise. The key thing to understand is that the app model, the business users model their own application, and they're able to deploy their applications. It's a logical representation of what their application is.
03:20
They're able to specify things that are sensible to business technologists, like, I want to build load-balanced web servers, and I want them to be Windows 2016, and I want three of them, I want them to have a couple of CPUs, eight gigs of RAM, and a hundred gigs of storage. They then make a call to the CloudFormation template that is the component tier that builds
03:45
these load-balanced servers. They pass all of that logical information into that component tier where it's physicalized. We've got Lambda functions, which for those of you who don't know, that's Amazon's
04:03
serverless platform, and we're able to take those inputs and be able to resolve for this business unit, let's say they're deploying a UAT environment in the U.S. East One region, we do a look-up and figure out where do we want to place that workload for this application.
04:24
Which VPC does that go into, and furthermore, what are the subnets that are configured to handle this workload? Similarly, when they say we want a Windows 2016 server, they don't have to worry about which machine image, because we maintain through our automation a continuous set of
04:45
gold images that are hardened, encrypted, and up to date. If they're using a relational database, if they want to model Postgres, they just say I want a Postgres server. They don't have to make decisions about is it going to be public or private, or is
05:05
it going to be encrypted. We make those decisions for them. They're private. They're encrypted. They don't have a choice. When they want to have their load balancer publicly facing, we don't give them a choice. The VMs that we build only have private IPs on them.
05:22
The load balancer has a public IP, but will only accept traffic from our WAF, and those VMs in the autoscaling group will only accept traffic from the load balancer we built for them. The application team doesn't have to worry about any of that stuff. They just say I want load balance servers, and I want to push them out. At this point, we've got our orchestrated environment.
05:45
We've deployed a set of machines and other components. That's good. What's next? What's next is what happens inside the machine, and that's why we're all here. We're using Chef. In particular, we're using OpsWorks for Chef Automate.
06:02
We have half a dozen instances up and running now for different business units. Anybody else using OWCA? Anyone in the crowd? Nobody? Anybody like the name OWCA? It is one of the most awesome services Amazon has.
06:21
We have found a great deal of value in using this, because it's very easy to hand to our team in India and say guys, build another one, and they're able to package everything together and make it delivered to the business for a set of people who had no experience with Chef at all. This was really powerful for us. It made it very easy for a business unit to start adopting Chef.
06:46
It's been really helpful, but the name, we've got so much brain power in the room, so many awesome Cheffers, I think we can crowd source a better name. Don't we agree? We have modeled and we have built probably about a dozen applications through OWCA,
07:08
and we've been able to reuse cookbooks across the different business units. We're in the process of standing up our own private supermarket to further foster that adoption.
07:21
For some application teams whose applications are more stable, they do quarterly releases of that, they're able to not only model and deploy the middleware through Chef, but the application code as well. We have applications that use Tomcat and also will push out the application code,
07:43
the war file through Chef, but for those that are using a more DevOps model that have more agile deployments in their plans. We've integrated with AWS CodeDeploy for being able to push that out.
08:02
Now, how do we get the config management piece and the CI CD piece, how do we get those modeled in the HOV lane? It goes back to the app modeling piece and that logical reference. So, an application team says, I want to use Chef, they pass it in as a parameter,
08:20
and I want to put my IAS version one, or type one, on the machine. And so, the components tier that we manage for the business, we use that to call another Lambda function to figure out what is the OWCA instance for this particular business unit
08:40
and create the client config on the VM on the fly. And we also get the bootstrap PEM file so that when that node registers itself with the Chef server, it can authenticate securely. So, we're able to be very confident that the machines joining our Chef server are supposed to be there.
09:03
We then create a run list dynamically on the machines. We put down a baseline for the operating system, enabling the InfoSec and operations teams to be able to put whatever agents they want, antivirus, there's a couple of others.
09:21
That way, they can manage this across the enterprise and push out that baseline across all of our OWCA instances and very easily ensure that everybody is at the same baseline. And that personality, which gets passed in from the app model, that gets put in to the run list as well.
09:41
And that is the wrapper cookbook that's managed by the application team. And if they want to use CI and CD, the application team just puts a marker in their cloud formation saying, I want to use CodeDeploy. And then all of the metadata needed to set up the S3 buckets, to set up the groups,
10:05
to set up the roles that are needed to enable it, and to have Chef push out the agent, the CodeDeploy agent, onto the VM as well. That just happens for the business. They don't have to think about it. They don't have to worry about it. All they know is when their build finishes, the output of that cloud formation
10:24
gives them all the metadata they need. So that if they're using Jenkins and they've got the plugin for Jenkins, they can take that metadata from the output, create a Jenkins job that will just automatically push code out to the cloud environment.
10:41
So for those in the audience mathematically inclined, you can figure that we have approaching 100 accounts and a half a dozen OWCA instances. So we've done a lot in this space. I think we've made very good progress in modeling our applications and coming up
11:01
with the framework, but there's a lot of work still to be done. So like others, we're hiring. So if what Eric and I talked about is interesting to you, if it's something you think you might like to be a part of, we're around and we'd love to talk to you. Thank you.