[Music] alright ok so the last talk before lunch break we've got Dolman and Roman has been busy be working on things that are going to make your life your life easier tooling and infrastructure things especially cash shakes and he's here to talk to us about that today so give him a warm round of applause all right hi everyone can you hear me well alright so yeah it's great to see such a big conference Nick's conf so I'm gonna start with the cheesy picture and a cheesy statement yeah this has been my passion to think about this question over and over again those that were here last year I had a talk how to running running Nick successfully for two years in production and during that time I've learned a lot and I think this is kind of the next step how do we make it scale in a way that you know multiple organizations can use that easily without hiring someone that that is you know in the core team of Nick's to be able to start using mix and and what are the gaps to get there and yeah I think I can personally break it out into two fields one is documentation there is there is a lot to be said on this topic this this talk is not going to be about that I will say that yeah everybody tries to do their best I also contributed a couple of I did a few experiments on the documentation topic but I think there is a second point and I started thinking about which one should come first and once we have the infrastructure I think the incentive to write documentation is going to come by itself more on that a bit later so I've decided to to spend more time on second point and get back to the first one once that's in a good sign

so I believe history what got me to this thinking and and what got us to to Kasich's and Hercules as Robert will present later I think that it's important to capture that so it starts in 2016 with Peter Simons the first person that kind of like put a documentation how to run your Hydra or CI fornix and yeah it's it's still there it's still valid it's there's a few things that change but if you go there you can set up your own Hydra and that was half year later I added this to to mix OS to nix packages Elco wasn't that happy about that because for a very valid reason heater was meant to be used for Nick's packages and it's very well crafted as this big build firm but it's not that well designed to be used in an organizational setting and those that might run one know the problems but I've said okay I'll help maintain this is the best we have so let's go with that right so it's there how many people use sudra all right so about 20 hands 30 maybe so yeah the first comment was now I'd finally have my own Hydra tried once had trouble didn't try again right so progress so in in about yeah few months later we figured out that Elka was right we need to as always right we need to we need to rethink this how do we how do we go forward so with Joe fish that's his nickname I don't I don't know I don't know how to pronounce his surname I'm sorry with Joe we started this project called hair color see I like them round idea was to rewrite basically to keep the SQL schema to keep everything except rewriting all the Pearl and the front end with an proper HTTP API and and I

created back then an elm application with a new front end and we did this is in free time and every time we had some you know free time after work or whatever we came back and we did some work after fixing Hydra we went back to reimplemented Hydra annum and then it

kind of died off for multiple reasons one is that it was a big project to do and second one we kind of needed to reverse-engineer Hydra and and we didn't really anticipate that that's that's not non-trivial so we try to save time on reimplemented because kima and asking some hard questions but we've wasted time on reverse engineering and also it was done in free time again so I started to ask myself okay how do we build something sustainable how do we build something that will you know not die again and what have we failed at this time and so yeah this is yeah this just took about a year or maybe less than a year to come to to a state that to a

state where it still wasn't working back then I was employed at i/o HK and I didn't have any time and then in April 2018 this year I left our HK and I said okay how do we fix this and looking around at different communities I found it the only way to do it is is to make it a product so that development can be sustained and that people can be assured that after some time it's still gonna be there and they can use it and it gets better and better and I don't okay it's just me right so what can I do I know that it's building us yeah at this point is hard and it's a lot of work even though it's like packaging right how hard can it be so this is just a different level so I said okay I drew the whole CI design and I said okay what's the most minimal thing we can do here and I said okay we can we can already have a binary cash as a service right you can register you get a binary Cash when you use it with your own CI your own development whatever you want right that's the idea so my first goal was yeah that setting up a binary cash shouldn't take more than a minute and you're good to go right whatever is you I use you plug it in and and and you start pushing binaries and you know your developers whoever can can go from there and use those binaries now step two is what's happening right now implementing garbage collection implementing different permissions multiple Reed keys multiple right keys and so on and that should be all easy right and I'm step three is it's finally document how to use cash sakes what are the you know what are the common things that people get into trouble when when starting to use necks and so on so we come back to to my to solving the first of the two points that I think are important so a bit of numbers it's running for almost 150 days it has more than 500 users registered it's using one point three terabytes of storage a lot half of users creates the binary cash and so far we had two hours of time time which turned out to be that some Haskell optimization actually in the compiler god yeah the machine down it used all the memory so this is still something I we want to pursue and fix but now at least if that happens again it should recover itself so hopefully that's that's fixed so yeah what I've been

working in the last couple of months is is private binary cash so for for people who don't want to publish their packages publicly and this is gonna be a paid feature random I'm gonna try to show that I have

a live demo and and a slide email so let's see which one works better how am i doing on time few more minutes and then alright so so now you'll be able to pick if you want your binary cash to be public or private the UI is not great this is just you know a way to show that

it works for now so you pick your your cash six subdomain and then you pick what is the team in your organization that will have access read access to two to the binary cash right the right axis is the signing key and I will also add support for multiple signing keys so who wants to have one for the CI and warfare developers or whatever that will be there so let's create an ex-con that was in an 18 and then yeah let's say you want to let's say that we want to give you know one team on github access whatever is in there I'll probably show something like members or something that's it's more clear what but who are you giving access to and then you create the binary cash

you get instructions how to install

annex how to install Kasich's you get instructions how to authenticate from your machine to tutor Kasich's and then you push the story pads one thing I want to make sure here the signing key is actually the secret key that you only get that is generated in the browser it's never sent to Kasich's and if you don't make a backup I don't have a backup for you so you'll have to create a new cache so I'll make sure this is this is more clear in the documentation and yeah that's basically how you push things with a signing key and then if

you go to to the link this is Mexican 2018 that caching start work well this is a development server so it's not actually there I you'll see that it's trusted by this github user that's the public key and then instructions how to use it so you can put you know inside your organization you can give link and again you install Nix Kasich's you authenticate because you need it's a private binary cash then you say cash excuse and that configures your Nix conf and your net RC configuration and you can use the binary cash basically so today my went well yeah if you go to

castings org there is a forum where you can sign up probably this weekend will will send out a private beta access for people to play with if you think this is cool and now Robert will talk about the second part of what we're working together the Hercules GI so yeah meanwhile though sorry before meanwhile he sets up you

can probably ask me a question or two [Applause] questions oh yeah over here so it's actually already relatively easy to set up private binary cashiers are three also so what's the main value add of cash X so the main value is that yes you can do that there are there's gonna be features that are gonna be on top of that Adele will make your life easier so like our bitch collection and you can you will be able to set up with the different single sign ons who has access to that and yeah you can use s3 for that but it has it's very limited on the other hand behind Kasich's there is an application server that which gives you possible which gives me possibility to do a lot of things like searching through through that through the binary cash and it's going to be connected to - Hercules so the bootstrap of of using cash is going to be faster right so it's yeah those are the the benefits in for now what's the cost model for Kasich's I do I didn't publish it yet and I I don't want to say what it's gonna be I'm still thinking about it most probably is gonna be storage oriented but yeah I'm interested too to come up with a way that it's it's feasible for people that it's not too limiting I always don't enjoy when when there is like an arbitrary limit and you just you know can use the service yeah still deciding on this but most probably everything looks like it's going to be storage storage size oriented but if you have some feedback about this I'm interested to hear it's it's not a no solution is ideal unfortunately or at least I couldn't come up with one maybe last question my epic watch is today going to tell me that I moved moved enough when I tried cash checks for the auto-updates to just to review them I found out that I need to add my own user into trusted users to use it although I'm only reading from the cache why is that yeah so it makes the binary cash can conserve you basically as Erica said in his talk right when you build the package the the actual result of the package can be arbitrary right the the the signing doesn't assure you that it doesn't sign the actual contents of a package right so if you are if if people would be able to just download things in your unique store from any user that's basically they could if I give you a you know non secure binding I mean adversary binary right if user guest one downloads can download from any binary cache then they can just put something in and then user you know guest you uses that package and it could be something not expected right so that's the the trust model UNIX wants to make sure that you there's on one machine cannot affect each other and that's why it's done this way you probably might want to have it with your hand it's probably gonna be too far away all right I guess I'll put it there for a demo all right so this is me this is what I look like and yeah I'm kind of somewhat kind of new to the next community so I I'm also on a bit active on stackoverflow by the way so if you asked a question there maybe you've received an answer for me I hope it was a good answer yeah so I'll tell a bit about myself some history yeah my story

started in 2016 when I was I had a neck stress on my on my desktop and yeah this was me deploying an enterprise scholar application on JBoss that didn't work very well but also in 2016 this was me deploying the enterprise Scylla application with Nick's on sense Wes actually yeah so since then I've I started freelancing and doing some some some research into how am I going to prevent these kind of situations improve things and obviously next was come to be part of that yeah so I added some side projects basically and yeah I started experimenting with an ex-gi and I figured out quite soon that building a

CI is not just a side project it really takes a lot of effort to build a good one so that summer I I talked to domin and we decided to work together and we redesigned ci design and this is what we came up with so we have Hercules CI and it's all reads of course it'll fetch sources from get up but they won't actually go to our service instead we'll have we'll provide you with an open-source agent that will fetch the sources build it you sky shakes to fetch dependencies use Kasich's to push them back when it's when it's done and report back to the Hercules CI service for notifications and a dashboard and so on so yeah these are like the distinguishing features for Hercules you will be able to control your own build firm so if you have some exotic things you need it should be easy to do so you can have as many machines as you want and yeah we strive to be the best and easiest CI for open source mix users and and for companies all right

so

here we go this is right this is our MVP and yeah so when you go to the to the application I just have to sign in with github will be

using our paired up integration to configure softlight permissions automatically

so now Hercules knows that I'm there and we can start configuring

so I'll install Hercules now on quickly CI itself we have a mono repo that has the actual code now it'll be set up so

that particular CI can can actually only access that single repository so now we can switch to Berkeley CI accounts and start configuring the agent

and it's just a little snippet of in this case mix-ups configuration we have a mix-ups module that helps to set this

up so all we have to do now is write that to agent up next file like this and deploy the agent all right

groceries showing up it's good so now

all we have to do is is making a change to trigger bills and I guess what I'll do instead is just push had to whatever

and you'll see it starts evaluating and now it's it should be building it's not finished yet the actual building is something that's almost done I mean as a feature so it'll be stuck in this state for a while but yeah we do have

evaluation working and so these are the attributes that are in in our project and you can see there's actually we found a problem by using Hercules it's its value yeah so we have a mix up as a as a development tool we're building it in our in our next file and yeah we linked it in the way that was suggested in the next obstacle mutation but we're not using next path - fine thanks packages that's actually a good measure - to make sure that you're building the right thing to make sure that some developers environment doesn't affect your project so yeah that that broke here turns out that looks like for documentation it depends on a path that's discovered - next path and it shouldn't so something will have to fix yeah so that concludes the demo [Music]

so you've seen this

right so obviously we're not done yet and we have some some further plans to make this a really great see I want to provide some kind of NIC shell functionality with Hydra for example you cannot do side affecting stuff at least not in a proper way and we want to to integrate this into Hercules side so we have a like a principled way to to perform side effecting stuff as part of your deployment pipeline yeah and we we're still looking into ways to integrate with container infrastructure like docker or kubernetes there's a huge design space we haven't really decided how how we want to do this and we would love to talk to you about about this and other ways you you are using NYX so we can provide the best experience yeah there's a private beta you can sign up for it we'll be sending out emails very soon to gather some more information and yeah if you if you sign up now you'll you'll be included and we love to it so thank you all right thank you very much two men and what and we have questions from the audience yes we do do my running thing again so what's your technological stack of which language this is written it and can we deploy it as well and the sub question did you manage to reverse engineer the Hydra cube we're on air so others can reuse it for their own CI attempts for example I micro CI and go right yeah so I stack is basically Haskell for the back-end services using um front-end obviously we're using NYX for packaging and deployments for now we're using NYX ops on AWS but we probably want to change that into something a bit more flexible yeah and so our 18 software is open source at least it will be when we start the private beta we will release the agent software and yeah it's open source so definitely have a look at it are we info are we invoke NYX so it wasn't entirely clear to me what the agents is for okay you said it again all right so that the agent that's the part that you're running on your own infrastructure and it's responsible for invoking Nicks for evaluation for invoking or basically mixed bills and yeah it'll fetch dependencies obviously and it'll upload those depend just built packages to a binary cash okay so the evaluation and building is not done by the Hercules CIE service yeah so at least at least for now like the hard work so to speak is it's all done on customer infrastructure we may implement a a more hosted version of the service in the future but right because that would be a very compelling feature yes yeah yeah so our first approach is to make it really easy to do it on your own infrastructure and probably in the future we will look into hosting the entire infrastructure yes for most people it's not required to to host their own infrastructure right because many applications are running on like commodity hardware so yeah for those people we want if we can make it even easier by hosting their agents we will do sir probably more questions nope doesn't seem so so yeah thank you very much again don't mind all that for your talk and so next up we've got a one-hour break for lunch lunch time enjoy