Hello, good morning. How's everyone doing this morning any hangovers out there? All right, I've got quite a full session, so I'm going to take the extra couple of two minutes

To try and make sure I can fit everything in So my name is Adam Murray. I'm from Sydney in Australia. I've recently started my own consulting company called Ticaboo and Today, I'm going to present to you about managing your git

And using VSTS for doing PowerShell module deployments So who's using git in the day-to-day development at the moment? That's good. That's very good anyone. That's not definitely needs to be there. It's amazing once you start using that tooling How useful it can be?

Who's using VSTS today? That's pretty good as well package management within VSTS Two not too bad Who's got binary components so DLLs or X's in the git modules today? Not too many okay, so the build process I'm going to present today

You we handle binary pit bits We actually put them in a separate package so that they're not clogging up your git repository So we had a few scenarios where we wanted to have selenium for example as part of one of our modules We wanted to n log in another module

The demonstration today, I'm just using a very small library that does some network calculations We once you want to put those things into your module you want to keep them separate from where your PowerShell code is I? Think Lee Holmes mentioned yesterday that of all the code that he's written all the PowerShell code equates to about 5 meg

and Once you start chucking binaries into your git repository That'll blow up very quickly start slowing down your git clone start slowing down your build processes so What we're going to cover so we'll we'll start from scratch we'll create a VSTS project we'll set up VSTS package management

I'll quickly walk through using nougat to push a package up into package management I use plaster to scaffold out a PowerShell module With all the basic bits that I need so that I can just start creating my functions and my tests It has all my build pipeline built into it

And all the other pieces that I need And that uses invoke build Along as as long as with plaster so plaster does the scaffolding and invoke build is used as the build component So invoke build is essentially very similar to P sake or cake if everyone anyone's use those

It's got a bit of a different Implementation in the background, but the principles the same where you define tasks Then we'll use VSTS to build the module with the binary component in it and publish that module back to a VSTS package package management feed so that acts as like an internal

PowerShell gallery for you, it's like a private PowerShell gallery And then we'll use PowerShell package management to actually install that module onto my machine We'll make an update and then I'll install the update So lots of people talk about the demo gods the demo gods don't worry me about don't worry me

It's the demo demons that I'm worried about And because we're using VSTS. There's quite a fair few opportunities for this to go a bit bit iffy Let's see how we go, so I've just got a clean a relatively clean VM that I'm using for the demonstration today

This is VSTS that you're looking at here. So VSTS is the Online version of TFS. I highly recommend using VSTS over TFS You don't have to worry about any of the stuff in the background. They're coming out with new features all the time Over the last two years. It's absolutely amazing to see what the team's done with VSTS

Amount of functionality has just been increasing at a rate of knots So I don't have any projects to find yet With VSTS you get five licenses for free. So any small team can use it for free if you have MSDN Those users are free as well package management has a 30-day free trial

It's provided by Microsoft as well But it's also free for five users ongoing if you have MSDN Enterprise Those licenses include a package management license as well So we're just going to call our demo summit 2018

We use git so the other option I wouldn't even consider So definitely don't consider that agile workflow, so it's going off and creating our project in the background VSTS now actually provides a one-stop shop for your whole development lifecycle. It has code management

build and release management, it has work management, and it also has a Wikipedia and you can add extensions such as package management So what whereas previously you might have been using Jenkins and TeamCity for your build or you might have been using

GitHub or Bitbucket for your code management you might be using JIRA for issue tracking and confluence for your wiki You can do all of that within the one interface which is very helpful So up here you can see some of the items so under code. We're just looking at our git repository

Work items so you can have backlogs, Kanban boards, whatever you want Build and release so you can do builds releases Libraries are like a way of just collating all your variables and common components that you might want to use during your builds and releases together

You can do automated some testing in here as well, that's something that I haven't actually explored very much And then the wiki function which is relatively new as well package management was introduced I think about mid last year and There's another feature that we're going to use which is actually using a yaml build file

so previously I had to go and define my build step-by-step in the GUI and VSTS is now caught up with that fire and like Jenkins pipelines I can version control my yaml build file and when I push that up. It'll actually create my build process Okay, so let's go and create some package management feeds

I've already got installed, but I'll just show you what the process would be you just done browse to the marketplace Search for package management, so there are a few downloads

25,000 so it's definitely it's definitely getting some use All you need to do is follow the bouncing ball you click get I've already added it into my account So it's going to probably say that it's already

existing but if it wasn't you just say add free trial and you get your 30 days free trial and then once that expires then You can assign the license to five users or buy it. Okay, so now that I've got it installed when I go to

My build and release I get this extra tab called packages so when you don't have it installed you won't see that packages tab Screens a bit small here a bit big so we're going to create a new feed

I'm going to call it summit bin so what we're actually going to do is create two feeds one for our binary pieces and then another feed for our private PowerShell gallery

Summit bin is for our binary pieces It's associated with my account you can actually have the package management within VSTS use an upstream source So you could have it if it didn't find the package you could go to new get.org I don't want that for what we're going to do in the demo today, so I just say I only use packages from our feed

Audio we've created a feed VSTS gives us some helpful information on how to get started I Have actually created a VSTS credential provider What this does is when you're using new get it will case your credentials every time that you want to talk to

VST VSTS package manager, so I've already downloaded this package It's just a zip file, and you just add it to the path It's just got new you get dot XE and some dlls for the credential manager So then what I want to do is I want to add

My my source to new get So this is going to be used in part of our build when we want to retrieve the binary And you get I'll actually call out and retrieve our binary okay, so we're going to add it summit add summit bin and

Just using that URL the endpoint that it gave us to new get So that's been added and while we're still in VSTS I'm just going to create another feed Or our modules

So we're going to call this summit module once again. Just use my own leading packages published in this food So it's very simple to create the package to create the package management feeds

now what we want to do is So we have some binary components that I want to Actually put up in that repository that we can use as part of the build process So this dll here. It was actually came from It's available on new get.org I downloaded it from new get.org and I expanded the new get package

and Then I'm created a new new get package that just has that one dll in it So when you download a new get package, it's actually just a zip file So you can open up the zip file and you'll see it has a lib folder which has the contents that I want These are the folders underscore rails package

They get created when you do a new get pack So what happens I could actually if I went into this folder and ran a new get pack I could generate that new new new package file It uses the new get spec, which is just an XML file

It has some metadata about the package etc So that's all you really need to know for the moment. What we'll do is we'll now do a push

So we're gonna push to our source that we added we're using the API key VSTS So whenever you want to invoke that credential manager VSTS is always the API key that you use

And then I just give the path to the new get package So let's see how we go here. You can see it's hit the credential provider. I've already authenticated against my account When I was practicing the demo, so it gave me a prompt to provide my VSTS credentials

If you use two-factor, it supports two-factor because it's just the normal Michael Microsoft type dialog login So the good news our package was pushed so if we go back to package management and we look at our summit bin

There we go, so That's What we're after So now that we've got our binary sitting in our feed that we've created in VSTS All we want to do is actually create our module. So I use

Plaster, so let's just have a look at the plaster template quickly. So Rob did a good session on plaster yesterday So if any of you missed that when it comes out on on YouTube

Definitely have a look because he went into a lot more detail than I'm going to go into So the main component If you think about plaster All it really does is take a set of files and it allows you to define Take some input so you can say ask a few questions get some input back and then based on that input

You can say copy files in particular folders create a manifest based on some of that input And replace content in some of those files Based on that input as well. So essentially it can do token replacement within your file So there's three sections to it the metadata which allows you to version control your plaster template

Parameters, which are the questions where you're getting input data and you can have default values You can have a multi-choice and you can even have a multi select which Rob covered yesterday

the other section is the actual content and Here so you can see the format of this Variable plaster underscore parameter underscore module name In the parameters when I asked for the module name That name gets tagged on to that prefix and that variable is then available for me to use in the template

So I've based my follow module structure largely sort of on what PS rambling cookie did a few years ago using public and private And then I dot source those functions. I find that makes a lot easier to do pester testing So that's that's the basic idea there. So let's kick it off and create this module

You're everyone using ps3 line as well So this searching I'm doing I use ctrl R and actually our ps3 lines built into Windows 10

And it keeps my history across all sessions and I can search back and forward and it has a lot of Enhancements that are much better than the normal history so I'm just pointing it at my plaster template and I'm going to put it into this folder see new repo Name the module it's going to be the same as our project. It doesn't have to be but it just makes things easier

The module version number I'm just taking a default here I do some tricks as part of my module But I'll show you in a minute. I Use visual studio code for all my development now So I set up a couple of settings in visual studio code to help me

We're going to use visual team studio team services for our build It can pick your name out of your git profile, so I'm just going to take my name from my kit profile we set up our summit bin and our summit module and

These inputs I'm taking here. I actually replaced in some of my invoke build scripts with farm so that it can actually connect to the right feeds audio The other thing I spit out at the end is I give you some help

On how to register that repository to point to the VSTS feed There's an issue with package manager in PowerShell where it's currently supports version 2 of nougat not version 3 so There's a few things here. I've changed the you are a little bit to have a version 2 endpoint and

I Have to do a workaround at the end when I'm publishing the module and I'll talk about that a bit later, so let's register this Looks all right

okay, let's Let's have a look at what we've created. Okay. There's one little step. I've missed which is to actually

sync my Code so I'm going to do that now

So what we're going to do is we're going to push. I'm just going to do a git init to initialize my repository You can see my prompt has changed. I use posh git any anyone using posh git

It's a very handy Extension for PowerShell to allow you to see from the command line. What's happening with your repository, so it's essentially taking your git status and Providing information around which files are on tracks which files are on

stage which files are uncommitted and whether you're in sync with your remote Origin with your remote repositories It's very handy so we want to Commit all these files so I'm going to do a git add star

So I'm going through that sort of free stage process where I add commit And then I'm going to set up my remote so that I can push So you can do those tasks from within VS code as well For those that don't want to play in the command line. Okay. Now we have our code here

Let's have a look at our new module So I have a subfolder with the module name which actually contains the module and then the other folders are the utilities that help me

With my build process to my testing of my module So I have private and public my module definition my PSM one file My tests I mirror my structure for my module. So I have a private a public I have a basic test as part of my scaffolding which just tests the module manifest. I

Have a git ignore which is very important because I don't want to go through all this trouble of Moving my binaries out of my git repository to all of a sudden commit them into the repository Because once it's in there, it's not coming out Well, not easily anyway We have the yaml file which we'll come back to in a minute. The packages config is a file that's used by new get

So if you're doing visual studio development You'd have a package just on config that would define any packages or libraries that you relied on New get can read that file and then it will actually go to the new get feed and try and find the package and download it

Locally, so this is a bit of the magic for handling the binary pieces There's a number of files that are to do with invoke build So this file here is the main file and What it does it defines a series of tasks

So I've got a task here called install dependencies, but on top of that I can have a task which I call test which runs a series of Subtasks, so it'll run clean install dependencies run tests and confirm test pass a Task dot is your default task, so it'll run test and publish new get

But there's a few little tricky things in here that I've just point out So here I do a new get restore It's pointing at my summit bin new get source that we set up So that's where it's able to find the DLL that I've that new get package that I've set up I then actually when that new get restored

It'll restore to a packages folder and I copy the DLL from the packages folder actually into my module into a lib folder So we'll see that when when I run it The other Most of its pretty standard I run script analyzer Which is this section here?

The tests is pester Jim did a very good session on pester and I think Glen and I think there's a few other sessions So if you're more interested in finding out more check those out What I do is I collect those results and I have a task Called confirm test past

And what it does it just reads the results of the pesters pester tests, and it also looks at code coverage So I can set a setting to say I will only allow to publish if I've got 50% code coverage or 80% code coverage So it sort of forces people to make sure that they're having a bit of an effort

There's a publish task here, which is if you wanted to publish it to an SMB repository I don't use that as part of this demo But this this task called publish new get is very key It's actually not publishing to new get what it is doing is updating our module manifest It gets the newest it gets a version

From our build server. That's so the build server is a hosted server essentially in azure VSTS has hosted agents for Windows Linux and Mac OS so you can be good in developing in any language and targeting any platform and you can use a Hosted agent you can also run those agents on-premise

So you've got a lot of flexibility with VSTS What I needed was as I publish a new module I need to make sure that my versioning is updated every time that module is built so I take an environment variable Which is called on? environment build build ID Which gets incremented every time I do a build within my VSTS account, so just make sure that

Every time I'm pushing my module. I've got a new unique version number on it without even me having to increment the version the other things The other thing I do there is

Because I'm using this public-private structure anytime I have a a Script in my public folder the name of that script matches the name of the function Which is also? Designed to be publicly exported so rather than me having to go and update my module manifest every time. I'm doing a build I

Automatically scan my public directory, so I'll just get all the items out of it and Then I get the base name so for example if I have get summit dot ps1 in my public folder that'll just return get sama and Then I pass that array into update module manifest

Functions to export functions, so it's doing that that step automatically for me, so I'm not forgetting to do it And you can see that it's also putting that new version number on The next step is Because there's a limitation with package manager in PowerShell. I can't use publish module to publish into VSTS today

There is a preview Release of package manager on the gallery that has version free support But I haven't had a chance to test it yet So in the future you might be able to use publish module and point it directly at your arm VSTS private gallery But for the moment what I actually do is I create the new get package that publish module will would do in the background

And I use new get to push that up to VSTS So there's a little bit of workaround, but essentially it's doing the same thing that publish module would do in the background So That's this new get pack step. That's creating an NU pkg package and

As part of my scaffolding I have a new spec file there So you don't actually have to do anything could you were to pick up my scaffolding? It would create all this for you, or you wouldn't have to worry about Okay, so what we don't have is any functions, so we need to fix that pretty quick

I've just got a couple that we're going to copy in so this is where The VSTS really comes in posh it comes in handy. I can see I've got two new files No, modified files and no deleted files and because it's in red. I know these aren't staged, so if we just

Have a quick look at these functions Get summit does nothing so just as a directory listing and outputs. It's now a New subnet it does actually use

My DLL to do a subnet calculation So what we also need to do is we need to add that DLL into our module using add type Go to our module

So every time our module loads it'll run this add type which makes that DLL available for my command line Okay, well the things I haven't talked about so I've got a couple of pester tests, but um There's nothing fancy about those

What I haven't talked about is this vs code folder, so this is in my scaffolding you can actually Scaffold settings that you want your whole team to have in common so for example here I've got a setting that says trim trailing white space is true. That's the only setting I'm currently pushing in my scaffolding But as you get more mature in your team you might want to put in more settings

To have our vs code do more things for you automatically and common across your whole team The other file in there was a task dot JSON and this allows you to integrate Build or test tasks natively into vs code So as I said I was using I'm using invoke build to do my testing and my build process

So what I can define in here is a bit clearer Just look at the test task here So I have a label test. It's a shell so it's going to shell out

And it's all it's going to do is run invoke build with the task test It's pretty simple Vs code has this idea of types or kinds of tasks, so this is a test task It's just a way of grouping tasks together, and I've said this is the default test task then for build

As I said that dot for invoke build defines the default task So it's exactly the same it's going to shell out It's going to run build and it's going to run the command invoke cart invoke build task dot So we're ready to go so let's see what that looks like that integration

So if I go to tasks I can then see the tasks in my invoke build build process So these are the ones that I had defined in this build settings

build settings so the publish new get Confirm test pass these will be the same things that I can see in this list Publish new guess confirm test pass the other thing with the defaults So when I said if I want to run the test task

I can just go ctrl shift P to bring up the palette and I can say run test task or I can go task spell Run build task and it'd pick up that default build task What we want to do is run the test task and make sure everything's working on on our machine So you can see the new get restore happened

So it actually went up to VSTS to our binary feed pulled down the new get package It's expanded that out. It's created a lib folder and copied the file in Kicking off our tests and hopefully

Yes, that's good So all our tests passed our code coverage was run 100% Everything looks hunky-dory on our local machine So let's just have a quick look at what's happened here in my folder structure So as part of that install dependencies

It's and the pest and the test run it's created this folder called artifacts, which has my results in it It created a folder called packages, which has my new get package file So say new get package it expands that out so that I can actually see that lib directory that I was talking about which is what I compiled into the new get package or

Packed into the new get package and then I have a step that copies From that directory into my lib fold up within my module So now the DLL is actually where I need it for my module. So when we're going to do the publish on VSTS We should be okay

So we want to commit our changes so we can do it in here updates to module How many people are using VS code? Any ISE users

Any other editors Okay So before I push this what I'm I need to just explain what that YAML file is doing

So this YAML file just details the steps that my builds going to run So once I push this back up to VSTS VSTS will automatically pick up file and kick off a build based on the steps that are defined in it I'll just run quickly for the steps. There's not too many of them

so I I use a step called use new get well, it's actually caught the task is called new get tools installer and It's prefixed with a version. So use at version number This YAML feature was only released in I think around November last year, so it's in preview

So you actually have to enable it on your account, so you go to your account All right preview features There's there's some preview features on on just you but this is based on your account

So you just have to turn on build YAML definitions, which I've already done Previously, so we install new get and puts it in the path. We do a new get restore Pointing at our summon summit bin feed It reaches we tell it to use the packages config file

So it knows go and get that network DLL that I want or that package. I then run a PowerShell command and Here I'm essentially Installing invoke build and pester on to my move build machine. So I'm using a host of build agent. It has Various packages and software already stored on it. You can go to VSTS and

You can actually find out what packages are on there It doesn't by default come with invoke build and it comes with an old version of pester So I install invoke build and update pester to the latest version I use this skip publisher check with the force and the scope current user because they can't install it on the whole machine and that gets around the issue with

the fact that they baked pester into the OS I then kick off My build which is just running invoke build install dependencies test publish new get it just runs those tasks on the hosted agent I then do a new get push

So the reason I'm not using new get as part of my invoke build for the VSTS part even though I did that locally on my own machine when I did the tests is because VSTS has native authentication to my package feeds So rather than me having to provide credentials to say connect to this feed

Because I'm doing the build within my own VSTS account. It knows how to authenticate so I do the push Here and Where we push into we're pushing to summit module, which is the second feed we created just for our modules

Then I publish test results and I publish coverage results So let's get that up there so it can kick off what we'll see under builds

Actually, I've run into a little issue that I found If you don't initialize your repository you have to actually make a change to your YAML file

To get it to trigger so if I had to actually clone my repository First and then pushed up with the YAML file. It would have been fine, but because I'd initial I didn't initialize my repository It didn't see the change in that YAML file, so let's By default it uses this queue anyway, so I'm just gonna get rid of that. I'll save it

This wasn't part of the demo, it's just that I went out of order so we'll see if we can we can recover

Hopefully I'm not great look at this You have a build running so all those two steps that I defined in that YAML file

We'll see we'll see those start appearing. There's some default steps where it clones the repository There we go now. We can see all the tasks that I defined and You get console you get a console view of what's happening as it's going along as I said you can use hosted agents

Or you can use on-premise agents as part of the free account you get 240 minutes of free Hosted agent running time I've run this demo a lot of times over the last couple of weeks

I think I'm up to about 35 minutes, so it doesn't take a lot of time the The other thing you can do is if I put a hosted agent It would actually go and pull out to be STS and pick up the job and then run on-premise

So we've actually used this process to do quite a few things in the past arm I had is I had a process that ran and took the windows to our server 2016 ISO And using an on-premise agent we would download the latest version of the Microsoft patch. We would then use convert

to VHD Patch that ISO offline and spit out a VHD that we could then consume each month onto our VMM library So it's pretty flexible So our builds run. We're just doing a push now So this is where we're pushing the new get package that is our module up to our

Internal our private PowerShell gallery that we created Publishing the test results and we're nearly done, and we're doing okay with time, so that's not too bad

All right, so that jobs finished Some of the nice features you get in VSTS is because I am publishing my test to test results I can see I can get information about the tests and any failed tests. I Can also see information about my code coverage in there as well, so it's pretty nice feature So if we go to packages, so we were looking at summit bin before let's have a look at summit module

Hey So this is a PowerShell module sitting in VSTS package management Audio so what I want to do if we have look under modules, so I don't have anything installed on my machine at the moment

So I want to I've registered that Feed so I should be able to find module repository What do we call it summit module yes spot on?

Is this going to work? Oh? So it didn't work because we haven't provided any credentials to authenticate PowerShell has a concept of PS default parameters. It's a standard variable that you can use to append to commands

default parameters that you'd like to use so what we can use So what I want to do. I want to use this command here PS default parameter values Well, not this command this variable takes a hash table So it's star. I'm putting in star hyphen module so that means for any commandlet that is star hyphen module

Pass into the credential parameter dollar cred as part of my profile actually I've I created a PS credential I used to expel it seal it seal it seal ixml to store that credential encrypted on my disk and

As part of my profile. I load that into dollar cred every time my module kicks off So so where this comes into play is? Now that I'm using package manager within PowerShell It needs to be able to authenticate to VSTS when I was using new get it was using the VSTS Credential provider as part of new get so I had to set up a access token

so under security You can create a personal access token I've already got one here test PM, but I'll just show you the process you go add you only need selected scopes So the only scope you actually need is packaging read

That's all it needs is because the only thing we're going to do with a PowerShell is read from that that package feed So I've already done that so we can just ignore that for the second And then I set it's not very easy to see here But our repository So I set the parameter repository to default to summit module so that I don't have to type it every time so let's run that

Okay, so now if I do find module I don't need to specify any parameters because by default it's now taking credential as dollar cred And it's taking our repository as summit module So let's do install module

summit 2018 And once again because I'm using those wildcards It's going to get passed to This command as well and scope we want to just do current user. So it just installs in my profile

Okay, so we can do get command module summit 2018 There's our two public functions. I have a look at the folder. I've got our version controlled Under lib there's my DLL wonderful. So now I can run

My new subnet just takes in an IP address and a subnet mask spits out some information about the network So that's actually the DLL the binary component of my module working If we run get summit, we don't get much at all. So because it didn't do anything

So now we've we've published one version of our module what happens as we want to make more changes We just go back to our functions So in this case So on edits

My get summit and we're just going to add in the traditional line. So we've changed our Function will update our test our tests was actually very dodgy here. So we run the command and it should be

Yes Save that Run my tests or control F control shift P run test tasks locally It's much quicker to find out if it files here before I push it to the STS

Okay, great. Everything's worked successfully again So let's Commit our changes. We'll push those up. So what we should see is another build will kick off

Build got a running build, which is great. So while this is running, is there any questions?

Because you're using a personal access token That because it's using that personal access token. It won't have two-factor authentication

Yes, that's right So for people that don't want the binary components You could change the plaster template and remove the nougat

restore bits So that you could just be using this for just your normal module development and versioning The binary bits was just a bit of an additional thing and then the issue that we're running in this We were getting into more complex modules the other thing you can do if you want to have a step at the end of this build process that push this to

PowerShell gallery you just change that final step instead of publishing to the VSTS package manager feed you just do publish module put your API key in as a variable and Call that and then you could push directly to the PowerShell gallery. Oh

Wonderful, or you could do both. That's right So, um, there's a lot of flexibility and I've just got the basic so in my scaffolding You could pretty much do exactly what I did Scaffold it create a function of the test and then you should be able to do the same get a build running

So my My scaffolding is available on github and I'm going to put a copy of just the out the final module that we had today I've won on github as well Okay, we've run successfully Now test passed. So let's just have a quick look at our packages now

Should look a little bit different. So under versions. I now have two versions. So this one created just now and so within PowerShell I can do update module

Summit 2018 and what we'll see happen here is we get our additional folder. We do we do

import module summit 2018 Force so that we can get a new command and then if we run get summit PowerShell VSTS rock, so we're running out of time. So I have a bit of a recap in the slides

So we just went through we set up the VSTS project. We did the VSTS as packages feeds Then we did the new get and we push that into our binary feed. We then did our local module setup Ran that locally And then we set up our yaml build by pushing that up into VSTS

And then we did the steps within PowerShell to import to install that module and to update that module So I had some information about the steps. I use chocolatey Absolutely rave about it. It's amazing. I

Mentioned posh git it's great. I just had this the other steps that I used on that machine I've got some links in the pack to the resources that have got a lot more information about what I've done and That's it Thanks guys