We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Application Landscape Management for Teams

00:00

Formal Metadata

Title
Application Landscape Management for Teams
Title of Series
Number of Parts
94
Author
License
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
How can you get an overview of architectural concerns such as inter-service communication, lifecycle, security, health in a new or big project? Usually the answer is: "Ask the architect" or "look into the documentation". And the usual outcome is well known. The talk demonstratesa tool which can help answer many of these questions. It is designed to automatically gather information from existing sources, like code repos or deployed services and to be enriched by code committed by the team. The goal of the talk is to show the current progress and get feedback from the audience. How can a new team member get to know the application landscape it is working in? Where to find insights on typical Enterprise Architecture Management concerns like lifecycle, security, health and so on? Usually these questions arise sooner or later in most projects, even smaller ones. To answer them, however, is not always easy. EAM tools will help, but are not always available because of costs or high ramp-up effort like learning a sophisticated meta-model or TOGAF. In addition, architects usually design the landscape, gather information and have teams implement it, which results in a gap between design and current state. At times there is no dedicated architect, nor up-to-date documentation. Or generic tools are used, such as graph designers producing nice visualizations which are outdated at the time they are published and require much maintenance. Same goes for wikis if not maintained with high efforts. As a result the team has to work with a partially incomplete picture of its environment at least for a longer period. Nivio (working title) is a free and open source tool to address this issue. It is Application Landscape Management for teams, i.e. a bottom-up approach targeted primarily at the working class (developers, ops, designers, ...) not working full-time on EAM. It can be run one-off as docker image on a local computer, generate reports and then be discarded. The idea is to use the information that is already available and enhance it with information which is known to some developers or admins, but usually not required in code repos or operational environments. The model behind Nivio is biased and rather simple, partially based on and inspired by pivio. The main concept is to put services into groups and describe their state and dependencies, data flow and so on. Configuration is mostly YAML, which is easily readable by humans and known to most targeted users. There is one configuration per landscape, containing things like the name and the responsible persons, and most importantly, a list of data sources. This list can contain git repos, docker-compose files, rancher or kubernetes (some of them in alpha state at the time of writing) and a proprietary format. The report produced by Nivio is an automatically formatted landscape diagram and a few html pages containing all info that could be found. In addition to that there is an web API. Future plans are to provide a Java plugin API for customisation of input and output plus some more automated reports. Visit github.com/bonndan/nivio to learn more. Daniel Pozzi https://programm.froscon.de/2019/events/2360.html
Data managementFreewareOpen sourceAreaComputer programmingMetropolitan area networkInteractive televisionNetzwerkverwaltungCartesian coordinate systemInformation technology consultingXMLUMLLecture/ConferenceMeeting/Interview
Streaming mediaProjective planeOpen setLevel (video gaming)Multiplication signExistenceEnterprise architectureProduct (business)Open sourceNetzwerkverwaltungLibrary catalogTwitterService (economics)Physical systemExpected valueLecture/ConferenceComputer animation
Slide ruleProjective planeService (economics)Data storage deviceWordCartesian coordinate systemOperations support systemPoint (geometry)Computer animation
Projective planeOperations support systemInstallation artConstructor (object-oriented programming)Lecture/Conference
Projective planeLimit (category theory)Open sourceMedical imagingProduct (business)Quality of serviceRevision controlCartesian coordinate systemService (economics)Computer animation
IdentifiabilityVirtual machineNetzwerkverwaltungMedical imagingVulnerability (computing)Cartesian coordinate systemService (economics)HypermediaGateway (telecommunications)GoogolProxy serverOnline service providerComputer animation
Cartesian coordinate systemNetzwerkverwaltungCommunications systemOffice <Programm>Lecture/Conference
Information securityComputer networkOperator (mathematics)Cartesian coordinate systemTelecommunicationSoftwareInformation securityLevel (video gaming)CurvatureNetzwerkverwaltungService (economics)Lipschitz-StetigkeitImplementationWordMultiplication signSoftware developerComplete metric spaceElectronic mailing list
Office suiteGraph (mathematics)Enterprise architectureService (economics)Moment (mathematics)AutomationOffice suiteLevel (video gaming)Lecture/ConferenceComputer animation
Office suiteGraph (mathematics)Enterprise architectureFlagGraph of a functionNatural languageCartesian coordinate systemNetzwerkverwaltungEnterprise architectureDemo (music)Lecture/ConferenceComputer animation
Office suiteGraph (mathematics)Enterprise architectureAutomationIntegrated development environmentCartesian coordinate systemOperator (mathematics)Multiplication signoutputGraph (mathematics)Natural languageSoftware developerProbability density functionTerm (mathematics)Server (computing)Physical systemFile formatProjective planeEndliche ModelltheorieClient (computing)Musical ensembleCoefficient of determinationService (economics)Workstation <Musikinstrument>WordChaos (cosmogony)Non-standard analysisLevel (video gaming)Computer file
Graph of a functionSingle sign-onEnterprise architecturePhysical systemBitGraph (mathematics)CircleCache (computing)BlogEmailWordGame theoryMusical ensembleService (economics)Computer animation
Plug-in (computing)Physical systemCache (computing)Service (economics)Interface (computing)State of matterEndliche ModelltheorieShared memoryDataflowComputer animation
Electric currentRevision controlDemo (music)Revision controlDemo (music)Software bugComputer animation
Revision controlDemo (music)Electric currentOpen sourcePoint cloudPredictabilityMaxima and minimaSummierbarkeitMetreComputer fileInstance (computer science)NeuroinformatikXMLComputer animation
VacuumText editorComputer fontOpen sourceDemo (music)Open setLine (geometry)Computer fontNatural languageComputer fileMultiplication signVisualization (computer graphics)InformationService (economics)Open sourcePhysical systemBitInternetworkingIdentifiabilityWeb serviceVirtual machineRight angleLoginSoftware repositoryLevel (video gaming)Game theoryPhysical lawVideoconferencingBus (computing)Forcing (mathematics)HypermediaElectronic mailing listElectronic visual displaySource code
Web 2.0Cartesian coordinate systemElectronic visual displayComputer fileNetzwerkverwaltungInformationGoodness of fitService (economics)Link (knot theory)File formatHypermediaEndliche ModelltheorieComputer animation
Line (geometry)Open sourceIdentifiabilityComputer fileService (economics)InformationLecture/Conference
Mutual informationEvent horizonComputer fileRepository (publishing)Open sourceSatelliteElectronic mailing listComputer animation
Open sourceLine (geometry)Open setWeb 2.0Medical imagingEvent horizonServer (computing)Multiplication signGroup actionService (economics)Video gameWeightAttribute grammarInformationComputer animation
Service (economics)Connected spaceDemo (music)Bit rateRectangleBranch (computer science)Different (Kate Ryan album)Group actionGraph (mathematics)Software maintenanceComputer animation
Line (geometry)Open sourceInformation securityGreen's functionSystem administratorArithmetic meanComputer animation
Local GroupDemo (music)Conic sectionHill differential equationInformationInformation securityInformation securityGraph (mathematics)SeitenbeschreibungsspracheNetzwerkverwaltungInformationDemo (music)Communications protocolTable (information)Water vaporComputer animation
Local GroupMIDIMaxima and minimaElectric currentDemo (music)Revision controlUsabilityOpen sourceLevel (video gaming)Operator (mathematics)Software developerNetzwerkverwaltungCartesian coordinate systemMultiplication signOpen sourceLevel (video gaming)Projective planeFeedbackResultantPhysical systemPrototypeOnline helpCoefficient of determinationComputer animation
Software bugSoftware repositoryCuboidFeedbackPlanningIntegrated development environmentCoefficient of determinationHand fanComputer animation
Projective planeSoftwareLevel (video gaming)Product (business)Gateway (telecommunications)Figurate numberMotif (narrative)Service (economics)VideoconferencingMultiplication signAdventure gameIntegrated development environmentInformation securityoutputObject (grammar)Measurement
InformationState of matterOpen sourceRevision controlFile formatAlpha (investment)outputTheoryService (economics)CountingPhysical systemExpert systemDifferent (Kate Ryan album)Information securityOnline helpEndliche ModelltheorieComputer animation
Maxima and minimaProjective planeLecture/Conference
Open sourceFreewareCartesian closed categoryEvent horizonXMLComputer animation
Transcript: English(auto-generated)
Welcome to my talk. My name is Daniel. I'm Freelance consultant living here in Bonn. And I had the motivation to invent
a tool which helps in application landscape management. And yeah, so let's start. I think you all have all read the introduction of text in the program.
So my first question would be to you. What did you expect from this talk? I will repeat the questions here for the streaming. My motivation was, as you have seen in the comic, that often in projects, there are several things not really working well.
One of them is often project documentation. It's neither up-to-date nor existing. Yeah, so why are you here? Do you have any expectations, or do you have any tools maybe you use? Yes, please.
OK, yeah, yeah, thank you. So yeah, yeah, so next time we'll try the micro.
So can you help me with that? So the question doesn't work.
Hello? OK, next time. So the question was, if this talk is about LinaX, or which tools I will present. In fact, I have worked for LinaX some years ago. And I really like what they are doing. They are developing professional enterprise architecture
management tools. And the product I'm in, they often lack resources for such tools. I will show later why. Usually, you don't have a full-time architect, or you don't have the resources to pay such a professional tool. So I went on and had a look into the open source market
to see what is already present there. There are some really nice solutions. Some of them I will name. But neither of them really satisfied me. So I went on and tried to improve one of those tools. Any other comments?
OK, yeah. So you may have seen the comic on the title slide. I think that's still true for most of the projects. And the point where I want to improve the situation
is, or what's concerning my tool is often analysts design an application landscape, a bunch of microservices or whatever. And the design is not really that, but it's really what is implemented later on. Or what maybe operation installs.
Architects don't necessarily know all that constrictions that ops have. And often the project is rarely documented, or the documentation is completely missing.
Often, in the situation, to be the new guy in the project, and I have to go around and ask everybody, what is this service doing? Why is it there? Why was it built like that? Are there any architectural limitations or something? And usually the guy who built this isn't in the project anymore, or nobody can answer it.
At least for me, these are typical situations. And of course, I want to build a competing open source version to professional tools. Yeah, what is application landscape management?
The definition is quite long. I can't remember it right now. I Googled it today, but it was really too long to paste it here. What you can do is to identify weaknesses in your whole application landscape, all the services in your landscape, all the machines maybe, or gateways, proxies, whatever there is.
They need to be documented. And if you have the full image of what is happening in your landscape, then you maybe can identify weaknesses or find opportunities to improve something. Otherwise, your knowledge is limited, and you might just not see some restrictions or so.
So that is why landscape management is important. And nowadays, you often build microservices. Everybody here, I think, does microservices because they are fancy. And suddenly, you have more and more deployments
and more and more communication. And it's hard to remember all that. Yeah, the whole system, the whole landscape gets a lot more complicated. So that is why it's important to document it. Application landscape management is, first of all,
about services. Services are anything you can deploy that has some business value or does something on a technical level that enables other services to do something, to deliver the business value. But application landscape management is also about networks, about data flow, which service
communicates to others. Can he do so? Are there any networks in between? Or what about security? What about costs, business capability, and so on? The list here isn't complete. But I think you can get the idea of what these are typical questions an architect has to deal with.
But I think it's also equally important for developers to know all these things. Because ultimately, the developers have to implement what the business side wants. And they have to know things like lifecycle. When does the service I need go out of business?
Or is a service I need maybe not healthy enough to contact it every day? Or if I need a new feature from another services I'm dependent on, I have to ask the owner. And who is the business owner of that service? And does he have the money to implement that feature? These are important questions.
So what can you do to do ALM? First of all, I've made up this chart. It isn't really scientific. I just made it up. On the basic level, you can do handmade art. You use a tool like Pencil.
There's also a tool called Pencil, which is actually really nice for drawing. You can use OmniGraffle on Macs. Or you can use Microsoft Office, Visio. These tools, there's DrawIO, which is a really cool tool. And they're using a library.
I'm using two. And there's Wikis, of course. But all these documentation you're entering is outdated in the moment you are entering it. Because it's totally not automated. It depends on what you know. And yeah, you can imagine that after two or four weeks
it's completely the opposite of what's really going on in your landscape. Then you have tools like PlantUML or GraphViz. GraphViz is for drawing graphs. You can script them. But it requires a lot more effort.
You have to learn these tools. You have to learn these languages. And although they are powerful, you still have to invest some more. And then, of course, there's LinaX, for example, or other enterprise tools which deliver the most value. These tools are focused at doing exactly that application, Landman, application Landman.
Landscape management concerns, they offer a lot of value. But they often, they have two prices. Either they cost in terms of money, or you have to learn some very sophisticated language.
They have some models. And this is usually not something a developer has time to do, or an ops guy or so. So what you need is something in between. A tool you can automate. And that will generate most of the documentation for you.
So you can answer these questions about, yeah, whatever, health also automatically and in an easy way. Yeah, and the tools I found, there are some more. For example, with texture, you have to look it up at GitHub. It seems to be a really cool tool.
It uses graphics, so it creates a graph. And I don't know, it uses proprietary input formats to draw your application landscape. You should really check it out, I think.
Then there's PVO. PVO is the predecessor, so to say, of my project, which is called NIVIO. So PVO was invented by some guys from APOST. And I really like their approach. They're using a very basic language. And they use YAML files to describe their landscape.
And they use terms which are quite common, but quite well-known to all of us blue-collar guys. But what I didn't like about PVO was it uses some server agent architecture. That means, as far as I understood it,
you have to deploy agents on every target system. And that's not something I want to have. I want to have a tool which you can run once and then end. And you have some artifacts like the documentation in HTML or PDF or whatever. And then if you need it again, you can run it again.
But you don't have to be dependent on agents or anything else. You don't necessarily need access to the production environment. Yeah, this is what NIVIO does. It generates graphs. So yeah, I will explain it a bit.
So these circles here are systems in my landscape. This whole thing is a landscape. And I have, for example, here a WordPress running my blog. And I have an Apache in front of it. I have some other microservice I don't know which sends emails.
And WordPress also uses a database. It maybe uses a cache. And then if you're an enterprise environment, you have maybe a single sign-on system before it. And yeah, maybe a dashboard is measuring KPIs or whatever.
So it doesn't really matter what's inside this landscape. What matters here is that this graph is more or less fully automatically layouted. So every service here has, for example, interfaces.
Or it has dependencies. Dependencies here to the cache. Without the cache, it wouldn't run. So it's based on how you model it. You could model it otherwise. It doesn't really matter. But it's up to you to say the cache is mandatory or not.
And you have also data flow, for example, which means that WordPress has been extended by a plugin and now sends JSON to that KPI system here. So all these things you can model. And these services can also have state and a lot more, which I will show now.
One thing I think I need to mention is that the current version is quite young. So yesterday evening and even this morning, I had to fix a lot of bugs. But I think I finally got the demo running.
So there's no official release yet. Yeah, but let's see how it works. What I did is I just written in Java. But that doesn't really matter. I have already started an instance. It's running on my computer here at port 8080.
And I told Nivio, now please go and start with a file. And this file here, can you see it? Does anybody know how to enlarge the font size with Microsoft Visual Studio Code? Because I don't. OK, I really like that.
OK, so this is a proprietary language used in PView and a bit modified by me. What does it do? It names an identifier for your language, which is more targeted at systems to exactly define. And this is the landscape. And perhaps there are other landscapes too.
You can give it a human readable name. And you can add contact info and other stuff. And then you have sources. Sources are references from where I can pull information. And that is one important thing of Nivio. Now, it can do push.
Or you can do push information inside Nivio. But the idea is to use pull. That means Nivio would pull information from whatever source there is. You can use maybe a GitHub repo or here
Docker Compose files from other servers. Or you can even grab stuff from a GitHub repo. And there are, of course, other things you could imagine, like having an existing Docker machine
and log into that Docker machine and have the services listed, or Kubernetes, or whatever there is. There are a lot of fancy stuff right now. Or maybe a monitoring system. All these sources of information could be used here as references.
And yeah, for the demo, I have told Nivio to read from this Docker Compose file, which is quite simple. You can always interrupt me if you don't know what Docker is. Or I have to explain a bit more than just say it.
So it has three services. There's one web service. There's one Redis. Then there's Datadog. I just copied that from the internet. What happens? It reads that file and displays it. So it says, OK, we have three services.
And I'm using the name from the Docker service name. And I know there's maybe a link between Redis and web. And so there's a dependency between them. And that is what it does. So that's not really much. In a Docker Compose file, you can't add that much information
to have a good application landscape management info. So you need a lot more data to gain some value from it. And so the idea is not to just use that one file,
but to add another file. Here, Docker Compose extra, for example. Here, I added just very few info. I'm using an identifier to pick one of the services from the Docker Compose file and give it a name.
So when I'm re-learning that stuff now, you see here that suddenly the name has changed. And that is the idea. You use existing sources, which are maybe not perfect for doing ALM. And you enhance it with stuff you can easily manage.
For example, I could put this file into a Git repository or host it on some web server and then have it automatically drawn into NIVIO. So every time I'm asking NIVIO to render that graph, I will get that nice image.
And I can now answer it with more. There are a lot more attributes. For example, a group. I can group services, and they appear in different groups. Or I can add info about service lifecycle. Has a service just been planned, or is it already end of life, and so on. So I'm saving it and hoping the reload works.
Yeah, what you see now is that I have added this group here to that one service. And now it's displayed in a different rectangle. And still the other connections in that graph are maintained.
One last thing in the demo is I can regard status as, for example, I can use any label I want, like security, and have status green. And it now will have nothing. And now I will have that status red. I think everybody understands what that means.
And hope reload works. So now FUBAL 2 has a security issue, and that is also presented in the graph. And finally, one thing I can generate is the graph. The other thing is have some rendered HTML document, which
I can also turn into a PDF and send that to management or whatever. And so you have here all this rendered graph and all the information that's available. For example, here there's FUBAL 2, and it's just in a table from where that info comes from.
Yeah, that's so much for the demo. OK, yeah. So what are my project goals? I think I already mentioned some of them. I want to have a high automation level
and perhaps no installation, if possible, or just Docker run and have all the rest done for you. Because I want to have it used by developers or ops guys or whoever with a blue collar is there. Because I'm a developer too, we don't
have time to do that documentation stuff full time. So other thing is no fees or costs, because often in the projects you needed some documentation, but you don't have the money for it. And you, of course, reuse existing sources is one of the greatest ideas in the project, I think.
And yeah, you name it, whatever it is. This talk is about getting feedback from you. If you're interested in application landscape management or you have to do it, you maybe want to have a look at Neveo and add some feature requests. Neveo has a GitHub repo.
It's bonden.bonden.neveo. Bonden, that's me. Yeah, and you maybe want to add some bugs here or feature requests or whatever you think can help you. So that's it, basically. My plan is to get feedback from you and then have some first adopters.
First of all, I would ask my friends. They owe me something, and they have to install it in their environment and then maybe get more feedback. And once maybe I'm hoping or dreaming that Neveo will become the most visually appealing tool,
because one reason also why I built Neveo was that most rendered stuff is really ugly. So I wanted to have something cute I can look at every day. Yeah, thank you.
Yeah, any questions? I think there's plenty of time. So thank you for the talk. Seems like an interesting project.
As I'm currently on the Kubernetes train, or however you want to say that, I could imagine that would be quite cool to have it in Kubernetes because you have already everything described in declarative resources and then maybe even combined with service measures
like Istio and so on, you could get some quite nice graphics out of it. And if you go further and implement GitLab or whatever, you maybe can get nearly fully automatically your full environment with security issues,
with out-of-date software and so on and so on. So also objectives that enterprise-grade software is doing already now and doing it manually most of the time. So yes, just a comment. Yeah, thank you. I think that's true and that's something also,
Linux is, I think it's, okay, it's done. Okay, that's, hello? Hello? Hello, hello?
Okay, I think that's already something that Linux is doing and that's why Linux is a great product. I'm just copying something off Linux, I just can say that because I don't want to compete with professional tools. I will never have a UI, for example, or I will never have some sophisticated authentication,
but I can use some basic features to, yeah. Any other comments? Thank you. We saw the Docker Compose files as input
plus that YAML format. We just heard about Kubernetes. Of course, there are also Kubernetes YAML files that could be used as input in theory. Are there any other formats that you either are using currently or that you are considering using? For example, I could imagine Swagger may also be interesting for different kinds of information
but also for helpful information. Yeah, please add that to GitHub as a feature request. In fact, I have an alpha version of Kubernetes running but I'm not the Kubernetes expert and I did some things wrong, which I have to correct now.
And of course, you can dream of a lot of things. For example, like Prometheus, so Prometheus is a logging system. It basically holds a count of every system there is and you could read that Prometheus stuff and yeah. Other formats, I don't know,
imagine something, add them to GitHub. One thing I'd like to add to concerning Prometheus and state, service state is an important thing. I mentioned also health and these are status as two, as health and what else is there?
Security, things like that. You can use Prometheus, for example, as a source and don't create new services but add some state information to services too. So yeah, there's a lot of things I want to do but over the time, that will happen.
Any more comments, questions? Okay, and that's it, thank you for coming. And please have a look at the project and leave some comments if you wish. Thank you.