Building Observable Streaming Systems with OpenTelemetry
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 69 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/67349 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Berlin Buzzwords 202117 / 69
11
25
39
43
45
51
53
54
60
00:00
BuildingSystem programmingContent (media)Presentation of a groupObject (grammar)Right angleXMLUMLComputer animation
00:44
Elasticity (physics)Software developerSystem programmingDatabaseOracleRed HatConfluence (abstract rewriting)EmailTwitterOraclePresentation of a groupRight angleMereologySoftware developerSoftwareData conversionElasticity (physics)Computer animation
01:28
Web serviceSystem callRight angleSlide ruleOpen setSoftware developerPresentation of a groupCartesian coordinate systemDescriptive statisticsBlock (periodic table)Service (economics)BuildingXMLUML
02:21
EstimatorElasticity (physics)Steady state (chemistry)SineSoftware testingState observerArchitectureMultiplication signRight angleFocus (optics)Spring (hydrology)Interactive televisionMessage passingAnalytic setNumberOperator (mathematics)State of matterDatabase transactionSystem callEstimatorDatabaseCartesian coordinate systemProgramming languageJava appletBootingSlide ruleThread (computing)Analytic continuationStrategy gameCommunications protocolSoftware development kitOpen setSoftware frameworkCASE <Informatik>MereologyWeb serviceSet (mathematics)MultiplicationBit rateComputer animation
05:48
Spring (hydrology)EstimatorGEDCOMState of matterTouchscreenSet (mathematics)Slide rule
06:27
Connected spaceWeb serviceRight angleSingle-precision floating-point formatState observerScripting languageCartesian coordinate systemVariable (mathematics)Computer clusterIntegrated development environmentAnalytic setEstimatorArchitectureProof theoryBootingComputing platformJava appletMobile appPhysical systemSpring (hydrology)Multiplication signBuildingOpen setProjective planeSoftware developerMereologyMachine codeComputer fileFront and back endsService (economics)Source codeJSONXMLComputer animation
08:52
Database transactionDistribution (mathematics)Integrated development environmentEvent horizonScale (map)Entire functionBlogSource codeAutomationMetric systemService (economics)Software bugBeat (acoustics)Type theoryElasticity (physics)Computing platformAsynchronous Transfer ModeJava appletBootingCartesian coordinate systemState observerDatabase transactionDoubling the cubeRight angleSpring (hydrology)Level (video gaming)DatabaseInteractive televisionAnalytic setBuildingOpen setRadiusSoftware developerWeb serviceEstimatorBitFocus (optics)WordXML
10:31
Hand fanBuildingGame controllerAnalytic setCodeComputing platformSign (mathematics)State observerMessage passingSource codeComputer animation
11:13
Software bugData typeBlogCartesian coordinate systemComputing platformRight angleMereologyDatabase transactionOperator (mathematics)ArchitectureState observerMessage passingInteractive televisionComputer animation
11:42
Social classFormal grammarDew pointSpring (hydrology)Configuration spaceBootingSystem callSocial classClient (computing)Right angleOpen setMessage passingCartesian coordinate systemCASE <Informatik>HookingInstance (computer science)Intercept theoremJava appletEstimatorClosed setMoment (mathematics)Category of beingSource codeComputer animation
13:58
World Wide Web ConsortiumComputer virusOpen setMultiplication signSoftware development kitDemosceneRight angleSource codeComputer animation
14:35
Link (knot theory)Electric currentInheritance (object-oriented programming)Gamma functionContext awarenessCodeMessage passingDatabase transactionService-oriented architectureBoiling pointData storage deviceImplementationDemosceneCharge carrierMathematicsRight angleStrategy gamePoint (geometry)Open setService (economics)Row (database)BuildingBlock (periodic table)Ocean currentOperator (mathematics)CASE <Informatik>InformationMultiplication signSoftware development kitObject-oriented programmingOrder (biology)ArchitectureSource codeComputer animation
18:21
Database transactionBitEstimatorState observer
18:46
Repository (publishing)Web serviceSoftware bugDatabase transactionError messageComputing platformRight angleState observerOpen setRevision controlZoom lensDisk read-and-write headBitInformationMessage passingDatabase transactionDifferent (Kate Ryan album)MetadataField (computer science)MereologyWordJava appletComputer animation
20:07
Interior (topology)Meta elementUsabilityIntrusion detection systemGraphical user interfaceError messageMereologyJava appletDifferent (Kate Ryan album)CodeSource codeComputer animation
20:31
Game theoryMIDIMathematicsLink (knot theory)MereologyImplementationLevel (video gaming)Message passingKey (cryptography)Multiplication signFunctional (mathematics)CASE <Informatik>WordRight angleCharge carrierSoftware development kitOpen setContext awarenessJava appletGoodness of fitAnalytic setDatabase transactionConstructor (object-oriented programming)PropagatorParameter (computer programming)Set (mathematics)PerimeterSource code
23:17
Software bugDatabase transactionEstimatorWeb serviceGamma functionRepository (publishing)EstimationInformation managementSign (mathematics)Menu (computing)Electronic data interchangeManufacturing execution systemGoodness of fitRight angleMultiplication signEstimatorState observerComputing platformCodeDatabase transactionMessage passingMathematicsAnalytic setInformationSystem callUniform resource locatorLengthChainMetadataOperator (mathematics)IntegerMetreMoving averageMetric systemRepository (publishing)Link (knot theory)Open setContext awarenessJava appletMatrix (mathematics)Computer animationSoftwareXMLSource code
27:51
Database transactionWeb serviceSoftware bugEstimationRepository (publishing)Cartesian closed categoryDigital filterComputer configurationOpen setState observerRevision controlEvent horizonMetropolitan area networkPulse (signal processing)CountingComputing platformState observerMultiplication signRight angleBitComputer animation
28:47
Parity (mathematics)EstimatorGroup actionMaxima and minimaTimestampMultiplication signProof theoryCountingOpen setRight angleComputer animation
29:44
XMLUML
Transcript: English(auto-generated)
00:07
Thanks everybody that is with us today and I'm looking forward that everybody is enjoying this amazing conference, which is Berlin buzzwords I'm pretty sure that there are a lot of content that
00:21
this year the conference is providing for the attendees and The objective here is always to not only to provide but also to share ideas. So feel free to after this presentation to Reach out and I'm more than available for answer any question to my habit regarding the subject or anything that has to do with
00:42
Technology, right? So for those of you that don't know me, my name is Oh For those of you that don't know me My name is Ricardo Ferreira and I'm working as a developer advocate in this company that pretty sure that you might have heard of which is elastic right so in elastic and part of the community and
01:00
For elastic I used to work for other software vendors as well such as Confluent or Co in Red Hat in the if you want to Reach out to discuss anything like I mentioned before You can send me an email I brought here my both my email from elastic and my personal one And if you happen to use Twitter, I'm also on Twitter available So feel free to reach out and I will be on the conversation right next year the presentation as well, right? So
01:29
If you read the description of the session, right the main purpose of this presentation is to show you How can you use first of all open telemetry API's?
01:42
to stitch together Services that somehow communicate using Apache Pulsar or I would say we're gonna use Apache Pulsar as an example here But what I'm gonna show here today can be applied for anything, right? So and when I was thinking about how I'm going to actually deliver this presentation
02:01
I thought alright, I'm gonna talk about API's but it makes no sense To use slides to explain API's right because API's these the beauty block the developer used to build applications So why not show an application for them, right? So and this is what I basically done here So what I'm gonna do today, I brought this. Let's call
02:23
minimalistic application right where I'm going to Okay I'm going to explain what I'm going to do step by step, right? So the application right now is up and running I'm gonna give you an overview about what are the application does but essentially TLDR
02:40
It is a simple API right that I've constructed using Java and spring boot, right? And the purpose of this API is to provide estimates so for example You want to estimate how much is the cost of a given car brands such as a Nissan or Toyota or Ferrari? Right, and then what this API does is to query those like skull estimates into a database
03:05
Which is employment using rads, right? So Typical micro service that you would see everywhere, right? in the What we what we are going to do here today is to Make this application that is currently not only accessing radis
03:23
But also is publishing a message into Apache Pulsar, right? So the first step we're going to do is to make this let's call transaction lag observable, right so everybody in the upside that is currently monitoring this whole transaction can see this operation because currently this is not being shown right I'm
03:43
Showed what is the current state of the architect rate as well as any downstream consumers that are pulling messages out of Pulsar This one is specifically I have written in using go, right? So that micro service over there pulls messages from Pulsar to do some analytics like counting the number of times
04:03
An estimate was given to Toyota for example, right? Very simple estimate but the focus is more like a not how to do it But what is the plumbing that is behind it? But the key thing here is that this whole interaction over here the publishing part as well as the consumption part is not being shown
04:22
right and you might be asking about right why not being shown here and everything else it is being shown because The open telemetry framework and SDK and set of API's allows you to expose pretty much everything that you currently have in your applications Right, but there are some gotchas, right? For example, if you were dealing with in this case over here
04:43
If you are dealing with something that is inherently synchronous like a spring boot or micro service It reading in any programming language. You are dealing with a protocol called HTTP, which is synchronous, right? So you can kind of Automagically instrument is that this API but the same doesn't happen if you are using non blocking
05:03
asynchronous Technologies such as for example Apache Pulsar and that could apply for anything that happens to have to behave the same way like you name it wrapped MQ active MQ Apache Kafka or any other messaging technology where all the Transactions spread over multiple threads, right?
05:21
There's not a single thread that provides the continuation of the transaction So we got to come up with strategies using open telemetry to make sure although it doesn't behave like this But it looks like it is the same transaction and this is what we are going to do here today, right? So
05:40
Let me start sharing like I mentioned before It's not a very good idea to explain API's using slides So probably this is gonna be the only set of slides that you're gonna see here today Let me show the current state of what I have here, right? So I'm gonna pull here my postman which is the UI that I used to send some requests to this API
06:03
I've got it here, right so you can see how it works But first of all, let me ask you this what if if I increase the font size to this is Everybody able to see my screen. You can actually put this on a chat so I can see
06:21
Um Maybe max not sure if you were following me there But can you see that this font size over here is good for everybody to see what's going on I'm gonna while you were answering this. I'm gonna issue a request here and oh The micro service is not running. So that's good
06:41
Can I got a connection refuses here which means that the micro service is not currently running which is good So this is the proof that it is everything alive, right? Okay, so what I'm gonna do is to start up my micro services, right so I have two scripts here One of them is let me increase the font size over here This is the spring boot application that I mentioned before
07:01
so basically I'm going to force a Maven build and this micro service application and Here's the interesting part. I'm going to download from the open telemetry project the Java agents, right? So if you're a developer and you want to instrument automatically all your microservices applications All you got to do is to use this Java agent along with your application, right?
07:24
Obviously, you have to provide how your let's call telemetry data is going to be sent to the back-end system So this is what I'm doing here I am exporting those variables and this endpoint over here is pointing to my elastic APM Which is my observability platform that I'm going to use here today for demonstration purposes, right?
07:44
so with that said I'm going to start my Brand estimator, right? and meanwhile, I'm going to actually start as well my Analytics layer which as I mentioned before is a go application. So this is a script
08:01
Let me show again So since this is written in go Basically, what I'm doing is exporting some environment variables and then I'm running this single go file over here, right? Which is going to generate a native application and probably in the temporary folder if I remember correctly And then it's going to be executed right? So both of them now are executed
08:22
Now let's issue one more time the request to make an estimate for this app So I'm going to click send right and now since the microservices up and running you can send here This is the brand and this is the price. So Right as expected right and if you look to the analytics leader, there's a cow here for the brand Nissan
08:42
So the whole architecture is the end-to-end architecture is working. But here's the problem Here's the gotcha, right if we look to the observability platform right now, right? And this is the observability platform Which is elastic APM if we go to the APM tab and I'm going to increase a little bit this
09:04
On side so everybody can see it, right? So you're gonna see that here on the APM There is this microservice called brand estimator that if we look to the service map You're gonna see that all all the interactions that the microservices does has been automatically discovered So yeah, the microservice is written in Java and access radis database
09:22
But more importantly if we click on this microservice, you're gonna see that there are some transactions Right that has been executed in the last I think this is 15 minutes. Let me double check. Yep Actually, let me decrease this for the last five minutes so we can have a more focused
09:40
Observability, right and then in the last five minutes, we've had this Invocation right here that if we scroll down We're gonna see that this is the breakdown of the transaction everything that you were looking here, right? It was automatically discovered by that
10:00
Agent that we've downloaded from the open telemetric and that's the beauty if you're a Java developer. That's the beauty of building Applications in Java that are automatically instrument because in the Java word open telemetry provide this agent So it discovered for example that this is a spring boot application It discovered they interacted with the radis database as you can see here over here. So
10:23
Everything was discovered automatic except for this invocation to this method called feed analytics that I Actually create I actually made this happen, uh, let me show you the code here of the controller so let me
10:41
so this is the end point that's Invoked when you call the API and if let me increase the phone signs over here one more time And if we see here, we're gonna see that as I fulfill that request I'm actually called this method called feed analytics, which in turn basically sends a message to push our right
11:01
However, how I made this available in the observability platform because I've used this annotation called X with spam So this is how why this feed analytics annotation here ended up being shown here on the ability platform But what matters is that as you can see here the where is the part where the application is sending a message to?
11:23
It's not being shown and that's the problem right this is what we are going to fix right now because we have to make this Interaction that is fundamentally part of the transaction right is part of the architecture. We have to make this interaction Observable visible to the operator. So let's see how we can actually fix this, right?
11:43
So what you're going to do, right? I'm gonna show here the what I call in a spring boot word, right the pulsar configuration, right? The pulsar configuration is the beam that is being used for instantiate The producer that is going to be injected and all the other beings on my spring boot application that can interact with pulsar
12:04
So specifically I am creating here and this pulsar client, right? So this is a beam right and then from that client I'm creating a producer right and as you can see here this producer is Using Jagen as schema, right and basically it's connecting to this topic called estimates and then I'm creating but
12:24
If you are working with pulsar and you want to bring instrumentation and open telemetry to the pulsar applications The best thing you can do is actually use interceptor. So I'm going to uncomment this right? So this is a Java class that I've already created. It's empty right now. We're gonna fill together
12:41
But if you create an interceptor in from pulsar, right? You can simply as you create your producer and consumers You can simply call the method dot intercept and then provide an instance of your intercept is that simple, right? So now that we've seen how we can hooked up into the or producer and consumers
13:01
Let's check into this interceptor right now So an interceptor as you can see here is essentially a class that implements producer interceptor Right very simple and then there are pretty much four methods that you have call back methods that you have to employment like the one that checks if the Instrumentation applies or not to a given message in our case gonna be always true, right the actual instrumentation
13:26
That's gonna happen before the message is sent to pulsar and if you want to hook up some specifically like alright I'm gonna do something in their exact moment that I receive an acknowledgement from pulsar so I can use This callback over here. And if I spin up resources in my Java class, I can use the callback close to you know
13:45
Close those resources, right? So the first thing we got to do here is Let me open up my check here
14:00
First thing we got to do is to come up with a tracer, right? So I'm going to put here a tracer, right? So let me show here one more time Why we are so easily being able to create a tracer here using the Alpine telemetry SDK because again you are Instrumenting using the agent. So the agent behind scenes is kind of a making the tracer available for you automatic
14:24
All you got to do is to retrieve it right and you give a unique name for it So that's what we are doing here. Right and then as I mentioned before we have to feel the before send Method so this method we are going to use this approach over here, right?
14:42
I'm gonna before returning the message to the caller. We are going to do this Let me show you here. How does implementation works? Oops, I think if you do this you are gonna be able to see so pretty much what we are doing here is right We are creating a span. Here's the tracer for the tracer you create a span, right?
15:04
So for those of you that know open telemetry this is how the API works and then there are some annotations that you can use to specify if this is a producer a consumer or things like that and then Every scene in the open telemetry or is boils down into the concept of context, right?
15:20
You have to be aware that everything you do you have to put it back into a context or you have to extract From the context, right? So because the context is the entity that is propagated over all the layers and services of your architecture, right? So you have to build up a context right and here we are retrieving the current context context. Sorry, and then we are
15:41
inserting this new span that we've created over here and then we are decorating this span with some labels or Annotations that just to make it like a visible for the operated operators to better understand what's going on behind the scenes, right? And then like I mentioned before everything is about the context So you have to come up with a way to give him the message from Pulsar, right?
16:03
Put that Context into the message because think about it, right? How a producer fundamentally communicates with a consumer, right? It's not about it's not with the broker the message acts as we call as the carrier Because it's going to carry the context from one point to another right?
16:22
So what we need to do here is to store this context into the message Which is going to be act as our carrier So that's why I created this math all store context on message. So what we are going to do here I'm going to save it and then let me explain the code
16:40
So pretty much what we are doing here. The message is the carrier, right? So here's the context here's the carrier I have to provide a way using the open telemetry SDK to Set that information right because again You can use this approach here this technique that I'm showing you for pretty much virtually any technology So you have to come up with your implementation in our case here to a setter that knows
17:06
how to put data Into a message and message is the building block from Pulsar But if we're dealing with file, let's say records for an Apache Kafka That will be the same approach and that's the beauty of what I'm showing here today because you can use the same approach over here
17:22
To come up with any strategy right for any technology that is asynchronous and no blocking So pretty much what I'm doing here is to come up with this callback Implementation called set that teaches how to set the message. Sorry the context into the message, right? With that change in place, right?
17:41
I'm going to pause my microservice one more time and I'm going to simply execute it again So because we've made some changes, right? We we've let's recap we created the interceptor and then we code the interceptor to create a span Decorate the span and put the span into a context and save this context into the message
18:01
so now the message being carrying this context over the Pulsar broker and Whenever any consumers downstream speaks up that message. Guess what? They're gonna be able to Continually the transaction. So that's the magic that we're gonna build here. So the microservice is up and running good So what we are going to do here right now is come up with a new invocation
18:25
Let's change the the brand right now Instead of Nissan, let's I don't know. Let's Make an estimate for a Ferrari for example. So yeah Ferrari is a little bit more expensive than a Nissan So but the estimate has been given good a new transaction has been executed now
18:45
Let's see in our observability platform, right? That's too big. Yeah, let's see in our observability platform in the last five minutes. What's going on? So, yep, we have a new transaction that happened So, let's see what's the differences in this new version of the transaction if we look at here right now, you're gonna see yeah
19:04
Everything else that was being shown before is being shown here again However, and that's the trick here it here is the that we call in the open telemetry word We call this a span, right? There is a new span a child span that happened on behalf of this method over here field analytics, right?
19:23
There was the trigger for that span if we click here on that new span and check the metadata This is the decoration or the information that we've put into the message, right? Which is pretty cool, right? Okay, I'm just seeing here that the zoom is a little slow so I'm gonna zoom up a little bit more right
19:43
Sorry, thanks for the heads up And so as you can see here We have everything ready for the decoration so let's do the consumption part right now because again the microservices that actually pull the message out of Pulsar is Written and go so we have to do pretty much the same process and go so we can see everything up and running, right?
20:05
So let's do this right now So the first thing I'm gonna do is actually to pause my microservice can go right so we're done with the Java part Let's now check the analytics and go the code here is pretty much done The only thing we have to do is create a tracer
20:22
So in a go ward, yeah, it looks like almost the same with a few differences So first you have to come up with a tracer. So this is part one and then you have to Fundamentally create a span that is going to where's the span? Here we go
20:43
yep, we have to come up with a span that's going to Demark that part of the transaction. So after we consume the message we are going to let me zoom up again So after we consume the message from Pulsar, we're going to create a span, right? So but here's the interesting part. First of all, we have to use that span. So
21:02
Right after we acknowledge the consumption of the message we can simply Receive the span dot end, right? So right now we are demarking the end of the transaction good But more importantly we have to come up with a way just like we've had to come up with a way to Put the context into the message now have to extract it, right? So
21:25
How we are going to extract this in the goal word so we have this some Functionalities in the go API that allows us to do this. So I'm gonna show you right here. I
21:44
Propagator and call the function abstract You can provide your current context and then just like we build a remember that in Java word We created a setter right now We have to come up with a getter Right an implementation of a callback that teaches the open telemetry SDK how to get data
22:03
From the carrier and this in this case. The carrier is the actual message. So let's see the implementation of this getter, right so Let me copy here. I'm going to put here writing the end of my
22:20
Code and let me zoom up one more time so you can see it, right? So what is the care? What is the gather and go right? So basically I create a strict right here that takes a message as a perimeter and a constructor and then those are the callbacks That I've created from that Strut, right? So there's three of them that have to implement, right?
22:40
You don't have to provide implementation for all of them, but at least in our case years the key is fundamental, right? So you have to teach how to Retrieve given a key how to retrieve the message from the key and also the set and also the keys, right? So now that we have right now that we have our Golang working as expected
23:03
Let's execute our analytics layer one more time because we've made it changes, right? We are we have probably a lot of six minutes to the end so it's we're on time which is good. Um Okay Alright, good. Good. Good. Good, and let's make another
23:25
Invocation to the microservice, right? It can be a Ferrari. No problem I'm going to call an estimate for a Ferrari one more time. It works as expected right because it should Presumably but now let's take a look on the observability platform and see what's what's different this time, right? So let's refresh
23:44
To see the transaction. So here's the new transaction that happened in the last five minutes after the chains in both the Java and Golang work microservice and now if we look this Transaction that was for the Ferrari the whole transaction took 23 milliseconds very fast But now you can see that voila right now by the time the feed analytics
24:06
Method is acute on the Java microservice side. It sends the message to Pulsar and then I'm terrible in math But probably like a less than 80 microservices microsecond microseconds after
24:20
The consumer which is written in Golang pulls the message from Pulsar and continues the transaction And as you can see here all of this both the producer and the consumer happens on the context Of the microservice and this is the beauty of you making the The whole transaction now observable because an operator
24:43
Thanks, Mike an operator Can now see that? Okay. Now it makes sense, right? Now I can see the whole transaction and then I can see that by the time I call the feed analytics and the Java microservice it sends a message to Pulsar which topic so the operator can click here and see the topic which is the topic
25:01
Estimates right and then it can also see that Oh, there's a consumer here that receives from the estimate and if you click here on metadata is going to see Oh, this is a topic. This is from Pulsar and this is a topic called the estimate, right? Uh, we could also decorate with a bunch of more information, but I've choose to be a little minimalist here, right?
25:23
uh, so Before we continue, let me do this, right? I'm going to provide to my dear friend Mike here that's helped me on the backstage A url of my of the github repository that has the whole code already implemented for you
25:40
All you gotta do is to clone the raffle docker compose up. Yeah, you have to stall docker I'm, sorry about that but you have to docker compose up and then everything we've all been running so you can study the code with more like a Because here i'm i'm doing things a little bit fast because we are time constrained right but you can study the code, right? And mikey if you could or uh provide this link for the uh, my friends that are
26:04
Are watching the session, please and then before we actually finish right since we've We have probably three minutes left. I would like to show you something really really interesting uh Okay Which is there is something that you can also do with open telemetry and i'm going to do this on the
26:25
microservice written in Bow, which is matrix pretty awesome. So what i'm going to do is to Use the api for an open telemetry to actually come up with metric so i'm going to create pretty quick here because we are
26:42
almost right now out of time Uh, where are my tracer here first after the tracer you have to come up with a metric and then You have to use the metric i'm going to copy the code and then i'll explain later All right. Let me put here the code first and then
27:01
We can simply see the code. So yeah, let me zoom in and then show you what i've done here So just like you create a mat you create a tracer You can also create a meter and then from the meter you create how many metrics you want it So here i'm creating a counter that is an integer, right? And the name of this counter is custom.metric.brand.count, right? And then what i'm doing here in the code is
27:24
after I actually Consume the message. I am Adding a plus one into that counter, right? So pretty much that's what i'm doing here on this code It's in the code that is on the github. So I'm going to execute this
27:40
Microservice one more time. So and we're gonna i'm gonna execute the estimate call probably three times So we can see the counter going on Um, here is where my postman here we go So here on postman i'm going to call this api one two three times, right so
28:01
Presumably if everything is working I have to have a new metric on my observability platform with a given name That the current count is three, right? So Let's check if this is true, right? uh all right, so Let me jump into my discoverer on kibana
28:20
and um Let me just minimize a tiny bit of the ui so I can see everything. So yeah, I have a Let me filter to see all the documents So I have three documents over here. Let me show an example of this document that is currently indexed on elastic search So as you can see here, the current count is three and then I have a ferrari brand, right?
28:43
And since we have everything here up and running, right? We can do things like this in kibana. Let me show real quick because we are probably running out of time ricardo So there's this tool called lance that we can use it. Let me create real quick here on kibana So i'm not here and i'm going to use a timestamp
29:01
In the one minute, thank you max and then i'm going to create calculate the maximum of that And then i'm going to group by group by where's the group by Group by by the name of the brand right? So here we go. I have a ferrari which is currently count is three so
29:26
That's the proof that now we can have stateful counters using the open telemetry api. So We have to wrap it up i'm, sorry for rushing things up I don't like to do this to be honest but I have to be respectful both with the conferences my dear friend max that's helping me on the backstage and
29:43
The other speakers that are probably waiting for this conference for the session to be done. So Thank you very much