Interactive, animated reports and dashboards in Streamlit with ipyvizzu.
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 141 | |
| Author | ||
| Contributors | ||
| License | CC Attribution - NonCommercial - ShareAlike 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/68688 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 202385 / 141
8
17
22
26
27
31
42
48
52
55
56
59
64
66
67
72
73
77
79
83
86
87
95
99
103
105
113
114
115
118
119
123
129
131
135
139
140
141
00:00
Multiplication signRoundness (object)Operator (mathematics)XMLComputer animationLecture/Conference
00:24
Graphical user interfaceMachine visionAbstractionINTEGRALShared memoryAsynchronous Transfer ModeScripting languageLaptopDifferent (Kate Ryan album)BitGroup actionCASE <Informatik>Task (computing)Codierung <Programmierung>Open sourceTwitterMereologyMathematicsLogicSimilarity (geometry)Multiplication signMathematical analysisResultantOperator (mathematics)IterationSmartphoneLink (knot theory)Point (geometry)Generic programmingOnline helpOrder of magnitudeCore dumpPlanningComputing platformInteractive televisionStack (abstract data type)Type theoryAbstractionMachine visionProduct (business)Roundness (object)Web browserLibrary (computing)View (database)Extension (kinesiology)Graphical user interfaceFocus (optics)Software developerSoftware repositoryRevision controlConnected spaceSet (mathematics)Interface (computing)SmoothingXMLComputer animation
07:19
Successive over-relaxationSoftware repository1 (number)Streaming mediaMessage passingLaptopSoftware repositoryLink (knot theory)WebsiteMobile appSource code
07:47
Computer configurationView (database)System callComputer fileKernel (computing)GUI widgetLogicPlot (narrative)ScatteringSystem of linear equationsCloud computingInformationWater vaporConfiguration spaceFaktorenanalyseSineIntegrated development environmentLaptopSource codeString (computer science)File formatSlide ruleComponent-based software engineeringFunction (mathematics)IBM Systems Application ArchitectureCellular automatonTouchscreenShape (magazine)Curve fittingCodeRevision controlCategory of beingImaginary numberShared memoryArchaeological field surveyGraph coloringFile formatSoftware repositoryConfiguration spaceSet (mathematics)Absolute valuePhase transitionMultiplication signComputer fileNumberPresentation of a groupCartesian coordinate systemPersonal digital assistantSlide ruleLink (knot theory)Object (grammar)Integrated development environmentFile viewerBitDifferent (Kate Ryan album)Right angleView (database)Direction (geometry)GeometryArchaeological field surveyComputer configurationCircleWordCellular automatonSoftware developerCounting1 (number)RectangleRevision controlSeries (mathematics)MereologyInteractive televisionEmailHierarchyClique-widthType theoryScheduling (computing)LogicCASE <Informatik>LaptopCrash (computing)Latent heatParameter (computer programming)SummierbarkeitScaling (geometry)Category of beingDefault (computer science)TwitterGroup actionMusical ensembleString (computer science)Event horizonSocial classForm (programming)Frame problemCoordinate systemComputer fontFamilyPolarization (waves)Stack (abstract data type)Streaming mediaMaxima and minimaAverageField (computer science)Element (mathematics)Visualization (computer graphics)TouchscreenCodeComputer animationSource codeXMLLecture/Conference
16:34
SupremumArchaeological field surveyMultiplication signUniform resource locatorComputer animation
17:06
Archaeological field surveyPresentation of a groupSlide ruleFile formatPythagorean tripleMoment (mathematics)MereologyDemo (music)Right angleMeeting/InterviewLecture/Conference
18:00
Plot (narrative)Configuration spaceFrame problemClique-widthEmailElectronic visual displayDefault (computer science)RSA (algorithm)Computer fileWindowFormal languageView (database)Codierung <Programmierung>Function (mathematics)MeasurementOrder (biology)Physical systemCoordinate systemPairwise comparisonProduct (business)Cartesian productCivil engineeringStreaming mediaMobile WebChemical polarityString (computer science)Electronic mailing listSlide ruleFile formatStack (abstract data type)VolumeQuicksortRange (statistics)Drill commandsMobile appCodeMultiplication signSampling (statistics)InformationFocus (optics)View (database)Zoom lensDrill commandsDefault (computer science)Category of beingConnectivity (graph theory)Link (knot theory)Configuration spaceMaxima and minimaRow (database)Object (grammar)BuildingFunctional (mathematics)Musical ensembleInheritance (object-oriented programming)outputSoftware repositoryGraph coloringGUI widgetFile formatProjective planeTouchscreenFrame problemPolar coordinate systemMechanism designFunction (mathematics)Streaming mediaWage labourPolarization (waves)Product (business)Different (Kate Ryan album)Game controllerVolume (thermodynamics)CuboidComputer-assisted translationPresentation of a groupComputer animation
24:00
WebsiteArchaeological field surveyTwitterComputer fileImplementationArchaeological field surveyMathematical analysisWeb browserProgramming paradigmTable (information)Open sourceLevel (video gaming)Computer animation
25:23
FeedbackVisualization (computer graphics)Right angleCategory of beingData managementProcess (computing)View (database)CodeCubeInteractive televisionTerm (mathematics)Presentation of a groupMobile appWeb browserBuildingFreewareDemosceneInheritance (object-oriented programming)10 (number)Front and back endsMultiplication sign1 (number)Goodness of fitTable (information)Open sourceInformation engineeringLibrary (computing)Bit rateGame controllerSet (mathematics)Machine visionSeries (mathematics)Software developerLecture/ConferenceComputer animation
Transcript: English(auto-generated)
00:04
Thanks very much, and thank you for coming. I really appreciate that. So, yeah, I'm the CEO and founder of this small startup. I came from Budapest, Hungary. We are really just small operations. It's been the three of us full time ever since. And right now we just managed to close our new funding round.
00:23
So we are going to scale up to have even 10 people, which is great news actually. But yeah, so it's a small operation. And yeah, before I get started, first things first. So I will try to show you a lot of things. Let me know if it's too quick or if you don't get it.
00:40
I'm happy to be interrupted and answer questions. There will be hopefully some time for questions towards the end. But again, I'll do my best. And I would also like to get to know you a little bit. So I have a couple of questions for you. And even during the session I will try to make you work. Be warned. So how many of you are data scientists? Please, hands up.
01:04
Okay, roughly half of the room. How many of you have to regularly present the results of your analysis to a non-data scientist or business people? Looks like almost everybody who put their hands up for the first question. Great. How many of you, I mean this is a sensitive one.
01:21
How many of you regularly screenshot your Matplotlib charts or similar charts and then just copy and paste it to PowerPoint when you do that? Yeah, a lot of people. Cool. How many of you use Streamlit regularly? Like ten people.
01:41
And how many of you have heard about Visu and IPyVisu before this conference, before this session? Four people. Great, five. Great, great. I love new crowds. Cool. So let me get started. So Visu, we built at Visu a generic chart morphing engine.
02:02
We basically, so based on the research of one of my co-founders, we employ a single logic to describe a plethora of different types of charts and because of this single logic we can interpolate between any of the charts that are expressed on our engine's interface. So that's the basic of it and that's why we can do these nice, smooth transitions between chart types that's pretty unique.
02:27
At least I haven't seen anything like it before. And so the next logical question is like, why would you do this? What is the purpose of all this? So basically we have a vision that we're working towards which is to build a graphical user interface for data.
02:42
We think that if we can employ animated charts and help people interact with data in a visual fashion, then it could be used for different use cases from exploration to storytelling to reporting and dashboarding, even to forecasting and planning. Now as I said, we're a small initial operation so we decided we need to focus
03:03
and we shifted our focus first towards storytelling because that's where the benefits of animations are, at least to our experience is the most straightforward, are the most straightforward. So why do you think animation means anything or makes sense to use?
03:21
First of all, static we think is abstract on one sense. So for example, these four charts show different views of the same data set but without your expertise and without being explained like what is we seeing on each and every chart, making the connection between them can be rather hard. Whereas if I connect these views with animation, it becomes very intuitive.
03:43
It's like no further explanation needed. Everybody understands how these views relate to one another. It's because basically we're using different parts of our brain to decipher movement than the abstract decoding task of charts.
04:00
A couple of examples of how animated transitions could be used. So like for example for drilling down in a, like in this case as you can see in a more complex way, showing differences or like showing one part of a stack chart and show if there's like one piece that goes against the trend.
04:23
You can change very nicely from aggregate values to distributed values. You can show change over time with the help of these racing bar charts that were so popular at one point that they were banned from the Data is Beautiful subreddit.
04:42
And yeah, so also you can actually turn one chart into a whole story. This one shows how many smartphones different companies sold back in 2016 and yeah, then we added the amount of money they make on this. So just one chart shows these leagues of companies
05:02
and gives you this sense of magnitude due to the animated transitions. So how we developed this technology and how we used it, actually the core engine, this generic chart morpher that I told you about that we built from the ground up is in C++. We use WebAssembly to compile it in the browser
05:21
and after we received our initial funding round in early 2021, we released our first open source product which is JavaScript library that it was intended to be used obviously by JavaScript developers to embed animated charts in their online products and then we shifted our focus towards data scientists.
05:43
We released our initial product last year in March which enabled everyone to use animated charts in Jupyter Notebook and similar tools. And then just last summer in Dublin in EuroPython 2022, I was happy to announce this extension of our tool called IPyvisa Story
06:04
that enables you to build, present and share animated data stories right from a Jupyter Notebook or a similar tool. So we have a ton of integrations now, a plethora of different notebooks and we are working together with PyScript, with BI tool mode
06:22
but most importantly I think the integration with Streamlit and lastly Panel opened up new use cases for us that I'm going to just showcase you a little bit. Actually the Panel integration is the newest among all of those. We would definitely need some people who could help us build more examples
06:43
using that platform because it's brilliant and I know that we can showcase a lot of the capabilities that our tool has and hopefully you can build useful stuff with it. So without further ado, let's get down to coding. I will go quickly through all the different iterations,
07:02
the first show you IPyvisa so that you understand the logic, then a more advanced version, then the story and then we'll get back, get down to the Streamlit version because I think that's how you can quickly understand how this is set up. On this bit.ly link you will find a repo folder that looks like this.
07:21
There is like a friendly welcome message there but even more importantly there are some useful links that will get you to the Streamlit app that I'm going to show you and our documentation site and our Slack workspace and in the notebook folder there are some notebooks that I've prepped for you. These are partly the ones that I'm going to just show you.
07:40
So if you want to follow along or if you want to later on take a look at everything that we do here, just go to this repo. Okay, so IPyvisa basically works with... Can you show the link again? Yep, absolutely. It's bit.ly slash visu hyphen euro python.
08:00
Okay. No worries. Can we move forward? Cool. Alrighty. So the basic logic of IPyvisa is you have one method, the animate method. You call it once, you get a chart, you call it twice, you will morph the initial chart to the second one.
08:22
It's easy as that. The animate method has three arguments. Sorry, I'm not a developer so sometimes I don't use these words properly. You have to add the data to it, you configure the chart how it looks and then you style the chart. You also have the option to fiddle with the animation options.
08:41
We try to make these transitions as easy to understand as possible but sometimes you have a specific use case and you just want to fiddle with those settings as well. So for initial example I'm going to use the Titanic dataset. You can use pandas data frame, you can install our tool from pip or conda so it's pretty straightforward
09:01
and I just declared that the P class thing is a string otherwise IPyvisa would think it's a value and we would end up having charts that we never intended to have. So yeah, that's basically it. We added a column of count which is one for every passenger
09:20
and so I create the chart object, I can set the width and the height and I'm going to call the animate methods right away. So I have the count, the value that is one per passenger on the x-axis and the sex on the y-axis. I also added the count on the label so it appears as a label on the markers
09:41
and I added a title to the chart and that's about it. So it was I guess pretty straightforward. The default geometry is rectangle so that's why you end up with a rectangular chart like this one. And as you can see, ipyvisa automatically summed up the count values for each of these categories.
10:01
We are going to, like in the next week or so, introduce new aggregation logic so you could count the average, the minimum, the maximum, actually do the count with that yourself but initially it was only the sum logic. And then in the next phase I'm going to just add the survive which declares if a passenger died or survived the crash
10:23
on the x-axis and also on the color scale resulting with a chart like this. So now we have a stacked horizontal bar chart. And as you can see, the label is adapted to it so now it's not on the right side but in the middle of the markers and we have the legend that appears telling us that the blue markers show people who died
10:42
and the green one is the one who survived. Okay, and lastly I'm going to just remove the survived category from the x-axis and add it to the y-axis, me ending up in a grouped bar chart. So we just rearrange the chart like this.
11:00
There are a couple of other parameters of the config object here. So we have the axis and the channels that you could give any continuous, any values, one of those and any number of categories to. You can set the geometry, the coordinate system, you can sort the markers, you can align them,
11:21
you can add a title as you saw. I mean we don't really have time to get more into it but if I set the geometry to circle then we get to circular elements. If I add the count, the same thing that's on the y-axis and on the label to the size of the markers then they would have different size and if I switch from the Euclidean coordinates to polar then I'll end up with a chart that doesn't have any sense.
11:42
I mean it still fits all the necessary requirements of a chart but we never use it because it's so hard to read. And so there were a couple of issues we faced early on. One of them is that people never used this kind of logic to build charts, this very generic like adding data series to axis and channels
12:03
and also you can build charts that don't make sense. You do want to build charts that you know what you will end up with. So we introduced presets very like quite after we did the initial API and here like for example with a stacked column chart it works like you add to the config part that you want to build a stacked column
12:25
so you set it up right away and then you have specific parameters for that type of chart. I mean it's pretty obvious. And by the way I have these playground notebooks in the repo folder. These are specifically prepared for you to just fiddle with the API
12:42
and understand how it works. So I'm not going to show you all the examples because I want to spend more time on the Streamlit bit but here we have a data set that shows how much revenue different music formats generated in the US. I create the chart and now I'm going to use a preset like I'm going to set that it's a bar chart and use those parameters.
13:02
I'm going to turn it into a group bar chart because I want the bars to be colored differently. So this is the styling bit. I'm going to change the font family, the font size, add a new color palette, change the number format and things like that. And as you can see everything I change when it's possible
13:20
will happen in an animated fashion so that you understand what you changed. And all the settings that you added to the chart object will get preserved for the next phase so you can count on those. And here this is how easy to turn it into a bubble chart, stacked bubble chart, and so on.
13:42
That's the presets. The other issue that we had is it's very hard to understand what the direction here is and going back and rerunning cells can mix your chart up very badly. So that's why we introduced this I5 is a story thing which introduces basically two new things, steps and slides.
14:04
And it's just like they are the same chart objects but you can make the animation stop there. So if I'm going to just quickly run this so that you understand what I'm talking about and I'm already running a bit behind schedule.
14:21
So we basically added these buttons beneath the chart that enables you to switch between the views that you generated instead of having to rerun cells and everything. So you have the presentation opportunity. You can go full screen, you can export it as an interactive HTML file with the story, with the data included so you can just send it in an email and so on.
14:41
And basically a step is, so like a slide is a view where I stop and one slide can have multiple steps as you can see on the picture while we go through before stopping at the final step of a slide. Okay, so that's the hierarchy. And yeah, so you can obviously build different,
15:03
we have event handlers to just personalise the charts to your needs and I'm just going to run this. So a couple of, and this is how you export it. So story export to HTML is the way you generate the HTML file. So now we have like for example this option that splits up a stack chart
15:24
and you can see this trend one by one. The filtering is actually the zooming bit. So I just added back the defence budget of the US to show the filter and here I'm going to just zoom in again with filtering the data actually shown on the chart and just rearranging them
15:42
so that the viewer can compare how the different bits changed over time. All right, now comes the time when I want to make it work. So, but it's going to be just like two minutes of your time. Survey time. Because we're working on a new API.
16:02
The newest issue we had is you don't like dictionary-based approach. You can't have the tab assistance in the IDE. It's harder to debug the code and everything. So we're working on now a new Pythonic API and we built a survey in which you can see actually one chart
16:23
and then under it three different versions of this new API. And we would just like to ask you to pick the ones that work the best for you. So coming back again it's just bit.ly slash ipivizu-api with capital API.
16:41
If you don't have the time for it to fill it now, I'm just happy if you just make note of this URL. But I'll give you a minute, okay, to get valve into it. And if anybody has any questions right now, I'm happy to answer while the others are filling out the survey.
17:01
Yes, please. Question was if it's possible to re-import the exported HTML. Right now it's a one-way ticket. So you build a story, you save it, and then it can be just shown, shared, presented.
17:24
But it's kind of like our first export format. There are others on our roadmap. So video obviously and the ability to export it to PPT is also there. We already have a tech demo exporting this to reveal.js, which is behindslides.com, that is an online presentation tool.
17:44
But that's an absolutely right question. Any other question we have at this moment? All righty. What about turning our attention to Streamlit, what do you say? Or you're still busy filling out the survey?
18:01
I'll let you do that.
18:23
Yep. The question was if there is a chance to export things to Matplotlib or is there any interoperability? Not right now.
18:40
So we are thinking about it. We'd be super happy to work with the Matplotlib project. The problem is that we haven't had the resources to do that. And yeah. Okay. Like I just had note of I have ten minutes left from the whole of the presentation.
19:01
And I want to give you some time for questions. So let's focus on the Streamlit bit, right? Okay, so Streamlit is a way to build data apps using only Python. Now on the right side you see an app and on the left side you see the code behind it. And basically there's, so first things first, I have to express a gratitude once again and I will do that plenty of times to Mr. Zachary Blackwood
19:23
who works at Streamlit and who built a bidirectional component via which iPyVisu could get well integrated into the Streamlit. And basically you have a new object called VisuChart that you want to use.
19:41
You create a chart. You add a data frame to it so it's pretty much the same. And then it's like in Streamlit you have these controls. So for example the ST checkbox is an input widget. It creates a checkbox that calls swap. And the whole code here says that do this. So create a chart with the cat, so the category on the X and the value on the Y.
20:04
And then if you click on swap then just exchange these. And that's what happens. So basically you just switch the chart. That's really like straightforward. And that's super easy sample. And there's an output. We're calling the chart.show method.
20:21
And then we just, you know, that's being shown there. Also because it's bidirectional you get some information about the value of the click bar here. So you could get value out of the Visu component where and what you clicked. There are a couple of apps here. You get to the same app if you follow the link in the repo folder that I showed you.
20:41
Another simple example is a slider which employs actually this filtering mechanism that I mentioned. Basically there is a slider here that says pick a year, minimum value 1973, maximum 2020. This is default value. This is here. And there is like one animate method where the filter is applied in a way that record year equals the selected year.
21:03
So if I change this then we get to a different year. And it's like pretty straightforward for anybody to use I guess. If I change this to have only like less than a year then it would be like up to that year.
21:22
And if I rerun this app then we'd have a lot bigger value because it would include all the values from 1973 up to 1988. It's that easy to change these kind of things. Okay? And we have some styling but that's not very important here. Another functionality that Zachary kindly implemented is the drill down.
21:44
It lets you just if I click on any of the bars I'll zoom into that and drill down to a different category. So I'll drill down to that year as easy as that. And if I click on anywhere else I go back to the original view. You can have a functionality like this.
22:01
And it's again pretty straightforward. So it's like there's the bar clicked. So we get from the chart the marker categories year. And then when there's if bar clicked none we get back to the original view. Else when the bar is clicked then we just filter to that year and rearrange the configuration of the chart. Ta-da. We drill down.
22:21
It's as easy as that. We can build explorers like this one where the user can actually themselves generate different views. This is just some bogus sales data. You can switch from sales to revenue. You can compare by product and region.
22:41
And you can also like switch to polar coordinates. You can rearrange the markers to be like sorted like this. And even the styling is available so you can set the background color to be very ugly. It's again super, super straightforward.
23:02
It's like actually it's just connecting these input widgets to certain configuration objects and then add those to. So this is basically where we say what should happen if this and that is selected. And then just begin with one config and then put it to the chart. So that's about it.
23:21
And the last example I wanted to show you and then we'll get down to questions. Is where you also have an explorer. It's a more bespoke one. I just make it full screen so you can see it. It shows once again the music formats. The user can select the timeframe when it is shown.
23:41
It's like you can adjust the value for inflation. You can switch from revenue to volume. You can stack the things by format instead. You can add more formats. You can sort them by value, whatever. But more importantly, so this is obviously something that any basic business user can do.
24:01
I have the opportunity to download this story as an HTML file. So download an HTML file. This is a very early implementation of this. And I have an animated data story of every analytical step that I took in the browser. This is super, super important for us because it shows the first step into a new paradigm
24:21
of intertwining analysis and storytelling. So all of this, sorry, all of it is free and open source. Hit it up. Use it. Come join our Slack and let us know how it sucks, okay? So thank you very much for your kind, yeah, for being here and heading for me,
24:44
like keeping with me throughout the days. There is a second survey. Sorry about that. But I have stickers that I'm happy to give you. And I'm actually in the OSS tables right around the food court. So you can get me there.
25:00
And lastly, I always make a selfie of myself in here. So if you don't want to be on it, just don't look in the camera. You won't be recognizable. But I need to prove the guys back at home that I did make it to Prague and I did make it to the stage. So if that's okay for you, I'm just going to ask you to say visu.
25:23
One, two, three, visu. I always suck at it. Okay, thank you very much. We still have, yeah, thank you. Good. It looks like we still have four minutes.
25:41
Thanks, Peter, for a rather animated presentation or assessment. So, yeah, we have like four or three minutes for a few questions. Just for that, we have two microphones in the middle here. So anyone, please come up to there and then we can have a few questions.
26:00
Yeah, thank you. Thanks for the presentation. What are your thoughts on building data stories without writing code for business users? Sorry, I couldn't get that. What are your thoughts on building those interactive data stories without writing Python code for business users who don't know Python?
26:22
Can you not use the mic for me? And I'll repeat the question. I'm sorry, maybe it's because it's facing that way, but I don't understand. What do you think of building data stories without writing Python code for business users who don't know Python? Yeah, I think it should be good for business users or at least that's what we're hoping for.
26:40
So if I understand the question correctly? Yes. Yeah, okay. Yeah, I mean, this is what the last example is about. And this is what we're working towards. Right now we have these free and open source tools and they work great and we need feedback and we want to make them as good as possible. Also, we started working on end user tools for Excel users
27:02
that would enable them to visually fiddle with the data, just how I said, that's the vision that we're working towards. These animated stories, I mean, I know people who got a raise because of using iPad Visual Story, presenting insights from the managers. So, yeah, they love it.
27:24
Hey, I know Asir mentioned that it's free and it's open source and thank you for doing that. But you also said you have investors and it's free people working on it full time. So I was just wondering, how do you make money? How do you eat? Yeah, lucky for us, we're just burning through our investors' money
27:41
but that has a deadline, obviously. So these tools that I'm talking about for the end users, those won't be free and open source. But everything that we build for you guys is like that and we'll stay like that. Got it, thank you.
28:05
So I'm just thinking about how much data can that handle because it's a dashboard and what exactly does Streamlit do behind the scenes? Yeah, so the question was how much data our thing can handle and what does Streamlit do behind the scenes.
28:22
So we have experiments with a couple of tens of thousands of markers that appear on the chart. We have two bottlenecks. The first one is rendering. We're using the canvas right now. We plan to implement, so like use OpenGL for that and that would sort that one out. And in terms of data management, we could do a better job.
28:44
So even with smaller data sets, you can get issues if there are too many categories put on the chart in the same time because we're actually building an n-dimensional data cube and that could get really big even with a small amount of data. With regards to Streamlit, they are the ones providing the control,
29:00
they are the ones hosting these apps and we provide the animated charting experience. As with any JavaScript developer, you can use this anywhere. So it's like we don't have a backend. It's like you download the package and then you can use it right away. Nothing gets back to us. Everything happens in the browser.
29:27
Actually, the more granular the data you add to Visu, the better because that's how you could generate the different views. So unlike other charting libraries, we don't want you to aggregate the data down to the views that you actually want to show,
29:41
but the granularity that encompasses all the views that the business users would want to generate. Usually that's pretty granular. And if you have any other questions, it's like our time is up. So please visit me at the open source tables. I'm super happy to continue the discussion.
30:01
And thanks again for keeping with me. If you have three more minutes to let us know what you think about Visu and about my presentation and how we could improve, just go to this link and thank you for your time and your attention. Thanks again.