TiTiler: Not just a tile server
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 351 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/69048 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Production Year | 2022 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
Server (computing)HistogramWell-formed formulaMusical ensembleStatisticsMountain passBit rateRaw image formatLine (geometry)AlgorithmMoving averageProcess (computing)Level (video gaming)Product (business)BitProjective planeData storage devicePixel2 (number)Zoom lensDifferent (Kate Ryan album)Cache (computing)MultiplicationDivisorOpen sourceSoftware frameworkMathematicsNeuroinformatikTerm (mathematics)TesselationModule (mathematics)SoftwareComputer fileInfinityDegree (graph theory)Set (mathematics)Fluid staticsWebsiteSoftware developerRaster graphicsMappingTessellationFile formatWeb browserSoftware repositoryFunction (mathematics)Software engineeringObservational studyFamilyMultiplication signLatent heatCombinational logicOpen setComputer configurationGraph coloringElectronic visual displayServer (computing)State of matterSuite (music)Parameter (computer programming)Mathematical optimizationPoint cloudContent delivery networkOperator (mathematics)Medical imagingClient (computing)Online helpPhotographic mosaicSymbolic dynamicsNumberRow (database)Object (grammar)Position operatorMusical ensembleComputer animation
07:15
PixelVector spaceInflection pointHistogramLogic gateCAN busWell-formed formulaSoftware testingServer (computing)Installation artPhotographic mosaicDemo (music)Symbolic dynamicsCore dumpVector spacePixelTessellationInfinityComputer configurationSymbolic dynamicsMusical ensembleModule (mathematics)StatisticsMedical imagingWebsiteStack (abstract data type)Combinational logicWell-formed formulaGraph coloringCodePhotographic mosaicPoint cloudDemo (music)DatabaseBuildingCore dumpBlock (periodic table)Binary fileMetadataCartesian coordinate systemLine (geometry)Social classOnline helpServer (computing)Installation artFactory (trading post)InformationHistogramLevel (video gaming)Electronic mailing listArithmetic meanProduct (business)Set (mathematics)Bound statePoint (geometry)CuboidPauli exclusion principleData typeComputer fileExpressionMathematicsSelectivity (electronic)Library (computing)PlotterTerm (mathematics)MappingResultantLinearizationMultiplicationData storage deviceRaster graphicsMaxima and minimaBendingGeometrySuite (music)Computer animation
12:13
Core dumpSoftware testingPhotographic mosaicReliefComplex (psychology)Process (computing)Default (computer science)Server (computing)Module (mathematics)Open sourceMetadataVector spaceDifferent (Kate Ryan album)TessellationSet (mathematics)Projective planeSoftware repositoryTesselationMean value theoremComplex (psychology)Multitier architectureElectric generatorPixelModule (mathematics)Medical imagingSoftware developerBitCuboidPoint (geometry)Cartesian coordinate systemSymbolic dynamicsProcess (computing)Level (video gaming)TwitterScripting languagePoint cloudTwin primeGreatest elementDigitizingGeometryDefault (computer science)Stack (abstract data type)Open sourceMetadataUniform resource locatorDemo (music)Active contour modelFamilyVector spaceDimensional analysisProduct (business)Photographic mosaicSocial classExpressionDatabaseSelectivity (electronic)ResultantFunction (mathematics)Server (computing)Raster graphicsGeodesicHill differential equationRight angleDynamical systemDifferenz <Mathematik>Computer animation
18:21
PressureComputer animation
18:48
Network topologySession Initiation ProtocolCAN busBendingLattice (order)Arithmetic meanComputer animation
Transcript: English(auto-generated)
00:00
hope everyone can hear me fine. So today I'm going to talk about Titaga. First I need to thank you to wake up early to be there and I'll try to keep you awake as myself. Um okay. So I'm a software developer at DevelopmentSeed. Uh some people call me Cogstar. I'm more like a yeah I'm a cog passionate and uh mostly a JPEG 2008er and
00:27
uh but yeah. I'm a self taught Python developer uh but I had great mentors during my career such as Sean Geese and Matthew Perry for example. Some people might know me because I created a website a couple years ago called remote pixel where I tried to ease the
00:44
access to earth observation data has on sat and something or two for example. I'm a geologist. I've got a master degree in earth science. I just put that to show that you can become a software developer, a software engineer even if your background is whatever. Um
01:02
and uh when I'm not coding or like spending time with my family I usually bike or get coffee so if you want to ride during the fiancee phosphogy hit me up. I'm always up for a bike ride. So I'm going to talk about what is Titaga, why it exists and how to use
01:24
it. Um and uh first Titaga is an open source project. It's hosted on on github and uh we welcome any contributors so feel free to uh check out uh the repo and open issue if you have any problems and or open comments uh a discussion if you need more help. So Titaga
01:46
is a Python it's a set of Python modules uh um and it's mostly based on uh a suite of uh other modules uh like ttyler like rio tyler mercantile uh all based on rasterio or
02:01
gdol uh like which is based on gdol and uh we also use pyproj. Um it's uh we are using uh so gtyler helps you create uh api's so we built ttyler using fast api which is uh like a new state of the art um framework to build uh python api's uh and so it's
02:24
uh pro production ready uh because yeah fast api is great. So as I say it's uh ttyler is a production ready uh module uh we use it in a couple of projects already and it's also used like in the planet i computer for example so just to let you know that it's pretty
02:42
performant. Um we support mostly club g majority but uh like all raster supported by gdol. Uh we also support stack items uh we can create tiles in different uh projections. We support like multiple output format if you want to display png or jpeg on your map or
03:03
like uh like even like support numpy tile numpy tile to play with the data directly um there is ogc wnts support uh full open api documentation uh thanks to fast api uh we also uh have a support for mojax uh via mojax JSON specification and uh we try to make
03:25
ttyler in a way that you can extend and customize it uh as much as you can as much as you want sorry. So yeah ttyler is is a pro project that help you make tiles to display on the map and uh tiles when we talk about raster uh it's like a gpeg or png files that
03:46
you display on the map. They usually uh size like 256 by 256 so yeah small um small image that you want to put on the map. Usually you create those like uh you pre-create those
04:05
uh and you store them on uh object storage like s3 so what you do it's uh you have a raster that you want to display on the map you create all the tiles in advance and then you use a a server or like a a cdn to store to to to serve those image uh to your map
04:25
clients. So you do all the processing, the reprojection, rescaling, the mat uh applying some mat algorithm to the image, cut all those tiles, put them in the storage uh for different zoom levels so you end up with millions of static files that you will then
04:42
serve to the to your web clients. And when we talk about dynamic for uh tiling for that we do with ttyler we we only store the the raw data so you uh call up your majority for example and there is nothing in the middle there is just a server there is no more
05:01
files and so every time the the web client will request a map tiles we will read the the original tf we will apply the reproj- the all the processing steps so reprojection, rescaling, mat algorithm and serve the tile. We can put a cache in the middle so it's fast for the second tile request but like there yeah there is no more like tiles to
05:24
store uh yeah on on on the object storage. So it's a bit slower but then it opens like an infinite uh uh uh possibilities like in term of rendering um so you can for example change the rescaling factor like uh for example this is an elevation data so
05:43
there is a different you can yeah apply different um color maps different rescaling factors and so you can do that on the fly so every time we change the the the the options we we generate the tiles and uh same for for example if you want to choose
06:01
like the bend combination of your data if your data has more than one bend you might want to change the the bend combination like you put the first bend at the uh like the bend one at the uh at the third position or whatever. So you can do that when using dynamic tiling. You can either oh sorry you can also like want to mosaic files like
06:25
if you have two uh or more like cloud optimizer if you want to create um tile output tiles that will merge the the the files together but you you can also for example choose the one to choose which pixel goes on the top of your tile like if you want the
06:43
oldest pixel to be on the top if you want the uh like darkest or like the brightest pixel you can do that uh when doing uh dynamic tiling you can choose like a lot of parameters and like we will just create the tile on the fly. So if you think about like
07:00
static tiling that you will have to generate like tons of millions of tiles if you want to support all this kind of operation but with dynamic tiling it's everything is almost possible. So and you can do also nice like nice things like like you can translate every pixels to vector tiles if you want like there is a like like I say almost an
07:25
infinite possibilities of options that you can do with dynamic tiling and you can also like do fusion like if you have multiple files that like not in term of mosaic but like in term of bands like for example the sentinel or the Landsat data they are storing
07:42
multiple files so you might want to to match those files first to to create a RGB combination. So for this this example is using a stack items so we read the stack items uh jessens and we see that there are there are multiple assets and we match the
08:01
assets uh dynamically and we let the user yeah choose which assets uh to to match. Um one important thing of a tiler um it's a python module okay it's a suite of python module so um the the thing I usually say it's don't fork it like just in install it and
08:21
import it uh and then you can extend it uh using your own python code. So there are multiple tiler modules so there is the core which has all of the building blocks for the building your API. We have tiler mosaic which uh adds support for mosaics. Uh we
08:43
have the tiler application which is like the demo application that has like support for cloud appreciative support for stack and support for mosaics and which is like production ready you can just you do pip install uh tiler application and you can almost launch your own server uh that with all the features uh that uh that you that
09:04
you need. And there is uh another one called tiler pg stack which is uh linking uh pg stack database so the stack meta data database and to create dynamic mosaics directly from that. So uh this is just an example of like a really small like python code that will
09:26
create like a full raster services uh from tiler. So we use something called tiler factories uh within tiler which helps uh the help python class that helps build and
09:40
register endpoints to your fast API application. So like it's like six or seven line of python code that will create like a full list of uh endpoints and then you have like support for map tiles so which I described earlier. You have support for pre previewing your data set uh to uh cropping your data set using bounding box or
10:03
geo checks and features. You can access the pixel values uh so there is a point endpoint and then some meta data endpoints like bounds info statistics and uh also the wnts ogc uh endpoints. One thing as I described before one thing important is you can
10:24
change like the bend selection you can choose the bend selection you can like apply bend math expression like ndvi or whatever um you can choose any color maps uh that uh come from matplotlib and also uh build your own custom color maps. You can apply
10:41
linear rescaling uh or like color formulas uh to uh to like play with the the the like color correction of of your data. So so before showing more tiles and more images uh so the the meta data endpoints are really great just to understand what's what's your
11:02
data. So you get to know how many bends the data your cloud up team adjective has like what's data type it is so you can understand what kind of options you can have then uh if you need to rescale your data if you need if you need to allow more bend combinations. So this is like the basic uh basic info about your your image and the statistic endpoints
11:26
give you like the min and max values of your data the like uh histogram of your data so you you have a understanding a more more deep understanding of your data and uh so and again you can choose which pen you want the which statistics or which bend statistics you
11:46
want you can like uh use expressions for example this is the ndvi histogram of this data so uh you can like again uh most of the endpoints supports like bend selection and bend expression. So the this is the point uh endpoint results uh so again you get all the
12:07
pixel values for all the pixel value for all the bends you can select the bends you can apply bend expression if you need. So this is the result for this is an example of the preview uh endpoints so again bend selection uh bend expression you can do whatever you
12:24
want. The crop endpoints um example so you post uh geodesism feature how you can also as I said before use a bounding box uh directly in the URL path and to create like yeah small images uh for your geodesism feature. So again uh don't fork it import it customize it uh
12:47
if you want you can exclude some endpoints if you want just the tile endpoints you can like uh yeah put it in your own application if you already have like uh a fast API application with other endpoints you can just register uh the tile endpoints that you
13:03
want. Um the uh as I explained before the tile application uh python module has support for cog stack and mosaic gsms by default uh so which is the one that uh is used in tile that is x x y x y z um demo endpoints uh so you can already play with
13:22
it if you want uh using this URL. Um and so some example of custom applications so there is the t-tail pg stack which connects to the pg stack database to create uh dynamic mosaics using stack items and so when I was talking about uh vector data creating vector
13:42
data from pixels from pixel value you can check out the t-tail mvt uh project uh on restore data and create like vector tile from it um there are other projects like t-tail uh digital twin t-tail rpds or rio vs that use t-tailer and customize it uh for
14:03
their own uh goal. So next on uh on rio tileer of on um t-tailer so the next development will be mostly on the low level like on the low level uh modules like rio t-tailer so we are going to make it more more more faster and so yeah we are going to work
14:27
on that in the next months because we know like sometimes sometimes it can be really slow so so on like when I say really slow it's like almost one second so for tile creation it's still okay. Uh we want to enable a complex post processing so if you
14:44
want if you if the user comes with a complex uh script to run on the tile or to run on the image on the output image we will enable that in t-tailer uh so that's coming in the next few months we want to focus on customization so to ease the user to do whatever it
15:05
wants with t-tailer uh classes um we need to work on the documentation so it is the access uh to anyone um and we also are going to add uh xar support because this is something that uh has been requested by a lot of users uh xar is already supported in
15:24
gdol so technically you can use um you can use t-tailer with xar data already but uh there is a tricky uh situation uh about xar when there are more than two dimension uh we need to read the sub data set of the xar data which is uh it's a bit complex directly right
15:45
now in t-tailer but uh we hope to use uh to to ease the access to xar data with t-tailer uh soon and we also want to add like non-geographic data uh as uh a talk like there was a talk yesterday about uh like oblique imagery uh non-geo diff data yesterday and so we
16:04
hope to add the t-tailer like non-geo data support in t-tailer uh pretty soon uh just on the bottom it's like example of custom uh script that you can you could apply for example on your data if you want to create a hill shade or like contour uh detection on
16:22
your data yeah so just to summarize a bit uh t-tailer is the python module so yeah you don't again you don't need to fork it you can just install it and like player and go from that it make tiles from cloud optimiziative but you can have more than tiles you can
16:41
have preview crop points metadata you can and you can and you should customize it and extend it it's production ready uh using fast api it support mosaics and stack and uh again it's open source and we welcome contributors uh so feel free yeah again to drop by the repo and uh or to to the the low level modules like kio-tailer or mercantile if you
17:05
have issue or question um so t-tailer is part of like uh a family of three project at development seed that we work on uh so there is t-tailer for the rest of services there is tmvt which is like um a vector tile a dynamic vector tile generation from pogess pogess
17:26
and uh we just released t-features which is uh ogc features api um all those three project are using fast api uh using the same style of uh coding so if you have uh checked t-tailer
17:41
there is no like different really big difference between those three project and uh the thing is you can create like one application with all those three project together uh if you want to have like a a set of endpoints that support raster uh features and uh vector for example so it's it's pretty fun uh all right reason in python using fast api um yeah
18:05
that's a that's a hit and if you have more question and and want to check out the the job at development seed you can and yeah and feel free to uh to reach out if you have more question uh either on twitter on on github um and yeah that's it oh wow that's
18:22
one more we have uh a conference we are hosting a conference in washington end of september about uh mostly about static data uh you can talk to bruno about it uh you will be there i guess and so um so yeah it's a great conference uh if you are in washington end of september you can also submit uh lightning talks uh up to uh early
18:45
september so feel free to check out uh this conference and yeah oh yeah i promised more gift in my in my in my keynote there so yeah that's all the gift i could find