Good morning Raise your hand if you are awake almost everybody good Raise your hand if you are not a Debian person Excellent, I'd like to have a few

So good morning, I'm Erica Vini Debian developer and I'm Kind of responsible for that tags, which is the categories Categorization system for Debian packages Okay, the my screen still shows the first slide confusing when you just woke up and haven't had breakfast

So, okay So the point of the talk is that I've recently Rewritten the the whole website that manages the categorization of packages And I'd like to talk about it because I think there's interesting bits and bits that can also be reused by other distributions

So that takes use the category system for them in packages about nine years old now It's faceted classification Which means that categories are grouped in in in in facets

Which is like groups of categories and each facet is homogeneous. Like all the categories in a facet are Represent the same kind of concept For example, there is a facet for What language the package is implemented in so this is implemented in C C++ and there is a facet for

What Language does this do software development so that the categories are like the same but in one case You know, you could have something written in C to to work with pearl

Like the pearl interpreter And then so by grouping categories in in different facets it things make a lot more sense It's a controlled vocabulary so it's not If you are used to tags in like delicious, that's folks on them. You can type whatever you want as a tag

But we went for a controlled vocabulary So there is a central vocabulary and in order to have a new tag you send an email to mailing list and then proposing the tag and then it will be committed in some version control system and yeah, and then you have it and

It's currently 614 tags in 31 facets. So it's rather extensive and and expressive The classification is anonymous. So the data is collected in a wiki like way But then there's a manual review because with There are tags which it's nice to give them semantic value

Like if you say something is a shared library, then you don't want to show it in a package manager but since you want to do something based on tags then you want to have a bit of a review to avoid rubbish going in the database and Firefox disappearing from the package managers because it's not

Because it's somebody tagged it as a shared library. Oh, there's no Firefox in Debian actually management anyway Okay, and It tags the whole Debian ecosystem or we're getting there. It's currently Debian and Ubuntu

and Perfect derivative from Debian, but we are now integrating Debian derivative sensors to have any Debian derivative distribution involved And it currently knows all of about well of 48,000 and so packages There's also robots tagging things, but I'll show that later

So yeah, there's a web interface that tries to make things as easy as possible so when you go and categorize packages you have in suggestions and we're going to see those later and the website I Wrote like five or six years ago was had a custom C++ back-end demon for speech that was queried by

CGI's and I could like somebody said oh, it would be nice to have that feature in the website and I was like Yeah, it would be nice to It's not a way. I'm going to do it on this code base. And so I've

written a lot using Stuff that's Rather standard at the moment than five years from now people say oh you were using tag Oh silly you but well at the moment that's really nice to work on So it's not it's possible to implement all sort of features on it

Okay, so maybe the introduction to depthx is getting a bit long and I should probably show the website

Okay, not that impressive

Okay, this is a list of facets all the groups of tags that we have so we start having an idea of

What we are talking about So this is like one of the main facets which is a role the role of a package in the system It's designed so that each package should have one tag from it and These are the tags. Well, it doesn't fit but

That's the idea. So you have programs and Shared like development libraries shared libraries and it should be kind of familiar. It's the usual stuff you find in a in a package repository

Now just having this These tags is enough to do quite a bit of semantical stuff like you can hide shared libraries or You can say I'm not a developer or

Then we have something which is a bit more Task oriented So there are sets of tags for specific uses like this doesn't apply to the whole

Archive, but it's very nice to provide a view of the games in Debian a new addition. I would like to welcome is the system which is

What kind of system this software is specific for so like a personal information manager for tablets Would be like system mobile and then a configuration management for cloud systems then system cloud and it's quite nice because it also allows you to see

the packages see how Broadly scoped is the package archive So, you know you can go from embedded control devices mobile systems and servers and So it's also nice to have a way to kind of improve the perception of

The packages themselves Okay So that's a browser of the vocabulary We can search For I don't know browser. I

made a little package search system and it's got the Faceted navigation on the right which you may be familiar from shopping websites. This is like a proof of concept

and I do want programs only. Thank you. And there you have it. There's also Possibility of filtering for like show me only desktop applications and that's just done what by a combination of tags

That is not a program So we click on it Not there We click on it and there's the classification editor and And

Browser we get with hypertext capability. It does not sound like a program although it does have a screenshot All that's complicated. So maybe it is a program Although I would like to talk with the person that packaged it. Do we have any pearl team people?

No, probably not Okay Never mind if we decide it's not the program. We just remove program and do submit And that's it what yeah anybody can do that It's Well, let's remove it whatever there's one change change to submit, uh-huh

Okay, we removed it don't Yeah, you can go ahead and clear the whole data set I have backups

So the system will suggest you tags it does suggest program but probably because it had program so it was like Used to be program. Maybe it's not bad idea

It mentions DK So it's a testicle that kind of makes sense and it's also suggest development library Which does makes a lot of sense? It should also say oh There's a chance that a whole program is missing probably because it says that

It's got an interface So it's got the next interface so it chances are it's a program It makes sense There are statistical algorithms that analyze the tag distribution and give you feedback like the shopping

Algorithm things like you both this you may want to buy that as well It's exactly the same thing that runs on tires and you thought it like that. Maybe you want that tag, too Okay, there's pop-ups with more information about tags and it's also

Roll documentation is rarely used in this way because you don't have usually a documentation with an X interface probably So remove tag roll documentation. Thank you And it says well you need to have some role for it

So it does have this smart way of guiding you through tagging. It's limited But it does get you to cover the basis like and and I can use this to give Hints to People who are tagging out of my experience

like if I want to set the target that every package must have a role tag, then I just had a hint for it to check and And it works Oh loads of suggestions here Okay, never mind well we'll get in touch with the maintainer of it. Oh, there's no link for the maintainer

It's funny. Also you are anonymous to the system, but since it correlates Edits together via a cookie I can see my patch history No big deal but and

And since there's loads of checks then I can build to do lists with packages that that fail those checks And these are sorted by popularity. So you see the obvious stuff that needs to be done first I try to make several entry points for people that want to help with the tagging to actually go in the site and

Start doing something. There's a getting started page That has several links to Places where you could do something there's also a tag cloud which are so 90s and it's also so ugly but

it was like it's a prototype this one and you could say yes, I want a whole program and No, I don't want you TVT's and Then you get packages and and you can navigate like that

And if you think there's packages that don't fit you can go to Tag editor and you can just remove them Oh, sorry Sorry, you can just remove edit the tags. Like there's not a program. That's not a program

Okay, never mind They are programs. So It mostly it's experiments of mine to to try to tag from different points of view Sometimes it's nice to take a package and say these are the categories sometime Sometimes it's nice to to say show me all

first-person shooting games and then see if there's something that doesn't fit and Having a flexible site now allows me to do all these kind of experiments quite easily

so Users of that tags that the whole point was back in 2003 that there were discussions on they've been developing Maybe it's like oh, we should stop accepting new packages because they've been is too big. I

Was like no Let's just make that be unable to cope with more packages and So one way one idea was to have a categorization system that could scale because the previous one was like only one section per package and It didn't scale that well It did have a very nice side effect that you can move pointers details away from package descriptions

it's like if you see a package whose description is this is a Utility to with the GTK interface to do something then you can just take away all that and say this is a program to do something and

Then all the detail this kind of technical details that can be represented by categories Then you just do it by categories. Those are shown in package managers. So there's no point in the world nobody Generally you're not interested in what UI toolkit this have but if you are it's in the categories and you don't clutter

descriptions with unneeded details or generally unneeded details There's some semantic aspects of tags then hide and interesting packages is quite useful Recently, there's been experiments or for linking the hardware categories to limited hardware

Detection so you could have a feature like you want to install this package But you do not have a DVD burner. Are you sure you want to continue? or I Have a DVD burner show me packages that can use it or something like that

It's an experiment but it's nice I find it interesting to Attach two categories some code that can kind of do or guess something about them There can be domain specific package managers

Let's see if I have go play installed. I've recently reinstalled my It's like a package manager for games which shows you games and you can choose what type of game you like to play and what interface well and

I want to play text-based interactive games and you have a list so I can on the terminal I can play

these kind of games or There's something wrong here, oh It was both there's something wrong going on there's more interfaces there's like 3d interface at least I'm not going to debug it now, but it's a general idea and

To play with things you could do the same thing and say go admin and Have the same but system administration tools Obviously you'd like to change the interface the screenshot isn't that useful

Maybe you want to give more descriptions of packages or something But it's it's kind of a prototype to hint at the idea of having task specific package managers The their surprise of in what you can do with task specific stuff that we recently found out that

Debian is like the most widespread operating system for neurosurgeons and there's like a Debian booth at the world's leading neuroscience conference and And it's very big there and like what? But yeah make sense and

You know that could be catered for Okay, and then as we've seen search functions that allow you to say yes, I want programs and yes I want them graphical and I want to work with this kind of file you can

I did something Well, there's the mime type database for that but you could Until we extract the mime information from packages in an index You could do something that points at the file detects the mime type and show all the packages that can open it

Okay, that's skip it late So a tour of the website We've seen a bit Okay

Their statistics we can see

Okay, the numbers are similar to what I've been showing before it does mention tagged by human and tagged by robots and this is a So those are tagged by

Humans and those are tagged by robots. Yes. No, yes That Means when a package is first seen by the site It's marked with a tag that that says this hasn't been reviewed by a human yet

And if you are a human and review it, then you're supposed to take that tag out So So the system can tell whether it's something that's under viewed by nobody or not Things that no human has touched yet

There's all sort of heuristics that are taxed to them like look at the package name and say well that looks like a per library Tag it per library something like that. So there is a basic of tagging Throughout the system, although it may not be perfect Once you remove the marker, then the robots will respect the higher intelligence and not touch the package anymore

And So they won't mess up the tagging after you reviewed it. Hopefully it's a higher intelligence, but Generally, it is. I mean system seems to work and And that's like the number of tags per package

And so you got like loads of packages with few tags and then very few packages with loads of tags Never mind. Not that useful except as a that's the list of Packages with the outliers packages with loads of tags and squid happens to have loads of tags

Anyway and That's all that the the automatic checks that are run on the system and but let's go back to the editor

Squid is a bit insane. I'm not sure it should Well, most of these things actually applies Anyway, there's the suggested tags and there's like all tags then you can

Tag that way You can search tags so and

You can search pretty freely those don't need to be keywords in the tag descriptions because I did something weird There's load images and it did say works with images It's it's a bit contrived the way it works, but it's really quite nice to

To find tags you need by just saying I want something kind of like that Because I cannot pretend that people actually know all those 700 tags. I need to provide ways For people to search them or for the tags to be suggested to them

And then I can do patch review which is the process to

Bring those submissions into Debian

Okay, these are the paths that have been submitted there's an example of somebody that doesn't do doesn't use Cookies in the browser Which I find annoying from this side of the

Barricade because then I have like Several edits of mdb tools when it's obviously it's obviously the same person and so it could have been shown like this So that's somebody who's been like tagging all their packages and and I can easily review it And say yeah, that makes sense. That makes sense generally. Yeah

Definitely looks like a sane person to me. No need to see them all Why it's not hardware video that one It looks like it is actually so yeah

But it seems to have that he or she seems to have a point there I don't know enough to be able to tell so I'll just leave it like that and let somebody else fix it

There's the password submit and then yeah, and then I can say submit somewhere And that should go in Debian after a couple of days

Eventually, so the workflow is actually this one Using cookies they are grouped by the same person. So basically when you review you can just make a judgment like

Sane submission or a person that was trying the website by clicking every button That's the two kinds of submissions we get and if it's saying then you just say okay and then Once that's reviewed it gets committed into version control system. So I get historical backups

And then eventually uploaded to Debian

Yeah, it will be at the bottom of the list of stuff to review There should be actually We removed the Lib TK pod Perl, that's the edit I did

And I can say no that person's totally insane There you go, so I can actually amend the changes before submitting them. Well, you can edit it again

yeah, there's no undo but There's an undo before sending So before you send

Let's go mess up Firefox. So there is Firefox in Debian after all. No, it could be in Ubuntu So I've edited something and it says there's three changes to submit Actually, no, it is a web browser and it does work with HTML and it's okay. Okay, then nevermind

so currently the review tags go into the archive and then And then

Packages are only tracked by name. So every version of that package will get those tags It works 95% of the times there's all the cases in which it breaks so The current tags of chromium do not actually apply to Debian stable

because or git It used to be called git core because there were such things as GNU interactive tools In the past that was called git and so there's been a name transition and so It doesn't always work to track stuff by name only

You'd need to track them by version, but it gets too messy So the idea is to allow tags to be overridden inside packages But that doesn't work yet There's a new workflow proposed, but it hasn't been discussed much. It's just an idea of

I'm trying to move to a way in which a package maintainer can Override stuff from that text Debian net if it makes sense But yeah that there's going to be more of that possibly in Debian lists once I managed to talk it through with FTP masters and

Provide some patch to the archive system and so on There's QA To-do list that we have seen we haven't seen the one per maintainer

So I maintain packages and so I have a page for myself with the packages. I maintain and to-do list. Oh You bastard go and do some work So it Indexes several distributions, but I can say I don't feel like tagging anything for Ubuntu today

And there you go. I hide all the Ubuntu packages and I also hide the Debian packages and Only there is a package, but that's a glitch that is going to be fixed There's going to be so many more distributions in this box and I have to figure out how to lay them out

I think there's about 50 in the Debian derivative census That's going to be interesting And so yeah, these are all my packages and then One that makes sense to

tag Aha this is an example. Can you read this is an example of a case where?

The hints are wrong That's a display manager Which just logs you in it doesn't ask who you are it's configured like for kiosk machines You could just say well when the system boots open a next session as this user without asking any password. I'm a kiosk

I don't people can't log in via You know a touch screen and then they don't need to went that it only runs a browser or it only runs the control interface for Industrial devices and it's that kind of display manager

So it is a display manager, but it doesn't have a UI toolkit because it doesn't open any window And in that case that the suggestion is reasonable but in this case wrong That there's loads of corner cases and that's where being an intelligent human makes sense. I can choose to ignore that

Whereas if it were automatically tagged then the system would just add it and it wouldn't be correct Because it wouldn't know which UI toolkit to use but anyway, so personal page That page is linked from

Like QA dot Debian the org and all the other per package report pages So one can have a look at the sort of personal to-do list in Debian and then there's also going to be tags

and then there's a bit of internals like It does export the whole tag set There's links these are just download links and

You can download all the under view tags and all the review tags the whole data set. It's available for download There is well these are like used by the editor and so on the vocabulary is a static file It's a static JavaScript file with all the descriptions

so I load it in the tag editor so browsers can cache it and Description don't need to be loaded over and over again Certainly things like that These are the the code with all the checks that are done that all the hints you get obviously There's also the source code to download over here

And so Using these links you can just set up Chrome jobs that upload that download the new version of the database and you can get the review categories for Ubuntu for example or any other distribution that's covered by this

Or you could just download this convert it via distro match This is a tool that can map package names across distributions And then you get tags for your non Debian style distribution as well. There is a bit of a web API

Which is used by the various pages One this one and it's documented a bit So there's like an API to submit patches to attack changes and that shows how to do it

And it will apply saying if the change was okay if it changed something and Old and there's also other API's for searching packages, but I haven't published them yet because

They're not that stable But I intend to do so and these API's can be called from anywhere there's a cross-origin resource sharing headers so that you have a any JavaScript from everywhere can call these and send patches and Score all the data. No problem. There's backups. I won't go that much into the internals

There's a bit of internal I'd like to show Dependencies It's free Python packages Python Debian is can be pure Python So there's no particularly that mean specific dependencies, it's Python

jungle sapien and a module to read some YAML like files It there's a configuration filed That you used us to put parts

And That's it you can run the server There is a procedure that downloads package information Since it needs to cover Debian Ubuntu and so on it does not make any use of the local apt database

For example because well, there's no up to database that can cover all Debian and Ubuntu And so it's it's quite easy I mean, it's definitely possible to pour this to another distribution You just need to export your package names and descriptions at the moment

It will import them in a yaml like format that's used by Debian It's not that hard to change the exporter if if you just don't want to convert it and want to read your native format and That's about it update data

update data Downloads package information source package information to see who are the maintainers?

Popularity contest is optional you can do without a Seed of stable tags, which is also optional and the vocabulary information and then runs the maintenance script that will index everything and

Update a PTX up in index that will also index everything and that that can also be run from other distributions, there's a wiki page showing how to have an X up in index of packages for non Debian distributions

And Also the theme the The graphical theme of the website is a pluggable Django application. I created the generic Django up

for the Debian website layout and so everything else just Derives from it and doesn't do any particular layout themselves. So it's rather easy to just switch that With a page that has a different branding If

Someone from Ubuntu would like to do the work of doing the branding then I wouldn't mind doing something that Says that if that says if it comes as that takes dot Ubuntu comm then Show different branding that can easily be done jungle. So I support for it. So yeah, that's

That's what you can do if you want to set this up on for non Debian distributions I'm happy to assist. I'm not happy to do the work for you. Obviously because I can't manage that takes installations for everyone, but

Yeah, I can definitely assist and then if you set this up we can trade submissions submissions are there's a patch queue of stuff that is submitted and I can just send you my Reviewed changes you can send me yours or I can just download your new database and make a difference from the previous version

and So yeah, things are tradable and shareable and stealable And that's the work I've done on my Christmas holiday, so I think I deserve a substantial applause

There should be limited time for questions

Okay, how many the question is how many reviewers do we have for the Debian version of that takes and are we looking for volunteers? So to submit patches to submit tags like the anonymous submissions

It's difficult to count them Possibly every maintainer can just take care of their own packages to review submissions then it's two of us at the moment I'm not yet looking for volunteers because I would like first to

To allow people interested in one field to review only the changes for that field I'd like to to have the the Debian game team Be able to review the game submissions then not have to bother with other things But I don't have that yet, so I'm not reaching out I can manage loan actually it's

Less work than these spamming your inbox So you can do it once a week, and and you be spam your inbox every day, so

Yeah, so the question is

Can I Search for tags using like after No, I didn't understand the question then sorry yes, oh

Okay, I get it right It's a good point so the what the purpose is to be able like if if a package contains four binary

executable files that to tag each one indeed because they could be different how do you oh So they could show in the application menu I tried reaching out to the free desktop years ago saying well, there's also those categories and

There's there's a category system in dot desktop files, and that's been standardized and There is already a category system for application menus

so these already categorized and I didn't sign that I may or may not like it, but it's already a standard so I Don't think nobody would listen to me if I propose something else. There's also talk about removing these kind of menus

anyway, so No idea. I did make something experimental that shows binary executable available in the system by the tags of the packages they are in it's obviously bit fuzzy when these things are different like a program that a

package that provides main application and set of utilities obviously The two have totally different tags one can be a command line tool and one can be a GUI application, but One can split the package if that becomes a problem or the no

Am I running out of time somebody should throw a shoe to me if There was a strain sign there. Yeah

Okay there it's Technically easy to do It hasn't been done yet There's Very good people that intend to do it

The point is that Tags have a silly name, which you don't want to show users like this one This one could just be used as a handle You don't translate this Everything else but then that's a string you show to users and so technically it's trivial to translate

But then I don't know much About get text so I'm not that good at doing it myself Other people may not know much about that text and we haven't yet Got together with enough

Common push to do it. But yeah, it's possible. I'm not sure I'm going to translate the website Probably doesn't make much sense to translate this website, but it does make sense to translate it for Applications definitely like you do want software center or whatever to show

Those kind of strings in your language because that software is translated there is yes

You can do something like

It does have Boolean things and it's fast. It uses a local. It's based on the local Which does index

Okay, I guess