Modernising FreeBSD package management
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 24 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/19206 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
1
3
6
8
9
10
15
16
17
18
22
23
00:00
VideoconferencingRepository (publishing)Video trackingData managementBinary fileFile formatLocal ringDatabaseLibrary (computing)Network topologyOcean currentOnline helpDefault (computer science)MetadataDirectory serviceScripting languageInformationNormal (geometry)Different (Kate Ryan album)Reverse engineeringCASE <Informatik>MereologyDirectory serviceFreewareScripting languageNetwork topologyPhysical systemBinary codeCategory of beingGrass (card game)File formatSoftware developerRevision controlTheoryBitStudent's t-testData compressionDataflowLocal ringDatabaseDefault (computer science)OrbitAuthorizationMultiplication signSummierbarkeitData managementData managementLibrary (computing)Form (programming)BefehlsprozessorComputer programmingCodeElectronic data interchangeString (computer science)QuicksortCapability Maturity ModelServer (computing)Execution unitCache (computing)Real numberMetadataRepository (publishing)Ocean currentCurvatureComputer fileTrailHacker (term)Installation artXMLComputer animation
07:43
Reverse engineeringCalculationBackupDatabase transactionDatabaseLocal ringRevision controlComputer fileLine (geometry)BitDebuggerDatabaseKeyboard shortcutBoom (sailing)Cache (computing)Set (mathematics)State of matterFront and back endsComputer configurationAreaFunctional (mathematics)Vulnerability (computing)Equivalence relationLevel (video gaming)BackupMetadataProcess (computing)InformationRepository (publishing)Theory of relativityGroup actionRootTranslation (relic)Library (computing)Scripting languageReverse engineeringCalculationMultiplication signPlanningDirectory serviceBasis <Mathematik>Single-precision floating-point formatDefault (computer science)File systemFormal languageRegulärer Ausdruck <Textverarbeitung>Forcing (mathematics)Standard deviationExistential quantificationForestComputer-assisted translationRepresentation (politics)Normal (geometry)Computer fontSummierbarkeitBridging (networking)ReliefElectronic data interchangeFile formatOrder (biology)Sampling (statistics)Sound effectForm (programming)Arithmetic meanTrailComputer animation
15:20
DatabaseInformationMaizeDualismLetterpress printingPlastikkarteElectronic signatureData compressionRepository (publishing)Scripting languageComputer fileData integrityRing (mathematics)Default (computer science)Installation artBinary filePhysical systemFunction (mathematics)Computer configurationRevision controlMultiplication signComputer fileConsistencyDefault (computer science)Functional (mathematics)DatabasePerformance appraisalGastropod shellString (computer science)Query languageMetadataLibrary (computing)Repository (publishing)Electronic signatureDifferent (Kate Ryan album)Electronic mailing listSoftware maintenanceBinary codeCombinational logicCuboidRemote procedure callSet (mathematics)BitInterface (computing)InformationCache (computing)Meta elementLattice (group)2 (number)SpeciesCalculationOrder (biology)Scripting languagePhysical systemMultiplicationMathematicsHeegaard splittingRegulärer Ausdruck <Textverarbeitung>Software repositoryOnline helpWhiteboardView (database)Software testingHypermediaState of matterFrequencyAuthorizationTheorySheaf (mathematics)Slide ruleEstimatorExpert systemTraffic reportingLevel (video gaming)Forcing (mathematics)SpacetimeOpen setTelecommunicationElectric generatorGradientIterationWebsiteCircleFault-tolerant systemComputer animation
22:58
Extension (kinesiology)File formatLevel (video gaming)Network topologyMultiplicationMessage passingError messageInformation securityAbstractionRepository (publishing)Library (computing)MultiplicationMessage passingDigital rights managementWebsiteRootLocal ringFunctional (mathematics)CASE <Informatik>Multiplication signComputer configurationLattice (group)Bootstrap aggregatingPhysical systemCuboidFlow separationSoftware repositoryType theoryComputer fileServer (computing)Computer architectureGroup actionBinary codeRevision controlPoint (geometry)Software bugLevel (video gaming)Directory serviceOpen setInformation securityDefault (computer science)Software testingInstallation artTerm (mathematics)MetreParameter (computer programming)Insertion lossResultantUsabilityService (economics)Goodness of fitBit1 (number)WordAddress spaceComputer animation
30:35
Term (mathematics)Task (computing)SpacetimePatch (Unix)Open setCurvatureFunctional (mathematics)Letterpress printingVariable (mathematics)Physical systemCASE <Informatik>Repository (publishing)Data managementMiniDiscDefault (computer science)CuboidIntegrated development environmentUniverse (mathematics)Computer configurationKernel (computing)Student's t-testRevision controlInstallation artMetadataLink (knot theory)Regulärer Ausdruck <Textverarbeitung>BitSystems engineeringComputer filePartition (number theory)Fault-tolerant systemPower (physics)Sound effectMereologyVideo gameGenderWave packetPhysical lawDataflowForm (programming)StapeldateiEnvelope (mathematics)System callSource codeJSON
40:21
Software developerForcing (mathematics)Default (computer science)MereologyDirectory serviceState of matterHeegaard splittingBitLibrary (computing)Multiplication signLevel (video gaming)Network topologyComputer fileComputer animation
43:20
DemosceneGraph (mathematics)SpacetimeRevision controlEvent horizonView (database)Reverse engineeringProcess (computing)Group actionMultiplication signDirectory serviceTraffic reportingDifferent (Kate Ryan album)Physical systemPhase transitionScripting languageCanonical ensembleGoodness of fitPhysicsPulse (signal processing)QuicksortLink (knot theory)Category of beingFunction (mathematics)Library (computing)GenderCASE <Informatik>VideoconferencingGradientComputer fileDatabaseInstallation artBackupState of matterRollback (data management)Web pageNetwork topologySoftware bugSet (mathematics)PurchasingArchaeological field surveyElectronic mailing listSource codeJSON
50:49
Reverse engineeringFile formatComputer fileMultiplication signComputer configurationNetwork topologyData compressionRevision controlBootstrap aggregatingDefault (computer science)Physical systemScripting languageDatabaseRepository (publishing)Electronic mailing listTouch typingLine (geometry)Observational studyInstallation artParticle systemSheaf (mathematics)QuicksortSet (mathematics)Arithmetic meanCategory of beingWordData conversionEuler anglesConsistencySource codeJSON
58:18
Dirac equationForm (programming)Execution unitDivision (mathematics)Directory servicePrice indexDirected setoutputSurjective functionIntegrated development environmentMultiplication signRepository (publishing)Line (geometry)Link (knot theory)Scripting languageComputer fileInstallation artInformation securityHeegaard splittingExtension (kinesiology)Network topologyRevision controlLevel (video gaming)Directory serviceDrop (liquid)Office suiteFlagConfiguration spaceTrailAxiom of choiceDatabaseRegulärer Ausdruck <Textverarbeitung>Software maintenanceWordMereologyQuicksortMatching (graph theory)Arithmetic meanTerm (mathematics)InformationAreaSet (mathematics)Image resolutionGroup actionExpressionFood energyFitness functionSource codeComputer animationJSON
01:06:22
Factory (trading post)Source codeComputer animation
Transcript: English(auto-generated)
00:07
Hello everyone, thanks for attending. I'm Baptiste Darusson, I'm a FreeBSD developer, I'm a member of the port management team, and I'm here to speak about the new package manager we are going to switch to on FreeBSD. So first, why did we choose to create a
00:27
new package manager? If you have a look to the tools we use now, you have a look to the code and you see some comments like that saying it should be changed, things have been written fastly and should be changed soon. It was about 20
00:43
years ago and it never changed. The tool we have currently has no safe upgrade support, so no upgrade support at all by the way, so if you
01:01
want to upgrade something you just remove all the package and reinstall them. It's missing a lot of metadata, you have a lot of information we can have out of the ports that you can't have when you have your package installed, like if you want to know what is the upstream website, if you want to know
01:20
what's the license of the package and stuff like that. It works pretty well with dependency tracking. It manages somehow to be able to track the dependency when installing the package, but if you are on removing a package there is some ugly hacks to be able to determine that the package is used
01:43
by another package and stuff like that. If you ever tried to run a binary-only package management on your you'll understand that it doesn't work at all and there are many other issues
02:04
with the current package. Package-ng is adapted to fix all the stuff, so the first thing we have done is creating a new package format that fits all the new metadata we want and that will work better with new
02:23
compression things. We rewrote the local database, it's no more any flat files and stuff like that, we have something consistent which is able to track dependency and reverse dependencies. We tried to wrote a simple and easy to use library, so to be honest the library currently isn't the
02:44
API, the library isn't considered as stable, we will consider it as stable for package-ng2 so that we can have some maturation of the library. We wanted to write a single binary so that you don't have to remember its package
03:03
underscore things or that you just have package which is a binary that deals with all the all the packages. We have binary repository support not just something pushed on an HTTP server and then you try to fetch the first
03:21
package and discover oh I need that, I need that. We have a real repository support with metadata you can fetch and you can search on your metadata, you can gather a lot of information out of that. We have binary upgrade support. One of the most important things was to be able to work
03:41
with the current ports tree so that we can just create the whole packages from the ports tree without having people needed to actually modify their ports or stuff like that so that we can now be able out of the ports tree to create packages for package-ng and for the old package
04:01
system. We have added to package-ng a lot of things that will help to improve the ports tree in the future. The one we will drop the support for package install will be able to add a lot of new features people are expecting for a long time in the ports tree just because the package tool do
04:20
support them. And it helps modernizing the way you can manage package for your desktop or for your servers or for stuff like that you can just real managing system on that you can create some script on top of puppet, on
04:46
The new format is still a tar file. It's compressed. We support all the compression that libarchive supports. In the old format you had a lot of
05:01
plus files which were the different metadata you can have. We have just unified them into a single one which is written in YAML because it's easy to extend, easy to read if you just extract it, if you it's human readable. We are able
05:22
to deal with empty directories. If you ever try to create a package that just need I don't know a cache directory somewhere then you can't package that cache directory because it's empty by default. So you'll have to install the package and then mcatheo5, the cache directory. With package-ng now
05:41
you can prepare all your empty directories. You don't need to have any entry for that or stuff like that. We have splitted the scripts in pre-install, post-install and stuff like that because with the current package tool
06:01
you have one script which is install and one script which is then install and then you have to write the code to say oh it's pre-install or it's post install. Now you can just separate that so that the scripts are easier to write. We added a new kind of script that doesn't exist in package
06:22
install which are the upgrade script because sometimes you want to do things after an installation and when you upgrade you don't want to do the same things. And sometimes you want to do things during an upgrade like I don't know, migrating your data from the old version to the new version and stuff
06:44
like that so that you can do that into the upgrade script. We had to reinvent the wheel about ABI because we want the package installation to be safe. So we want to be able to determine when we're installing, on what
07:00
kind of system we're installing and if the binary are compatible with that. The problem is with all those embedded CPUs you have a lot of different ABI available so you can just say okay this package is for 3bsd 10 and your CPU you have to say it's 3bsd 10 my CPU it's 32 byte 62
07:28
byte or it's little and young big and young it uses the old ABI the new ABI and stuff like that. So we just created the new string which is which you can extend if you have a new things to add to be able to determine a
07:43
new ABI you just had it at the end of the line and when package.ng try to install the package if you have a package that could work on all those ABI at the time you just have some globings option on it so you create a package and you say it's for all 3bsd 10 because it's just
08:03
shell script so it will work everywhere you write 3bsd 10 and stop. The local database we decided to use SQLite for the local database so that you just have a single file with everything it's relational so we are
08:23
able to do easily and really fastly the reverse dependency calculation it's easy to backup you just take your file and put it on your backup and you can get it. We also still provide a tool to create a backup which is not an SQLite file so you can if you don't trust SQLite you can still
08:45
backup it and restore it. It's safe we use translation everywhere so that if something goes wrong during the installation during an upgrade or during something like that you won't have broken database anything will just
09:03
roll back to the previous state which was okay. It contains all the metadata you can have out of the packages so that now you can know when the package has been installed you can know which option has been used when the package has been built you can know a lot of things like that.
09:21
It's extensible we we can create our own functions so we just don't write simple SQL to create database we have function for us saying for example I want package with versions higher than this version which is not just
09:42
mathematic it's a bit more complicated so we just have a version compare function which is mapped into the SQL and we can extend it as much as we want we can safely upgrade that if tomorrow we change the information we want in the database we can just determine that the local database was from an
10:04
older version and do everything we want to be able to upgrade to what the new database should be. The library itself we try to push all all the things into the library so that if you are if you are willing to create a
10:23
front-end which is different from the one we provide you don't have to deal with anything with about the package or about upgrade about the repository everything is boom body inside the library you just have some high level functions for you it's written in C so you can write easily bindings for
10:45
any language we try to keep it safe all the action I don't in the library so you you just prepare your action and you say okay from now do the job you
11:01
don't do it in your front-end we try to maintain a simple API I think we managed to do something great in that area I think the API is really easy to use but it could be a lot simplified so we will we will simply simplify that for package ng2 the package command is the front end you have to install the
11:27
package by default it have a lot of sub commands which are all the actions possible on your package so you have package add which normally a normal user won't ever use is just to add the package out of a repository you
11:42
have package somewhere in your file system you that do package add this package this is mostly to be able to scrape some things or to for the pottery to be able to add package and things like that but it's not designed for end users package audits is there to allow to for you to to
12:01
check your local packages against the known vulnerabilities you can have it's the equivalent of port audit so it uses exactly the same the same files to to get the vulnerabilities so if you switch to package ng you don't need another sub passage to do that because in package ng we are able to
12:24
track if you have installed some stuff as a dependency of other things we are also able to remove all the offense you if you just remove Firefox for example you have tons of dependencies that you don't need anymore you just do package auto remove and it will remove it for you package backup is
12:43
just to extract all the things that you have in your local database into a tar file plane of yaml files so that you can just back up and restore them package check is to be sure that your local database is consistent if you
13:01
decided to remove some packages which are dependent and you force the removal and then you discover you still need it you just do package check it will discover all the all the things that are missing and propose you to reinstall them it does also other things like checking for the checksums to be sure that you still have the files yeah it does it does that so you
13:31
package check minus s and then it check all the other checksums of the files to be sure that this are still the one you had with your your package
13:45
you can do that on package basis on all the database you have switch minus a all the database just my name your name you can do globing you can do regular expression it can also recalculate all the size of the package imagine you have modified something you are happy with the file
14:04
you're modified but you still want to know how much size is taken by by this you can just recreate or recalculate all the the size taken by the package package clean is there because when you install from a repository we have cache directory with all the package that has been downloaded and then you
14:24
sometimes want to clean that up to avoid having all the old packages you don't use anymore so package clean will just clean up your cache directory package create I think I don't have anything to say about it package delete the same oh yes package create has two ways to create
14:42
packages you can create packages out of your local database but you can just provide it a fake root somewhere with your files in it and you just give it a p-list and it creates the package out of that so you don't have to have the file inside on your file system to be able to create package package fetch is basically if you want to do to prefetch every
15:04
package you want to install you can just do package fetch and then you have all that in your cache and you can reuse them outside package fetch can also be used to mirror only a set of package you say I just want those
15:20
package I package fed them it will fetch the dependency you take everything from your cache and you can create your own repository which just package you want package info is pretty straightforward package install is the new the new way to install packages you don't add them you you use
15:41
package install and package install we ask the remote repositories for the package you want and it will just do the calculation to be able to know what dependency has needed if your package was already installed and there is a new version remote it will also automatically detect it and do the upgrade for the given package package query is mostly like package info but
16:04
designed for for scripts or for stuff like that you have some kind of evaluation string when you can say I want only this is this and this information formatted like that and I want only those package if this meta
16:21
is like that and this one like that I say a bit more about it later package search is pretty straightforward package set is not to be used by normal users it's just to allow you to modify things in the database without having to open the database itself so you can for example change you can check for
16:43
the origin of a package saying package says minus oh this is my the new origin of this package and package ng will try will think that the package is not the same anymore package register is done is only to be used from the from the pottery you can also use it if you are installing by hand a
17:05
package you're completing it by and is you're completing something by hand you install it and then you say oh it would be great if it was tracked in the package just create a pally at least and you say package register please add it in my package database package repo is the command to create your
17:23
database for the remote repository so you have bunch of packages package repo on it package shell is basically a SQLite shell you can open on the local database if you want to directly do things on that it's not recommended at all but it's still available you can help debugging also have all the
17:44
functions we have added to SQL already in it so it could help package shell libraries is we have an option which is off by default so that the package can track all the shell library it depends on we analyze the binaries
18:02
inside the package and we discover all the shell libraries it will try to load and then we can have a list of them so if you need to know exactly which shell library not which package but which shell library is needed for your package you can just list it using that package update is just to fetch the
18:21
lattice metadata package upgrade is what is recommended each time you have a new version available for your packages you just it's recommended to just upgrade the whole set of package you have you can do this package by package using package install but it's recommended to upgrade everything so
18:41
that everything keep consistent package version is the same that packet the old package version package which is just you have file somewhere you don't know where it comes from you say package which and it will say this is this package that installed its own as I said we have a lot of more metadata
19:00
that we we used to have before so if you install for example XPDF you want to know everything about it you just use new minus F option to package info and then you can have all of this you can also you can also see
19:20
that all the option used while building the package are tracked in the package itself you can have the same information from the remote repository so that before before installing different telling a given package you can check if it has the option you want you can also know the
19:42
size of the package and how much things will be needed to be downloaded the size on your system this is the to query interface so we have the normal query which will query the local database the remote query that will query the remote repositories with the remote queries you can specify to
20:07
query only a given issue of multiple repositories you can just specify I want this one what you can see is that we have an evaluation street string where you said I want to check all the first query is I want to check
20:22
all the package maintained by me and I want to print the output like the name dash the version and then the comment on the remote for example here I want you to check all the package which maintainer is globe with
20:42
bat in it and where the name is different from fossil you can have a lot of combination like that and you can create a lot of things about it if you don't want to write your evaluation string you still have minus a to create the whole thing you still have in you still have regular
21:01
expressions the repository themselves are also an SQLite database we have the ability to have a signature you can sign them you can also not sign them if you don't want any new signature checking you can do you can
21:22
use a lot of different repositories it's still considered as dangerous because we can only grant you the consistency of the packages on for our for one single repository but if you want to test I don't know the latest Firefox version you just add a new repository with the package for the latest version and
21:42
you can test it out of box most of the data the metadata are present in the remote repository in fact the only one that is missing in the remote repository are the list of files otherwise the repository will be too big we are thinking about adding a second file a second repository with list of
22:01
files so that if you don't know which package to install that provide the given file you just query that file but deactivated by default it's compressed and it's searchable so as I said just earlier both command install
22:21
and upgrade can upgrade packages it's not it's real upgrades it's not just the installing the previous version of the package reinstalling the new one it's being able to only remove the files that won't that will be will
22:41
need to be removed it overwrites all the other files it is able to determine that a package has been splitting in two species and not have conflict when doing that and order this right way it has a lot of people wants to be able to I upgrade an Apache and I want then to I want to
23:06
automatically restart it in the end of the upgrade so we have a support for that which is deactivated by default because I don't like it I think it's too dangerous but a lot of people are asking for it so we had it it's you
23:21
just you just have an option to say okay I want it on how to upgrade your system basically you just have to package update to save sliders the lattice meter data and then you package upgrade and it it's done with the next version that will come in few days you don't need to run any
23:43
more package update it's automatically done by package upgrade without any modification we are able to work on the current pot 3 on pointy hats we are able to run about 22,000 packages which is what we are expecting with the old
24:05
package so we are almost equal no no problem we also added some new features that will help in the that will help to create to add new features to the pot 3 itself for example if you want to add a new keyword we don't
24:23
have to change the package tool so that it understands the new keywords inside the pot 3 we can just create a small yaml file with the name of the keywords and the yaml file just have some actions which are function inside of package ng so case drm will just treat the arguments of the
24:45
keyword like if you have written in your p-list add drm and then it will add to action while in post install one and in post the install one of the features we want to see in long term the pot 3 is the ability to have a
25:02
stage directory package ng is already aware of that so you can just say instead of my files are already installed on my system my files are in this fake roots the roadmap for now package ng has will we still live in
25:24
the pot 3 it will never go into base but we need base to be able to in to have package ng so we created a bootstrap which basically is just a package binary and if package ng is not installed it will try to fetch it
25:43
from the official repository and install it with itself so it pretty much straightforward is you just install your new system type package install some package ng is not there it will fetch it install it and do the installation you asked for it after it's installed you you have to specify a repo yeah it's
26:11
by default it's it knows what are the official package the official repositories so it's automatic to have to specify anything but if you if
26:23
you want to specify your own repository it respects package site package root and stuff oh yeah user local yeah of course it installing user local yeah
27:12
yes that's basically what it does what still need to be done to one oh
27:21
is improved a lot our user messages we still have a lot of messages which are mostly technical warnings like the conflicts SQL SQL conflicts or stuff like that which are good messages in which are good warnings we know that
27:41
if we are at that point there is a problem but for the users it's really not useful so most of the time is just changing detecting that problem and instead of printing what SQLite say to us or what LibRK say to us just having a user-friendly message so the more people are testing package ng
28:01
right now the better the message could be we have a lot of fixes recently in that and we need to have a lot of testers to be able to find as many bugs as possible we are really close to this candidate and I want to be able to have like a new package ng 1.0 for the summer so the more people just
28:25
report bugs and the better we'll have 1.0 release for the future we have will soon open package ng 1.1 tree so that we can hack on the new features and
28:40
we have in my lot of things one of the things is a security in general and capsicum is one of the things we want to work on about security we want to be able to add better architecture support it's not directly linked to package ng because package ng is able to to have globing when testing the
29:05
architecture but we want to be able to separate the packages into this is not architecture dependent this is just AMD 64 this is just I don't know what so that we don't have to to duplicate any packages we are thinking about
29:26
something like a provide feature so that if you have nginx install and something needs an HTTP server it won't try to install you Apache because you just wanted at that time nginx it still need to to be design
29:44
because it's quite complicated to deal with but it's something we really want to see we want to be able to split the packages so that we just built one package and instead of having all the dependency for that we can just say
30:03
okay this is just the x11 shared library push that into separate packages and stuff like that and we want to have multi ABI package so this is mostly for the port 3 more than with package ng but we want to be able to
30:22
install for example wine32 on the AMD 64 box and stuff like that before going to the question I don't know if I have time left yes yeah so I have prepared a simple demonstration to show you some of the features of package ng
30:49
yeah I don't know where we turn off the lights yeah so what I what I will
31:29
do is create a vanilla jail out of nano and install Apache on it with package ng so it's done on this small EPC box so it could be a little bit
31:43
slow but we'll see so I just prepared jail what we need to do once the jail is
32:09
prepared is to create a package.conf file with the link to the repository it could also be just an environment variable we check for both so the disk
32:31
is a bit slow port master is just to deal with the port 3 with package ng you don't have to
32:52
deal with the port 3 at all but we have yeah we have a patch for port master then it can work on top of package ng there is also a patch for
33:05
port upgrades which will should be committed pretty soon that port upgrade will also support package ng and most of the tools that deal with the port 3 have patches to work with package ng out of box so I start the jail so the
33:27
first thing to do is to fetch the metadata so I just we have an option to deal with jail directly from the host if you do minus G the name of the jail then package ng will run inside the jail without having the
33:41
need of having package ng in the jail sorry yeah yeah it doesn't care in fact
34:06
it just use the jail function so if jail support it it support it you just have to have the the compatibility in your kernel and it's okay so now we have the metadata we are able to search for example apache so
34:23
by default the search is using regular expression so that you have everything that contains apache on it but you can switch the search to have globing or to the search to have extended regular expression we'll just go to install the latest apache so when you install you don't
34:47
need actually to to do to give the full name you can just give apache but in that case we have two packages named apache so we have to specify the version or if you want you can also here add the origin of the package if
35:01
you do www apache 2 2 it will works the same way currently with 1 0 you just have a conflict which will be print out to the user saying you have to choose one we are expecting to have more a smarter solver in the
35:29
future but it's quite a complicated task you have we have a lot of possibility for that there is the solver from open suzy which is BSD license and works pretty well there are also some university students that are
35:45
willing to work on this task so I pushed this to package ng2 so before installing anything it will show you all the dependency you will have to install it will order it for you it will say to you okay I need that this
36:04
space currently it doesn't check if you have sent this space available but it check if the download space available is okay so that you don't start to fetch anything and then it stops saying no space left on the ice in the future we want to be able to check also the space available where
36:23
you will install the package but it's a bit more complicated because we don't know in advance what the partitioning you have done we couldn't know but we have to do things for that you can just do dash y if you don't want to have the prompt yes there is yeah popmaster can can be fully
36:55
automated you can have some yeah in package.conf you can had assume yes
37:05
and then you're done you can also do it as a viable yes every options we have in package.conf could be either in package.conf or an environment
37:21
variable so that if you're in the system you're running everything out of puppet for example and you don't want to push any package.conf on your remote host you just add the environment variable and then it's okay yes
38:28
for example done a much wiser it's a different approach
38:42
generally a PBI is just a flat package without dependency so just create a flat package ng package without the dependency and then you'll get it I've been I've been able to to create flat packages quite easily with package ng I just use the package that are already built and then I extract them
39:04
and I pack all this and I say this is a new package which is a flat one and everything is bundled in it so I don't need any dependency and to work yeah I
39:33
prefer the package manager to still be able to do both yeah as a system
39:45
engineer I'd work I prefer to have small packages and dependency and stuff like that than flat packages but it's true that if I want to provide to any customer package with something and I want this to work I will in that case do flat packages and give him a flat package saying okay it works so it's
40:03
different approach we need both I think so now we have installed Apache we just need to figure out if it works forget the name of the jail so I can start Apache and I see that it works
40:23
so we can install the package and actually see that it's not just installing them is there also works yes you could oh you got you you'll have
40:53
to manually query currently you have to manually query if you want that kind of features to get in package ng just go to our website github slash package
41:04
ng slash package ng and add the issue with will soon open the development currently the development is frozen so that we can release one oh but all the crazy idea you can all have about package ng just feel an issue on github so that when we'll go to package any one one or package ng 2o
41:23
we'll just have that in mind and try to go that way if possible you just need to keep in mind that in package ng we want everything to remain as simple as possible I prefer not having a feature that having a feature that will work most of the time but not all the time or if we had read it we had it
41:42
off by default
42:02
yeah but that could be solved by one of the things that package ng will bring to the ports tree is in the features like installing stuff in stage directory and then if we have a stage directory we could imagine splitting the package and the reason why DB for DB for is has problem with DB for seven
42:24
that is they provide the same files so if we split them and just provide the library then you say if you can install the latest Apache with DB for four and then you can install I don't know what you use with DB for seven yes one of the other thing I wanted to show is an upgrade yep that's here
43:24
so yes I just launched the video so that people can look at the upgrade while you ask the question I made a video because I wasn't sure we have things here to be able to upgrade so yes currently it's several time on the the
44:01
varchage DB of every day that is it can be fixed by just nullFS that directory yeah yeah and all the command are documented we have the
44:40
main page that are up to date now all the command are you we have the
44:45
command so if you don't know how package install works you just do package help install and then you have it will fire up the main page corresponding yes no currently you you will just have to package it at minus
45:11
F to force the deletion of the previous version and install the purchase list on the version you want you can just specify oh I want to downgrade this one we have a bug right now in the version that sometimes some
45:25
drive downgrade our output in in the upgrade process so yes we know we know how to detect the downgrades and we know that we know we could provide this but I think it's it shouldn't be easy to do because most of the time
45:42
dangerous it will just bring you a package which is not consistent with the latest package set we provide to you so I prefer not having something too easy because a lot of users will just use it and then report bugs everywhere saying okay this don't work this don't work I just don't credit this but it's not consistent with the package that we have it's it's not it's not
46:10
difficult just not straightforward you can just package it at minus F so that it doesn't say oh it depends on by something so I won't delete it you just force the deletion and you add it and when you add the old version your
46:23
version will be linked with with the package that used to depend on the new version the dependencies are cleanly can you handle but if if the I don't know the library is income is not compatible between the new version the
46:41
old version if you don't write then your dependency will be will be broken yeah so this packaging in that case the dependency are still still tracked right way yes yeah one of the feature I want to be able to do is to be able to
47:15
roll back like it's done in FreeBSD update I want to say okay you've done an action you're able to roll back to the previous state before this this
47:22
action but it's quite complicated because if you have a huge set of packages it will require a lot of this space to be able to do that rollback so we need to find something to to deal with cleanly and yes in the future
47:40
there will be a package rollback something like that yes
48:20
yes yes all the directories we used are configurable so you can say my local the my local database is in slash user local it could be a good idea but I just prefer not changing too much thing at the same time so first we get
48:42
package ng and then we can do some survey or stuff like that about this idea and just switch at for package one one or something like that but it's doable with with package ng currently and yes using ZFS for the rollback was one of the thing I was thinking about like it's done on it's not open
49:06
salaries but yeah the package system which is a crap but he handled the FS very well and I'd like to be able to have some kind of hooks or thing like that so that you can do the same things out of package ng so before
49:22
installing you just snapshots user-local and then you can roll back and stuff like that yes yeah but one of the things we're thinking about is having hooks about everywhere in the library so that you can just provide all the
49:40
script for if you want to manage ZFS or if you want to manage if you want to install a package without some files like dot edge dot a dot stuff like that you can just say I took the normal package I and before installing and just ignored all those files it's something we're thinking about for one one or two oh nine years a question yes you can't you just go to package ng one shot we
50:23
have a script which is called package 2ng which converts the old database to the new one but you can't go back so prepare your backup that's it prepare your backup or stuff like that and if you are switching to package ng do not
50:42
and still using the the pot tree do not forget to activate packet ng in the pot tree to avoid having both because it's one shot it's not incremental we can't continue using the old package and then from time to time saying oh should I have new package so please upgrade to the database yeah we use we used to do that but I remove that code because
51:15
because the the old package format is really for example the P list is not a
51:23
list of files in the old files it's a script we check that thing from there there there and there and then now install a file and stuff like that so we can move from the old to the new one saying okay all the script goes to post and style or goes to print solve a going the reverse is almost impossible because we are not able to determine that pick that particular script should
51:45
be for example in Python should be executed in first in the beginning of of the of the file extraction but after this file is extracted something like that so no we can't create the old package but we are working on the script
52:03
that will do the reverse it means you already have some old format and legacy format packages and you want to just convert them into package ng so that it's straightforward to you to install them on the new system and it's pretty easy to do I just didn't find time to write the script yes
53:18
yeah so for that particular problem
53:24
you still have to use package register to register your stuff but we are thinking about something like held command we say okay this package don't touch it this is for this is something I know I have an install and I don't
53:41
want to be upgraded from from Russian but it's it's complicated because it can lead to inconsistency in the future the best thing to do to deal with that is to create your own packages and set your own repository and then provide the package in that so that when you install a PHP you just bum
54:02
the port revision name in it and then it's always higher version than the version that is in the official repository and then you have it installed correctly one other thing we want to do in the future is just to also add an option saying okay this package comes from this repository if
54:21
this repository is still there then it will it can only be upgraded from this repository it's not done yet but it's something we're thinking about too and if the if the repository is not configured anymore then go back to the normal way you can is no you don't have to set up your own repository
54:50
package register is done to do exactly the same thing as package add or package install but from the pot tree so you go in your post like you do currently you just do make install and then it's installed has package
55:01
engine the only thing you have to do currently is to activate it because by default it's still the old system so to activate it is just adding who is package-ng equal yes in mag.conf and that's all
56:10
yes
56:40
yeah and most most of the time the option and the post really just had a new new file into the package so we can if you can split the package most of the option will be gone except for things like nginx or or mplayer or stuff like that but most of the time we can deal with that and maybe with the provide feature we want to see in you'll be able to also do some
57:02
kind of flavors and say okay I'm installing Apache dash that option and it's recorded as Apache and everyone looking for Apache will be happy with
57:34
it okay yes I'm waiting for for the end of
57:57
line of 8.1 because 8.1 is not able to use XE compression as soon as 8.1 is
58:08
out I'll MFC the bootstrap into 8 and I could start to create official 8 packages by the way the thing I haven't said is just currently we have
58:24
we have some repository available on pkgbeta.previously.org so if you want to be able to if you want to be able to test try to update them once a week sometimes it's longer sometimes it's shorter this these are just team links
58:44
to the new ABI line so that it's easier for people to it's easier for people to find out I'm MD 64 yes you want that I do auto remove but there
59:04
has been nothing in it okay yes package check minuses I just did it
59:40
Apache and then I can do auto remove it detects all the packages are not
59:50
needed anymore
01:00:06
Yeah, and it's recursive, so you don't have to run it ten times, and the dependency tracking is also done from the port tree, so if you keep just using the port tree, if one of
01:00:20
the ports is installed as dependency, it's tracked in the database. Because when you install a package, we just add a flag in the database saying this package has been installed as dependency and this package has been installed on purpose by the user. You know, when you do package install apache, I know that
01:00:43
apache is the package you want, not everything that comes with it. If you had the globing, all the package that matches the globing will be marked as wanted by the user. It's integrated in the port tree. If you just go to a
01:01:15
single port, you have fresh installation, you activate package-ng, you go to a single port, make install, it will install a bunch of things including libtool and stuff like
01:01:23
that. Once it's installed, you just do package to remove and all the things that were only build dependency will be dropped. Yes? Yeah, with
01:01:50
package set, you can say package set, we have some keywords which one of them is percentage A for automatic, percentage A equals 1 and then you mark it as
01:02:01
automatic and if something goes wrong, I don't know what reason, but you have all the automatic stuff that has been dropped, you can say package minus you can say automatic equals 0, you can change it by hand and as usual you have the choice to do that on the whole packages or just some
01:02:21
packages using globing, using a regular expression. No, you just say this file while removing, it says this file has been modified, I won't drop this file but
01:02:41
the rest is removed. Yeah, it does keep it except if you have modified the thing on purpose and then you are willing to track it normally, you can with package checks recreate all the checksums because I know what I do and then the files won't be considered as modified anymore. Yeah, yes? It's more
01:03:23
about, for example, if I want to be able to split packages, I want the post tree to be able to understand what is a stage directory and doing that with the old tools is pretty much very very complicated, so if I can drop it, it
01:03:41
will be easier. The other thing is that I want to kill most of the scripts we have in the post tree. We have a lot of things like exec, unexec, and stuff like that which are not reviewed by security officer, it can't be reviewed because there is everywhere in the post tree and I want to be able to add a lot of new keywords and in those keywords we have the script which is
01:04:03
human readable and then we can, if we have any problem with security, we can just ask any security officer or someone that knows about security to review things in one single place with just the script there and in a very long-term goal I want to get rid of any script at all, providing something
01:04:20
else to be able to do cross-installation of packages and stuff like that. That's done with the current packages using scripts at exec
01:04:47
something. What we've done in that, in package-ng you can create just the file calling it .packageconf and if we find this file and there is no file without this extension we just create a new one and if there is already a file
01:05:03
we keep the the configuration done by the users. Yeah we have the feature, I don't know how it will be used by the
01:05:22
maintainers and people like that but we have the feature to avoid the script that did that before. Yeah but after that we could imagine some scripts that just detect that there is new version of the configuration and and prompts you you have to merge this like merge master or stuff like that. Sorry? Yeah
01:05:50
Yes, the upgrade scripts are also a reason why we can't, currently we can't use upgrade script if we want compatibility with the old package install because the old
01:06:02
package install don't know about upgrades. So if you want to push the upgrade script in most of our ports and most of our package we need to get rid of package install for example. Okay, thank you.