We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Pharo3: Status

00:00

Formal Metadata

Title
Pharo3: Status
Title of Series
Number of Parts
199
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Pharo 2 was released in March 2013. Not even a year later, we are close to the release of Pharo3. With over 1200 issues fixed and many deep changes, it is the release with most changes yet. This talk will give an overview of the changes and improvements done and present some examples of what can be done with Pharo3
Point (geometry)Projective planeData miningFormal languageEndliche ModelltheorieLogicSummierbarkeitForestIntegrated development environmentCompilation albumObservational studyProgramming languageDirection (geometry)SpacetimeCodeComputing platformDebuggerLibrary (computing)Just-in-Time-CompilerBitWindowComputer animation
Software developerSinc functionString (computer science)Multiplication signScaling (geometry)PlanningReading (process)Logical constantGreatest elementSoftware testingScheduling (computing)BitArithmetic meanSoftware bugGraph (mathematics)Computer animation
Continuous integrationCentralizer and normalizerError messageSoftware developerSoftware testingRevision controlProcess (computing)CASE <Informatik>Projective planeINTEGRALServer (computing)Computer scienceMassPhase transitionTask (computing)Student's t-testSet theoryComputer architectureMultiplication signAlgebraic closureBranch (computer science)Computer animation
Server (computing)Projective planeNormal (geometry)MathematicsINTEGRALWordQuicksortObject (grammar)Medical imagingNeuroinformatikPhysical systemData structureMereologyDescriptive statisticsWeb browserComputer fileContext awarenessRevision controlWebsiteContinuous integrationSoftware bugComplete metric spaceSource codeComputer animation
Endliche ModelltheorieQuicksortRevision controlPhysical systemLecture/Conference
Analytic continuationCore dumpContinuous functionLeakImplementationSoftware testingSemiconductor memoryDisk read-and-write headMedical imagingVideo gameNeuroinformatikMultiplication signPhysical systemStatement (computer science)Algebraic closureForestAbstractionLevel (video gaming)Social classInstance (computer science)Block (periodic table)Computer programmingAbstract syntaxTunisState of matterSource codeCodeFunction (mathematics)Theory of relativityUltraviolet photoelectron spectroscopyStandard error1 (number)WebsiteReal numberRadical (chemistry)Text editorInterpreter (computing)Web browserComputer animation
Different (Kate Ryan album)WindowObject (grammar)MorphingMultiplicationSound effectNetwork topologyTouchscreenMenu (computing)Web browserEndliche ModelltheorieHierarchyState of matterMaxima and minimaProgrammer (hardware)Demo (music)Normal (geometry)Point (geometry)QuicksortData storage deviceGame theoryInternet forumSpectrum (functional analysis)Video gameComputer animationProgram flowchart
Raster graphicsIntermediate languageVector graphicsRight angleCore dumpForm (programming)Software testingBenchmarkPhysical systemProjective planeMathematical optimizationGraph (mathematics)Artistic renderingVirtual machineAdaptive optimizationIndependence (probability theory)Front and back endsBitSoftware frameworkComputer hardwareGraph (mathematics)Vector spaceComplete metric spaceComputer animation
Vector spaceRaster graphicsPhysical systemGreen computingSound effectSoftware frameworkMultiplication signElectronic mailing listImage resolutionTouchscreenDemo (music)Complete metric spaceShape (magazine)Zoom lensComputer programmingArtistic renderingComputer animation
Social classBitCompilerFront and back endsObject (grammar)Variable (mathematics)Basis <Mathematik>Abstract syntax treePhysical systemLevel (video gaming)Instance (computer science)Representation (politics)MathematicsBytecodeDifferent (Kate Ryan album)Sound effectBusiness modelComputer fontWordConnectivity (graph theory)Form (programming)Integrated development environmentImplementationMereologyInverse elementDivisorMedical imagingComputer animation
Mathematical analysisInstance (computer science)VarianceDifferent (Kate Ryan album)Variable (mathematics)Physical systemSource codeComputer animation
Form (programming)BitPresentation of a groupVideo gamePrinciple of maximum entropyLine (geometry)Integrated development environmentSelf-organizationGroup actionRevision controlSoftware testingLevel (video gaming)Multiplication signDirection (geometry)Endliche ModelltheorieReal numberInverse elementIndependence (probability theory)MereologySoftware developerForcing (mathematics)Physical systemCore dumpExpressionInternet forumCASE <Informatik>Dependent and independent variablesPoint (geometry)Hecke operatorInterface (computing)Set theoryImplementationDebuggerScripting languageServer (computing)Letterpress printingBuildingCodeMedical imagingSoftware bugView (database)Structural loadProgrammer (hardware)Source codeComputer animation
BitWebsiteQuicksortPower (physics)Point (geometry)Information technology consultingBasis <Mathematik>Direction (geometry)MereologyAssociative propertyComputer animation
Library (computing)Right angleEndliche ModelltheorieFocus (optics)Exception handlingRevision controlVisualization (computer graphics)CASE <Informatik>PRINCE2Enterprise architectureWhiteboardSoftware bugRegular graph
Presentation of a groupOnline helpControl flowProjective planeSet theoryPoint (geometry)Binary multiplierForm (programming)BitCentralizer and normalizerOrbitEvent horizonPhysical lawWordInheritance (object-oriented programming)Arithmetic progressionRule of inferencePhysical systemKernel (computing)Real numberSoftware testingComputer animationLecture/Conference
Transcript: English(auto-generated)
So next speaker, Marcus Denker, talking about FARO 3. Yes, exactly. Yeah, FARO 3, which is the next release of FARO. And before I show you what actually FARO 3, what we did and what it will be, a bit in general about it.
So you have already seen FARO already in the first talk, which was already FARO 3 because the release is very soon and so people are starting to use the latest now. So what is FARO? FARO is a language and an environment where environment means a full programming environment
with code editor, debugger and everything. It's a simple language, small talk. Right now very much compatible to the original small talk. And it's object oriented, it's dynamic and especially it's extremely reflective. So that means it's implemented in itself and you can change and inspect everything of the language directly.
So the idea actually is that we want to build the ultimate programming environment. So of course a lofty goal and we will see how it turns out. But that's the end goal. So FARO is MIT licensed. Our main platforms are Mac, Linux and Windows. There are some people looking at mobile space
but that's a bit difficult because they don't allow JIT compilation to be done and so that means it's a bit slow. We have a growing and really nice community. Very important, FARO is improving steadily. So actually there is almost no day where there is not something being improved.
Often it's of course very simple but sometimes it's big. And we have a lot of libraries and external projects that use FARO like we saw in the first talk for example this BabyMOC project. So FARO was started in 2008. But we had our first release only relatively late in 2009 in October.
And since then we had almost five releases where 2.0 is stable which was released last March. So last year's talk here at FOSTA was actually about what we had done for FARO 2 and that we would release it soon. And the interesting thing is that we managed to do almost everything
or most of the things that we were actually planning for FARO 3. The idea is that we want to do one release per year for the next 20 years or something like that. So that means that we... Later in the FARO 4 talk I will discuss a bit what that means from a time schedule.
So we started in March 2013. If you look at the bug tracker and you filter for all issue tracker entries tagged FARO 3 then you see that it's 2021 that were closed since March.
This is quite a lot so that means this is not even 300 days. And that means these are really multiple things happening every day. So when people submit an issue tracker fix then this fix is automatically verified by running all tests.
And then we combine four of those into a so-called update of which we have around one, two, three per day. And since March 18th last year we had 733 updates for FARO 3. So that means a lot of things happened and we still have a bit to lose.
We have 40 issues tagged that we need to fix until we can release. But we hope that we will do that until March. Lots of activity means... So this graph I made in December so it's already a bit outdated. This is the release date of FARO 2. This is shortly before Christmas.
And you can see that the open issues stay more or less constant. So we have always a constant open issue of around 500, between 450 and 550 issues. And it's really difficult to get that down. So I always try and we always have sprints where we try to get this down. But it's very difficult.
But the interesting thing is that it stays constant. So even though people open a lot of issue tracker entries they get close too. So we managed to close 2000 issue tracker entries within the development of FARO 3. And one of the reasons why we were able to move so much faster than in the past
is that we now have a working continuous integration infrastructure. So this is actually provided by INRIA which is a French government research lab for computer science. And they have a central continuous integration infrastructure built on Jenkins.
And we can use that to build up our infrastructure and we use that extremely a lot. So that means every fix that someone submits is validated automatically for the latest version of FARO, latest development version. Which means it runs, in this case only on Linux, all 16,000 tests.
And only if these tests are green this issue will get tagged to be a verse of human review. And we run even kind of lint style checks to see. So for example if someone forgets to put a comment in a class or if there are certain small errors they are checked automatically too.
So that we do for every fix. And then when an update happens of FARO itself this update is validated on all three architectures automatically. So that it runs all those 16,000 tests three times. And of course after that there was a new version of FARO and that means that all issues that have been already checked to be reviewed
they will be retested. So that means we have a lot of activity going on on this integration infrastructure. One update push of FARO triggers a lot of jobs. One is to rerun all the tests on the proposed fixes. And the other thing is we have now in the FARO contribution CI server over 80 external projects that are validated.
Often on both FARO the stable version and the unstable version. So that means that one push of an update triggers directly 80 builds on the contribution server. And the contribution server looks like this.
So it's a normal change installation but with 80 projects. Besides the continuous integration we build a lot of other infrastructure over the years. Most of it really got stable last year which is very interesting. So that everything comes together and pushes you forward. So we now have a file server, files.faro.org
where you have every FARO image that ever was. So the FARO release of every intermediate update is available there. So if you want to check if from the version that you used two weeks ago if this bug really happened there and not now and you don't have this release anymore you couldn't go back. There was a whole history available there.
We have get.faro.org which is an easy way of installing FARO especially in a server context. So it's a website with a description where you use wget to directly install a complete FARO system. So if you go there with your browser you can see how this looks like. Another one is Smalltalkhub which is a special kind of GitHub-like system for Smalltalk
which instead of using Git uses Monty Cello which is the traditional revision control system that was invented for Squeak a long time ago. And this one starts to get quite used to lots. We have over a thousand users and due to our wonderful build infrastructure
we get around 15 million downloads of versions every day. So what did we do with all these 2000 update issues and lots of updates? So we had a lot of small cleanups of course. So when the philosophy in FARO is that nothing is too trivial to be done.
So if someone while reading code finds a typo of course that is not really relevant for anyone. I mean a typo, missing i in a comment, who cares. But people actually when they read that they actually stumble and say someone should fix that. And if they then do the work to fix it we will integrate it.
So there is no, but this is trivial, we will ignore it. There are lots of code-based tunings like performance improvements, memory consumption improvements especially because we managed to get a lot of memory leaks in some things.
There were a lot of improvements in FARO3 and overall lots of small improvements on the code base. So we have now a lint style checker in the system that provides a couple of I guess around 80 or so checks of things that should never be wrong. So for example if there is an instance that nobody uses, why is it there?
Just makes people think of why do you need this kind of state in this class? And then people do not understand anymore and then they search around and they find out it's not used and they lost 10 minutes of their life. So we are trying to improve all these small things. So lots of things were done there.
But of course there were a lot of larger improvements too. And not a lot of them I can go into detail. So these are actually the ones I will not talk about. So for example we added the closure class that used to be in C site. We added that into the system so now you have a closure implementation that you can use directly without loading C site.
We have now real terminal output for standard error and for Unix very nice. We cleaned up a lot of code with source files. We have an AST interpreter now in the system that allows you to very nicely experiment with small talk like systems by changing it. It's a visitor over the abstract syntax tree, very simple but complete.
So it can actually run the whole test successfully. We added lots of AST related things. So for example there is AST based navigation in the browser so that you don't in the code browser or in the editor you don't need to go character by character but you can say ok go to the next AST abstraction.
Go to the next statement. We added a nice commit tool. Lots of small speed ups and things like that. Yes? Which one?
Ah continuation, yeah you are right. Continuation class. Blocks are closures but if you want to program with continuations you always need to do it yourself and this simplifies.
So one of the bigger improvements that are actually visible to the users if you download faro is the new inspector. I actually want to show you that quickly if I have the time. Ok so let's quickly take the latest faro and start it up.
Done. So if you are in faro there is the, in morphic in general not only in faro you have the whole what models the screen is actually an object that is bound to a global variable called world and you can of course inspect that variable.
And so you get an inspector and you see that this inspector looks a little bit different than the, or if you know older small talks you see that it is a little bit different because it has a selector at the top which allows you to switch which kind of inspector you want to use. So small talk already since I think small talk 80 is originally reset the concept of multiple kind of inspectors but it was always a bit hidden.
So normal end user programmers even didn't understand that when they looked at the dictionary that this was a special inspector that was not showing the real world but how it would look like that it looks dictionary like. And we made this now available as so that you see that there are multiple kinds of inspectors for certain objects.
And for these morph objects we now have always four different inspectors available. One that is the standard inspector, one is a tree inspector which is what used to be the explorer which shows you all the state of this object nicely in a tree. Then we have a hierarchy explorer which only shows you the morph object and the hierarchy that the graphical things have on screen
so you can see okay this is a spec window and it has a grid morph and all these things. And then we have just as a demo something really fun which just shows this graphical morph and renders it inside the browser.
And as this of course renders what you see on screen you get a really nice effect of an endless inspector of the main screen. And if you open now here a menu then this will immediately be copied step by step into this so it's kind of fun.
Will this run out of memory? No because it just renders the, I think it stops at some point or something like that. I think it has maximum depth so kind of nice. And this new inspector actually has already been proven to be very interesting.
So it is used a lot in our virtual machine experiments to give us special inspectors for pastries, intermediate representations. So there is work going on on adaptive optimization which will be very interesting
and we hope that we will see something around the end of the year working. Which the benchmarks you saw that Stefan showed which will hopefully be improved a little bit for Coq together with this optimizer. And there it's very useful to have this inspector because you can inspect all these things.
These SSA graphs that get optimized you can introspect very nicely. So that's very nice. Another thing that we are working on but which is a big project is a vector graphics backend that should replace the Stephanie style bitmap graphics that drives the system right now.
So it's a new API for vector graphics, modern API with independent backends. For now we use Cairo. There is a balloon 3D backend for debugging but it's because balloon 3D is not complete in the sense of a complete vector framework it's not really useful. It cannot do some certain things. And in the future we would like to directly go to OpenGL and use hardware rendering.
But for now it's Cairo. And we actually have this working in the current 3.0. There is a preview. It's not yet used to render the desktop because that is a lot of more work to be done. But it works already like this. Kind of nice. So how is this called?
Tiger... VTiger demo run demo. So you can see here this is a... Lots of shapes for the vector data. And this is rendered to make this relatively smooth animation all the time completely while being copied around as a bitmap quite a lot.
So to get this into the morphic screen it gets copied again. And there is a lot of copying going on. And that's why it's not really that fast but it works. And people are starting to use that as a specialized canvas inside of Cairo 3. But the future will be to actually replace the canvas that renders the whole desktop with this graphics framework.
And then we would have a vector based system completely. Then we could easily implement zoom of this so that when you have higher resolution that the graphics do not get smaller and all these things.
And of course it should speed up the whole system quite a lot if we do that. So that's quite nice. So in Pharaoh 3 we have a canvas for that that you can use in your programs but the system itself does not yet use the canvas to do the rendering of the desktop.
So then we actually did a lot of big changes in Pharaoh 3 that are completely invisible. So one is that we replaced the smalltalk compiler. So the compiler that takes text and makes bytecode. That was completely replaced by a reimplementation which uses the abstract syntax tree from our tools which is very high level so it has lots of things for...
So it was actually implemented for refactoring tool. And now we use it in the compiler which is quite kind of interesting. It gives you a lot of flexibility. And then it uses the backend which is kind of like a high level assembly for bytecode which is very nicely usable standalone if you want to do experiments. And you can actually decompile the whole system into this intermediate
representation, manipulate it and then re-image bytecode and that's extremely fast. So decompiling the whole system takes less than a second using the IR representation. But it's very nice if you want to do bytecode manipulation. All in all the compiler is much easier to change because it's built on visitors, one after the other.
And it really uses the basis for a lot of really nice things that we've seen in Pharaoh 4 which we will see later. A bit on the same level is that we replaced the class builder. The class builder is a system that is there when you want to make a class because making a class is not that simple. Because if there are already instances you need to migrate the instances.
There are different kinds of classes so it's quite a lot of work to be done. And the class builder was really complicated so actually I don't think that anyone understood it. And now we replaced it with a class builder that is much easier to understand and
much more flexible because it makes a lot of things into objects that weren't objects before. And one of them is super slots. So now we have for every instance variable in the system we have an object that describes that instance variable. Right now it's purely for the same that it is in small talk. So it's an instance variable but you have an object for it.
But it's kind of nice. So if you look for example in the current 4.03 and you say slot all instances size. Then you see that we have right now in the system 7,272 different instance variables. So for every instance variable we have an object. And in the second step this will allow us to do a lot of really crazy stuff.
Which we will see a little bit in the 4.04 presentation. On the same level we have a complete new implementation of the debugger. Which now separates the view from the model and we have even two views implemented. So we can prove that it's independent. The model is scriptable so you can actually script it using small talk. It's understandable so you can actually hack it.
And there is a lot of nice research going on in Switzerland and Berlin on this. And they have really fancy debuggers. So the idea is that they want to provide you a toolkit. So that you can provide your own debugger for your own complicated system. So kind of programmer implements their own debugger to debug easier. Very nice.
Another thing that we did is command line interface. So because of the build server and because people are starting to use Faro a lot in real world settings. It needs to talk better with the outside world. And one of the things that is really nice to have is a command line interface. That makes sense.
So if you make a subclass of command line handler then you can implement. Just look at it, it takes two minutes to do. Then you can implement your own command line handler. So I did one last week which is called clean, which runs image cleaner. And it is really nice because this one now can be used in the build server quite nicely.
So you can just say Faro update and then it loads the code updated in print version. And it prints out the version and then Jenkins can use that. While for example we can run the tests from the command line which is used in the build server of course. You can do image cleaner or you can just evaluate a small talk expression.
Very nice. So that was already about the small talk system itself. And a bit on the infrastructure level but not technical infrastructure but more human infrastructure. We are starting to do a lot of things too. So one idea that we made real last year too, so it is part of Faro 3, is the Faro Consortium.
And the other idea is that there are companies using Faro for their work. And there should be an infrastructure set up so that they can actually invest into their own future. Because if they use Faro for their day to day work, then there should be interest that bugs get fixed,
that things get improved and that we move forward in general. And if you are a company then you always need to do the things that you need to do to get your customers happy. So you don't have the time to do that at all. Sometimes you have but often not. And so the idea here is that the companies can actually give money to one organization
and that organization should be able at some point to hire someone full time to fix bugs in Faro so that they don't have to do that themselves. Right now we have 13 members that are real companies. We have six academic partners and three sponsors.
So sponsorship is the kind of, you are not a member but you are contributing. And it's not yet enough to pay someone full time but it's already quite on a good direction. So we will hope that this will continue in 2014. And here are the companies that are right now a member.
Sadly our sponsors are cut off a little bit so have a look on the website to see them. Yeah, very nice. This is really going in a good direction. So originally we wanted to have both companies and individuals being part of the consortium but the problem is that the consortium is managed by Inria and Inria as a government entity was not able to handle individual contributions.
And so we decided for now to split these two things. So we have a consortium for companies and institutions and we have an association for individuals which has now around 60 members. And the idea is that at some point the consortium and association may be merged into one entity.
We will see in a couple of years. Another thing that we did is we released the second book which has more, so we have already the Faro by Example book which needs to be updated because it uses Faro 1.1 I think as an example. And we now have more advanced topics in the second book.
So it's not really for introductory things but it's more things on top of Faro like external libraries that are described like ORASA, the visualization engine, things like that. All really deep topics. I think there was something about exceptions and things like that. Oh, they implemented how they work.
There is a third book in preparation. If someone is interested to write something, it's always interesting. And the third book will be about Faro in the enterprise. So more like what do you need to know if you want to deploy Faro professionally. Or what kind of libraries do you have that you need in this world.
Before I stop, one interesting thing we are trying to do is to have sprints regularly. And sprints are that we meet with 10 to 20 people with the goal to fix boring bugs. So to really look at the bug tracker and fix all the things that nobody ever wanted to do because it just sounded boring or complicated and not that important and things like that.
And this is starting to get really efficient. So the last sprint, we managed to fix 30 bugs before Christmas. So that starts to get really nice. And we try to do it in many places. So if you want to organize one, you are welcome. Of course, we try to do one every month.
Not all of them are really publicly announced. But around every two or three months, we want to make a really big one with external participation. And if there are, for example, conferences, then we try to set something up. But everyone can do that if you are interested. So that's my presentation for Faro 3.
And if you are interested, help is always welcome. I will present what we will do after Faro 3 until next first and then after the break. Any questions? Yes, maybe that's what is already done for Faro 2.
I have to admit I haven't tried to look at Faro 3 now. But as I remember, the packaging in the system was very connected every package.
Has this been resolved with Faro 2? Yeah, so we started to work on that. So there are very interesting two projects. One is about bootstrapping Faro from SARS-CoV-2, which I will talk a little bit in Faro 4. And the other one is about shrinking Faro down. And those projects, as people were working on that, they submitted improvements.
So I think we improved a little bit the situation. But the problem is we have not yet infrastructure in place to make sure that it does not degenerate anymore. So when we then said, okay, now Faro 3 is nearly finished, some people were actually looking at these things and then said, hey, but this is getting not better, but even worse than it used to be.
So in Faro 4 we need to really build infrastructure. We need to test that there are no... Because it's so easy. You fix something and two weeks later someone else tells you, yeah, but you are using now in the kernel this package and this is a problem. And then you think, yeah, but I didn't know. So it's really a problem. Ongoing work then.
Ongoing work, but there will be, I think, a lot of progress in Faro 4. Because now multiple people actually said, oh, we need to do something. And that should be based on the packaging system that was already there from the beginning maybe? Or are there ideas to use something else? So for the real, yeah, we need at some point a real package system with imports and all these things.
But this is a bit difficult to get it right. So it's very easy to get it wrong. And that is always the danger so that we are a bit reluctant of saying, okay, this is the solution and then it's wrong. We will see. So at some point this will come and we need to do experiments. The first thing I think we will do is to put rules into place that check these things.
So that we define this is the kernel and if the kernel uses something that's not the kernel then the test is correct. So that we need to add. I think we will do that in Faro 4. Thanks. Any other question?
Thanks very much.