Ceci n'est pas un MOOC. How to build awesome, open, dirt-cheap online courses

Video in TIB AV-Portal: Ceci n'est pas un MOOC. How to build awesome, open, dirt-cheap online courses

Formal Metadata

Ceci n'est pas un MOOC. How to build awesome, open, dirt-cheap online courses
Title of Series
Number of Parts
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
There is an incredible opportunity to build new and interesting ways to learn together online. MOOCs dominate the conversation, but only scratch the surface of what is possible. Better learning will happen when more people, not just academics and universities, gets involved in building, experimenting, and innovating. There is a lot of interest in MOOCs these days, but many of the early courses seem to replicate the worst part of the traditional campus model of education. And they are expensive to create and sustain. In this session I will talk about work I have been doing at P2PU and the MIT Media Lab to create alternative approach to supporting large online courses:using the web as a platform instead of building new portals;using open source software and free services instead of closed systems; and supporting group work and community engagement rather than delivering content to the maximum number of people. And doing it all on a shoestring budget and having a blast.
Network topology Metropolitan area network
Adventure game Point (geometry) Multiplication sign E-learning Spacetime
Point (geometry) Dependent and independent variables Link (knot theory) Observational study View (database) Sampling (statistics) Student's t-test Graph coloring Cognition Different (Kate Ryan album) Moving average Energy level Proxy server Resultant
Word Universe (mathematics)
Multiplication sign Factory (trading post) Order (biology) Set (mathematics) Surface of revolution Physical system Social class
Adventure game Medical imaging Mathematics Different (Kate Ryan album) Moment (mathematics) Right angle Writing Connected space Power (physics)
Point (geometry) Right angle Grass (card game) Food energy
Mathematics Mathematical analysis Online help Bit
Adventure game Slide rule Mereology
Trail Moment (mathematics) Bit
Moment (mathematics) Shared memory Water vapor 2 (number)
Right angle
Slide rule Data conversion
CAN bus Personal digital assistant Direction (geometry) Universe (mathematics) Projective plane Data conversion
Moment (mathematics) Bit Pressure Spacetime
Software Direction (geometry) Moment (mathematics) Universe (mathematics) Videoconferencing
Adventure game Universe (mathematics) Routing Wave packet
Adventure game Building Projective plane Open set Row (database)
Web 2.0 Hypermedia Different (Kate Ryan album) Projective plane Universe (mathematics) Range (statistics) Content (media) Bit Open set Endliche Modelltheorie YouTube Spacetime
Axiom of choice Musical ensemble Connectivity (graph theory) Characteristic polynomial Student's t-test Mereology Power (physics) Element (mathematics) Peer-to-peer Hypermedia Hierarchy Energy level Software testing Endliche Modelltheorie Multiplication Uniqueness quantification Projective plane Fitness function Bit 3 (number) Student's t-test Performance appraisal Peer-to-peer Hypermedia Video game Spacetime
Web 2.0 Service (economics) Open source Connectivity (graph theory) Projective plane Social class Bit Mass Free group Number
Rounding Code Multiplication sign Content (media) Similarity (geometry) Line (geometry) Content (media) Formal language Goodness of fit Website Right angle Free group Video Genie Form (programming)
Email Process (computing) Level of measurement Roundness (object) Strategy game Multiplication sign Projective plane Survival analysis Bit Open set
Arithmetic mean Sign (mathematics) Email Hypermedia Hypermedia Line (geometry) Videoconferencing Bit Student's t-test Mass Free group Social class
Musical ensemble Email Scheduling (computing) Open source Forcing (mathematics) Content (media) Number Sign (mathematics) Word Googol Computing platform Computing platform Computing platform Form (programming)
Collaborationism Group action Different (Kate Ryan album) Core dump Interactive television Computing platform
Musical ensemble Multiplication sign Universe (mathematics) Archaeological field survey Video game Amenable group
Point (geometry) Personal identification number Musical ensemble Distribution (mathematics) Link (knot theory) Mapping Building Tape drive Multiplication sign 1 (number) Materialization (paranormal) Bit Limit (category theory) Directory service Binary file Spreadsheet Different (Kate Ryan album) Auditory masking String (computer science) File archiver Data structure E-learning Spacetime
Musical ensemble Videoconferencing
Online chat User interface Open source Multiplication sign Expert system Videoconferencing Virtual machine Video game Data conversion Event horizon Online chat
Degree (graph theory) Slide rule Arithmetic mean Different (Kate Ryan album) Universe (mathematics) Student's t-test
Musical ensemble Connectivity (graph theory) Data conversion Mereology
Point (geometry) Mechanism design Goodness of fit Software Inheritance (object-oriented programming) Multiplication sign Moment (mathematics) Expert system Videoconferencing Frustration Freeware
Building Open source Software developer Expert system Self-organization Theory Spacetime Twitter
Multiplication sign Touch typing
it it it it
the the but I'm coffee
lank uh this reality comes out once lies near minus Cory Doctorow and in dozens of trees a man in the taught as a how about some this is something which we didn't uh so it has been applied to Deutsche is of English readers and I just could
not in Boston have landed off exactly at some and I'm going to speak about adventures learning and I have to
say we both are speaking about books but I have a very different uh fascination with learning and online and
books and I wanna make 2 points uh 1 point is that learning can be the greatest adventure that we can make the experience and that we should be thinking or we could be thinking about learning as an adventure and that may give us new ideas for how online learning could work and secondly that this is 1 of the most exciting times to experiment with learning have because of some of the possibilities that exist today coming from technology but also the institutional interest in this space for innovation right now that wasn't there 10 years ago that was there 20 years ago and I hope that more people get
involved and start experimenting and start prototyping and start building things are very excited about them so this is an interesting study of experiment that was done at the place where I work now and where the students were given these responses that measure at some kind of a proxy for engagement or activity and cognition and then they wore them for a while as a very small sample size that don't view this preliminary but they wouldn't them for a while and then essentially uh if you look at that and if you can see from than they would if you look at the results a kind of you can see the activity level spiking at certain points and then there's a different color bars and below them and the uh the kind
of frustrating or disappointing thing is that some of the lowest levels of activity across all of those are when the students go a class as so they have higher activity even when they do show as and when they watch T V E uh much higher when they sleep um and so it it uh it's kind of a a summary
really out of this situation uh accept that obviously this is more
scientific uh but and it tells us something about
the way that we're doing education right now may not totally be working what these not for everyone and also this is from MIT in MIT is a great university the idol of MIT necessary as much better teachers than other
universities but I I would say that they're not words or significantly worse than other teachers and so the I think this is something to kind of keep in mind and the other thing
is that all a lot of the ideas that we have about education how education should happen and where it should
happen in what order are really from a time when people were prepared to work in factories and industrial revolution a
lot of the education system is the Industrial Revolution's education system and and we kind of live in a different time now some people talk about the creative
economy or and the trade of class that's emerging in that people need to be uh up-skilling faster and they need to think about and being lifelong learners and all of those ideas I think are not reflected by a lot of the institutional settings that we're still dealing with some so I really like this image and for me um most of the things
that I care about and learning I can describe using just this 1 this 1 image and also there is a personal connections and that's not me on the arm for but for there are certain things that I see in this picture I think are interesting so 1 is when you learn to ride a bike psych the world changes like is that
moment when you you wobble along and then like he takes flight and you you write your writing and so you can never go back to the nobody can can learn to ride a bike and for me that's that moment of epiphany that we have 1 great learning happens when we have a new idea we look at the world in a totally different way and we can never go back we can never unlearn nest and so for me that's 1 aspect of a kind of a learning adventure and the 2nd 1 is at riding a bike isn't extreme extremely powerful right if you're a young person and now you can ride a bike you can go to places you couldn't go to that before so the world is a much smaller place because you can now ride a bike and in the same way if you like good learning makes the world a smaller or bigger place but in a different place and kind of gives you may be a
different viewpoint on on what what's happening around you and then the 3rd the 3rd point about riding a bike it's a great example in uh learning through failure right uh in in this picture uh there's the grass and is
the helmet and so you know you fall off and you fall off and you fall off and you can you cry and through failure and
trying again and try trying out different ways that you learn and I think that's very important that people have challenges in and it can be hard but that ultimately it's safe and you're not going to die from an energy I learn to ride a bike
on on the street and so for me it was the 6 or a little bit higher but I did you know I'm OK I am not a ride a bike and then the last piece of kind of my learning story with this picture is really is a very personal experiences it's I taught my sister how to ride a bike analysis is 4 years younger than me and she she lives in Berlin she's a curator at the hamburger bun hold which is an amazing museum and you should all go there and look at her exhibitions them but I taught her to ride a bike because of my so she was struggling like somehow father could explain it to her and she was she was afraid and she she did wanna get on the bike but I was close that you're right I learn to ride a bike just a few years before so I understood that she was afraid I understood what she needed kind of where she needed help but what you help with and that experience of like being
invested in someone else's learning has been just incredibly powerful uh um uh I guess the change in how I perceived of learning where when you are starting to
teach someone else you probably learned a lot more than you learned when you were learning in the 1st place and so adamant that applies solely
to the bike but my sister and I became very close we trusted each other and so there's all this other stuff and learning that I think is magical and that's part of this adventure that we don't often talk about
the contest and and quizzes and so on it would be kind of if we remember the slide with the where everyone
falls asleep when peoples talk on stage we kind of hypocritical if I just continue to talk on stage so I thought it would be interesting is to do a little bit of an
experiment and to take a moment and for everyone to understand maybe 10 or 15 seconds and think about who you are today
and what you like and what you do we passionate about and then think about a learning moment that made you good at who you are today so if if if there's kind of a of a preparatory track to become the person that you are then what is the moment in that history that you feel like
you really learn something that made you the person who you are and so that we can take 15 seconds to think about that quietly and then I want everyone to turn to their lives neighbor on either side and share stories and we can take 1 or 2 minutes and we just I just want people to share their learning moments and stories and I'm gonna drink water when you
do this and the yeah the kind of a kind of of the of the of the of the of the of the course of
yeah right right right right right right right right on the
it right right right right
right OK this is great I maybe I should just leave and you can you can carry on and solve for the people have started
conversations and hopefully those conversations will carry on and at the end I am my Twitter handle at the end of the slides and if anyone wants to share stories I would love to hear them but am I suspect I've had lots of
conversations with people were asked them this question and often the 1st answer is a story about a very inspiring teacher or professor but often in school or in universities and I think that's a really important piece of it that we meet uh these people who inspire us to go in different directions but then when you talk more about kind of the the the experiences and you know how you became more you are often the can of next things that come out the door in many cases haven't happened in schools or universities that happened by working on projects ordering internships for doing a thing with your friends and they often uh based on
passions that people had so I have an inch
I had this unquenchable thirst for knowledge about space or whatever it is whatever you're interested
is it's like that's your somehow that led you along and that made you learn something and it's often it's so we talked a little bit about this so it's very rarely that you're sitting at a desk when those moments happened and and it's very rarely that you under pressure to learn that you may be
working very hard and there may be a high risk but it's often not that someone else tells you to learn this but it's more that you really really wanna
learn this thing on and so I feel like when we talk about these things we we know what the learning is that we all felt it and we can all put put a finger on
it mn so often we look at the institutions of how we organize learning we seem to forget all of that and we say well this how we've always done it is the guy on the stage he
tells everyone at you know either long videos on short videos essentially I'm and it I feel like we're at this at this moment now where we can go to directions and with technology 1 is the
direction of that I would call transport which is basically the university becomes this highly efficient kind of road network you wanna go from Berlin to come book The
University will get you there on the fastest possible way by train radial streets is that you get to choose where you wanna go but then it's kind of like you to follow all the roads there's no way you can view of the
roads and the other thing that I'm more excited
about is what I call adventure which is so you wanna get to the top of the mountains and mountains also great metaphor for learning but you wanna get to the top of the mountain and maybe others have come before you and they've mapped out the route but there's no where which says you can't go another way and actually the better you get the more you wanna go a different way the more you wanna like people have come up with all these names for the different pieces here's all different people like they discovered like little pieces of this thing and
altogether they build this learning pathway and it was a huge adventure and then the places where we learn should look more like this right it's like the new challenges you get to the first one you go to the next 1 but if you fall off it's OK because you're hanging on the rope and there's a guy in them into being the uh the standing there and he's going to make sure you don't kill yourself and then when you get really really good you should have an
experience like this right that's what learning should be like where I mean that must be amazing imagine you that guy and you've not climbed up this building and you don't need the rows anyone anywhere so maybe I I get a little carried away and so I I guess my mission has been how can we create adventures like this that involve learning and technology and openness and I found project
co-funded with a lot a bunch of really amazing other people a project YouTube University and the basic premise was really just
that we will all kind of dueling along in the open education space and we we just sent to each other here there's the web there's all this amazing content out there and there are millions of other people that you could learn with
and wall let's just do something and was so we started offering courses it's all what run by volunteers and we've experimented with a range of different models the most of it hasn't worked and I'm happy to talk about all the things that haven't worked and then some of the things have worked and I talk about some of the things that have worked in a little bit and I now and base of this place and and so is the lady who sits right there and and this X you can see where my said right there if you ever come to visit in that little corner
there I will be sitting there looking out the window and and the Media Lab is an interesting place because most of my work has been outside of institutions because I've always kind of poked fun a little bit of the existing models and I want to challenge them and the media Lab is an institution that challenges the idea of the institution and it has 3 ways that it measures but what it's doing them into your non-hazardous but its impact its uniqueness and it's magic so any project that the media does has to have those 3 characteristics and if you have an evaluation of your project you can use those
3 characteristic and say my project had had maybe not so much impact it was pretty unique but had an amazing amount of magic and magic is 1 of those things like finding the you know when it's there maybe you can do a rubric or multiple choice test to see if it's magic enough but you know it when you see it and then also what what drew me to the media that is the media has a very very um strong uh philosophy of learning partly because of some of the amazing people who work there and on research like Mitch Resnick who runs the Lifelong Kindergarten Group which is the group that I work with and but the the philosophy of learning at the media lab centers around 4 components 1 is you always have to make stuff like this nor you can have an interesting idea but unless you make something with the idea it's not a you know it doesn't really count you need to make it because when you make something you learn a lot more than me just think about it and usually the 1st try goes wrong and so then you learn a lot about doing it better and and iterating on that secondly you follow your passions you find out you find something you are really really interested in and then you just go for it and if that doesn't fit into the curriculum doesn't matter there's space for pretty much anything at the media 3rd element is this element of playfulness which by playfulness I don't mean we're uh it's all fun necessarily but it's something where you can fail and it's safe to fail you can you know like computer games you can do something doesn't all you get killed you try again you try again you try again eventually you get it right so there's that kind of that playful uh approach to learning new things and it's peer-to-peer peer-to-peer doesn't city mean that everyone knows the same thing I see quite the contrary the peer-to-peer is just describes the power hierarchy between the people engaging with each other there's always someone who knows more than you in a certain area but if you talk to them on the same level you both learn a lot more and so at the media have to for the most part the enemy of isn't perfect but for the most part people do engage on this peer-to-peer level and is often hard to know if someone is a student or professor or researcher or just visiting some guy walked in from the street and and I think that's kind
of an important component of this um OK so I wanna talk what was so that we talk about moves is I wanna talk what to projects that we played around with uh and um and I should say that I am I
wasn't so excited by the books and I still am not but um the 1st and then that explains maybe the 1st project we did which was really more of a joke and so we saw all the smoke being developed huge amount of money being spent on them every was very excited because of the massive numbers of people that signed up I just felt like we already had all those pieces in place and just using the web and a little bit of open source after we could build things that aren't like the moves but there are kind of similar them and so we set out to build what we call the mechanical mook and the idea was you know we were just going to use resources that already exist we weren't going to build anything else we were going to build any real technology just use services that anyone can access and we're going to see what we can come up with and and see if people wanna do it and so what we what we did is we
we chose a topic that you know is very attractive it's learning Python which is programming language that a lot of
people who want learn and we picked 2 really good content from MIT OpenCourseWare that's available for free on their website and we picked tutorials from called Academy which is a start-up that helps people went to college and you can do these little experiments on line you you submit your
answers it's kind of the the similarity of the many quizzes except you are actually writing code them and then the we use OpenStudy which is a big question and answer form for people learn online and so they can go there they can ask a question other people may help them with 2 kind of comes through it we've run it 3 times it's at about 10 thousand people signed up it's cost almost nothing and it's now we have we have these donations like in week 7 such basically what the way it works is you sign up and we send you emails that's it and we just point you to the right
places and you go and you work with the other people and you figure out the rest yourself and in week 7 we ask if anyone has a good time or has had a good time were still around in week 7 you wanna drop a few bucks into our ordination jobs that
would be great because we've to Paterson of these e-mails and some people don't so 1 is that the survival strategy for open
education projects based on small individual donations does not work for these not fast and the amount of money you get is ridiculously low but the cost you have to run something like this is also ridiculously low like this doesn't cost more than 200 bucks now had to run for each round just for sending out the e-mails that we don't really do anything anymore and we get enough donations to cover the cost of e-mails we build a sustainable book it just happens to cost only 200 dollars to run them so we did the 1st 1 was a bit of a
joke of it we were kind of surprised by how well it went and also we got a little excited about it we write while if you have 10 thousand people sign up you can do really interesting things with them you don't just have to send an e-mail them and so we get a lot more ambitious and we said why we try this with a media course and so that's what we did we pick the course that's taught at the media lab to students once a year called creative learning technologies we call the learning creative learning them and uh this is the professor and also the person work with Mitch Resnick amend and that it's a coincidence but he looks like the Lego plow in the in the background of the video which is which a lot of people commented on and which is kind of became a bit of a joke or mean of the course and and uh he teaches a course to 15 or 17 students I think this semester almost over we have 1 more session next Monday and we we thought well what happens if we open this up but we open it up kind of in a way that would make sense for the online participants because the class is very hands-on like there's a
different topic you work on every week and how you know how could we translate that online and we also knew that we didn't really know how to do it like that this is going to be a massive experiment lots of things were going to go wrong and it actually says that here and it says like all the 1st 4 5 e-mails we send
everyone we just repeated this over and over this is a big experiment things will break book that was it we don't have all the answers sometimes we plan and the nodes as anymore but it's basically we invited people to help but to figure it out with us and they really responded to that they've they were much more
engaged in a kind of bought into this idea that they can build the scores with us which is great because we didn't we really don't know how to do this and sold because we weren't sure how to do it we also than 1
vessel of money in building a platform of tools so we used a bunch open source stuff like a sign a form and we use a the same e-mail schedule that we use for the mechanical moved away we could just mass e-mail people in small groups and then we use Google + tools that we use Google + communities and we use Google + hangouts extensively I'm not because we think there are necessarily the best tools but what we liked about Google + as it starts with this idea of social rather than content there are lots of other platforms that we could have used like Google has a course a platform that's that's you run these large forces that starts with content from and we wanted to start with something that feels differently so we started with Google + because we like the social aspect and we love to hang out some and it doesn't cost you anything to do this you can and you know anyone could just create an accountant and and and run it through google +
some so that they may be 1 word about number so we had about 25 thousand people
signed up for the course and then in the Google + community which is really the core of where the the collaboration happens 10 5 just under 11 thousand people now and the other people just get the mouse and they they work on their own we wish to be honest we don't really know what they do
but they do not they weren't interest in joining the community and and then we invited people to start their own little groups because we feel like it's really important that if you have this large community that's also but if you have a small community it's a very different quality of interaction is you get to know people over kind of the weeks of the course you learn to trust them you get much more invested in their learning and you become friends right I
and maybe the best in with some of the best outcomes of of learning and education are that you make life on friends so we all amenities for me I have some of my best friends are people I met at school and university and just because you spend time with them and your learning together and you can of get to know each other
and so we feel like the small groups are really really important and we so this is 1 of the things that we really haven't figured out yet not have looked at the survey data and like some the small groups is a problem we haven't forgotten but was also
was formed in 50 people started small groups so they were for at least 450 small groups and those are the ones that we knew about and we only knew about the because 1 of the participants made a spreadsheet and invited all the other participants to put their links into the spreadsheet so they created an archive and directory of all the groups and then they started giving the group's names so they had the team names and then they started kind of making jokes on each other and anyway so you had all the interesting things happening that happen but when you have lots of people in small groups and distribution
was really the um both global and the interesting thing about this map as we did make this map um uh participant from Italy made the map wrote a little tutorial how do you put your pen and then in the map and then became the curator of the map and over weeks people put the pins in the wrong place needed other people's bins and have the honor was or was there and he like very gently explained it over and over again and he made the map and and when we have the 1st exchange he said on the Google + he said you know be amazing if there was a map and we can also see where everyone else is and we we just said the that's an amazing idea you should do it and he made it and then we featured and he kind of became has become a really important member and collaborate this course but there so the point I want to raise this global but then there's also local and I think local is important so this is a picture we get from a local school where people meet once a week in the work on the course materials together and the way the course works is the little bit different from typical online courses this some PowerPoint weirdness over them and so every week we introduce an interesting concept from the creative learning space and and then we invite people to do an activity so they can still read about it we give them but we give them articles and if they wanted dig into those that's that's fine but we also we always start with the activity and so people than like experience the idea hands on and they reflect on the experience and we if we feel it's a much better way to learn and also builds community in a different way and my favorite 1 is the 1 we did in the 1st week which is the mask marshmallow challenge them and I'm kind of running out of time but anyway it's amazing it's you should
do is you look it up online the marginal charge you can do with your friends or you with 2 kids with your students or with you work work uh people and the the the best thing about it is the group that's really really bad it is business executives they the terrible the group that's really really amazing added kindergarten graduates to recent graduates of kindergarten amazing and from anyway
we did weekly seminars so we did kind of act we did a lot of things wrong but we did long videos that people had to watch um these
almost an hour long and it where there was conversations we invited experts and then we had a religious lively conversation with your friends and people could watch it I The life all recorded at is is made no some ends it was interesting is there was a lot more interest in the event character with people wanted to watch it at the same time than we had anticipated but we also gave them because it's hard to watch a video over a long period of
time and you feeling kind of you know alone we give them a very a backchannel and we've been experimenting with this open source package that lets you run these chat rooms around videos and you can embed the video here we can start and stop the video for everyone was in the chat room and and people really love to chat room so we after the 1st session which was machine on the
previous slide so this is the director of the Media Lab Joey till this is Mitch Resnick and that's Joyce sister Mimi Ito was an academic at University
of California Irvine and they were talking about interest based learning and they were talking about their personal differences meaning is like the poster child
uh mn students always great in school of school has PGs from all these the offensive this is
Joyce the opposite almost dropped out of school never finished college doesn't have a degree
and and both of them are just these passionate learners and and so there's a great conversation and afterwards in the backchannel people were saying you know that was a really good precision but the backchannel was much more interesting than Julian Mimi and we loved it and we told we told you in the they also loved it because this is like this that community that's kind of making it work for themselves and I think this is this is part of the reason why words like if you can make new friends through learning it you know that's a really important component of a of a great learning experience and um 1 1 last example of what people can have
contributed there's 1 participant who I think she's in the Netherlands with the she she made these illustrations after each seminar she made an illustration she shared it with the whole group and people really love them and that there is always a long discussion about and what you left out what you put in to to do the right way but just like the fact that people felt invited to participate and
we so the first one a mechanical move cost almost nothing to build this 1 we had a few people kind of keeping an eye on it but we didn't build it we didn't go fancy software and we
didn't produce high quality videos we did it was not an expensive thing and it was fun and I think all the like Mitch loved it and the other people he runs the course with had a really good time and they connected to a much larger audience and so I think it's a good example for experimentation prototyping there you don't have to be an expert you don't have to be a super d you can use tools that are out there that are available to anyone free of cost and and you can just be playful and you can come up with new ways of doing this and I feel like this is such an amazing moment for more of us to do this and all of you have skills and experiences with learning maybe frustrations that you could start building learning experiences for others uh if that's something you interested in I'm not saying everyone should do this but if that's something you're interested in and you wanna get started the you have an idea like I'm super happy to talk to anyone about this and self like my to this my 2 points are really that I would like to see an education future that looks more like adventurous rather than transport and there is an opportunity
to build that right now I and I feel like it's have 2 important to to let the opportunity slip away and then secondly like practice over theory like
if you wanna get involved you wanna build something don't feel like how held back by maybe you're not an expert or maybe you haven't studied
pedagogy haven't studied pedagogic actually most of the many of the very interesting people in this space are not education specialists their open source developers they come from community organizations they just bring a really different lens to this anyway there's goes up to Twitter
handles that you can find me at Schmidt fee is a little more personal quirky and inappropriate people use more professional learning stuff and then please reach out and get in touch and I think we have and we have a minute for question or are we out of time we have time sorry I anyway thank you very much
if I think the the