Class learning analytics with Raku
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 287 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/57091 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2022244 / 287
2
4
6
8
12
17
21
23
31
35
37
41
44
45
46
47
50
62
65
66
67
68
71
73
81
84
85
86
90
92
94
100
102
105
111
114
115
116
117
118
121
122
124
127
131
133
135
137
139
140
141
142
145
149
150
156
164
165
167
169
170
171
172
174
176
178
180
183
184
189
190
192
194
198
205
206
207
208
210
218
220
224
225
229
230
232
235
236
238
239
240
242
243
244
245
246
249
250
253
260
262
264
267
273
274
277
282
283
287
00:00
Computer fileMereologyStudent's t-testPersonal digital assistantSocial classSoftware testingUniform resource locatorPhysical systemArithmetic progressionForm (programming)Right angleNumberObject (grammar)File formatMultiplication signCodeOrder (biology)Maxima and minimaProcess (computing)Projective planeDrill commandsPoint (geometry)BitAnalytic setPoint cloudTest-driven developmentRepository (publishing)Table (information)Row (database)Revision controlLatent heatData managementDiagramComputer animation
07:05
NumberDifferent (Kate Ryan album)Set (mathematics)Total S.A.Object (grammar)Social classMessage passingStudent's t-testDecimalLocal ringBitWeightPresentation of a groupFault-tolerant systemComputer fileNamespaceVirtualizationMultiplication signFrequencyMathematicsFlow separationProcess (computing)QuarkPseudonymizationLatent heatComputer animation
14:05
Multiplication signCodeStudent's t-testData miningObject (grammar)StatisticsReal numberINTEGRALRegular graphProcess (computing)Line (geometry)Streaming mediaAdditionSinc functionComputer fileSocial classRepository (publishing)InformationView (database)Point (geometry)Group actionSet (mathematics)CASE <Informatik>Latent heatElectronic data processingVideo gameExpressionPower (physics)Analytic setFamilyDivisorScripting languageSingle-precision floating-point formatUML
21:04
Social classEstimatorObject (grammar)Sinc functionData managementMultiplication signProduct (business)Goodness of fitSoftware developerModule (mathematics)Student's t-testComputer fileLibrary (computing)Order (biology)Computer animationMeeting/Interview
23:56
Computer animation
Transcript: English(auto-generated)
00:08
Hello, here I am again, JJ Morello. I'm going to talk about class learning analytics with RAT. So, I'm teaching cloud computing in, well, right now, in 2022, 2021.
00:25
What I'm going to try and do is, well, I'm going to try to make students learn, basically. What I'm teaching is agile development, and then a little bit of DevOps, and test-driven development.
00:41
That's not really so important. As long as you're teaching anything that's related to programming, or anything else for the matter, this is going to be pretty much the same. But from a point of view, and I have always defended that everywhere, is that learning is practicing, so if you don't practice, you will not be able to learn, you will not be able to remember.
01:02
So, I use something called project-based learning. Essentially, it's that the students are doing not stand-alone drills or assignments that have nothing to do with each other, but they are progressing through our project, which is a project for the whole semester.
01:22
Then, this is something which is called flipped class. Flipped class means, essentially, that the time that you are giving, broadcasting some teaching is reduced to the minimum, and you are going to give only some, you're going to teach some students on demand.
01:44
I'm going to have some questions or whatever. Theoretically, flipped class says that the students should prepare all their material at home, and then they should ask whatever is in there. In practice, that doesn't happen because, of course, students don't do that.
02:01
But what I have done in practice is that I give very small conceptual classes, like 20 minutes or 30 minutes, and then the rest of the time is interacting with the student. And then, this year, for the first time, I have used something called formative grading. That means that
02:21
grading and assignment is not the end of the learning process, it's only part of the learning process. So what you are looking for is students reaching some objectives, and you will have every student reaching their objectives at their own pace, so whenever they want.
02:43
Of course, they have to reach a minimum amount of objectives to pass the subject, but everything else is, you know, you are not telling them, no, this objective is not reached. You're out, but you tell them what they need to, in order to support. What we do is, every one of those things needs some code.
03:05
That code has been written around four or five years. This year, I have done a part, a very important part of it, essentially. If you're going to do project-based learning, you have to get some tests, so that when
03:21
a student submits their objective, you test whether at least some basic part of it is right. Of course, you will have to check out another part, basically, as in any pull request. So the students submit a pull request, and then they pass some tests, and then we have to review it to see if the objective has been reached or not.
03:45
And then, if you're going to use flipped class and formative grading, you need to control the individual progress of every student. So you need to know what every student is doing to offer some help, or to actually know where everyone is, because that class is not progressing in lockstep.
04:03
You have everyone reaching objective number one, and then some of them failing, then objective number two, some of them failing, and so on and so forth. But you might have people going all nine yards, from objective one to objective nine, or even ten. Right. So, basic idea is, I want the students to learn in practice, so instead of having some form or some learning management system
04:31
or whatever, we just pull GitHub, and then they create a pull request in their own repository, so that I can review that pull request,
04:43
and then they submit the URL of that pull request to my repository, so that I can pass tests, and I can be warned, or I get an alert saying that there's a student that wants some grading.
05:03
The tests are basically wrong in Perl and JS. They are wrong in JavaScript because it's there, they are wrong in Perl because it's there. I've already given several talks on why Perl is in every workflow or CI environment, so, well, this is what I do.
05:28
Right, but then, we do code reviews, and when I say we, it's not only me, or I don't have any teaching assistant this year, so it's essentially me, but also increasingly the students are doing the reviews.
05:45
Of course, you want your Raku, and I'm going to give you your Raku. It's going to be right here. We're getting there little by little. I mean, the analytics part is the one where we're going to use Raku.
06:01
Why? Because, well, it's very practical. Since students, what they do is they submit pull requests to specific files. I use that file to mark whether the specific pull request passes or is accepted, so I
06:24
accept pull requests in their own repository, and then I mark that pull request as, you know, check. It's passed, and it's marked in the same row, and students always have the same row, so they, in order to avoid conflicts and things like that,
06:46
they know which row of the table they have to modify, and even in one objective, for instance, they have to pull request to a different repository. Even in that case, what we are doing is exactly the same. They always have to choose the same
07:04
row, so I know that if a student, like for instance, you can see here, if a student has, alcantado means reached, has reached another objective, I know that there's going to be a mark, a check mark in that specific row, and that's essentially the raw data I have to know if there's some objective, fulfilled or not.
07:26
It's public. It's there. The students can use a pseudonym if they want, or they can use a class-specific account. I mean, that's on them, so if they want to use their own account or their Quark account or whatever, I'm not against that, so it's up to them.
07:45
I try to show them that GitHub is a big resource. It's their portfolio, so if they really want to leverage whatever they do in this subject for their work interviews or whatever, job interviews or whatever, I mean, they're very free to do so, and I really encourage them to do it.
08:04
At any rate, what you see is I have several markdown files, which are like this, right? So, I mean, the whole idea is that you need to have the students to progress towards every objective, right? So, you review, they submit that, it passes original test, I say, okay, if it passes test, I can
08:27
go review it, I review it, and then I tell them, well, I request changes, and then it changes. So, objectives can be not submitted because the student hasn't known it, not evaluated,
08:40
or it hasn't passed because I have requested changes and it's not reached yet. It can be done, or it can be on Falfit, so eventually when the class period is over, I can't offer them the possibility of resending and resubmitting and changing stuff all the time, just because it's not legal to do so.
09:07
So, if by the time of the final grading is submitted, it can be on Falfit, so that's another option, right? But we need to know how students are progressing their objectives, so that's the basic idea.
09:23
I need to know if some students have dropped at a certain objective, for instance, number two or number three or whatever, I need to know also how many students have passed every objective. I mean, in general, kind of analytics, so that I know how difficult is every objective, if I need to
09:43
give them eventually a different weight in the final grade, whatever, so that it's essential to know what's going on. So what I have essentially is a set of, the raw material is a set of markdown files, and then I need to know how many students are in every objective. That's essentially, it's not much more complicated than that.
10:08
And that is where I use something that I call IV. IV is the infrastructure of virtual, virtual infrastructure is the name of this address, so that's why I've used that namespace.
10:21
And then stats, because it's essentially statistics. Again, this is just an objective, the markdown files with objectives, I'm not paying grace on anything else I want, so maybe eventually I want to know something more complicated, like for instance, how long does it take from initial submission to passing objective, that's something I intend to do.
10:44
When the subject is over, which will happen, well actually it's happening as you're listening to this, because the final presentation was a couple of days ago. Well, so, I want to know what are the grades, so I have, as I said, I have
11:05
a weight assigned to every objective, and I want to know the grades, and it's as simple as this. It's a bit more complicated, because when I obtain the values, they are in the American or English locality, I want to pass it to the Spanish locality.
11:23
We use a comma for decimal separation, instead of a dot. But that's essentially it, so this prints something, and I can copy that directly into a spreadsheet, which is what the students see. As I said, I was interested in knowing someone has dropped out, because you have to
11:46
follow every student individually, so if I know that someone has dropped out in objective number 3, I send them an email, a telegram message or an email, telling them, can I help you, anything where I can suggest some additional material, some specific thing that you don't know how to do, whatever.
12:07
So that's one of the things. You also kind of get to know, personally, every one of the students, and encourage them to continue. Or, if you see that the time needed for doing some objective is kind of running out, because classes are about to end, you tell them,
12:28
Well, probably you should try and start with this other objective, because you're not going to run out of time. So, I use this simple thing. Again, it's simply an object that sets stats, which, as I
12:44
create the object, I read the Martin files, and then I check out a couple of things. Bajas totales, which is the total number of people who have totally dropped out, or bajas objectogetivos, for people who have submitted their objectives, but they haven't passed, so it's still in progress.
13:06
It's essentially a set difference. It's not a complicated thing. Every objective, what I have is a set of students in every objective, so the only thing I need to compute is a set difference.
13:21
So, as you see here, it tells me that from objective number zero to objective number one, there were five students that dropped out, and actually what they did was not even submitting the following objective. In objective number one, they simply dropped out because they hadn't passed.
13:42
And then, from objective number two to number three, there are differences. There are some of them who didn't submit it. There are some of them who submitted it but didn't pass, and that's what I observe here. So, I use all the emojis and things like that, so it's a little bit clear. It's actually not decoration. It's not static.
14:03
I want to kind of see standing out what's happening there. This is actually very useful, so I do this from time to time, every week or every couple of weeks. I check how everything is doing. I contact them. I tell them what's the matter, so I don't want people to be left behind.
14:28
In some cases, it's essentially due to factors that are not totally unrelated. Actually, in most cases, it's totally unrelated to the class itself because they are working. They have some family problems or anything that's really not related, or simply they have other classes that take much more
14:48
time, and they are maybe less risky because at the end of the day, you have to work to achieve the objectives. Some of the things that I'm interested in is how many objectives have been reached. This is interesting because, as you see, I put it in a JSON file.
15:07
So, in this case, essentially I'm taking every set for every objective, I'm converting it into a list, and I'm putting it into a JSON file so that I know that objective 0 has been passed by this set of students.
15:21
Objective 1 by this set of students, and so on and so forth, right? So, the good thing is that we can put this inside a GitHub action. So, I don't need to compute every time how many students have passed everything. So, whenever I modify a Martin file, I call this script, and it generates a new JSON file in this way.
15:48
So, you see that it's running in a GitHub action, and it's run in Raku natively, so it's a specific GitHub action that is run inside a Raku container.
16:04
And that's something that I have been talking about, or I will talk about, in my other talk in this Dev Room. So, most of the things that I did to create and be able to work within Raku,
16:20
I mean, within a container with Raku, having applied, and this is the use case for that. And this is a nice use case, actually. So, in many cases, what you're going to have in a repository are artifacts that have been generated from what you have. Again, what I have is Martin files, what I want is a JSON file that helps me do something else with the information I have.
16:47
So, that JSON file is going to be committed and pushed to the repository, and it's essentially useful for one thing. It's because by the end of the class, what we were doing is for every pull request that was done,
17:03
the name of three students was generated randomly, and they needed to be three students that have already passed that specific objective. So, you make a pull request and say, okay, so I have tried to do objective three. Then, this JSON file contains students that have already passed objective three, and then it selects three randomly.
17:29
Actually, it goes a little bit further because it puts it into a file, and it generates automatically a comment in the pull request of the student. So, it's telling the student, these three other students are going to review your file, and since it's
17:44
a comment, they are going to be notified that they have been selected for doing that kind of thing. They do that, of course, for additional credit, and the experience doing so has been really, really awesome. Why? Because you realize whether the students are internalizing, first, the art of doing reviews, which is not easy at all.
18:07
Second, how they have really understood what the objectives were, and they are able to explain it in a code review to one of their colleagues or fellow students. So, all in all, very interesting experience. I think it's been, I have evaluated how it's worked, and it's worked pretty well.
18:35
Only here, essentially, what you try is transparency for time. So, you know, you have
18:43
guaranteed that you're going to reach a certain grade, but you have to work for it. So, the students are not unhappy, they mostly have liked it, but from the point of view of Raku, what are our conclusions?
19:01
First is that we have very easy data processing and handling in Raku. As a matter of fact, I'm working with the mardown files, but I don't even check them using mardown. I'm using some very simple stream processing or regular expression, so it's not a big deal. It's
19:21
easy, and it's fast, and can even be included in a GitHub workflow. So, you know, easy peasy. Now we have various integrations. This is something that I coded in the last month or so. So, this integration with GitHub Workflows
19:40
also gives you a lot of room to be able to do lots of things, you know, generating all kinds of files, charts, whatever. You know, I mean, you name it, it's something that can be done. Besides, this stats has zero dependencies, but we could very easily add some real
20:03
statistics or work with Git and check out, you know, do some Git mining and see how students were doing the pull request, and so so forth. So, many things can be done. And as you see, it's awesomeness in a few lines of code.
20:23
All the scripts that I have created on the library, maybe all together, they are 70 lines of code. So, it's, you know, being able to work with sets, being able to process things in a single line of code is really awesome. It's something that, you know, gives you all the power and expressiveness of Striker, which is what I like. I don't know if class
20:48
learning analytics is really a killer app, but it's really something that has made my life easier, and hopefully also the life of the students. And that's it.
21:02
Thank you very much.
21:27
Well, hello there. So, this is going to be back to the end of the video, so I might as well say something. So, the thing is, since then, that was like a couple of weeks ago, and I really wanted to do a few things with it after that.
21:45
So, the main interesting thing is, I was interested in knowing first when people were submitting the objective, because, you know, they have absolute freedom of when to do so. But I wanted to know when they were doing it, actually, so that I could kind of estimate how much time was needed for every one of the objectives.
22:06
And in order to show what I did was to create a specific library that does something like Git mining, so it looks at history. And after looking at history, what it does is it tells me when every one
22:26
of the objectives is submitted, and when every one of the objectives is fulfilled or not. So that I, it's relatively easy for me to know when one of the things have started.
22:43
And that's a very awesome class analytics, because when I started with this experience, I didn't know, I still didn't know what the, how much time was going to be needed for every one of the objectives. I mean, I could say, okay, this is going to need a week or a couple of weeks, but you don't really know how long it's going to take to actually, you know, do so.
23:06
So, I created another module. It's called git file history, git colon colon file, colon colon history. And then after that, I added another module to this IV stats, so that I can use that directly to do so.
23:25
And, you know, it's really been awesome. So, I mean, the fact that it's so easy to work with data and do that kind of thing is, you know, is very, I mean, reduces the amount of the development you need to.
23:41
And also, I think that is a very good example for students to see that we are actually using, you know, some best practices to work with them and we are as product managers.
Recommendations
Series of 6 media