Open edX Meetup Cambridge, MA January 25, 2017

Just another WordPress site

Open edX Meetup Cambridge, MA January 25, 2017

So we opened the registration for the conference today, right Joel? >> Yep >> Yeah, so the next Open edX conference is happening in Madrid, which is in Europe >> [LAUGH] >> That should be exciting It’s May 24th and 25th, which is a Wednesday and a Thursday of that week So I’m gonna be there and Joel’s gonna be there Joel by the way, is the manager of the Open EDX team here at EDX And George, where’s George? [CROSSTALK] He’s sitting at the front, the guy that was checking you guys in That’s George Song, he’s actually remote from Portland, but he’s in town for a couple of weeks and he’s also a member of the team The Open EDX team, our goal is to enable and encourage the community to use Open EDX, which as you probably know is the open source software that powers edXorg So edX has a goal of teaching a billion people by some point in the future, and I’m hoping that half of those billion are on sites other than edXorg Right now there’s about 300 sites around the world, that are doing online education using Open edX, which is very exciting because, in addition to other high quality institutions like MIT and Harvard, there are all sorts of non-universities, and companies, and kindergartens, and pilates courses that are happening on Open edX around the world Stuff that we would never do as edX But if you put the software out there and let a 1,000 flowers bloom and do more education, so I’m very excited about that as you can tell What else do we want to say? >> There’s a call for papers for the conference which is open today as well Is there a deadline for when that closes? >> The first March 1st >> March 1st So we have about five weeks to get proposals in for presentations that you’d like to give at the conference in Madrid And we’re interested in all sorts of topics There’s technical topics, there’s of course offering topics, there’s [INAUDIBLE] topics, there’s operational topics, there’s how to make videos kind of topics Anything that would help people do online education with Open edX is fair game for a topic So if you have any inkling at all that it might be an interesting topic, make a proposal, and we’ll work it out, and see if we can get you up in front of people Madrid What else? Anything else? Yeah, okay So the topic tonight is how MIT uses Open edX And this is of interest to me, personally because as you may know edX was founded by Harvard, at MIT So, MIT had a big part of our starting, and MIT has courses at edX.org, but they also have people who write software for Open edX And they also run courses online, and campus, and every once and awhile I hear about another thing at MIT that’s happening I thought, as well as you guys learning about, if I could get Peter and Jolene here to come and talk about it, then I would learn about it And so that’s gonna help me to understand the whole MIT landscape You’re gonna tell us about it, right? >> The whole MIT [CROSSTALK] >> The whole MIT landscape >> [LAUGH] >> All of it, yeah? >> At least all that I >> As Ned mentioned, you’re from MIT I am Peter Finch, I am the Associate Director of Engineering at the MIT Office of Digital Learning, MIT ODL And joins one of our MIT learning fellows in Physics, and we’re going to try to tell you everything MIT is doing Not a chance I’m hopefully going to be able to tell you a lot, and then you can follow the next week more stuff So first of all, what is ODL? It’s Office Digital Learning We have a mission statement, the important words here are everywhere, digital, and education So that includes on campus and across the world, I’ve got a wonderful org chart In my experience a new box gets added to this every six months, but there may be some familiar names here These are some of the groups that are using Open edX as part of the MIT Office of Visual Learning You might have heard it MITX, that’s the group that works on MOOCs Got a couple folks from MITx here if you wanna talk to them about that We also have what’s called MITili, that’s MIT Integrated Learning Initiative, and that’s a group that’s supporting research Fundamental research on learning, and as they do research and support projects on our digital learning, it’s also being incorporated into new courses So there’s a cycle involved with that We also have a group called the Digital Learning Lab,

that folks like Jolyon are part of And these are scientists and learning fellows who are embedded in the departments at MIT, and advocate and support the use of edx.org And Open edX in the individual departments, and because they’re familiar with the content, they can really push the platform in any direction, which Jolene will do later We also have a residential education group, and this is really our biggest user of Open edX We run our own instances of Open edX on campus, right now about 90% of MIT and graduates have taken at least one course of some content through Open edX Now that’s, I’m forgetting the exact number, but that’s a couple thousand so, we’re not contributing that much to Ned’s goal, but it’s an important part of the MIT experience And not only is it a great learning platform, but it also helps faculty in a lot of cases, teach more effectively and efficiently Open Courseware, you might have heard of them, they just were celebrating their 15th anniversary And since opening Foursquare is representing the entire MIT curriculum, they actually bring in some of the content from Open EDX on campus And convert it into their platform and distribute it with the open licenses and And then the last on this list is my group, we’re the engineering part of the Office of Digital Learning And we run Open edX on campusm, so we’ve had operational experience with Open edX And then we’d also do software development, mostly centered around integration, but also some content and other stuff, which I have talked about [COUGH] There’s a lot of stuff, hat MIT does with open edX This was a sampling that I put together, and I’m trying to come up with some themes about the kinds of projects that we take on And it kind of fell into three groups We’ve done some work on new assessments within the platform, filling in some gaps that were peculiar to our needs This list is not comprehensive, for sure But these are some of the things that my engineering group has worked on, and I’ll dribble it down into a couple of these We’ve also been very interested in content reuse We’ve been running Open Ed X courses for, I can’t count, almost five years now So we have a tremendous amount of content, and trying to figure out the best way to find it and reuse it Effectively it’s very important for us, and then of particular interest for my group is integration, so we run it on campus We have to integrate with campus systems, things like grade books, and authentication, and publishing systems, so we spent a lot of time thinking about APIs and stuff like that So just running through these really quickly, I’m going to go into some more details on these, but in the assessments space, we work on staff grading If we have time, I’d like to show you this sketch input assessment, which is really cool It’s developed by one of our, well, many folks in our math department What we’ve been doing a lot with exams on campus lately, in content reuse, one of our first projects was, Letech to EDx How many people use Letech? >> [LAUGHS] >> That’s more answer than I expected actually >> [LAUGHS} >> There’s a lot of Letech at MIT, and when you think about all the great teaching content and when you’re delivering on to EDx and fit that into that platform CCX, which I’m gonna talk about later, is custom courses on EDX It’s a very light weight approach to content reuse, and I’m not gonna go into a lot of detail about LORE, it is a more

detailed learning object repository for education It’s a tool for finding, and curating, and cataloging EDX resources And then on the integration side, we’ve done a fair amount of work integrating EDX workflows and authoring with Git again Maybe not something that most course authors don’t wanna do, but some of the advantages of version control and distributed authoring are really powerful Another thing I don’t have a lot of time to talk about today, but I can take questions about, is our Micro Masters Portal It bridges the gap between the MicroMasters courses that we’re delivering through edx.org, and MIT’s program in MicroMasters It’s actually leading into a master’s degree So there’s a lot of exchange of information about which courses people pass, then whether they’re applying to which program And we’ve written some libraries, we’ve done some work in LTI And Jolene’s gonna talk about the clickers project, and maybe some other stuff So I thought I would drill down into staff graded assessments, because this is something we have been working on for a long time, and it’s still evolving There’s a brief history here of all the different ways and the history of EDX, you might have attempted staff grading One of the challenges with a Mook is, it’s designed to grade a lot of people, but when you run a course on campus, you’ve really gotta be able to do the basic thing of, submit a homework, review the homework, mark it up with a red pen, and give it back to the students So, that was something that was missing in the platform in the early days, EDX I think like peer grading through ORA So we were at x block called SGA, staff created assesments And then EDX iterated on ORA and then we iterated on the staff grade assessment XBlock, and then EDX added staff grading to ORA That was great, but still wasn’t everything that we wanted [LAUGH] So our latest version is what we called SGA LTI This is my description of the original version of SGA This is actually part of, it’s an X block but it’s delivered along with Open EDX now So if you install Open edX, you get this x block It’s not turned on by default, so you have to turn it on, and its basically, this lets students upload a file The instructor can download it, mark it up, and upload it, and say agree So in the current state of affairs in ORA2 with staff grading, we ran into a couple of issues that weren’t working for us So the ORA grading workflow is the same as peer grading workflows, so there’s a list of submissions, and the staff person gets one item, they get to grade it, and then they put it back and send the grade back to the student And then they take the next one off and then send it back to the student We had a course team where they wanna be able to look at all the submissions at the same time, and download them all, and be able to compare them to each other so that the grading is consistent Another thing about ORA is you can’t respond to the file Our case again we had, this is actually the philosophy course, we have Philosophy courses at MIT They wanted to be able to download the assignment, and then mark it up, and comment on it, and then deliver it back to the student Again there was like this two-way flow And then I, something that’s a little bit more of a serious issue for the platform Something that we’d really like to do some work on Is having a way for some people in the platform to be able to do grading, who aren’t staff So right now at Open edX you basically have students or learners, and staff

And then there’s like an admin role that we’ll look at more advanced than that But the challenge is that the same people who have been grading and reviewing student submissions, also can change the course and that is not ideal for us So we wanted a kind of way to be able to segregate those two groups Being MIT we referred to those people as human graders >> [LAUGH] >> So, we thought about trying to enhance our X block but that was going to be lot of work, particulary because of this issue about access control So we decided to use LTI which I don’t know if everyone’s familiar with, but it’s a great standard For integrating educational systems And it’s a very nice way to extend edX It’s basically a way that you can run an application anywhere on the internet and embed it in your edX course For other elements as for that matter if it’s a standard and it will send where you getting [INAUDIBLE] great sent back from the application as well So we just put it off on another server, we get to make up our own access controls at that point And I think I got some screen shots here but I can actually try to show this No and it went all black There you go Real quick, there’s a student view There’s a button where you can upload a file I have some legal notices that I’ve uploaded here And then there is a staff view where you can quickly see everybody who uploaded files And see their status And we also had a distributed system cuz we thought we could have like a lead grader And then multiple other graders and there would be a way for the lead grader to look over the shoulder of these part-time philosophers who were gonna do the grading And it worked very well There weren’t that many students in the course, but it was very effective, and the ones that took it were happy, as well as the graders So we did that this summer Let’s see if I can get the presentation to work The timer started over That’s too bad Guess I will have to talk that long Let me talk about content reuse As I mentioned before a lot of our content reuse originally was focused on reusing content from other formats We had a lot of projects with a two in the middle There’s edX, we have an open courseware translator that sorta works There was even a Moodle translator, but I’ve kinda disowned that at this point >> [LAUGH] >> I should mention though that all our projects are open source So if they’re all on GitHub, some of them are maintained better than others But I’d be happy to talk about any of it We spent a fair amount of time in our first year or two working on how to move course content around repositories When we started that was the only way to write a course Studios make things a lot easier now but it’s still useful to be able to do that And some of our more recent projects have been kind of on a spectrum And I have a problem cuz I have to hold the microphone You think about content reuse, I think of it on a spectrum of, in some cases, you want something that’s really dead simple You just want to take your course, and let somebody else use it

Let somebody else run it You’re working with the whole course, you want a push button solution And at the other end, you have advanced authors who are trying to do things like find the best single problem that has been written in edX on momentum And integrate it and drop it into a larger course that they’re writing So some of our content reuse is people who are just trying to deliver a course And some if it is people who are trying to really offer a course And these two projects are at the opposite ends of that spectrum CCX is designed to make it very simple and LORE is designed for people who are course authors This last one here is actually part of the edX platform and I just mentioned it so that you know that it exists There’s a feature called content libraries in edX and it’s integrated into studio And it lets you reuse content Although right now it’s limited in the way that you can reuse it It’s basically designed for things like a quiz pool or a pool of questions that you would use in a quiz, and it pulls them randomly The technology is there to do other things, but that’s the only that’s been [INAUDIBLE] I wanna talk a little bit about more in depth about CCX This is something that we’ve been working on for a few years now And it started with this fact that 30 plus percent of our learners for the MIT and Harvard MOOC specifically, identify themselves as teachers And we get a lot of questions from them of, how can I use this great content in my own teaching And it sounds like an easy concept, especially cuz the MOOCs are open registration But when you actually try to take a MOOC and try to integrate it with an actual course that someone’s teaching, like a blended course, there’s a lot of hurdles to that MIT’s schedule is pretty aggressive If you’ve seen how much we try to teach in 12 weeks Not to mention the fact that most of our courses are 12 weeks In a lot of cases the content isn’t scoped in the way other teachers would want it to be We cover a lot more stuff, or different stuff than they would want If you think about enrollment fluke a teacher who’s teaching a face to face class can’t actually control who has access to it Or, maybe more importantly, they would wanna have a discussion forum in the course but limit it just to their students And there’s no way to do that in an open enrollment MOOC The teachers can’t see the grades, only MIT can see the grades And licensing is a big question It still is but CCX has given us a framework for how we might handle that CCX has been in development for a while, but I think it was part of the Dogwood release It does require that you use SplitMango, but that’s an operational detail And in a custom course, we introduce this A new role of a coach, a course coach And I gotta be honest, there’s still some mixed feelings whether that’s a good label to use But the idea is the person who is going to deliver the course, the custom course that suit The coach has a control of a number of different elements, of course they can control who can access it, they can control the schedule, so you can take an MIT course and slower it way down If that’s what you wanna do and spread it out over time The coach can also access the grades, so they can keep track of how the students are performing, and the grading policy And something to note here is there’s been some recent enhancements to custom courses, that actually wasn’t implemented by MIT It was done by Davidson College and Open Craft And they tied in a bunch of the other features of the instructor panel So things like re-scoring students and

unlimited access to analytics and things like that But some important things, the coach can’t alter the content So remember we’re at the end of the spectrum where it’s just a delivery The reuse is more of a delivery than an offering So we don’t let the coaches actually change the content And that actually has made the licensing concept easier for folks, cuz they don’t have to worry about the, original authors are less concerned about their material remixed And there are also some size limits We don’t want people creating another course the same size as the original So I’m gonna see if I can flip over again To browser That’s not the right tab So Right now, this is the coach view of 700x, it’s the Eric Lander introductory biology course at MIT that’s been around 20 years And right now I’m logged in as a coach So, I’m someone who maybe took the MOOC and said, ‘I’d really like to use some of this with my AP Bio students.’ And, I made an arrangement with MIT to allow me to do this And I now have a tab here called coach that’s available to me And when I click on that I get more tabs Just basically four simple tabs here I can control who is enrolled in my course So I can add people by email address or I can have a list here of who actually has access to my Tuscan course And I also have this schedule where I can build up a course by just using the elements that I want from the original Eric Lander course So let me show you So the original course has more than a dozen modules to it, these were roughly weeks in the original presentation Huge amount of content And I can actually juts pick out the things that I want I can choose them by section I can also, very exciting exam, I don’t know why I picked that Let’s try something else here And you can actually drill down to the level of a unit, and take out the actual unit that you want And then add it to the schedule And there’s just the simple tree here of all the things that I’ve added I also had the opportunity to change the dates This is actually right now a lot of the default dates, cuz this course was originally delivered in 2013 But I can change those to anything that makes sense for my students And then we also have a very simple access to the student grade book And then for advanced users, you can modify the grading policy And I mentioned that the instructor dashboard is still available And then, when one of my students logs in They have my custom course, and instead of this list that originally went down to here of content It’s just the stuff that’s been assigned by my instructor, my coach So MIT’s used this with maybe a dozen custom courses Actually it might be closer to two dozen now, particularly some of the entrepreneurship courses have been used a number of times as well Actually, the biggest user of this technology has been Davidson College, which is up to, I think, five or 600 custom courses at this point They wrote a number of courses designed for AP topics in high school

And they worked with a very large number of high schools to get those distributed Berkeley has also used it with their computer science So that’s one of the ways people can And I think those are the things I wanted to highlight I’m gonna hand it over to Julian And let’s try to pause for questions >> Yeah, let’s do some questions now people >> Okay, sure Anyone? Yeah >> So anyone can access this CCS? >> And so the tool is part of the platform It’s been in Open edX The content requires some kind of relationship with the content [INAUDIBLE] We’ve been working on something that we were hoping would sort of streamline the way system process and like build a catalog of courses that were available, but that isn’t ready yet So it’s still a little ad hoc in that way But courses like biology, a couple of physics courses and in particular the entrepreneurship courses have been licensed a number of times this way >> And is it just the MIT content is available through it now? >> Besides Davidson and Berkeley I don’t know anyone else who is doing And their approach has been a little different They’ve been more focused on like large scale funded research as opposed to licensing But we’ve talked to some other places We’ve talked to a number of people It’s just been trying to get some of the things off the ground >> Yeah, so when you have residential courses like your philosophy course, right Was that running in parallel with a normal kind of chalk and talk, or whatever you >> Yeah, we have a lot of write and we really leave that to the faculty to decide how best they want to use the online courses So we have a number of courses that are mostly interested in the assessments and they’ve moved problems sets and weekly assessments onto the platform And those are popular with students because they get instant feedback There are some other courses where they’ve taken more of a flip classroom kind of approach and emphasis on the video So, students use the platform to watch a lecture and then when they go to the class meeting time they actually do more project-based work instead of spending that time in a lecture And they’re other- >> [COUGH] >> There’s a wide gamut of how it’s been used So sometimes only a module from a course will be put online as a way to let people spend more time with a particular topic then they might be able to spend in the classroom There’s a few courses that have tried experimenting with going completely online >> Yeah, I mean I’m also interested in what kind of comparative metrics you have for the effectiveness of using this versus the sort of more traditional? >> I don’t know off the top of my head Do you have anything that you would say about that? >> There are some groups that are setting up courses to construct [INAUDIBLE] education So actually get a [INAUDIBLE] how well the students are doing >> Right, right, yeah, yeah, I mean, and also you have satisfaction >> [INAUDIBLE] students have satisfaction [CROSSTALK] We do measure a lot The students generally love having [INAUDIBLE] >> I would say something, another thing that we have that I kinda think you need to add to the edX platform to is that we have courses that sometimes start as a residential to ramp up to go into a MOOC So it can kinda be like a launching board for them to get the class ready and try to run it residentially And then when they have somebody more stable then they can grow to it >> And the couple went the other way too >> A couple the other way too >> [LAUGH] >> I think we tested out the online material globally and then showed it to the MIT students

[COUGH] So I, and >> I mean, not the most exciting metrics necessarily, but there’s also been a lot of uses of online platforms just to automate and make more efficient some of the tasks, like grading and things like that So we’ve had a couple of courses where they’ve moved the quizzes, and periodic assessments, and even a final exam >> Code graders >> And things like code graders too in computer science and that’s made it possible for them to be a lot faster in grading and be able to get students feedback faster than they would otherwise >> Yeah, so the introductory Computer Science courses use the Python code grader, so that’s the same as 601x and 602x We also use MatLab And that’s been on MOOCs and also residential And people talk about more exotic things, but I haven’t seen anything in [INAUDIBLE] yet for us There’s a number of other code graders in the community But we have new singulars >> Thanks >> If anyone knows a good grader for optimization problems like advanced Excel spreadsheets That’s something that we’re looking for >> Can you just talk a little bit about the analytics that course teams do at MIT on Open edX open courses and how much of that is built in edX analytics and how much is homegrown stuff at MIT? >> Yeah, I can’t talk about that in too much depth But we’ve had, and maybe you can chime in here if I need more, but we’ve got a project, sort of informally called xanalytics, that has been somewhat parallel to edX Insights So covering some of the same ground, but focused a little differently on some of the measurements and processes that we were interested in And frankly, also, I think we nvested time in that so that we could also For lack of a better phrase, double-check edX’s results so that we have independent assessment of the outcomes And so we ingest all the tracking logs that the raw data that edX generates, and then run it through this xanalytics system We use Google BigQuery for the analysis of it, and that might be all I can say accurately >> Is xanalytics open source? >> It is I’m not sure how easy it is to use out of the box, but I could also put you in touch with some people who deal with speaking about it And we collaborate also a lot with Harvard on that too >> Did edX compare data insights? >> Close enough I mean there’s been some conversations, but >> [LAUGH] >> Yeah, I mean it’s never exactly the same when you’re doing something like that, but it’s in the right ballpark, for sure Yeah? >> How [COUGH] When the faculty want to use it, is the platform up and running? And of course, how do they use it, the on campus faculty and what kind of training do they have to have or do they just train themselves, that kind of thing? >> MIT faculty are unique bunch And we’ve been working with really a lot of the early adopter types And they’ve also been the kind of folks who really push the platform, and the flexibility of edX has been very appealing to them And I think one of the things that we’re trying to figure out now is how to expand the use of the platform outside of those early adopters We do some outreach and some training We rely on the edX documentation a lot, which is great, and we’ve also, we are starting to make some investments in the instructional design side of it but that’s been actually a new endeavor for us

So, It’s been a learning experience [LAUGH] And really, one of the biggest ways has been our use of these digital learning scientists and fellows like Joylon Who are established in the subject matter And Figure out how to adapt the platform in a way that makes sense to any particular department And I’m going to use that as a segue >> [LAUGH] >> I’ll be around for more questions after [INAUDIBLE] >> I need this one >> I think it worked before >> It did work before >> [INAUDIBLE] >> That there is informative Okay One thing, I learned how to make and 9 by 16 So, with that extraction, my name is a Jolyon I am a physicist at heart I tend to spend a lot more time doing computer programming nowadays >> Comes with Java, yes And I’ve been involved with teaching in the physics department for the past four years, I guess Part of that teaching has been using the platform to advance our residential courses To enhance the on student experience and student satisfaction And to Optimize the use of instructor time and resources So what I want to do today is give you a kind of scatter shot approach to different things that we’ve been using edX platforms to do This is a little bit incoherent, but I tried to cram as much as I could Most of the edX platform use in physics has been in the very introductory courses, 8.01 and 8.02 These are courses that basically every MIT student has to take and pass We range from 500 to 750 students in these courses So putting together a set of resources and a set of instructors And getting all the professors together and so on, to get these courses running is somewhat of a large endeavor And getting as much of it automated as possible is one of the goals that we’ve had These courses are taught using the technology enhanced active learning approach That we use A lot of open space Whiteboards on every wall And lots of and stuff like that So it’s quite fun to teach in that learning environment And as I’m hinting We, in the past few years, have been steadily using edX more and more heavily The first time we used edX for teaching these courses, we said, right We’re going to do them a little bit We’re going to now problem sets online, and that will be it No more And from there, we built up to the edX pages Our one stop shop for absolutely everything in the course And we have even developed instructor tools that go on top of that edX course, to try and help the people who are teaching in front of the whiteboard So starting at the beginning, then The obvious thing to use edX for is automated assessment And we typically use a variety of different assessment types on the edX platform Each week in these courses, we’ve got some reading questions for students to do These are your multiple choice, check the boxes, or very simple calculation type things Where students give it a couple of goes If they still don’t get it, we show them the answer And we’ve used these to basically replace readings and reading questions that our data shows that students weren’t actually doing We found that by turning it into an interactive environment that’s on the computer Rather than having them write something and read textbooks or do things they don’t wanna do We get significantly higher student involvement with these questions types We also have Friday problem solvings Which is basically our recitation sections Where we’ve moved towards taking all our problems that we were doing in those problem solving sessions, and putting them on the platform So students can check their answers as they go And they know something right before moving to the next part

And even more so here We’ve been able to use the platform to say, whoops Looks like you got that wrong Perhaps you’d like to look at this resource, which kind of explains what’s going on here And my favorite story here is that we’ve been recording some videos Targeted at difficult aspects of some of the problems that we do in these problem solving sessions So that, as soon as students hit that bump where we’ve identified a There’s going to be problems here We’ve given them a two-minute video to say, right Now you’ve seen this How do you do it? We’ve got your attention Let’s go through it and discuss it So then you can apply it And My favorite anecdote here is that if one of the people who recorded some of these videos was teaching the class one day And he walked up to a group and said, right, how are you doing? Do you need any help? And they said, no, we don’t need you We’re watching a video of you >> [LAUGH] >> [LAUGH] >> We also have problem sets that are due on a different day But these are the problems that really bring the material together And we really want students to grapple a lot with it And we want students to actually write up their solutions and hand them in So we can give them feedback on their problem solving strategies and approaches Not just on whether or not they got the answer right But we do let students check to see if they’ve got their answer right And this has been a great motivator for students who can tell when they’re done with a problem set Or if they need to a little bit more And this is very popular If you’ve ever seen a survey to undergraduate students on anything You’ll be impressed to hear that the immediate feedback on the problem set problems has a 96% approval rating And it’s almost impossible to get that type of approval rating on any measurements Okay >> So this is the stuff that you could do with edX out of the box What I’d like to move into is, what can you do when you start pushing the boundaries of edX a little bit? And what are some things that we’ve tried to implement? Okay, so I’ve touched on this one already We can give instant responses in the classroom So, we’ve been using this in our problem solving sessions We’ve also been using this for things like experiments in class Where instead of having students work their way through an experiment on paper, and do their calculations, and get the And have no idea whether the wavelength of that laser is 600 nanometers or 600 kilometers We can actually give them some feedback along the way as to how they’re going So, some of this requires a little bit, so far, of technology We can do some contextual parsing of what students have put in Box A and Box B And if we multiply them together in the right way, we can check that what they put into Box C actually make sense or not So this allows us to really give some type of feedback to students on how they’re going, and where they might’ve started going wrong So here is an example of how we’ve used this in experiments This is a fraction experiment where we put a laser and diffraction grating, and the students make a few measurements, and by computing an appropriate quanity, they can calculate the wavelength with the laser So here they made a reasonable measurement for a length, another reasonable measurement for a different length But both were checked as, do your numbers fall within reasonable ranges, nd then we said right? Did you divide these two numbers and get the units right, and multiply it by the appropriate constant to find the answer we actually want Moving beyond the immediate capabilities of the x platform, I’m gonna talk a lot about exploiting the mighty Iframe For those who don’t know, an IFrame is basically a website within a website, and so you can take a website and put in a expanicle, and it looks like it’s just part of your website, or the x web page Make sure you install it correctly, and I’m gonna talk about three different ways that we’ve used IFrames within our residential teaching to our advantage The first one is Google Forms Ed x is a lovely platform for doing training and that type of stuff, but it can be a real pain if you want to extract the data from it, that students have put in quickly And sometimes you just want a quick solution, to give a student a survey, or you want anonymous feedback, or you want a variety of simple things where students enter a form, press submit, and they get a email to you, or something like that

And so you can very straightforwardly embed a Google form, that dumps its data into spreadsheets and is very easy to get the data out of And we’ve used this for anoymous feedback, conflict exam requests, excused absence, spreadsheets, surveys, and a few other things It’s just a quick way to extend the capabilities of ed x The one thing that, using a IFrame to embed a form like this loses is that, that form has no idea who the student is, what course they’re in, or any of that type of information So it’s mostly anonymous information that goes into here It has absolutely no access to who is this student, where have they come from If you want to start getting into who is this student, where have they come from, you need to start using the L-T-I tool that he talked about it a little bit L-T-I stands for learning tools interoperability, and it is standard for how to communicate between embedded modules in a sense Basically in an L-T-I component is a, web site that you have set up on a different server Not ed x, that gets embedded in your page, and that website gets two bits of information First it gets the course that it came from, and second it gets a anonymized student I-D And if you’re at M-I-T, then there’s a few tricks that you can play to actually get the student username, so you can actually figure out who is who The benefit of working with L-T-I is, that you have a complete web server at your disposal And if you can code it in Python, or P-H-P, or H-T-M-L, or Java Script, or whatever you wanna do, you can make it work in a L-T-I component There are a couple of quirks to LTI, one of which is that, Safari has coding security settings by default for L-T-I So a bunch of L-T-I stuff doesn’t work if you use cookies in Safari, and we haven’t figured out a way around that unfortunately So one thing that we’ve used L-T-I for is, to enhance our usage of the humble clicker in our classrooms So if we use these rapid response devices to get student feedback, instant student feedback, to questions within a class, it is great for instructors to see To poll the class, and see what the students are thinking and kind of [NOISE] At some point in time, if you want to actually do anything with that to the answer that the students are generating, you need the students register their clicker, and that’s our L-T-I comes in Once students have logged into M-I-T-X, we know who they are already or at least we can get that, and we might as well let them register their clicker, and we can ask them for more information like, okay what class are you attending? What table are you sitting at, what’s your group number within that table? And then we can start doing some fun things with job So as I mentioned these were the 500 to the 800 student classes, you can imagine the level of scores that we get from M-I-T, for dividing these students up into groups and so on [INAUDIBLE] And once students have registered, then we can actually go and generate useful class lists for the teachers of the individual sections So here is an example where we have three students in table two, a couple groups in table three, and probably you can just go and generate these things On the flip side we can also take back that clicker that we’ve generated, and then parse it as we will, and then push grades back into the ed x platform, if we want to The other fun thing that we’ve been able to use L-T-I to do is, to track student progress through a class in real time So for a normal course, as the course finishes, you can go back and look at the click screen log, and see where students have clicked through and all that fun We really wanted to get real time information on how students are progressing through a given exercise in the class So what we’ve done is, for a given exercise, putting a dial home L-T-I component, just at the bottom of the vertical, and kind of make it as unobtrusive as possible And this just says, this just pings home every time that vertical loads and says, student is currently at this place And when we use this in conjunction with conditional x Blocks, this allows us to well, I guess I should explain a conditional X Block

Where we can say you have to answer all the problems on this vertical correctly before you can move to the next one So we can use this to figure out which vertical the students are currently working on, and thus track the progress through the exercise in class And at the end of the exercise we provide a survey, that’s got information on all the students, and how far they got through that exercise And if we want to, we can then push grades back to some landing page in that next platform, to say this is how far the students have to cross with this exercise And one of the benefits of doing this is we can actually implement group work, where multiple students in the course are only working at one person’s log in And because we can push right back, the grades go right back to all the students who were involved in that group So we use this for in class experiments where students form groups, they work through the experiments one vertical at a time And it’s useful to know what student progress is through those experiments, and then doing it to be able to upload student grades based on how far they got through that work So, here’s an example of an experiment, we’ve got I guess four variables with things that students need to do and the last one is basically a congratulations, you’ve finished These are some things that you might think about So the very first thing that students need to do is to login and basically give us their usernames of everybody in their group And [INAUDIBLE] you can group which is otherwise difficult to work with Here’s an example of the little dial home components just embedded in the very bottom of the vertical And as I said we try and make it as unobtrusive as possible If students haven’t logged in then this gives us a nice bright red flashing message saying you need to go and login or you don’t get credit And it’s these little LTI components, that dial home and continually update the server on the student progress And then we can use that to do fun things like this Where we can say, this is map of our classroom, each square represents a group of students, and we color that square based on how far through their experiment that group has gotten So, a quick look at this, and I can say, well, I probably need some of my teaching assistants to table one if the students lagging a little bit there We can always pick on a group and find out who those students are Actually, surprise them by knowing their names when you go and say hi >> [LAUGH] >> How do you know which student are sitting where? Cuz they all sound the same, the same person >> So it depends on how you set up the course In this room there are actually three computers at each table And each computer has a static IP address And we use that to figure out which table is which, which group is which Otherwise it’s none students at a table at most, you will be able to figure out who they are by looking at the pages certainly So at one point in time I went and took a screenshot of this math, every 30 seconds following and put together another video So let’s see if I can get this to play Here we go, okay So here, one second corresponds to one minute in class time This was a very full class A lot of students, here Each square is usually three students, sometimes two, sometimes four But this gives us a quick birdseye view of how is progress going in our classroom, right now Instructors really love this capability I think one more square goes No, two okay Very good >> [LAUGH] >> Okay, so the conclusions here are basically that the scaffolding for implementing a lot of technological services in the classroom are really becoming, More and more present within the platform and even externals platform Most applications that you’d want for a classroom have some sort of technological implementation ready Some of them are straightforward to implement Some of them, okay, you might need to write from scratch But most of the time it doesn’t require a huge amount of tech know how to get some of these things to work

And I’m going to particularly point to this last point of if you can use an i-frame to make your life better, then give it a go Cuz they’re really quite okay Thank you, and I’ll take any questions >> [APPLAUSE] >> So, a lot of publishers, when it comes to physics, develop a lot of these online homework systems, where the students go online, they do their homework, it’s automatically graded So is what you’re doing replacing the online homework system, or are you still using those types of systems in conjunction with the new developing edX? >> We aren’t using any online homework systems, other than what we’ve gone edX in this stage >> Okay >> We’ve been a little careful about putting absolutely everything in this online system, because you think of one of the important things in physics is the process by which you get to an answer Not just that you got the answer right And, so we do want students to actually generate so that we can give them feedback on that process But for most intents and purposes, what publishers have put together and we’ve typically reproduce stuff in the edX platform at this stage >> Okay >> Are there any other questions? For Peter? >> Yeah, I’d like to ask some more about this question about early adopters versus the potential for mainstream something like this and really putting it out Because one of the sort of early statements about edX was that it was too democratizing education So and to me, this technology of early adopters, and music until the cows come home But if you can’t really, you know, scale it, use it in other venues then you will, that’s a goal that’s going to be really, really tough to achieve So even inside MIT, right, if you have this kind of segregation to go there Any others? How do you see that developing? >> I should clarify my statement about early adopters was more about on the faculty side >> Right >> On the student side, we’ve essentially gotten almost all the undergraduates, at some point, using this platform >> So, you think it’ll be a generational thing? >> I think that’s a part of it There are a few MIT faculty members who still don’t use email [INAUDIBLE] >> [LAUGH] >> I think it’s a question of, also, what tools are appropriate for the teaching material And drilling into that a little bit that even in an introductory physics course there’s some material that doesn’t have a good analog online yet The other thing that we need to do, is continue developing the tools One of the things I didn’t show was the sketch input tool that our math department uses So their courses a really fundamental thing is students being able to draw simple graphs that demonstrate that they understand the concepts, in and out, the formulas, and that’s not an easy thing to do necessarily on a website But they’ve done some really creative stuff to be able to facilitate that, integrated with the educational experience >> [SOUND] >> I’d be happy to show that to you, but I don’t want to take up too much time >> Other questions? Well, okay, I wanna thank the speakers >> [APPLAUSE] >> We can hang out here until about 8 o’clock There might be leftover food There are some snacks and some drinks over there There are a number of edX people here right now Raise your hands if you work for edX So all of these people >> [INAUDIBLE] [LAUGH] >> All these people have strong opinions about the democratization of education >> [LAUGH] >> So we would be glad to talk to you about those ideas or any others So stick around and meet people and whatever Thanks a lot >> [APPLAUSE]