anyone using Google Classroom yet?

[kirby urner]

This is a somewhat meandering essay that asks some questions while considering the whole notion of "classrooms in the cloud".

I wonder if any subscribers here are testing / developing with Google Classroom.

We've been using in-house software to knit our courses together:  content; student; mentors.

http://courses.oreillyschool.com/  (our courses, in the past included calculus)

Many "learning engine" models leave out the mentor, instead envisioning a "theme park" where the rides just run of their own accord, students ride them, watch videos, get educated, and prove it with metrics.

However, education is not all about working with robots even though that's a big part of it. 

As I was pointing out to a friend in the car earlier today:  Python itself is a robot, a conversational one, as is any computer language with a REPL.  So I would never deny that my students face a lot of automation in tackling our topic area.

But just as math teaching is not all about a student + textbook + calculator, minus a teacher, so is math learning not an entirely an automated process, when done right.  Other humans add a secret sauce, one the machines don't know about.

All that as preamble to what I gather Google Classroom is about.  Beyond that I know little.

What's true about my work with students is it's not in real time, but with the delay of inter-office correspondence.  That's partly what keeps it languid and paced, not in some on the spot high anxiety mode more characteristic of real time classrooms. 

"Asynchronous means bliss" for a lot of students as they're not under the gun to think on their feet as much.  That's because this is programming, not theater, not improv.  Different walks of life take different skills.  No one said we can't also have a conference track where we meetup in real time.

Of course parents or other locals with requisite skills may jump into the loop as providing mentoring, so those of us who "just" write curriculum may nevertheless posit mentoring / teaching, perhaps by family members, as a part of the overall picture.

One needs several overlapping kinds of talent to foster a working school:  content, those willing to communicate the content, interact with students, serve as guides.

But some of us just provide the content and are not taken up with the business of presenting that content in any day by day sense.  Authors and teachers have different roles, though of course some do both.

Kirby

Further reading:
http://mybizmo.blogspot.com/2015/08/google-classroom-vs-ale.html

[Rakesh Biswas]

Thanks Kirby, for sharing this very stimulating essay.It made me check out your other blog posts on ALE. :-)

I am a long term fan of ALe (where 'e' could even be environment instead of engine?). We have a blended learning 

program at our institute http://promotions.bmj.com/jnl/bmj-case-reports-student-electives/ where we spend 9Am to 4 PM in an offline synchronous learning environment 'SLE'  (where a large part goes into capturing data and loading them online) and the next part of the day in our 24x7 program is spent in an online ALe with a global learning network. 

best, 

rb

https://lnctgroup.academia.edu/RakeshBiswas

--
You received this message because you are subscribed to the Google Groups "MathFuture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mathfuture+...@googlegroups.com.
To post to this group, send email to mathf...@googlegroups.com.
Visit this group at http://groups.google.com/group/mathfuture.
For more options, visit https://groups.google.com/d/optout.

[kirby urner]

What has excited me (some past posts here speak of this) is Michigan State's experimenting with supervised work spaces within Nexus Academies (the brand) that a high schooler would go to during school hours.  Interactions with peers and support staff is a given, including cafeteria and gym.  However the teachers are not on campus but serve from a removed setting, which is not to say no real time communication occurs.

My professional day job has been as an asynchronous mentor and I've been envisioning helping lots of Michigan high schoolers learn Python.  Whether that materializes or not depends on many factors chief of which is what counts towards earning academic credits in the State of Michigan.  Would our training programs be certified to be worth anyone's time if aiming to fulfill specific requirements (in terms of credits).

That firewall between adult-oriented training outfits and high schools is difficult to bridge.  Case in point:  Google Classroom.  Wearing my "personal trainer" hat, as someone who works with adults working out in our "gym" (on-line asynchronous learning environment), I am ineligible to play around with such a classroom as my company is not a 501(c)(3).  We're definitely a for profit people first type outfit, and believe in both pro bono and Open Source cultures (somewhat related in terms of ethos), so, like Google, we may support Google Classroom as an initiative and concept.

However, as a would-be customer for Google Classroom, I've had to don a different hat:  IT Clerk for my regional network for Monthly Meetings for Business, which sounds awfully business-oriented but is actually a nonprofit and actually categorized as a religious institution, so more like a branch of Buddhism in that way.  Wearing that hat, I'm able to propose which start up a few Google Classrooms as year-around "Interest Groups" (an prartice we already have).  Big wheels turn slowly however.  I have no direct authority over IGs, so my introducing Google Classroom in this ecosystem is more a matter of educating business partners in how we might use Google's technologies (we already use Google Groups, Translate, Chrome, Android and other services).

Thanks for writing and confirming that we're in the same ballpark with our acronyms / abbreviations.

The field is wide open for these on-line classroom frameworks to spring up, some Open Source, some in-house / guarded / owned by a smaller set of committers.  It's not either / or i.e. we find that companies with in-house solutions are nevertheless willing to collaborate on Open Source versions, finding that giving the public opportunities to hack on something close to their hearts is a great way to groom future full time developers.  That's just one of the many pay-offs.  So you'll find companies from Netflix to Walmart to PayPal supporting Open Source projects in areas close to their core business interests.  Perhaps Pearson has plans to throw its hat in the ring, or have they already, around Raspberry Pi maybe?  I should do some more digging.

Kirby

[Rakesh Biswas]

Thanks Kirby. 

These are very interesting insights into 'business open-sourcing?' 

:-) 

[kirby urner]

Thanks.

That was an abbreviated version of some analysis / insights spelled out in more words

on edu-sig, one of the Python sigs (we have sigs and wgs mostly, wg = working group).

https://mail.python.org/pipermail/edu-sig/2015-July/011289.html

Kirby

[Roberto Catanuto]

Thanks Kirby for raising this point. My current school has a clear and increasing interest in computer mediated learning environments for high school Math learning.

I wonder if any of you has experience to share about it. I'm trying to dig deeper into a few of them, like moodle, canvas, google apps etc.

--
You received this message because you are subscribed to the Google Groups "MathFuture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mathfuture+...@googlegroups.com.
To post to this group, send email to mathf...@googlegroups.com.
Visit this group at http://groups.google.com/group/mathfuture.
For more options, visit https://groups.google.com/d/optout.

--

Roberto

[kirby urner]

On Wed, Sep 16, 2015 at 2:14 AM, Roberto Catanuto <robe...@gmail.com> wrote:

Thanks Kirby for raising this point. My current school has a clear and increasing interest in computer mediated learning environments for high school Math learning.

I wonder if any of you has experience to share about it. I'm trying to dig deeper into a few of them, like moodle, canvas, google apps etc.

I do computer mediated learning for my day job, teaching Python programming, and have for several years, for O'Reilly Media, the publishing company.  Our in-house solution is a not-open-source program called Nano which I gather has much in common with Google Classroom in its use of JavaScript to govern workflow, though in many other respects they're quite different.

The system I'm using was the brainchild of University of Illinois grads who wanted to give classroom teachers a solution, just as Google Classroom does, but with a twist:  a Wolfram Mathematica engine was part of the deal, with Useractive (later the O'Reilly School) permitted to hold the license and share it with the teacher and students, who then individually would not need a license. This was all in place by 2005 but with few takers.

The company came to the attention of the founder of Make: magazine, a prime mover behind Maker Fairs and the ongoing DIY movement, and he purchased it for O'Reilly, bringing the principals, Scott and Trish Gray, to the Sebastopol campus.  The course list was repurposed from calculus (Scott was a math professor) to the kinds of things O'Reilly publishes about:  the LAMP stack i.e. Linux, Apache, MySQL, Perl, things like that.  Python was starting to surge and that's when I joined the company.  The mentors were overwhelmed, especially in Python.  JavaScript and Android have also been popular.

Fast forward to 2015 and U of I founders have moved on, with our school set to close next year, in part because Nano is by this time too big a code pile to any longer manage in startup fashion and is of dated design (shows its age).  Amazon Web Services etc. were still in the future when Nano was initially crafted.  

I've been retained to work with remaining students along the Python track (other faculty likewise), but we're not taking new enrollments.  The parent company is reconfiguring, learning from past mistakes and moving on, but not because computer mediated learning doesn't work.  It works well.  Reconfiguring is normal in the high tech sphere. Stay tuned.

We were also hobbled from an abortive attempt to become a "real" school (dot edu, with transfer credits, diplomas and everything), meaning jumping through a lot of hoops that at the end of the day injured our business model and did not serve the students.  We had to insist they complete course X before taking course Y, even though colleges encourage taking courses in parallel, and even though it's often better to learn something in conjunction with something else, taking advantage of synergies. We had become far too restrictive, in attempting to bend over backwards to satisfy various authorities.  Live and learn.  Long story (I expect something to show up in Wikipedia, probably not by me).

As a mentor and as a track, we get high satisfaction rates and the administration and principal have blessed us for our good work, so we do not feel unappreciated.  We were well treated and given many benefits as a crew, yet it's true the ship is going down in its present morphology, as emblazoned on its website (oreillyschool.com).  

Students really liked having human mentors versus feeling abandoned to some theme park with only robot rides.  Classes and trainings offer that, theme parks don't. I'm eager to continue mentoring and sharing the role with more people.  I think the software projects in this area are still in their infancy.

I've been doodling the core ideas of computer mediated learning as I see them, based on all this experience with one instance, and adding imaginative components mostly from the travel industry i.e. I'm looking at a course as an airplane flight, as that gives me such "objects" as a "flight recorder" (keeps stats on this course, a record of transactions....).  I could go on and on.

Anyway I've got a lot of those doodles on-line here:

https://www.flickr.com/photos/kirbyurner/albums/72157654417641521

(I'm imagining starting over from scratch with a hypothetical version of the school in Russia, just to give myself an imaginative landscape and a freer hand to add science fiction ingredients for a futuristic next generation version)

I've been recommending that we cannibalize Nano for its ideas but remove any dependencies on Windows (we had a hybrid setup with Windows running Eclipse for IDE back ending into Linux).  Using a Windows client in the cloud, a remote desktop for each student, is a working configuration but had to do more with our history with Mathematica than with LAMP stack teaching. Today, Mathematica runs on Raspberry Pi, so I assume on Linux more generally right?  I'm no expert in that area.

I like the setup with Eclipse though.  As a student, I drag and drop a project into a submit box and voila, it zips up and unzips in the mentor's Eclipse, same way, but a copy.  Now I'm free to comment, nitpick about format, change the code here and there with explanations and send it back.

Often I just send comments with suggested code hints embedded therein so the project files themselves make a one way trip -- but then based on comments, the student may submit again, and again....  

I'd like to take courses in Jython, Java and Clojure in such an environment myself, then mentor them.

However in all these years, I have not had the experience of using computer mediated learning of this kind inside a real classroom, ala the early Univeristy of Illinois model.  

Their idea back then was to sell this to college professors:  here's a complete calculus course, with vectors and everything, in Mathematica, complete with a way for students to share work.  You could have a real time regular calculus class with this workflow built in.  

As it turned out, the U of I professor who actually took it on, Bruce Carpenter, made it into an early distance education option instead, successfully using it asynchronously i.e. students worked at different times and did not need a regular meetup.

This all happened before I joined the company, however I was privileged to meet this early cast of players.

Also, I have not given out a single letter grade in all this time.  Although we billed ourselves as a school and granted a credential, we only passed or returned a given quiz or project, everything open book, asking for fine tuning, fixes, more attention to this or that.  

Some students might take ten times longer to complete the work, or perhaps drop out in frustration, realizing this was not for them at this time in their lives.  

But then I had little insight into their schedules unless they chose to share those details (any student is free to write to the mentor about whatever in seeking advice, a collegial guidance counselor type function, again individualized -- there's no group interaction in this particular implementation).  

A really good student might also take a long time, just because doing many other things.  Beginners need the extra time as the learning curve is steeper for them.  If you already know Perl, Ruby seems a lot easier to get your head around etc.

A lot of puzzle pieces were missing from our solution.  There's more work that needs doing.

I'm hoping a generic open source version, which I call Asynchronous Learning Engine (ALE), might distill out the parts that don't need any real time interaction, which I'd like to handle with existing solutions already on the table.  

We already have real time audio, video and chat solutions in abundance.  

Taking out the real time communications aspect is actually more classroom friendly because we assume students and teachers are right there, communicating, and so don't need those facilities.  Or:  we assume real time communication is simply not required, which was the case in my on-line school.  Either way, it's a logical way to lump capabilities.

Kirby

On Sat, Sep 5, 2015 at 4:21 AM, kirby urner <kirby...@gmail.com> wrote:

Thanks.

That was an abbreviated version of some analysis / insights spelled out in more words

on edu-sig, one of the Python sigs (we have sigs and wgs mostly, wg = working group).

https://mail.python.org/pipermail/edu-sig/2015-July/011289.html

Kirby

On Fri, Sep 4, 2015 at 6:45 PM, Rakesh Biswas <rakesh...@gmail.com> wrote:

Thanks Kirby. 

These are very interesting insights into 'business open-sourcing?' 

:-)

--

Roberto

[Rakesh Biswas]

Thanks Kirby, Yes this looks very much like the future. . 

Liked the doodles. Are they all in your handwriting?

best, 

rb

[kirby urner]

On Wed, Sep 16, 2015 at 6:49 PM, Rakesh Biswas <rakesh...@gmail.com> wrote:

Thanks Kirby, Yes this looks very much like the future. . 

Liked the doodles. Are they all in your handwriting?

best, 

rb

Yeah those doodles are hand drawings by me.

One of the puzzles we've had a hard time solving is smoothly scaling to a larger size. 

Our employment model was one of hiring new people on contract, with the intent to have mostly full time employees, which is great for those wishing to teach courses many hours each day, which is what I've been doing (Python is popular, and I enjoy teaching it full time).

Another academy I've taught with is Saturday Academy based here in Portland, Oregon. 

Their strategy is to post a catalog of offerings having lined up instructors for all of them, but then a threshold of at least X students is required before the course is a go.  A listing in the catalog suggests the possibility of a course ("you could sign up for this") more than a commitment.  Since Saturday Academy courses tend to involve going somewhere physically, there's also the dimension of that booking a venue / classroom for real (versus virtual).

In the case of Saturday Academy the instructors are always on contract, never full time with benefits.  That's the other extreme.  I'd like to see more of a full spectrum, versus either / or.

Somehow a school or training service needs to match swells in student demand with corresponding faculty and I'd say we had a hard time solving that puzzle.  People with strong skills in IT are able to jump from job to job and often don't want to tie themselves down, or may want to work on a startup.  Our pool of would-be mentors would increase were we to go after such people.

It's also not snap-of-the-fingers easy for a veteran Perl coder to suddenly get good at Python or vice versa.  The concept of a small pool of full timers who keep up with demand puts a lot of stress on faculty to take the more popular courses internally with an eye to mentoring them (which I'd like to do vis-a-vis Clojure for example). 

However in reality few of us can reshape our brains that quickly, to become mentors of a new language in a short time.  I was able to hit the ground running with Python, given prior experience, but was so busy doing that, that I was not able to make much headway on most of the others that were popular.

What I envision for a next generation implementation is a subset of full timers who wish to do this kind of work a lot, mixed with those intending to become full timers, mixed with those who definitely do *not* want full time work but *do* want:

(a) some income for teaching content they enjoy and are already good at
(b) some resume kudos for having done so and
(c) an arrangement to "jump on and jump off" i.e. teach a couple courses over a summer, then take time off, come back later and so on.

In my diagrams, this is what I symbolize as an Oasis or Faculty Lounge, where we have people on tap.  We wanted people who mentored a specific course to also take it first.  The author and mentor roles were separate.  I collaborated with the author of the Python courses in some areas.  So that's yet another way to net some income, as a course designer.  I show that here:

https://flic.kr/p/wuz4Nf  (designer compensation and mentor compensation or separate line items)

Mentors need to schedule with a dispatcher regarding what courses they want to teach.  The dispatcher role is one of load balancing.  Work to give mentors the amount of work they can handle.  Some courses sit on a shelf indefinitely until the right mentor comes along who wants to teach it.  Then, if enough students sign up, go for it.

Another conversation within our school has been whether to go multi-lingual in the human languages sense (obviously we're already multi-lingual in the computer languages sense, having offered courses in C#, C++, Java, Perl, Ruby, Python and JavaScript). 

I've been pushing for a Portuguese version of the Python track for use in Brazil, maybe a Russian one and so on (the student depicted on our latest catalog is a native Russian speaker and took our Java courses from Kyrgyzstan before moving to the US).  Not being multi-lingual myself, I would not personally expect to mentor these courses.

I am not of the school of thought that computer stuff always needs to be taught in English. 

On the contrary, engineers worked hard in the last few decades to make sure we could have source code in Unicode, now the standard for many languages.

Kirby

[Maria Droujkova]

Interesting stories here. 

This year, we are changing how we are doing courses at Natural Math. I think it's time to say all the experiments trying to make MOOC courses interactive have not worked to my satisfaction. People perceive MOOC format as broadcast media, and maybe they are right. But within the large courses, small groups of people meeting and talking LIVE emerged, and those were my personal favorite parts.

So we are going to take these small live circles as our model. They will work like gamer groups (tabletop, LARP, or MMORPG - these all have relevant models) where 4-10 people have definite roles, and the whole campaign depends on everybody playing that role. I want to move from 5-10% active participation to 100%, but in small groups. Then scale up by making the course a conglomerate of such small groups, running the same system of the world.

Cheers,
Dr. Maria Droujkova

NaturalMath.com
-- .- - ....

[Linda Fahlberg-Stojanovska]

i am reading these posts with great interest. thank-you all!

Sent from my iPad

--

[Ted Kosan]

Kirby wrote:

> I do computer mediated learning for my day job, teaching Python programming,
> and have for several years, for O'Reilly Media, the publishing company.

<...>

> Students really liked having human mentors versus feeling abandoned to some

> theme park with only robot rides. <snip>

Does your school use automated grading software for students programs?

I teach a beginning programming course at a university, and this
semester I started using automated grading of student assignments.
Here is an assignment I gave out earlier this week:

http://206.21.94.61/tmp/automatic_grading_example_v.09.html

The automatic grading works by having each %mathpiper exercise fold in
a MathPiper assignment worksheet be accompanied by a companion
%mathpiper_grade fold. The %mathpiper_grade fold contains code that
grades the code in the exercise fold. The student can evaluate the
grading code as often as they would like to get nearly instant
feedback on how well their program matches the requirements of the
exercise.This collapses the time a student needs to wait to get
detailed feedback on a given program from perhaps a couple of weeks to
a few seconds.

So far the students really like having a grading robot for each
exercise that can point out to them how well they are doing as they
create their programs.

Ted

[kirby urner]

On Thu, Sep 17, 2015 at 7:40 PM, Ted Kosan <ted....@gmail.com> wrote:

Kirby wrote:

> I do computer mediated learning for my day job, teaching Python programming,
> and have for several years, for O'Reilly Media, the publishing company.
<...>
> Students really liked having human mentors versus feeling abandoned to some
> theme park with only robot rides. <snip>

Does your school use automated grading software for students programs?

We don't use any robo-grading, and put that out as a selling feature
to our students, suggesting they spread the word.  "Real humans R us!"

But then of course the computer itself is a robot, a source of automatic
feedback, and we do encourage running the program again and again
as part of the normal process, before submitting for evaluation.  When

a program contains errors an interpreter could have caught, I ask:

"did you run this first?" 

That's what's so great about learning programming, is the tools are

designed to give feedback, raise exceptions, flag errors.  A piano doesn't

do that.  But then if you have a recording of someone playing a piece

flawlessly, that gives a feedback loop.  Feedback loops are what's

critical.

Our projects will often contain the example output we expect, given
example input.  Words in a specific order, a histogram, a GUI.  But it's

up to the instructor to actually run the code, which is why having it

show up in Eclipse (IDE) as a project, same as on the student end,

is so dang useful.

We also suggest students submit code because the project unfinished,
and they're stuck, with comments about where.  Or maybe the project
seemed unclear so they write something and say "am I on the right
track here?".  We do not convert "number of attempts" into a grade
either, so there's no sense of being penalized for turning in unfinished

work with questions.

Our spiel is work in the real world is by successive approximation, iteration,
and the idea of submitting once and for all for a letter grade is a fiction
of the school world, not the real world.  We hype "realism" is one of our

selling features.

I see a spectrum, going from all robo-grading to none at all.

I could see using some robot features in the future.

However even on quizzes, which tended to be *not* multiple choice,
with sometimes open-ended questions, we're looking to engage. 

All handed in student work is an opportunity for a relationship with a
peer-like individual who wants to see that student succeed.

I teach a beginning programming course at a university, and this
semester I started using automated grading of student assignments.
Here is an assignment I gave out earlier this week:

http://206.21.94.61/tmp/automatic_grading_example_v.09.html

The automatic grading works by having each %mathpiper exercise fold in
a MathPiper assignment worksheet be accompanied by a companion
%mathpiper_grade fold. The %mathpiper_grade fold contains code that
grades the code in the exercise fold. The student can evaluate the
grading code as often as they would like to get nearly instant
feedback on how well their program matches the requirements of the
exercise.This collapses the time a student needs to wait to get
detailed feedback on a given program from perhaps a couple of weeks to
a few seconds.

We would not allow "a couple of weeks" to go by.  Three business days tops.

That's a fairly hard and fast rule, baked into the workflow, because yes,
feedback should be quick. 

Students, on the other hand, are not required to work on a schedule (in
our model), though there is an overall course deadline.  I often encourage

them to "slow down" or to "sleep on it".

We live in a culture where titles like "Learn Java in 21 Days" give a false

impression.  Then we give the idea that one can master a skill simply by

reading about it or watching lots of videos about it.  That's where the

do-it-yourself (DIY) philosophy kicks in:  hands-on is the way to learn,

first and foremost, with other modalities / media secondary.

Our founder, Scott, was therefore highly suspicious of Khan Academy,

not because the videos were of low quality, but because in his experience

too many people were missing too much of the hands-on component.

Perfunctory multiple choice tests based on memorizing (regurgitating

information) were in Scott's mind a terrible travesty purveying only the

illusion of learning.

 

So far the students really like having a grading robot for each
exercise that can point out to them how well they are doing as they
create their programs.

Ted

Does a human ever see the final result?  Fine points of style need to be

caught by the human eye sometimes.  Like sometimes they use one-letter

variable names, a bad habit inherited from mathematicians.  :-D

Kirby

[Ted Kosan]

Kirby wrote:

> Our founder, Scott, was therefore highly suspicious of Khan Academy,
> not because the videos were of low quality, but because in his experience
> too many people were missing too much of the hands-on component.
> Perfunctory multiple choice tests based on memorizing (regurgitating
> information) were in Scott's mind a terrible travesty purveying only the
> illusion of learning.

I have followed Scott's blogs for a number of years now, and I have
learned quite a bit from his writings. However, I don't know much
about the details of the software he created. Did any of the
math-related educational software he create have step-by-step problem
solving abilities? For example, could his software solve high school
level equations the way a human typically would and show all of the
steps it took to arrive at a solution? In my opinion, any math related
educational software that does not have step-by-step abilities like
this (either in the tutorial software or in the automatic grading
software) is also only providing the illusion of learning.

> Does a human ever see the final result?

Yes. The students submit their worksheets using an LMS, and I am
looking at their programs for now. However, I am grading their
exercises using the same grading code that the students have. As I
look at a student's code, if I see a property of the code or the
result it returns that I forgot to have the grader check for, I create
grading code for this property and add it to the other grading code
for this exercise. My goal is to make the grading code so good that I
don't need to physically look at student code.

Beyond this, each time a fold is evaluated in an assignment worksheet,
a copy of the code the fold contains is placed into a log file along
with a timestamp and other metadata. This log is turned in along with
the worksheet it goes with so the following information can be
determined:

a) How much time a student spent on each exercise in an assignment -
This information is helpful for determining if a given student is
having trouble grasping one or more concepts that a specific exercise
was designed to relay. If a significant number of students took a long
time to finish a given exercise, this indicates that the educational
materials that relate to the exercise need to be improved.

b) When the student worked on each exercise - This information is
helpful for answering questions such as: Did the student start working
on the assignment soon after it was given out, or did they wait until
just before it was due to start it? What time of the day did the
student do most of the work? Did they complete most of the assignment
in lab, or did they do it late at night in their dorm room?

c) How did the student evolve the code to arrive at a solution? For
example, did they build up their program from small separately tested
parts, or did they try to write the complete program before testing
any of it?

d) Did the student cheat? - If a complete and complex program that
does not produce any errors appears in an empty fold, there is a good
chance it was pasted into the fold.

> Fine points of style need to be caught by the human eye sometimes.
> Like sometimes they use one-letter variable names, a bad habit inherited
> from mathematicians. :-D

Automatic grading code can check for style properties of students
code. For example, in the following MathPiper code, the code in the
%mathpiper_grade fold checks to make sure that only variable names
that are longer than a single character are used in the %mathpiper
fold:

%mathpiper,name="Exercise 2",subtype="exercise"

a := RandomInteger(2);

If(a =? 1)
{
b := "HEADS";
};

If(a =? 2)
{
b := "TAILS";
};

b;

%/mathpiper

%output,parent="Exercise 2",sequence="28",timestamp="2015-09-18
14:28:50.355",preserve="false"
Result: "TAILS"
. %/output




%mathpiper_grade,name="Exercise 2"

FoldGrade("The program uses variable names that are longer than a
single character", 1)
{
UnFlatten(MapSingle(Lambda([variable], Length(ToString(variable))
>? 1), VarList(foldCode)), "&?", True);
};

%/mathpiper_grade

%output,parent="Exercise 2",sequence="29",timestamp="2015-09-18
14:29:06.52",preserve="false"
Result: False

Side Effects:
FAIL: The program uses variable names that are longer than a
single character. (0/1)

0/1 points
. %/output


If there are other properties of code that you think would be useful
for an automatic grader to check for, I would love to hear about them
:-)

Ted

[kirby urner]

On Fri, Sep 18, 2015 at 12:06 PM, Ted Kosan <ted....@gmail.com> wrote:

Kirby wrote:

> Our founder, Scott, was therefore highly suspicious of Khan Academy,
> not because the videos were of low quality, but because in his experience
> too many people were missing too much of the hands-on component.
> Perfunctory multiple choice tests based on memorizing (regurgitating
> information) were in Scott's mind a terrible travesty purveying only the
> illusion of learning.

I have followed Scott's blogs for a number of years now, and I have
learned quite a bit from his writings. However, I don't know much
about the details of the software he created. Did any of the
math-related educational software he create have step-by-step problem
solving abilities? For example, could his software solve high school
level equations the way a human typically would and show all of the
steps it took to arrive at a solution? In my opinion, any math related
educational software that does not have step-by-step abilities like
this (either in the tutorial software or in the automatic grading
software) is also only providing the illusion of learning.

I'd like to see more of the early calculus stuff myself.  In case they

write that Wikipedia article I think the company history starts with
University of Illinois and Dr. Jerry Uhl, then NetMath, then UserActive,
and then O'Reilly School.

By the time I joined O'Reilly School, the majority of our courses
were already of the LAMP stack variety versus Mathematica-based,
i.e. we had repurposed Nano (the in-house workflow solution) to
send homework from a desktop-in-the-cloud S to the mentor's
computer M, another desktop -- with no Mathematica present, just

the interpreters of LAMP fame e.g Perl, Ruby, Python and PHP,

plus C family / Java compilers.  We don't have any LISP family

languages on tap e.g Clojure, which runs on the Java engine.

How much is all this "mathematics" I hear readers thinking? 

That's a hot debate these days. 

Clearly this LAMP stuff is not algebra in the sense of computer
algebra systems (CAS), which may or may not "show steps". 

Whatever it is, this computer stuff, the schools are ravenous for
it (in South Africa too) and the states are insistent that it be

phased in i.e. yesterday was not soon enough.

Today's news:

https://www.facebook.com/mayordeblasio/videos/10153514877316166/

(computer science is joining the "3 Rs" as uber important,

New York mayor says -- *all* students will be getting these skills

within the next... 10 years was it?).

 

> Does a human ever see the final result?

Yes. The students submit their worksheets using an LMS, and I am
looking at their programs for now. However, I am grading their
exercises using the same grading code that the students have. As I
look at a student's code, if I see a property of the code or the
result it returns that I forgot to have the grader check for, I create
grading code for this property and add it to the other grading code
for this exercise. My goal is to make the grading code so good that I
don't need to physically look at student code.

I may just glance at student code, or I may dive in.

A lot of our code assignments have short solutions.  We're teaching
the grammar / syntax of the language, not dissecting or constructing
big algorithms.

Typical beginner projects:

* User types in words, program spits them back with those words

containing uppercase before any with no uppercase.

* Program reads a text file and reports on how many of each

wordlength e.g. 267 2-letter and 3-letter words in US declaration.txt

More advanced:

* You have two tables in MySQL, one of animals (with id field) one

of foods that animal eats (one-to-many).  Report on (a) which animals

if any have no food in the food table or (b) confirm all animals have at

least one food.

These are actual projects.

Sometimes I dive in and rewrite parts of the solution, showing how
a pro might do it. 

As we get more advanced, the GUI parts get gnarly. 

By Python2 they're doing event driven programming (widgets with
handlers), using Tk, a toolkit running in another process.

Browser-based tools would make this difficult if not impossible which

is why we stick to a "one desktop per student" model in the more
advanced courses.

Managing a filesystem is part of what it means to develop academically
in the New York sense. :-D  Every 5th grader needs to know about file

paths.  Smartphones are good at hiding the fact that those even exist.

What we start showing in Python2 is how to write testing code in

parallel to the requested code, and we require those tests to be

submitted as well. 

As the mentor, I run the test code the students themselves wrote. 

This is part of the Agile philosophy:  test driven development i.e.
write some tests first, then write code to pass them, and so on. 
You have to develop a certain mindset i.e. the ability to ruthlessly
test your own stuff -- like learning to play chess against yourself.

Good skill, resume-worthy.

However, I don't think one size fits all, in terms of workflow and "how

it should be done". 

In comparing different workflows, I don't think we're trying to narrow
it to the one best one.

The kind of supervision a student needs will vary greatly depending
on exactly what walk of life we're talking about.

Therefore, whereas I might on occasion relay Scott Gray's suspicion
of Khan Academy videos and/or this that or another teaching tool,
my own view is videos have their legitimate place even in a serious
workout and diet.

I'm eager to produce more video content myself -- but not with talking

heads so much (Khan vids are not talking head -- he talks, but we
never see his head). 

I prefer animation (cartoons).  Khan's stuff is actually quite toon-like. 

I'm a fan of Vi Hart's as well (we see her hands a lot, drawing stuff
-- I like that style (no coincidence she and Khan have collaborated
some)).

 

Beyond this, each time a fold is evaluated in an assignment worksheet,
a copy of the code the fold contains is placed into a log file along
with a timestamp and other metadata. This log is turned in along with
the worksheet it goes with so the following information can be
determined:

a) How much time a student spent on each exercise in an assignment -
This information is helpful for determining if a given student is
having trouble grasping one or more concepts that a specific exercise
was designed to relay. If a significant number of students took a long
time to finish a given exercise, this indicates that the educational
materials that relate to the exercise need to be improved.

b) When the student worked on each exercise - This information is
helpful for answering questions such as: Did the student start working
on the assignment soon after it was given out, or did they wait until
just before it was due to start it? What time of the day did the
student do most of the work? Did they complete most of the assignment
in lab, or did they do it late at night in their dorm room?

c) How did the student evolve the code to arrive at a solution? For
example, did they build up their program from small separately tested
parts, or did they try to write the complete program before testing
any of it?

The text of our Lessons is on-line, for example here is Python2:

http://courses.oreillyschool.com/Python2/index.html

The Lessons assume the student is typing in the code and has it right

there, because then we go:  delete this line, insert that, run again,
you should get this, try this other thing, change that... and so on. 

They're using the keyboard at the same time they're reading our
instructional materials. 

We want them to develop these exploratory instincts vis-a-vis whatever
interpreter.  In Scott's days with the school (he left a couple years ago),
that interpreter might have been Mathematica.

By the time a Lesson is finished, there's a pile of code, and quite

often the Project that follows makes use of that exact code, or code

very like it, so the student feels they're at least in the ball park.

There's always more to do though, and sometimes we throw in

a surprise new concept, which almost forces a student to contact

the mentor. 

That's good.  Some students are shy and need to face that brick
wall that gets them to get attention.

I'll see all the attempts in chronological order, with my comments,
as they submit them, going back, so I can see the trajectory as
student has followed.

But I only see handed in work, I don't look into their personal file
systems -- though I will gain access to their MySQL databases if

they share the password (otherwise I can use mine).

Given our students are from all walks of life, from experienced
programmers to scientists to curious dilettantes (lets presume),
there's not much to conclude from precise time measures so we
rely on students to be articulate and write directly to the mentor
about frustrations and mental blocks. 

Contact your mentor, don't be shy.  Tell us how it is for you.

Autobio welcome.

As a deadline for the course approaches, I might start alerting the

student that I'm noticing that, in case she or he is not paying attention,

but really, time management is up to them. 

I'm not a nanny, just an evaluator of turned-in work. 

If they decide to not do the work, maybe it's because they got
the dream job they wanted and the course is no longer relevant.

I have a mechanism for sending "nudge notes" if I've heard nothing

in two weeks (we call it the Hospitality Queue).

Actually we have several forms of followup, student surveys etc.  All

our correspondence is recorded.  I probably have several thick volumes

of mentor-student interaction on file. 

However in my case it's all in writing, nothing by voice (though other
mentors did use voice some -- we gave teachers a lot of latitude in
how they wanted to use any synchronous media).
 

d) Did the student cheat? - If a complete and complex program that
does not produce any errors appears in an empty fold, there is a good
chance it was pasted into the fold.

This is an interesting area.  Our general sense was why would it make

sense to cheat?  You're going to a gym to get strong, and first day on

the job they'll ask you to lift some heavy log, and if you've been "cheating"

at the gym i.e. just standing there, or grunting but not really efforting, well

good luck lifting that log!

In other words, since one can hardly fake programming abilities, it

stands to reason that cheating would be self defeating.

That being said, in a crunch a student might search StackOverFlow

and find another student had "asked for help" by posting one of our

homework assignments.  That would require filling out a form and

submitting it to StackOverFlow proving copyright infringement, in

which case they'd take it down.  A formality but also a pain.

I was looking with interest (still am) at the Nexus Academy model in

Michigan.  The students are in a supervised / proctored setting.  There

is the possibility of doing distance education where one still has a place

to show up as a student, at least for proctored tests.

 

> Fine points of style need to be caught by the human eye sometimes.
> Like sometimes they use one-letter variable names, a bad habit inherited
> from mathematicians.  :-D

Automatic grading code can check for style properties of students
code. For example, in the following MathPiper code, the code in the
%mathpiper_grade fold checks to make sure that only variable names
that are longer than a single character are used in the %mathpiper
fold:

 

If there are other properties of code that you think would be useful
for an automatic grader to check for, I would love to hear about them
:-)

Ted

That's all pretty interesting and unfamiliar territory to me.  The software

looks pretty capable.

I can diagnose common problems at a glance and have canned responses

which I mix with custom text, so my workflow is still maybe 1 minute per

assignment on average, with some taking much more time (subtle bug).

I'll typically get through 80 assignments per day, a mix of quizzes and

projects.  The load balancer gets stats on mentor throughput and if mentor A

is starting to burn out, new students go to mentor B and so on.

My "next version" ideas about load balancing incorporate what I'd

consider "Lessons learned" such as we needed to lump students into

"virtual cohorts" even if they completed at vastly different times (or

failed to complete), not for their benefit but to help admin and mentors

keep a handle on load balancing.  We were missing some critical

feedback cycles and tried to do way too much manually.

How many students did I have at any one time?  It'd be hard for me to

say.  It's like Grand Central Station with lots of people milling about and

mentors at the ticket counters.  Some people are always lined up, ready

to do transactions, but many more are just reading the paper or sipping

coffee or whatever, not currently engaged.  

If all my students were to suddenly turn in work at once, it'd be like a
run on the bank and I'd be overwhelmed.

But that hardly ever happened (we made some rule changes that created

a deluge at one point, when admin was trying to please some authorities).

Now that the school is fading away, I'm eager to see where we go next

with the genre of software (Nano i.e ALE).

Again, our focus was realism.  The next stop after our courses was often

a job. 

I'd be a little concerned with code folding feedback where that's not available
on the job. 

Also, in not using grades I had more ability to use my own judgement.  When
the code is immature, I can hold it back and demand more. 

If the student is already strong, I might push that  student to become even
stronger. 

In other words, I'd be reluctant to relinquish too much of my judgement to
software, nor would I want students to have that kind of feedback past
a beginner level, if that was not a kind of help they would get in the real
world.

However it sounds like that is not a big concern in your setting.

Kirby

[kirby urner]

I can diagnose common problems at a glance and have canned responses

which I mix with custom text, so my workflow is still maybe 1 minute per

assignment on average, with some taking much more time (subtle bug).

I'll typically get through 80 assignments per day, a mix of quizzes and

projects.  The load balancer gets stats on mentor throughput and if mentor A

is starting to burn out, new students go to mentor B and so on.

In looking over what I wrote, I realize if we do the arithmetic it sounds

like I only worked about 80 minutes a day, which is not correct, as anyone

hanging around me can attest. 

I seem to be always "grading", even though we don't use grades.  We still
use the verb "to grade".  I'd say: "I supply a gradient, not grades".  You'll

find me in coffee shops, at my home office, in an airport... getting through

those quizzes and projects, answering technical emails (those take time!).

The activity is a lot like knitting, and when mentors would meet, we'd

all set up our laptops and continue grading.  But we never had to "go to

class" and that was a perk of the job.  Nor did students.  The asynchronous

nature of it all is what gives time to reflect, and a way of getting into a flow,

a routine.  Some people are night owls.  No rush hour traffic.

Lets say six hours of focused concentrated work of this kind is quite a bit
-- one needs breaks, change of venue, change of topic.  It's like working
out, a sport.

Six hours / 80 = 6 * 60 mins / 80 = under 5 mins per task on average.

OK, that sounds about right.

Plus the job of mentor comes with other responsibilities besides grading.

There's another feedback loop where we file issues reported by students

and continue to upgrade the track.  New versions.  There's keeping up

with the skills, continuing professional development.

In my travel industry picture, mentors are like pilots yet behind every

successful flight there's a whole cast of unsung heroes, the flight crew
and ticket people, booking agents, maintenance crew i.e. student services
and IT.  They get a cut per course in the doodle diagrams.  Our "airline"

needs to be self sustaining. :-D

https://flic.kr/p/utuuj9  (mapping a school to a travel industry metaphor)

https://flic.kr/p/wuz4Nf  (expenses per course / class / flight)

Kirby