AMATYC postition statement on Distance Education

[Mary Beth Orrange]

AMATYC's position statement on Distance Education falls under our area
of responsibility. The last time it was reviewed was in 2002. There
have been many changes in this area since then; so we need to review
it with a careful eyes.

Please read it, comment about its relevance so we can determine if we
need to make changes. There has been much discussion about integrity
in online and seated courses; we should determine if that subject
should be included in the statement in disctance education or if it
should be made as a stand-alone statement, if at all.

It was also suggested to me that we combine the two statements; the
teaching and learning with technology statement and the distance
education statement. Keep those thought in mind as we discuss this
position statement.

http://www.amatyc.org/documents/Guidelines-Position/DistanceED.htm

If you are interested, the full set of position statements is found
at:

http://www.amatyc.org/documents/Guidelines-Position

Thank you ... again for your support and help!

[Nolan at Chemeketa]

In my opinion, the essence of our recommendation should be that
quality of a distance course is expected to the same as that of a
traditional face-to-face course.

We all support having the same quality of instruction, the same
assessment standards, the same learner outcomes, the same level of
student engagement, the same integrity and academic honesty regardless
of the delivery method.

If that principle is clearly stated then we don't need to go down the
road of suggesting specific procedures for assessments--individual
math departments can decide how they want to carry that out.

However, if a statement is required on proctored testing, I'd
recommend something that rates the different options in a most-
desirable to least-desirable way rather prescribing a single method.
We could use the following as a starting point. I adapted it from the
fantastic 2004 article by Neil C. Rowe, which everyone should read
(see http://faculty.nps.edu/ncrowe/dlcheat.htm)

----------------------------------------------------------------------------
SAMPLE STATEMENT ON ONLINE ASSESSMENT
----------------------------------------------------------------------------
Online assessment currently raises serious security issues and may
facilitate many new methods of cheating. For these reasons it is
recommended that:

1. Human-proctored traditional paper-and-pencil tests with traditional
security procedures remain the preferred method for major assessments
in distance learning.
2. If manual grading is too burdensome, human-proctored tests taken at
a computer are a second-best choice provided that the computer's
software and networking capabilities are tightly restricted.
3. If human-proctoring is not feasible, it is critical that assessment
questions are drawn randomly from a large pool and reordered randomly
and/or programed with changing numeric parameters. Appropriate time
limits should also be enforced.

It is anticipated that online assessment will become a normal
assessment option as new identification technologies are developed and
cheating countermeasures become more sophisticated. Until that time,
online assessment in distance courses should be done with caution.
----------------------------------------------------------------------------
END OF SAMPLE STATEMENT
----------------------------------------------------------------------------

-Nolan
Chemeketa CC

[Nolan at Chemeketa]

Concerning the existing document, I'm not sure I see the value in
listing the different varieties of distance courses, especially since
that list could change at any time. So I'd suggest leaving out the
line

"Distance education modalities include print, audio teleconferencing,
radio, audiocassettes, videoconferencing, videocassettes, computers,
on-line via the web, and other multimedia and interactive options. "

-Nolan
Chemeketa CC

[Orrange, Mary Beth]

My thoughts are similar to those Nolan expressed; that the quality of courses taught at a distance be equal to the quality of courses taught in more traditional circumstances. Part of that quality includes assessment standards.

No one has required that we develop a different position statement, but rather several concerned committee members asked us to address the issue. What I hear from folks in a variety of situations there is that there are areas of concern; incorporating some of Nolan's suggestions in the existing statement might be enough.

The existing statement includes:

Distance education programs must maintain high standards. Mathematics courses provide a large part of the basis for learning in many other courses and disciplines. Therefore, distance education mathematics courses must meet the content, pedagogical, and assessment standards used in traditional on-campus mathematics courses.

Perhaps this could be moved up to the top and/or expanded?

By the way, thank you for sharing the paper by Neil Rowe. It obviously directly supports our discussion.

Mary Beth

CONFIDENTIALITY NOTICE:
This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by e-mail and destroy all copies of the original message, including attachments.

[Dave]

I would completely agree that any online class should be as good or
better than a brick and mortar class. That is the focus of my proposed
sabbatical proposal...which will hopefully get acted upon next week.

We spend a lot of time talking about the effect of technology on how
content is delivered in online classes and how technology can promote
new ways of communicating (teacher to student, student to student, and
student to teacher). What I think has been lacking is the effect
technology has on how we assess and what we assess in mathematics.

There has been a lot of discussion about WolframAlpha, MyMathLab, and
WebAssign...their impact on what we assess and how it is done. I agree
that their are security issues with online testing...I want to point
out that those issues are not "fixed" by simply requiring students to
be physically present for an assessment. Many of you will remember my
experiences at the University of Arizona when I caught several
students taking exams for other students. They had what seemed to be
valid ID (they were fake) and had been paid to take the exam. his
became less of a problem as our class sizes dropped and I had more of
a personal relationship with each student. I knew who they were.

Unfortunately, if your states are at all like Arizona...you know that
class sizes are increasing. Particularly at public universities and
colleges. With fewer adjuncts and grad students, many classes have
exploded in size. I have found that the enrollment in my online
classes have increased with students attending universities and
colleges. They would rather take an online class than sit in a lecture
hall with 250 of their closest friends. The issue of testing security
is going to get worse not better. And it won't matter if it is in a
brick and mortar setting or an online setting.

I don't want to open a can of worms...oh what the heck...do exams
really assess what we really want students to know? I am not singling
out in person exams...I see the same problems with online homework
systems like MyMathLab or WebAssign.

With all of this technology that we are all fired up about, it would
seem that we could figure out better ways to assess students. What
percentage of the grade you give to students an exam percentage (by
this I mean short answer without justification of how the answer was
arrived at). Hmmmm....makes me think a bit more. There is an
assessment where no work is required to be shown, and then there is an
assessment where you look at the steps and give partial credit. What
do we value in students in terms of mathematics? Are our expectations
different for online students versus face to face students?

I have a hard time even phrasing the question regarding
assessment...maybe some of the rest of you might be better at phrasing
this. For my students, 50% of their grade comes from MyMathLab
Homework and Quizzes and the other 50% comes from other types of
assessments that require a sustained effort, solution strategies with
multiple steps and a sustained effort over several weeks. I do this in
my face to face classes as well as my online classes. In fact, my face
to face students have the exact same requirements as my online
students. The only difference is how they interact with the content in
the class (and perhaps with each other).

To me, the class is the class, A grade of B in College Algebra means
they earned a B in College Algebra, not a grade of B in "online"
college algebra.

Dave Graser
Yavapai College

[wmackey]

I really like Dr. Rowe's and Nolan's statements but I'd like to switch
it around a bit. See my version below. With apologies to Dr. Rowe.

wayne

Quoting Nolan at Chemeketa <custo...@gmail.com>:

[Hide Quoted Text]

----------------------------------------------------------------------------
SAMPLE STATEMENT ON ONLINE ASSESSMENT
----------------------------------------------------------------------------
Online assessment currently raises serious security issues and may
facilitate many new methods of cheating. For these reasons it is
recommended that:

1. Whether human-proctoring is or is not feasible, it is critical that

assessment questions are drawn randomly from a large pool and reordered

randomly and programmed with changing numeric parameters. No two
tests given should be exactly alike but must be equivalent.
Appropriate time
limits should also be enforced of course.

2. Human-proctored traditional paper-and-pencil tests with traditional
security procedures remain the least preferred method for major assessments
in distance learning as well as F2F classes but may be necessary with
a shortage of available computers.

It is anticipated that online assessment will become a normal
assessment option as new identification technologies are developed and
cheating countermeasures become more sophisticated. Until that time,
online assessment in distance courses should be done with caution.
----------------------------------------------------------------------------
END OF SAMPLE STATEMENT
----------------------------------------------------------------------------

Quoting Dave <David_...@yc.edu>:

> --
> You received this message because you are subscribed to the Google
> Groups "AMATYC-ITLC" group.
> To post to this group, send email to amaty...@googlegroups.com.
> To unsubscribe from this group, send email to
> amatyc-itlc...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/amatyc-itlc?hl=en.
>
>

[wmackey]

Dave,

I also believe that the only way to find out if someone has learned to
understand a concept or knows what a symbol means or has acquired a
skill in application or can solve problems is be asking them. That is
why assessing learning is different from giving grades for most
people. To give half a students grade based on a presumed great
effort is assuming that learning has taken place as a result of that
effort. It is fine for a grade but isn't assessing learning.

wayne

Quoting Dave <David_...@yc.edu>:

[Bruce Yoshiwara]

I do not accept the assumption, implicit in Wayne's two statements, that assessment should be based on machine grade-able questions.

Bruce Yoshiwara

> Date: Fri, 12 Feb 2010 15:22:08 -0600
> From: wma...@uark.edu
> Subject: Re: [amatyc-itlc] Re: AMATYC postition statement on Distance Education
> To: David_...@yc.edu
> CC: amaty...@googlegroups.com

[Nancy Rivers]

I stand with Bruce.

I definitely do not agree with the statement in Wayne's email copied below:

2. Human-proctored traditional paper-and-pencil tests with traditional
> security procedures remain the least preferred method for major assessments
> in distance learning as well as F2F classes but may be necessary with
> a shortage of available computers.

To me, equality of online and face to face classes is a perfectly valid reason for testing online students with the same assessment instruments used in the seated classes.

Nancy Rivers

>>> Bruce Yoshiwara <byosh...@hotmail.com> 02/12/10 5:20 PM >>>

[Nancy Rivers]

[Pauline Chow]

All of our online math students are required to take two paper tests and a paper final exam in our math courses, with (may be) one exception. We have three testing periods lasting between 3 to 4 days at 5 campuses. Online math faculty at the largest campus, Harrisburg campus, share the proctoring responsibility. Other campuses administer the paper testing through their test centers and by appointment. The Virtual Campus coordinates the scheduling, distribution, delivering, and collection of paper tests to all 5 campuses, and off-site proctoring. Our system has been working like a charm for the last 10 years. Please click on the links for more information:

http://www.hacc.edu/VirtualCampus/CurrentStudents/StudentResources/Testing-and-Proctoring.cfm
http://www.hacc.edu/VirtualCampus/CurrentStudents/StudentResources/Paper-Testing-Locations.cfm

Pauline Chow

>>> "Nancy Rivers" <njri...@waketech.edu> 02/13/10 10:21 AM >>>

[wmackey]

Bruce,

What is meant by "machine-gradable" tests?  

wayne

[wmackey]

Nancy,

That was what I was trying to say. That neither F2F or distance
classes should be tested in the old traditional ways. Over the
centuries students have figured out those security tricks. Both
should be tested the way I described to be completely fair about it.

wayne

[Bruce Yoshiwara]

A machine grade-able question can be scored without human intervention.  Such questions are typically found in homework grading systems like MyMathLab, MathZone, WebAssign, WeBWorK, etc.  For example, any question for which the required answer is an algebraic expression is machine grade-able.

In contrast, a question whose answer requires students to write sentences in English (or other natural language) cannot currently be usefully graded by machine.

Bruce Yoshiwara

---

Date: Sun, 14 Feb 2010 17:56:12 -0600
From: wma...@uark.edu
Subject: RE: [amatyc-itlc] Re: AMATYC postition statement on Distance Education
To: byosh...@hotmail.com
CC: amaty...@googlegroups.com

[Behnaz Rouhani]

Has anyone tried Maple TI? It can grade short answer questions. I am scheduled to pilot that in my online classes.  

 

Behnaz
----------------------------
Behnaz Rouhani, PhD

Department of Mathematics & Computer Science
Georgia Perimeter College Online

 

Email: behnaz....@gpc.edu

Website: http://facstaff.gpc.edu/~brouhani

[wmackey]

Bruce,

I agree that (I always called them essay questions) some questions like the ones you describe are not grade-able by a machine unless the machine had passed the Turing test.  However I don't think that assessment of mathematics learning can only be accomplished by essay tests.  I think, unless it has changed recently, that most tests given in developmental math courses require answers like numbers(integers, fractions and decimals), polynomials, rational expressions, equations, words or phrases, matrices, and other mathematical objects.  Aren't they?

[R Collins]

Wayne and all

 

I have noticed that the computerized testing though does not mix and match the problems but keeps them together by type.   Students are usually practicing not only the same concept but the exact same types of problems in batches. Sure they can do those problems after doing ten in a row but by the next week that knowledge is nearly gone (it would be a stretch to call this knowledge anyway) and by the next semester it is as though the student has never taken the previous course.  The students would say “ I never had this before… I never saw this before” and in their brains, they are right. It was in one side and out the other. I think that they call this parrot math in the lower grades.

Ruth

[Alison Schubert]

That may well be true in some classes but in my experience looking at only a final answer does not give you an adequate idea of the students understanding in many courses. Yes this is a judgement call but could you look at the final answer to a limit definition of the derivative and know that the student used the limit definition and not the shortcut rules?

It seems to me in a position statement we need to be carful and make sure our position applies to all levels of mathematics from developmental all the way through calculus. As a result I personally stand with do not feel computer graded exams are better then pencil and paper exams and would not like to add that implication to our position statement.

Alison Schubert
Wake Technical CC

>>> wmackey <wma...@uark.edu> 2/15/2010 1:19:40 pm >>>

Bruce,

I agree that (I always called them essay questions) some questions like the ones you describe are not grade-able by a machine unless the machine had passed the Turing test. However I don't think that assessment of mathematics learning can only be accomplished by essay tests. I think, unless it has changed recently, that most tests given in developmental math courses require answers like numbers(integers, fractions and decimals), polynomials, rational expressions, equations, words or phrases, matrices, and other mathematical objects. Aren't they?

wayne

Quoting Bruce Yoshiwara <byosh...@hotmail.com>:

> A machine grade-able question can be scored without human intervention. Such questions are typically found in homework grading systems like MyMathLab, MathZone, WebAssign, WeBWorK, etc. For example, any question for which the required answer is an algebraic expression is machine grade-able.
>
> In contrast, a question whose answer requires students to write sentences in English (or other natural language) cannot currently be usefully graded by machine.
>
> Bruce Yoshiwara
>

> -------------------------

[wmackey]

Ruth,

I agree totally.  The ones you've seen I imagine are the ones Bruce listed.  They are totally unacceptable as assessments of student learning.  I wasn't talking about them, just mine.  I'll show you in Boston if you come by my booth.

wayne

Quoting R Collins <Lv2...@comcast.net>:

Wayne and all

I have noticed that the computerized testing though does not mix and match the problems but keeps them together by type.   Students are usually practicing not only the same concept but the exact same types of problems in batches. Sure they can do those problems after doing ten in a row but by the next week that knowledge is nearly gone (it would be a stretch to call this knowledge anyway) and by the next semester it is as though the student has never taken the previous course.  The students would say ? I never had this before? I never saw this before? and in their brains, they are right. It was in one side and out the other. I think that they call this parrot math in the lower grades.

[wmackey]

Well if all I asked for was the derivative, no I couldn't tell how
they got it but I bet I could come up with some questions about the
limit definition of a derivative that would allow me to find out if
the had an understanding of it which is what you want to know anyway.

wayne

[Behnaz Rouhani]

Hi Mary Beth,

I think the position statements for technology and distance education should be separate. The technology statement applies to the technology used in f2f classes also. If we keep the two statements then the topic of integrity, assessment and proctored testing can be brought under the umbrella of distance education.

In my school several disciplines that offer online courses do not have any proctored exams and as such it makes it hard to assess those and compare their student performances with the f2f courses. In other words, only the f2f courses are assessed not the online!

Behnaz

http://www.amatyc.org/documents/Guidelines-Position/DistanceED.htm

http://www.amatyc.org/documents/Guidelines-Position

--

[R Collins]

There are similar situations in other courses. We have one campus site where
there is an extensive lab with anatomy and another campus site in the same
state where there is no lab. The schools know this and will not take the
course to transfer because they do not know if the student had a lab or not.
We are fighting ignorance about distance education and unfortunately some
truths here as well. Maybe a split is a good idea.

Ruth

[Behnaz Rouhani]

This is an excellent organization for the online testing Pauline. All our online college level courses require one proctored final exam and the learning support courses two. Starting this semester, We will no longer have paper tests and all proctored exams will be administered using computers.

Our testing centers cannot accommodate our students, thus we have put together a system whereby students go online and pick a time/day they wish to take their exams. If one instructor teaches 4 classes then they do NOT have to be there four times. You might say this is not a good system as a we do not get to see our own students at the final exam time. There is a solution for that too. When students sign up for times/day, name of the instructor that proctors that particular exam is there therefore they have a choice to pick the time/day their own instructor is proctoring an exam.

Behnaz