Re: Week 4: writing feedback
3 views
Skip to first unread message
Vanessa Crary Vaile
Jan 17, 2009, 10:53:18 PM1/17/09
to tipstri...@googlegroups.com
Both but the emphasis is on document files. Checkers and more sophisticated text analysis software are part of what we can do but not the only thing - and an option to be used carefully and never never depended on. More important, rather than just teacher using checking software, I that students learn to use checkers in word processing programs and online rater software to identify mistakes - not to depend on them for corrections (double off for checker errors in the final version). They correct and revise initial drafts themselves. No software can replace the human reader. Students trained in peer review can give feedback - not correct mistakes - about how something reads. What works, What does not work. What makes sense. What is clear or not clear. |
David
Jan 18, 2009, 1:44:44 PM1/18/09
to TipsTricks2009
Vanessa can you steer us to the free rater software you mentioned --
I'd love to have a look at it (and apologies if you already posted the
link and I missed it).
Sorry about the audio problems today too. We could hear you but it
seemed like you couldn't hear us...... And your name was grayed out
after I gave you audio rights... that's not right.
David
I'd love to have a look at it (and apologies if you already posted the
link and I missed it).
Sorry about the audio problems today too. We could hear you but it
seemed like you couldn't hear us...... And your name was grayed out
after I gave you audio rights... that's not right.
David
efler
Jan 19, 2009, 9:36:27 AM1/19/09
to TipsTricks2009
Dear all,
You can see a demonstration of automatic writing grading at
http://www.writetolearn.net
This software automatically grades student writing and works like
this:
A large number of student essays are marked by human experts.
The software processes these essays and scores.
New essays submitted on the same topic are compared with the existing
bank of essays and scores.
Results are provided immediately.
Automatic marking of student essays is already being used in high-
stakes exams and results are said to be as reliable as human markers.
Like me, probably many of you have been involved in high stakes essay
correction. Apparently humans can easily mark 99 essays perfectly but
be way off on the 100th. Standardizing scores so that two human
markers give the same grades is also tricky.
There are various research papers listed here:
http://www.pearsonkt.com/resPubAutEssayGrad.shtml
Best wishes
Steve Chadwick
You can see a demonstration of automatic writing grading at
http://www.writetolearn.net
This software automatically grades student writing and works like
this:
A large number of student essays are marked by human experts.
The software processes these essays and scores.
New essays submitted on the same topic are compared with the existing
bank of essays and scores.
Results are provided immediately.
Automatic marking of student essays is already being used in high-
stakes exams and results are said to be as reliable as human markers.
Like me, probably many of you have been involved in high stakes essay
correction. Apparently humans can easily mark 99 essays perfectly but
be way off on the 100th. Standardizing scores so that two human
markers give the same grades is also tricky.
There are various research papers listed here:
http://www.pearsonkt.com/resPubAutEssayGrad.shtml
Best wishes
Steve Chadwick
David
Jan 20, 2009, 11:04:02 AM1/20/09
to TipsTricks2009
Thank you very much Steve! :)
I'm going to try this out with my students (or maybe write an essay
myself and see how I did...)
> You can see a demonstration of automatic writing grading athttp://www.writetolearn.net
I'm going to try this out with my students (or maybe write an essay
myself and see how I did...)
> You can see a demonstration of automatic writing grading athttp://www.writetolearn.net
Vanessa
Jan 20, 2009, 6:38:39 PM1/20/09
to TipsTricks2009
Steve -
About 5 or so years back, I did a bunch of research on LSA, one of the
mathematical models used in automatic marking systems. Back then,
university hosted LSA research sites had writing rater and summary
programs - open to use for free, Beta testing you might say. There are
positives and negative about raters. They are tools we can use but,
despite superiority to standard grammar check software, still flawed
and would flunk Thoreau in a heartbeat.
I would like to see other sources. The Pearson site cannot be
considered unbiased or likely to post research that does not support
using their product.
Vanessa
> You can see a demonstration of automatic writing grading athttp://www.writetolearn.net
About 5 or so years back, I did a bunch of research on LSA, one of the
mathematical models used in automatic marking systems. Back then,
university hosted LSA research sites had writing rater and summary
programs - open to use for free, Beta testing you might say. There are
positives and negative about raters. They are tools we can use but,
despite superiority to standard grammar check software, still flawed
and would flunk Thoreau in a heartbeat.
I would like to see other sources. The Pearson site cannot be
considered unbiased or likely to post research that does not support
using their product.
Vanessa
> You can see a demonstration of automatic writing grading athttp://www.writetolearn.net
efler
Jan 21, 2009, 6:07:48 AM1/21/09
to TipsTricks2009
Hi Vanessa,
ETS uses software called e-rater and Criterion to score TOEFL essays
www.ets.org/research/erater.html
There is an article in the Journal of Technology, Learning, and
Assessment
http://escholarship.bc.edu/jtla/vol4/3/
Regards
Steve
Vanessa
Jan 22, 2009, 6:05:02 PM1/22/09
to TipsTricks2009
Steve thanks for the article link. As for the other, not new - known
about it since git-go. ETS introduced AWE scoring for the GMAT first,
primarily because that was the only writing test ETS offered wiithout
option of submitting a handwritten version. There was a lot of
discussion surrounding the decision, some of it still interesting
reading. Now GMAT scoring by AWE is well accepted - and as to be
expected, GMAT prep materials include extensive instructions and tips
on how to game the software. Although under consideration for SAT
essay scoring, ETS is being much slower to adopt it there. IMO only a
matter of time.
AWE is here to stay - inevitable since the earliest prototypes in the
60s - and potentially as useful tool. I wish the were more free online
raters were available, but now that it's established, accepted and no
longer in testing, most (and no doubt the better ones) are no longer
free. Both Criterion's and e-Rater's free sites included ones
specifically for middle school and high school writing, ideal for
using with intermediate ESL writers. Even the UC and CSU systems had
them up as self-evaluation tool for potential students - a handy tool
for advanced ESL writers.
Any tool is just that - a tool. It's up to the tool-user to wield it
well and above all wisely.
Vanessa
On Jan 21, 4:07 am, efler <s...@examenglish.com> wrote:
> Hi Vanessa,
>
> ETS uses software called e-rater and Criterion to score TOEFL essays www.ets.org/research/erater.html
>
> There is an article in the Journal of Technology, Learning, and
> Assessmenthttp://escholarship.bc.edu/jtla/vol4/3/
about it since git-go. ETS introduced AWE scoring for the GMAT first,
primarily because that was the only writing test ETS offered wiithout
option of submitting a handwritten version. There was a lot of
discussion surrounding the decision, some of it still interesting
reading. Now GMAT scoring by AWE is well accepted - and as to be
expected, GMAT prep materials include extensive instructions and tips
on how to game the software. Although under consideration for SAT
essay scoring, ETS is being much slower to adopt it there. IMO only a
matter of time.
AWE is here to stay - inevitable since the earliest prototypes in the
60s - and potentially as useful tool. I wish the were more free online
raters were available, but now that it's established, accepted and no
longer in testing, most (and no doubt the better ones) are no longer
free. Both Criterion's and e-Rater's free sites included ones
specifically for middle school and high school writing, ideal for
using with intermediate ESL writers. Even the UC and CSU systems had
them up as self-evaluation tool for potential students - a handy tool
for advanced ESL writers.
Any tool is just that - a tool. It's up to the tool-user to wield it
well and above all wisely.
Vanessa
On Jan 21, 4:07 am, efler <s...@examenglish.com> wrote:
> Hi Vanessa,
>
> ETS uses software called e-rater and Criterion to score TOEFL essays www.ets.org/research/erater.html
>
> There is an article in the Journal of Technology, Learning, and
Debby
Jan 23, 2009, 9:21:09 PM1/23/09
to TipsTricks2009
We had a lecture on this subject last year at Georgia State
University, and the pro-software group insists that research proves
that the software correction is as accurate or more accurate than
human correction. To me, this means that we really don't care about
the "artistic" quality of writing. Students are being training to
write boring little basic 5 paragraph essays with the topic sentence
first in each paragraph, and following through a fixed format of
content development. I know that with ESOL writers, especially
beginners, we aren't looking for creativity, but I feel like we're
squeezing the soul out of writing and teaching our students cookie-
cutter automatic writing. I wonder how some of our great authors would
have fared with this kind of testing.
> > > > Steve Chadwick- Hide quoted text -
>
> - Show quoted text -
University, and the pro-software group insists that research proves
that the software correction is as accurate or more accurate than
human correction. To me, this means that we really don't care about
the "artistic" quality of writing. Students are being training to
write boring little basic 5 paragraph essays with the topic sentence
first in each paragraph, and following through a fixed format of
content development. I know that with ESOL writers, especially
beginners, we aren't looking for creativity, but I feel like we're
squeezing the soul out of writing and teaching our students cookie-
cutter automatic writing. I wonder how some of our great authors would
have fared with this kind of testing.
>
> - Show quoted text -
Vanessa
Jan 31, 2009, 10:57:35 PM1/31/09
to TipsTricks2009
Debby
What I'm thinking is - use the raters to free up teacher time for
meaningful feedback and suggestions for improving their writing -
whether as a preliminary scan to check mechanics or to train students
to use as a checking tool to look for mistakes, not for evaluating
content or style.
And yes you are right about how great writing would fare. The
Gettysburg Address flunked. How's that for perspective?
Vanessa
What I'm thinking is - use the raters to free up teacher time for
meaningful feedback and suggestions for improving their writing -
whether as a preliminary scan to check mechanics or to train students
to use as a checking tool to look for mistakes, not for evaluating
content or style.
And yes you are right about how great writing would fare. The
Gettysburg Address flunked. How's that for perspective?
Vanessa
Reply all
Reply to author
Forward
0 new messages