On-line rebuttal, first-hand experience
gvc
experience.
"In a 2063-character text box, respond to criticisms like this:
"While this paper presents some impressive and interesting way to
improve
spam filtering, I think there remain some problems with the writing,
theory
and experiments.
"[...] the so called theory is purely some imagination without any
theoretical or experimental support. [...] I expect to see some
proofs or
some verification in the experiments.
"I don’t think the experiments are appropriately done, especially with
respect to the previous works by the same author. Although I’m not an
expert
on spam filtering, I tried to read the author’s papers on KDD 2009 and
SIGIR
2009. After that, I was surprised to see that they don’t use the
ranking method
in SIGIR, neither compare against their own proposal in KDD. I hope
the
authors to respect their own studies and keep their paper consistent
with the
existing results"
[This message is already over 1/2 the space allocated for rebuttal,
and I've
quoted only one referee's [totally off-base] criticism. There's only
space
to snark:
-- We mean theory in the scientific sense, as a model used to
make testable claims, like "this method will improve global
spam filter results." Then the testable claims are validated
by laboratory and human experiments.
-- The current work measures filter performance using ROC curves,
which are standard. AUC area cannot be used as a summary because
we are comparing a point and a curve, not two curves. Similarly,
point measures like accuracy cannot be used. So we project the
point and the curve to the y axis and take the ratio. What's
wrong
with that? Our SIGIR and KDD papers were concerned with ranking
the effectiveness of filters; this paper is concerned with
comparing
a specific method to a baseline. So ranking approaches are
inapplicable.]
[That's 2000 characters for one of four reviews, and not nearly
enough detail for you or the committee to decide whether I or the
reviewer is correct. It's just a pissing match.]
Using high-stakes conferences to bless results and researchers
is just plain stupid. A lightening rebuttal round just makes it
into "American Idol" with you standing up to be insulted.
I guess we'll see whether "Impressive and Interesting" wins out
over he-said she-said over methodology. I doubt it, because
pathological risk aversion has taken hold of the review process.
Is "Impressive and interesting and we can't tell if it's correct"
grounds for acceptance or rejection? I'd think that'd be a great
role for conferences. Much better than being ersatz journals.