About this Group

[Jinyoung Kim]

This is a web board for EPS2011 workshop.

We'll share slides and ideas before the workshop, and continue the
discussion after the workshop.

Here are a couple of my ideas of what each participants (including the
organizers) can prepare for the workshop.

------

I think the biggest goal is for participants to be agreed on a
specific evaluation method in the end, and I'm a bit concerned that a
free-form discussion won't result in anything concrete.

Therefore, in the following discussion, I think we can structure it to
some extent, and one way is to ask the participants to take a
position. Here are several positions I could take from our
submissions.

P1 : Let's improve simulation techniques

P2 : Let's perform an instrumentation study with shared infra (code
and users)

P3 : Let's create an real dataset that's sharable (by anonymization
and annotation)

Participants already expressed their position to some extent in their
paper, so it won't be difficult for them to do. They can take one
position, or a combination of a few. They can possibly do that before
the workshop begins, so that they can prepare for their argument. For
instance, we can ask participants to send us the response to the
following questions (in a page or two):

- What would be the strength and limitations of each position, and
what was the rationale behind your choice?
- Can you describe a concrete evaluation plan based on your position?

We can be even more be specific and let the participants think about
potential limitations of the each position.

- For simulated evaluation : how would you overcome the lack of
connection between the user and the collection?
- For instrumentation study : what would be a reasonable starting
point (e.g. Google Desktop API) on top of which participants can
implement their algorithms?

If we can start the discussion with these materials at hand, we are
likely to end up with a concerte outcome, even if we perform a free-
form discussion. In a sense, I think the success of this workshop will
depend on to what extent we can let participants be prepared in
advance.