designoracle towards CSCW
0 views
Skip to first unread message
Paul André
May 2, 2013, 11:43:48 PM5/2/13
to crowdcamp-c13 Creativity Project
Hi all,
First, apologies for my reduced contribution so far; turns out job search is like adding a full-time job on top of everything else. and to that end, I'm away again Sat - Tue, so will miss our meeting. sorry! (feel free to bump me to Acknowledgment section of whatever we end up doing!)
But second, and more importantly: what are we actually doing?! From what I can gather, there's been a lot of great progress in individual parts, but I at least don't have a sense of what the forest looks like. (and maybe everyone else does, but my fear is that each of us has a slightly different idea of what the overall contribution is). This is really intended to just open a discussion so we are all on the same page, and we have in writing somewhere a clear goal.
- What problem are we addressing? what is our goal? what is novel about this? what are the compelling use cases for our system?
- I've heard creating paragraph short stories, as a more general example of complex creative artifacts with the crowd. but... when was the last time you read a paragraph short story?
- Other option is comparing methods for... eliciting creativity from the crowd? for exploring a design space? creativity support tool? What is the role of evaluation here, across different techniques?
-- from what I can tell, there are a suite of techniques developed/tested that we could focus/explore more: Yes/No; Snowflake; Save the Cat style 15 beats.
As an aside, I am reviewing a UIST paper about "Divergent, Synchronous Collaboration for Creative Story Writing". I'd like to share the entire thing, but that's probably not ethical. Maybe I can give highlights once I've actually read it.
Apologies if I'm just late to the party, but from the meetings I have been in I still don't think we're clear on this, and to move forward (and to have any chance at CSCW), we need to be able to articulate our specific approach and contribution. PLEASE REPLY via e-mail! This is something that needs some thought and context, and probably can't be hashed out over skype.
cheers,
Paul
First, apologies for my reduced contribution so far; turns out job search is like adding a full-time job on top of everything else. and to that end, I'm away again Sat - Tue, so will miss our meeting. sorry! (feel free to bump me to Acknowledgment section of whatever we end up doing!)
But second, and more importantly: what are we actually doing?! From what I can gather, there's been a lot of great progress in individual parts, but I at least don't have a sense of what the forest looks like. (and maybe everyone else does, but my fear is that each of us has a slightly different idea of what the overall contribution is). This is really intended to just open a discussion so we are all on the same page, and we have in writing somewhere a clear goal.
- What problem are we addressing? what is our goal? what is novel about this? what are the compelling use cases for our system?
- I've heard creating paragraph short stories, as a more general example of complex creative artifacts with the crowd. but... when was the last time you read a paragraph short story?
- Other option is comparing methods for... eliciting creativity from the crowd? for exploring a design space? creativity support tool? What is the role of evaluation here, across different techniques?
-- from what I can tell, there are a suite of techniques developed/tested that we could focus/explore more: Yes/No; Snowflake; Save the Cat style 15 beats.
As an aside, I am reviewing a UIST paper about "Divergent, Synchronous Collaboration for Creative Story Writing". I'd like to share the entire thing, but that's probably not ethical. Maybe I can give highlights once I've actually read it.
Apologies if I'm just late to the party, but from the meetings I have been in I still don't think we're clear on this, and to move forward (and to have any chance at CSCW), we need to be able to articulate our specific approach and contribution. PLEASE REPLY via e-mail! This is something that needs some thought and context, and probably can't be hashed out over skype.
cheers,
Paul
Yotam Gingold
May 3, 2013, 2:19:39 AM5/3/13
to crowdcamp-c13 Creativity Project
Good luck on the job hunt, Paul. Finding a job is a full-time job, without the salary (or benefits in the traditional sense).
Here is my take:
A system that gathers and refines (iteratively) creative story outlines from a crowd of not-very-creative people. This is evaluated in two ways: (1) by comparing the gathered stories using our technique to stories gathered in a straightforward manner (not questions-based), and (2) by comparing the iteratively refined stories to initial gathered stories.
Who needs creative story outlines? I think that working designers want to browse interesting ideas and be inspired. When I do graphic design (I make web pages a lot more often than I write stories), I like to browse examples.
Yotam
Kurt Luther
May 3, 2013, 1:10:54 PM5/3/13
to Yotam Gingold, crowdcamp-c13 Creativity Project
Hi Paul, thanks for asking these questions. I agree it's still a
little unclear what the goal of the project is.
Yotam, the goal and experiment design you describe makes sense to me.
But, I'm not as convinced about the motivation for the system. If I
understand you, the claim is that the system will help writers by
providing very short stories that might inspire them in their writing.
I don't know much about how writers develop new stories -- do any of
us? -- but it seems unlikely to me that they lack sources of
inspiration. Surely the published books and short stories they read
every day, or their own personal experiences, provide plenty of
inspiration? What evidence do we have that this is insufficient, or
that our crowd stories would be better? If the goal of our system is
providing inspiration to writers, how do we measure that? I guess we'd
ask people to write stories after exposure to our crowd stories, and
then try to determine how much influence they had -- this seems
tricky.
A different approach might be to say the goal is having crowds write a
compelling short story, and these are early steps towards that goal.
Then the focus shifts away from the outcome and towards comparing
different processes -- "ok, none of these stories are great, but which
is the best and why?" If we take this approach, and a crowdsourced
short story is our ultimate goal, I would argue the single greatest
challenge to a crowdsourced short story is maintaining consistency
across a large body of text. (Compelling characters, exciting plot
points, high quality writing are all irrelevant if the story is
incoherent and inconsistent.) I *think* some of our work could be
repurposed for this framing, because yes/no and Snowflake both provide
useful, yet different, approaches to maintaining consistency. But,
both would run into big challenges as the story got longer than a page
or so, and we haven't gotten far in addressing those challenges yet.
Just my two cents... what do you think?
Kurt
--
Kurt Luther
www.kurtluther.com
little unclear what the goal of the project is.
Yotam, the goal and experiment design you describe makes sense to me.
But, I'm not as convinced about the motivation for the system. If I
understand you, the claim is that the system will help writers by
providing very short stories that might inspire them in their writing.
I don't know much about how writers develop new stories -- do any of
us? -- but it seems unlikely to me that they lack sources of
inspiration. Surely the published books and short stories they read
every day, or their own personal experiences, provide plenty of
inspiration? What evidence do we have that this is insufficient, or
that our crowd stories would be better? If the goal of our system is
providing inspiration to writers, how do we measure that? I guess we'd
ask people to write stories after exposure to our crowd stories, and
then try to determine how much influence they had -- this seems
tricky.
A different approach might be to say the goal is having crowds write a
compelling short story, and these are early steps towards that goal.
Then the focus shifts away from the outcome and towards comparing
different processes -- "ok, none of these stories are great, but which
is the best and why?" If we take this approach, and a crowdsourced
short story is our ultimate goal, I would argue the single greatest
challenge to a crowdsourced short story is maintaining consistency
across a large body of text. (Compelling characters, exciting plot
points, high quality writing are all irrelevant if the story is
incoherent and inconsistent.) I *think* some of our work could be
repurposed for this framing, because yes/no and Snowflake both provide
useful, yet different, approaches to maintaining consistency. But,
both would run into big challenges as the story got longer than a page
or so, and we haven't gotten far in addressing those challenges yet.
Just my two cents... what do you think?
Kurt
--
Kurt Luther
www.kurtluther.com
Kathleen Tuite
May 3, 2013, 7:10:34 PM5/3/13
to Kurt Luther, Yotam Gingold, crowdcamp-c13 Creativity Project
This is good that we're having this discussion!
To agree with Kurt's 2nd paragraph, I was thinking during the last meeting that comparing the 2 or 3 different methods of snowflake, yes/no, structured yes/no might be a reasonable endeavor. The point being, again as Kurt said, investigating how to extract a creative and interesting story from the crowd.
But... is the crowd component there yet? Don't we still have one person at a time doing the yes/no questions or expanding one snowflake sentence? I know there have been things where the yes/no story output of one person was translated into a paragraph by another person, and people marked the critical yes/no questions of others, and there was talk of expanding the snowflake one level deeper... am I missing something? Does that exist yet? Can I see it and play with it?
What I'd like to see is something that starts off with the yes/no question thing being seeded by one person, and then another person coming along and building off of that story, maybe pruning it or maybe expanding it or maybe even changing a no to a yes or vice versa.
Brief anecdote about Björn Hartmann's cat book: I bought it because I like the crowd and I like cats. The stories in it were pretty weird though, and it made me think of all these cat stories I personally had that I thought would be way better. If these people writing their own cat stories had got to soak in other people's stories first and see what was written and maybe have in mind a level of quality/humor/etc that they wanted to meet or surpass... maybe the stories would have been better. I think the question of "how can we get the crowd to (collaboratively?) produce quality creative content" is pretty nice, although the motivation of writers lacking inspiration or something doesn't really speak to me.
- Kathleen
Robin
May 5, 2013, 10:14:18 AM5/5/13
to Kathleen Tuite, Kurt Luther, Yotam Gingold, crowdcamp-c13 Creativity Project
Hello all,
Similar to Paul, I was a little confused about the goal of the project. Combining Yotam, Kurt and Kathleen's responses, comparing crowdsourced story elicitation methods to help crowds produce quality creative content seems to work. I think one motivation would be how to get crowds to complete more abstract tasks so they're able to more sufficiently complete a variety of crowd jobs in the future.
p.s. I've modified the code to the last site to fix that 2yeses+no error and to display the current element as people ask questions. Next question I have (which can be answered tomorrow) is will this be modified to include snowflake and in what way
Yotam Gingold
May 5, 2013, 11:19:56 AM5/5/13
to Kathleen Tuite, Kurt Luther, crowdcamp-c13 Creativity Project
On May 4, 2013, at 1:10 AM, Kathleen Tuite <kathlee...@gmail.com> wrote:
> This is good that we're having this discussion!
Agree.
> This is good that we're having this discussion!
> ...
> Wrapping up...
> I think the question of "how can we get the crowd to (collaboratively?) produce quality creative content" is pretty nice, although the motivation of writers lacking inspiration or something doesn't really speak to me.
> Personally, I am all for *enabling* collaborative creativity in random folks in the crowd. What does a creativity support tool *for the crowd* look like? What does that even mean?
I like Kathleen's summary here. I do want to add that although "writers lack inspiration" sounds contrived to me, too, I am fairly certain that graphic designers and visual artists love having design inspiration and spend actual time collecting it. So a visual analog of our process, such as the forsaken poster design, wouldn't be contrived.
> I think the question of "how can we get the crowd to (collaboratively?) produce quality creative content" is pretty nice, although the motivation of writers lacking inspiration or something doesn't really speak to me.
> Personally, I am all for *enabling* collaborative creativity in random folks in the crowd. What does a creativity support tool *for the crowd* look like? What does that even mean?
Yotam
> ...
Yotam Gingold
May 5, 2013, 11:22:12 AM5/5/13
to crowdcamp-c13 Creativity Project
I also agree with Kurt:
> A different approach might be to say the goal is having crowds write a
> compelling short story, and these are early steps towards that goal.
> Then the focus shifts away from the outcome and towards comparing
> different processes -- "ok, none of these stories are great, but which
> is the best and why?" If we take this approach, and a crowdsourced
> short story is our ultimate goal, I would argue the single greatest
> challenge to a crowdsourced short story is maintaining consistency
> across a large body of text.
and Robin:
> comparing crowdsourced story elicitation methods to help crowds produce quality creative content
Yotam
> A different approach might be to say the goal is having crowds write a
> compelling short story, and these are early steps towards that goal.
> Then the focus shifts away from the outcome and towards comparing
> different processes -- "ok, none of these stories are great, but which
> is the best and why?" If we take this approach, and a crowdsourced
> short story is our ultimate goal, I would argue the single greatest
> challenge to a crowdsourced short story is maintaining consistency
> across a large body of text.
> comparing crowdsourced story elicitation methods to help crowds produce quality creative content
Kanya(Pao) Siangliulue
May 6, 2013, 11:51:01 AM5/6/13
to Yotam Gingold, crowdcamp-c13 Creativity Project
To echo many of your thoughts, I think it would be interesting to look into having crowd generate a good story as a goal in itself. The motivation on my part might come from a different angle.
While creativity can be considered as an individual activity, creating something with others can be quite fun (like when you play music in a band). However, there is no existing platform/method that actually helps people get together to micro-write without stepping on each other's toe. I imagine that our project can contribute by exploring different structures that can move towards that goal.
Talk to you all soon,
Pao
Kurt Luther
May 6, 2013, 1:13:56 PM5/6/13
to crowdcamp-c13 Creativity Project
Hi all, here is my attempt to synthesize all the responses:
* Motivation: very broadly, we're interested in exploring how the
crowd can do complex, creative, and/or abstract tasks. Less broadly,
we are interested in (1) how crowds can do high quality creative
writing. We are also interested in building tools to support
collaborative story writing (2). Both of these goals are further down
the road, though. Our current project is meant to make small but
important steps towards those goals. We are not making any claims that
the stories produced by our experiments are valuable per se. Our
specific motivation is understanding which story eliciting technique
produces the best results from the crowd.
* Method: we will run experiments with several techniques for
eliciting stories from the crowd. These may include yes/no, yes/no +
prompts, branching, and Snowflake. It will be important to make sure
these techniques are collaborative in some way, e.g. crowd workers
building on each other's work, pruning ineffective branches, etc. The
exact experiment conditions are TBD.
* Evaluation: most or all crowd-produced stories in the experiment are
too short (or too poorly written) to be comparable to a recognizable
genre of writing. But, we can still compare stories generated from
different techniques. Our current evaluation approach is to have
crowds rate stories using Likert scales along several dimensions
(surprise, character, plot, etc.)
Does this work for everyone? Am I leaving anything out?
--
Kurt Luther
www.kurtluther.com
* Motivation: very broadly, we're interested in exploring how the
crowd can do complex, creative, and/or abstract tasks. Less broadly,
we are interested in (1) how crowds can do high quality creative
writing. We are also interested in building tools to support
collaborative story writing (2). Both of these goals are further down
the road, though. Our current project is meant to make small but
important steps towards those goals. We are not making any claims that
the stories produced by our experiments are valuable per se. Our
specific motivation is understanding which story eliciting technique
produces the best results from the crowd.
* Method: we will run experiments with several techniques for
eliciting stories from the crowd. These may include yes/no, yes/no +
prompts, branching, and Snowflake. It will be important to make sure
these techniques are collaborative in some way, e.g. crowd workers
building on each other's work, pruning ineffective branches, etc. The
exact experiment conditions are TBD.
* Evaluation: most or all crowd-produced stories in the experiment are
too short (or too poorly written) to be comparable to a recognizable
genre of writing. But, we can still compare stories generated from
different techniques. Our current evaluation approach is to have
crowds rate stories using Likert scales along several dimensions
(surprise, character, plot, etc.)
Does this work for everyone? Am I leaving anything out?
--
Kurt Luther
www.kurtluther.com
Robin
May 6, 2013, 1:31:56 PM5/6/13
to Kurt Luther, crowdcamp-c13 Creativity Project
Sounds good to me. P.s. I'm on the way home. May be 10 min late to the meeting
Robin
Sent from my Android device
Krzysztof Gajos
May 6, 2013, 2:03:04 PM5/6/13
to Kurt Luther, crowdcamp-c13 Creativity Project
I largely agree with this emergent picture.
Reply all
Reply to author
Forward
0 new messages