I took two late days

[Yotam Gingold]

Aside: I browsed a paper, "Gender Differences in the Effects of Anticipated Evaluation on Creativity" (John Baer 1997 Creativity Research Journal) [ http://users.rider.edu/~baer/Gender&Evaluation.pdf ]. He asked three expert judges to evaluate every story and poem created with these instructions:

>>>

"There is only one criterion in rating these tests: creativity. I realize that creativity doesn't exist in a vacuum, and to some extent creativity probably overlaps other criteria one may apply-aesthetic appeal, organization, richness of imagery, sophistication o f expression, novelty o f word choice, appropriateness of word choice, and possibly even correctness of grammar, for example-but I ask you to evaluate the poems [stories] solely on the basis ofyour thoughtful-but-subjective opinions of their creativity. The point is, you are the expert, and you needn't defend your choices or articulate a definition of creativity. What creativity means to you can remain a mystery-what I want you to do is use that mysterious expert sense to evaluate the poems [stories] for creativity."

A 1.0 (low creativity) to 5.0 (high creativity) rating scale was used, and judges were encouraged to use the full scale; that is, they were encouraged not to concentrate their ratings around a single score point. There were no specific limitations about the numbers of papers that were to fall in each scoring range, however.

<<<

For stories written from the same prompt, he had good interrater reliability. Did we look at interrater reliability only for people with the same prompt?

[P.S. The Braer paper found that extrinsic motivation decreased the creativity of 8th-grade girls but not 8th-grade boys versus intrinsic motivation.]

=== Some ideas for experiments and research questions they would answer ===

Given a prompt (what to do with a paperclip; generate app or story ideas; what happens next? to help with writers block; what is the surprise ending?) and optionally some facts or context:

    Ask people to suggest one (or more) ideas.

    vs

    Ask people to ask questions to guess my idea.

Research question: Does one elicit greater creativity by telling people the work is already done?

For example:

    "Write a one-sentence idea for a story."

    vs

    "I have thought of a story; guess it with yes/no questions." At the end of the task, "Please summarize the story in one sentence."

---

Control: Have Turkers iteratively mutate a story (adding to it, remove from it, changing something) into a hopefully longer and longer story. Turkers are shown the whole thing or just the end.

Variation: With every mutation, the Turker also updates a one-sentence summary that is displayed.

Research question: Can we improve the consistency of the writing with worker-maintained context in the form of a summary? ("Kurt's statement that context is the challenge of all crowdsourcing.")

---

Variation 1:

    Start with a tiny image.

    Iteratively: Scale it up and get people to add detail.

    vs.

    Scale a tiny image up to a large size and get people to iteratively fill in details.

Variant 2:

    Whether the "tiny image" from Variation 1 is actually a blank canvas.

Research questions:

    Variant 1: Does iterative improvement for drawing tasks work better with a level-of-detail approach?

    Variant 2: Does "priming people" with a small fuzzy image produce aesthetically more pleasing results?

---

Given a one-sentence story idea, please write a 100-word story.

vs

Given a one-sentence story idea, please use the snowflake method to expand it into 5 sentences, then (same person) summarize as a 100-word paragraph.

Research question: Does the snowflake method generate 100-word stories with more well-developed plots?

---

(Off the deep end) Simple algorithmic measure for creativity of one-sentence stories.

Possibility: Does a histogram of word-rarity correlate well with traditional measures of creativity?

Research question: Can we find a computational measure for a one-sentence story's creativity that correlates well with traditional measurements, like asking experts to evaluate an artifact or asking many people whether an idea is "novel and useful"?

Danger: There is work in this domain I am not familiar with.

=== The part where I sign my name ===

Yotam