Quantitative and Qualitative Copy/Content Measures

85 views
Skip to first unread message

Suz Bednarz - Kish

unread,
Apr 25, 2012, 2:58:23 PM4/25/12
to content...@googlegroups.com
I am tasked with producing some sort of metrics on our copy/content quality.

Easy to measure are things like  # of words, typos, adherence to style guide, nomenclature, literacy level,  but how do you truly measure value of copy/content to the true consumers of that content? 

Webby Awards for website content gives the criteria of "Content is the information provided on the site. It is not just text, but music, sound, animation, or video -- anything that communicates a sites body of knowledge. Good content should be engaging, relevant, and appropriate for the audience. You can tell it's been developed for the Web because it's clear and concise and it works in the medium. Good content takes a stand. It has a voice, a point of view. It may be informative, useful, or funny but it always leaves you wanting more."

All well and good but how do you go about determining if your content is engaging, relevant, appropriate, etc.?

Has anyone here cracked this nut? Do you use surveys? Focus groups? Opinionlab Feedback? Some other tool? 


=============
I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel. - Maya Angelou

Rick Yagodich

unread,
Apr 25, 2012, 3:23:57 PM4/25/12
to content...@googlegroups.com
Well, Suz, this one is pretty easy to answer (blatantly stolen from
Scott Abel and Rahel Bailie):

Scott would point you in the direction of ifixit.com - where every piece
of content is directly tied to revenue. They can take a purchase, and
see the instructional article the user was reading immediately prior to
selecting the product. You have a problem, look for a solution, get
fixing instructions, and can buy the tools/pieces immediately. The
quality of the content related directly to how much money it generates
(ok, also taking into acount the frequency of the particular problem).

Rahel will tell you that every single piece of content MUST be able to
justify its existence. It must facilitate a user activity, in order to
achieve a business objective that fulfils a board-level goal. I wrote
something up based on her model as described at CS Applied:
http://think-info.com/2012/04/16/the-content-testing-ground/

- R
> --
> You received this message because you are subscribed to the Google
> Groups "Content Strategy" group.
> To post to this group, send email to content...@googlegroups.com.
> To unsubscribe from this group, send email to
> contentstrate...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/contentstrategy?hl=en.

Suz Bednarz - Kish

unread,
Apr 25, 2012, 3:31:32 PM4/25/12
to content...@googlegroups.com
Thanks. I am a fan of Scott and Rahel so appreciate the links. Will check them out.


=============
I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel. - Maya Angelou
To post to this group, send email to contentstrategy@googlegroups.com.
To unsubscribe from this group, send email to contentstrategy+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/contentstrategy?hl=en.
--
You received this message because you are subscribed to the Google Groups "Content Strategy" group.
To post to this group, send email to contentstrategy@googlegroups.com.
To unsubscribe from this group, send email to contentstrategy+unsubscribe@googlegroups.com.

Milan Davidović

unread,
Apr 25, 2012, 3:36:41 PM4/25/12
to content...@googlegroups.com
On Wed, Apr 25, 2012 at 2:58 PM, Suz Bednarz - Kish
<bluest...@gmail.com> wrote:
> I am tasked with producing some sort of metrics on our copy/content quality.

This may be of interest, even if not a complete match with what you're
supposed to do:
http://www.slideshare.net/IntelligentContent/beyond-visits-and-page-views-how-to-develop-actionable-web-metrics-reports-and-analysis

--
Milan Davidović
http://twitter.com/altmilan
http://altmilan.blogspot.com
http://ca.linkedin.com/in/milandavidovic

Suz Bednarz - Kish

unread,
Apr 25, 2012, 3:37:48 PM4/25/12
to content...@googlegroups.com
Thanks Milan. 

=============
I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel. - Maya Angelou
--
You received this message because you are subscribed to the Google Groups "Content Strategy" group.
To post to this group, send email to content...@googlegroups.com.
To unsubscribe from this group, send email to contentstrate...@googlegroups.com.

kjohnson

unread,
Apr 25, 2012, 6:27:18 PM4/25/12
to Content Strategy
I've been think quite a bit about the quantitative measurement of
content quite a bit in the past few years. It's particularly
complicated in our case because I work on a higher ed site, and our
main revenue-generating purchase points (applying/accepting a space
and donating) are built on years and decades of interaction with our
institution, respectively--interaction with everything from news
stories to degree program marketing to attending events to buying
sweatshirts. (And of course, being a nonprofit institution we have
some pretty substantial limitations when it comes to resources, which
calls for a lot of resourcefulness on our part. Including that we
don't yet have a CRM to track constituents' behavior over time.)

(And there's always the challenge of interpreting data--whether a
higher bounce rate means that users are getting to the page they need
and finding the information they need, or whether the page isn't
relevant. Whether a higher time on page indicates more engaging
content or content that's harder to parse and navigate.)

The best answer I've found so far is to be very clear about the goal
of every piece of content, and to do some creative reasoning about
what numbers can measure that specific goal. We have a number of
information pages for prospective students, for example, and no single
purchase path that users follow to get to the application process. So
I'm tracking a ratio of unique visits to applications ultimately
submitted, in order to get an overall sense of how our content is
relating to behavior. And some purely informative pages with the type
of information that we know students tend to want earlier in the
recruitment cycle when they're gaining awareness--essentially
compiling lists of their options to delve into later--I'm more
comfortable with higher bounce rates there than higher-touch pages
that come up later in the cycle. For relationship-building pages for
alumni, regularly timed brand/attitude surveys are helpful. I keep
looking for new tactics, so I'd love to hear any suggestions.

Long post short, I don't know if there's a single good answer to the
quantitative measurement question. For some e-commerce sites it may be
relatively clear cut. But in every case, I think it involves a lot of
thoughtful analysis and understanding of customers and context.

Kate Johnson


On Apr 25, 1:37 pm, Suz Bednarz - Kish <bluestokk...@gmail.com> wrote:
> Thanks Milan.
>
> =============
> I've learned that people will forget what you said, people will forget what
> you did, but people will never forget how you made them feel. - Maya
> Angelou
>
> On Wed, Apr 25, 2012 at 3:36 PM, Milan Davidović <milan.li...@gmail.com>wrote:
>
>
>
>
>
>
>
> > On Wed, Apr 25, 2012 at 2:58 PM, Suz Bednarz - Kish
> > <bluestokk...@gmail.com> wrote:
> > > I am tasked with producing some sort of metrics on our copy/content
> > quality.
>
> > This may be of interest, even if not a complete match with what you're
> > supposed to do:
>
> >http://www.slideshare.net/IntelligentContent/beyond-visits-and-page-v...

Rahel Anne Bailie

unread,
Apr 26, 2012, 3:47:45 AM4/26/12
to content...@googlegroups.com
I agree with what Rick said (so interesting to hear one's words boomerang back to you - in a good way) and if I weren't on vacation with a flaky internet connection, I would elaborate. It sounds like you've gotten some good advice, though, so I'm glad for that.

Rahel

Jay Greenspan

unread,
Apr 26, 2012, 10:44:50 AM4/26/12
to content...@googlegroups.com
> Long post short, I don't know if there's a single good answer to the
> quantitative measurement question. For some e-commerce sites it may be
> relatively clear cut. But in every case, I think it involves a lot of
> thoughtful analysis and understanding of customers and context.

Well put, Kate!

I spent a lot of time working on ways to quantify content quality on
an educational site I developed and managed. I asked for tips on this
list and got some great feedback. The site was almost entirely rich
media (video and interactive games) and the users had to log in, so we
had excellent data to work with.

For the videos, I looked at two key metrics:
1) engagement. Most video platforms have graphs that show you how
long users stay with a video. When we saw a steep drop-off, we knew we
were failing our users.
2) "bounce content." likelihood that a particular video would be the
user's last.

I put a lot of effort into tying specific pieces of content to revenue
over time, but that proved to be essentially impossible.

With these metrics, it became pretty clear what our liked, and it had
a big impact on our editorial decisions. We became far less concerned
with the prominence or on-camera abilities of the instructors. We
focused instead on titles and structures that our users wanted.

I found this effort really edifying. We got to know our customers a
whole lot better.

Sadly, the company was indicted and revealed to be a huge financial
fraud before I could really determine the impact of the change we
made. That sucked. But it's still better than working for a company
that charges $500 to fix a typo :)

-j

Jill Stuart

unread,
Apr 27, 2012, 9:57:26 AM4/27/12
to content...@googlegroups.com
Late to this conversation but more thoughts:
I'm totally down with using metrics tied to whatever call to action you're interested in, as described in the 'piece of content directly tied to revenue' description below--but I'm not a huge fan of 'last-click' as an attribution model.

To give just one counterexample: one piece of content might have great performance as a 'last-click' for a certain revenue source, but I'd lay a substantial bet that higher priced purchases are to more intensive engagement with content before that last click, and very likely repeat visits--and that content might never be identified as contributing to the bottom line if you're only using a last-click attribution model.

On Wed, Apr 25, 2012 at 3:23 PM, Rick Yagodich <ri...@excolo.com> wrote:
To post to this group, send email to contentstrategy@googlegroups.com.
To unsubscribe from this group, send email to contentstrategy+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/contentstrategy?hl=en.
--
You received this message because you are subscribed to the Google Groups "Content Strategy" group.
To post to this group, send email to contentstrategy@googlegroups.com.
To unsubscribe from this group, send email to contentstrategy+unsubscribe@googlegroups.com.

Adrian Howard

unread,
Apr 27, 2012, 1:49:33 PM4/27/12
to content...@googlegroups.com
Hi Suz,

On 25 Apr 2012, at 19:58, Suz Bednarz - Kish wrote:

> I am tasked with producing some sort of metrics on our copy/content quality.

Apologies for answering a question with a question - but my instant response when metrics are brought up is "Why?".

Imagine that you already had the perfect metric - what would you do with it?

Is it about culling content? Improving content? Grading authors? Allocating resources? Scheduling content reviews? Something else?

I generally find that once you know the "why" it's much easier to figure out what you should be measuring (or, in many cases, not measuring :-)

Cheers,

Adrian
--
http://quietstars.com adr...@quietstars.com twitter.com/adrianh
t. +44 (0)7752 419080 skype adrianjohnhoward del.icio.us/adrianh



Colleen l Content Science

unread,
Apr 27, 2012, 2:31:05 PM4/27/12
to content...@googlegroups.com
Hi Suz,

Kudos to you for tackling evaluation! For quantitative metrics, definitely check out Marko Hurst's blog about content analytics: http://www.markohurst.com/

I share some of my philosophy on evaluating content (including measurement or metrics) in my book Clout and also outline important measurements and techniques to get them, including surveys, remote tests, analytics, and so on. As far as measuring the value of content to people, bring together the mix of qualitative and quantitative measures that show
  • What do people think about your content, such as
    • What do they say about it on social media?
    • How do they rate it on a survey?
  • What do people do with your content, such as
    • How many people share it?
    • How many people return to it?
Those are simple examples, but you get the idea. Why does what people think about content matter? Because content often is not trying simply to change a behavior (what people do), it's trying to change a perception--or even create a perception that wasn't there before.

I completely agree with Kate. (Wish I could "like" her response here!) I don't see any way around experimenting with what combination of measures will best tell the story about whether your content is meeting your goals. Every situation is a little bit (or a lot bit) different. But, Marko and I are working are setting aside some time each week to crack this nut wider open. That's all I'll say for now to avoid jinxing our headway. ;-)

Evaluating content seems mostly an afterthought or too hard to tackle. I'd love to see evaluation be a priority for content governance. I shared a few thoughts on that here: http://content-science.com/expertise/content-insights/my-goodness-good-governance

Colleen

COLLEEN JONES 
Principal, Content Science
Founder, Atlanta Content Strategy
E: jones...@gmail.com  l  col...@content-science.com
T: @leenjones 
P: 770-296-0121 


--
You received this message because you are subscribed to the Google Groups "Content Strategy" group.
To post to this group, send email to content...@googlegroups.com.
To unsubscribe from this group, send email to contentstrate...@googlegroups.com.

Marko Hurst

unread,
Apr 27, 2012, 3:48:57 PM4/27/12
to Content Strategy
Hey Suz,

Just adding my $.02 here.

You asked, "how do you truly measure value of copy/content to the true
consumers of that content"?

What you are looking for is somewhat complicated in that there isn't
simply a set of "content analytics" that translate value to a consumer
to the business one-to-one. Something I'm trying very hard at solving,
but that's for another day.

From a quantitative perspective do the following:

What you need to be able to do is choose the right metrics that show
content consumers are or are not "enjoying" the content and how those
metrics correlate to increases or decreases in one of the following:
1) Increase revenue
2) Decrease costs
3) Increase satisfaction

Often it's a combination of these three, but by always starting here
it will help you narrow what KPI (key performance indicators) and
metrics are appropriate to measure, i.e. measure what matters. If you
want to get "advanced" you can continue that same idea and determine
the business model, industry, user path, etc. If you're less familiar
with quantitative (web analytics) measures don't worry about them for
now. The more you dig and get into it the more complicated it will
become. Arguably more accurate, but in most cases that's not the case,
so keep it simple. Do keep in mind that anything, I mean absolutely
ANYTHING, can be measured. So keep at it.

That being said, if you are looking for a qualitative way try this:
1) Choose a few characteristics or dimensions of what (you think)
makes content "valuable" or good. Findability, Context, On-brand,
Professionalism, Sharing, etc. I'll refer you to a post from Colleen
Jones (back at ya!) I've used: http://www.uxmatters.com/mt/archives/2009/04/toward-content-quality.php
for more
2) Simply assign a value to it, e.g. 0-3 (my preferred), 0-5, 0-10,
etc.
3) Add up the total score given across all dimensions and divide it
by total possible. e.g 9 of 18 = 50%

Now you have used a qualitative method to quantify content across a
series of dimensions that is scalable, repeatable, and while not
'exact' its consistently inconsistent. Which is why I prefer the 0-3
method scale as it limits the amount of variation. You can then
monitor changes in your content over time and also correlate them to
quantitative metrics for a well balanced "qual - quant mix" of content
measurement.

I'll be posting a more detailed version of this method on my blog,
www.markohurst.com over the next week.

Cheers,
~Marko

Marko Hurst
Author, Speaker, Consultant, & Director of Content Strategy @ HUGE

Content Analytics
e: ma...@markohurst.com
t: markohurst
s: www.markohurst.com

Hilary Marsh

unread,
Apr 27, 2012, 4:04:48 PM4/27/12
to content...@googlegroups.com
Wow, these are great points!

The only thing I'd like to add is that the organization isn't looking at the content itself, but what people think or do as a result of reading that content -- registering for an event, buying a product, participating in a program, etc. So it's program/product analytics, not necessarily content analytics.

That's one of the biggest challenges of content, is that it's what it's about that needs to be measured. Of course, the better the content in terms of meeting user needs, speaking users' language, being compelling, getting published where and when it makes the most sense, etc., the better the content and the ultimate source will perform.

--Hilary

--
You received this message because you are subscribed to the Google Groups "Content Strategy" group.
To post to this group, send email to content...@googlegroups.com.
To unsubscribe from this group, send email to contentstrate...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/contentstrategy?hl=en.

Reply all
Reply to author
Forward
0 new messages