light-weight analytics

8 views
Skip to first unread message

George Siemens

unread,
Sep 2, 2010, 12:30:47 PM9/2/10
to learning...@googlegroups.com
Hi,

I posted a few thoughts on SNAPP as a light-weight learning analytics tool: http://www.elearnspace.org/blog/2010/09/02/light-weight-learning-analytics-tools/ .

What are you using for tools for simple analysis? What is the balance between educator-controlled simple tools and more comprehensive system-level tools?

George

Viplav Baxi

unread,
Sep 4, 2010, 4:37:21 AM9/4/10
to Learning Analytics
Hi George,

SNAPP looks like an interesting tool. My experience in analytics,
apart from the BI stuff, was mainly around my work architecting an
online platform for K12 learner back in the year 2000. We had tracking
codes built into pages, discussion forums, questions, activities etc.
that used to get collated at different levels - student, subject, time
period etc. This provided invaluable information around usage and
patterns and informed both our customer support and sales teams when
they were forming their strategies for go-to-market. Power laws were
evident then as well. But this was a closely coupled system,
obviously, unlike some of the network analysis tools that we see today
that can loosely integrate with multiple platforms.

The thing I have learnt about such analytics is that they indicate
quantity rather than quality, which really is a subjective judgement
and varies from person to person. They can inform an educator about
activity levels, level of cohesion and networking etc. which are
useful pieces of information. To indicate quality, the underlying
activity must be constructed in a way that can lead to quantitative
analysis.

For example, if trust is a strong measure of relationships, can we
somehow measure it? Or if there is a really good debate happening, can
we measure the quality of that debate versus another in which
participants are equally active. How much of that value judgement
should the educator weigh in and how much should the students weigh in
could be another area of research or judgement.

Regards,
Viplav

On Sep 2, 9:30 pm, George Siemens <gsiem...@gmail.com> wrote:
> Hi,
>
> I posted a few thoughts on SNAPP as a light-weight learning analytics tool:http://www.elearnspace.org/blog/2010/09/02/light-weight-learning-anal....

George Siemens

unread,
Sep 4, 2010, 10:00:42 AM9/4/10
to learning...@googlegroups.com
"The thing I have learnt about such analytics is that they indicate
quantity rather than quality, which really is a subjective judgement
and varies from person to person. They can inform an educator about
activity levels, level of cohesion and networking etc. which are
useful pieces of information. To indicate quality, the underlying
activity must be constructed in a way that can lead to quantitative
analysis."

Absolutely! In fairness, we are still in the early stages of analytics in learning, so it makes sense that the first stage of analytics consists of low-hanging fruit such as number of logins, pages viewed, time on task, etc. To be effective for interventions and decision making, analytics have to move beyond this.

Paul Lazarsfeld statement (from 1940s!) can serve as a target for learning analytics - at least in terms of the social dimensions of learning: "Who talks to whom about what, and with what effect". Currently, LA is focused on "who talks to whom"...as LA progresses we'll get to the "about what" and "with what effect". This of course doesn't get to interactions with content, but I think it serves as a helpful guide in learning analytics for the near future...

George


Aneesha Bakharia

unread,
Sep 4, 2010, 9:14:59 PM9/4/10
to learning...@googlegroups.com
Hi

Dr Shane Dawson and myself are the creators of the SNAPP tool. I am
quite pleased to hear it described as a light-weight.

My masters research (soon to be converted to a phd) focuses on finding
themes within student generated content and will result in tools that
integrate with the LMS in a similar manner to SNAPP. I tend to hear
that SNA does not provide an indication of 'quality' a lot and am now
looking for evidence of this in literature. Please pass on any
references.

Regards

Aneesha

Viplav Baxi

unread,
Sep 4, 2010, 11:19:50 PM9/4/10
to Learning Analytics
Congratulations, Aneesha on building a great tool!

I came across an article recently which might interest you titled
"Measuring Behavioral Trust in Social Networks" (http://www.cs.rpi.edu/
research/pdf/10-03.pdf). The authors state:

"We will focus on two particular behavior patterns as an expression of
trust: conversation and propagation. Specifically, if two nodes
converse, then they are more likely to trust each
other; and a prolong conversation reinforces this conclusion. If one
node propagates information from another then it suggests that the
propagator trusts the information. Similarly, a repeated propagation
makes the conclusion stronger."

and,


"The measure of conversational trust will be based on the
conversations in C, obeying the following postulates:
• Longer conversations imply more trust.
• More conversations imply more trust.
• Balanced participation by A and B implies more trust.

Note that one could add other requirements, for example, if people who
did trust each other stop keeping in touch, their trust will likely
deteriorate over time - i.e. more spaced apart conversations implies
less trust. However, the above three properties are a good starting
point."

This is just the only type of value judgement that SNA can make -
based on quantitative data that can be captured about activities in
the network. It's akin to treating each conversation as an atomic
transaction, indifferent to it's actual properties.

Or look at HP LABs' recent research around Influence and Passivity in
online Social Media (http://www.scribd.com/doc/35401457/Influence-and-
Passivity-in-Social-Media-HP-Labs-Research) in which they looked at
measures like retweeting rates to study popularity vs. influence.

Of course, Valdis Krebs has a wealth of information and analysis on
SNA at http://www.orgnet.com/sna.html.

To understand if these measures could be an indication of quality
would be to ask - have we captured enough sources of data to determine
trust or influence or are we happy doing trending and graph analysis?,
do our data sources accurately define the subject of analytics (is
influence = number of times my followers retweet me? or trust = longer
conversations?), does our model, whatever that is, give us sufficient
basis to make predictions (my followers will on average retweet any of
my posts irrespective...), and does our analysis capture the longer
term changes in user behavior rather than analyzing static snapshots?

As George says, these could be a good starting point. I agree with
that. But at the same time I want to focus on building tools that act
as sources of data for qualitative analysis. Apologise I don't know of
any research that took a hypothesis from SNA and proved it to be wrong
on the qualitative dimension, but I am not sure if I want to search
for one in the light of the questions above,

Regards,
Viplav


On Sep 5, 6:14 am, Aneesha Bakharia <aneesha.bakha...@gmail.com>
wrote:
> > George- Hide quoted text -
>
> - Show quoted text -

George Siemens

unread,
Sep 5, 2010, 12:20:06 AM9/5/10
to learning...@googlegroups.com
Hi Aneesha - I believe I met you while I was visiting Brisbane in December...congrats on SNAPP. 

As Viplav notes, we can infer that length and frequency of connections result in (are the result of) closer ties or greater exchange of information. Similarly, dense networks influence behaviour, information sharing, life choices, and even health (Christakis & Fowler in Connected...or http://www.scientificamerican.com/podcast/episode.cfm?id=behavior-influenced-more-in-denser-10-09-03). Additionally, you're likely familiar with the work of sociologists like Wellman and Watts...and mathematicians like Barabasi and Strogatz. The work of these individuals has provided us with a good sense of the patterns that networks exhibit (small world, power laws), the attributes of connections (in early 70's, Granovetter explored connections from a perspective of tie strength and impact on new information), and the social dimensions of networks (for example, geography still matters in a digital networked world). SNA is concerned with interaction and communication, so it only makes sense that educators have a keen interest in the topic, especially with the development of tools like facebook/twitter that make those connections explicit.

However, SNA is only one way to view a situation or sequences of interaction...and, can be limited when an individual/teacher needs to decide to act. The fact that a connection exists says little about the nature of that exchange. This struck me several years ago when I was at a conference where everyone had submitted interests/connections prior to the event. Once the conference started, they had huge posters on the wall revealing how conference attendees were connected to each other. People were excited to see their position in a network...but then...nothing. What was the actionable value of seeing positioning in the social network? I found it disappointing to see that the SNA of conference attendees wasn't better tied to some kind of action. As Viplav states, the longer term actions beyond a connection snapshot are important. 

In a tool like SNAPP, knowing which learners are talking to which learners is helpful on one level - outliers may be encouraged to join in conversations, faculty can follow up with additional out-of-course contact, and so on. SNA will be more helpful, however, once we a) get a better sense of the content/information exchange occurring in an interaction, b) the long term impact on learning and conceptual knowledge development based on an interaction, and c) tie analysis to personalization and adaptation of content. Two learners in the same course *should* receive different content based on their competence. 

 George

Leigh Blackall

unread,
Sep 5, 2010, 7:02:56 PM9/5/10
to learning...@googlegroups.com
George, I'm trying to capture some of the information being shared here - to the Wikipedia entry. You have made several mentions of historical context, but I'm struggling to draw it out...

sociologists like Wellman and Watts...and mathematicians like Barabasi and Strogatz. The work of these individuals has provided us with a good sense of the patterns that networks exhibit (small world, power laws), the attributes of connections (in early 70's, Granovetter explored connections from a perspective of tie strength and impact on new information), and the social dimensions of networks (for example, geography still matters in a digital networked world).

You also mentioned something a few days ago.. something from the 40s?

Could you spare a moment and help construct sentences around these references? I'm placing it in the history section of the Wikipedia entry?

Thanks.
Reply all
Reply to author
Forward
0 new messages