Chang Lee
Rob (@rharrigan on Slack) is the one man data team at 247sports, a CBS Sports affiliate, and one of his job is web analytics for the site. I am also working on clickstream data now, so I hit him with questions to know how people practice web analytics outside of my team.
For example, we talked about how they capture data and track users from different sources like Facebook and Google, and what do they do with the data.
One of my problems was how to remove bot-generated traffic from web data. He told me that I can tell from the user agent. (I'll need some clarification later, as the data I'm working with doesn't have a meaningful field for the user agent.) Another was how to identify unique views/users. He said that the industry is pushing toward counting the number of sessions instead of counting views, and building user profile from data other than cookies and GUID is still difficult.
They process data with Splunk and store on AWS. I don't have any experience with AWS, but he also gave me some cool articles, like how to capture page scrolls with Amazon Kinesis:
We chatted for only about 30 minutes on Hangout, but I learned a lot about how people do web analytics -- something I never learned in school! Thanks Rob!
Chang