Hi folks,

Need your advice and best practice ideas.

I scrub my web hit log everyday to look for keywords. The good news is
that this is a highly effective method of coming up with new keywords
and triangulating the viewer's habits and intention. The bad news is
that my daily log is getting pretty sizable whereas I used to spend 10
minutes a day to go through the log, it now can take 1 hours depending
on the length.

As the saying goes, this process of eyeballing the daily log is not
particularly "scalable" and I don't have 1 hour everyday to scrub the
log.

My question is how do you do it? My strategy right now is to sample of
the log so that I would look at it every few days. But, to the extent
that I seem to find something "interesting" in every log, I do wonder
how much information I am missing out. Equally, I wonder if there are
other methods of combing the data that can take less human time?

Look forward to your thoughts.

Warm regards,  YFJ

I wish had helpful input but in searching for new phrases and for
possible negative keywords I do pretty much the same thing you
described.

Anyone else got a better way?

Um, this is what computers are for, so you don't have to do all this
manual labor.  ;-)
Seriously, though, use keyword tools on the front end for keyword
research and web analytics tools on the back end.  If you don't have
any in place, start with the free tools on my site.
______________
work: http://apogee-web-consulting.com/adwords.html
play: http://apogee-web-consulting.com/tagman/

Not to take anything away from the value of the keyword tools
available, but at some point, which I have hit, the aggregated data
from these tools are not yielding additional insight. So, I have set up
a number of scripts to scrub the daily log. What is particularly useful
is to a keyword comparison using the actual keywords and the google
{keyword} tag.

However, since I have a number of SEO'd pages that are attracting
google traffics that does not come with the {keyword} tag so, I have
not figure out how to "process" these hits without eyeballs.

The other problem that I have not had a good solution for is for the
terms that are not derived from one of the existing keywords. They are
useful for getting additional insights on related markets and
applications. This is not something that a script can deal with and I
am still stuck with reading the daily log.

Anyway, just a few more things that I am wondering if people have a
solution for.

Regards,  YFJ