Redis Page View Counter

[Okanis]

Hello,

I want to use Redis to store a view count for each time a article page is loaded. And then I want to show view counts on dashboard.

But I should show:

• Total View Count And Total Uniq View Count of the article
  
• Last 5 minute view count of the article
  
• Today, Yesterday, This day last week, Yesterday week 5 minutes view counts of the article
  

For example: Article ID => 1009854

• Today - 15:05 => 1585 View Count
  
• Yesterday - 15:05 => 1500 View Count
  
• This day last week - 15:05 => 1532 View Count
  
• Yesterday Week - 15:05 => 1545 View Count
  
• Last 5 minutes => 1652
  
• Total View Count = > 52014
  
• Total Uniq View Count => 35045
  

I can't seem to find the right structure because I have not used Redis before

What should I design data structure for that ?

Thank you

Best Regards

[Tuco]

I think a single redis DS won't suffice for your use case, considering you also need the unique users which is normally approximated using the pfcount command in redis.

I am assuming that you need the last 5 minutes at anytime, so that at 12:07 PM, you would need the data from 12:02 - 12:07 PM.

For each article, you could have the following pattern of keys.

1. art:{<articleId>}:<dateinYYMMDD> eg art:{1009854}:161216 of type hash and will TTL set to 7 days(604800 seconds)
2. art:{<articleId>}:unq eg. art:{1009854}:unq of type string and data added using PFADD.
3. art:{<articleId>}:stats eg art:{1009854}:stats of type map, which could store the stats of the article like total count till now.

art:{1009854}:161216 can have keys of the form HHMM and values in type int, meaning the number of visits of that article during that minute. Additionally you can have another field in this key named todayTillNow which will be total article views till now for today.

art:{1009854}:unq can have the data added using PFADD and can be used for calculating the total unique users who have visited the article. Note that it is approximated, and is accurate upto 99%.

art:{1009854}:stats can be used for saving the stats of the article, like the total all time visits of the article. It is kept as a hash so that it is extensible, and can store other info/stats/metadata about the article.

Whenever a user with id user1 visits the article, we have to execute a lua script and pass it the articleId, dateinYYMMDD, timeinHHMM, and userid.

The Lua script will do the following.

1) HINCRBY art:{articleId}:dateinYYMMDD timeinHHMM 1

2) HINCRBY art:{articleId}:dateinYYMMDD todayTillNow 1

3) EXPIRE art:{articleId}:dateinYYMMDD 604800

4) PFADD art:{1009854}:unq user1

5) HINCRBY art:{1009854}:stats totalViews 1

Thereafter, all the data can be obtained from the above keys, ideally through a lua script which can be passed, the articleId, dateInYYMMDD, timeinHHMM.

1) today last 5 minute count => use HMGET on art:{articleId}:dateinYYMMDD to get the data for fields related to appropriate time for today, and add them up.

2) yesterday last 5 minute count => same as above, just pass yesterday

3) this day last week => same as above, just pass last week day

4) yesterday last week => same as above, just pass yesterday last week

5) last 5 minutes today => same as 1, just pass last 5 minutes.

6) totalViewCount => HGET art:{1009854}:stats totalViews

7) total unique view count(approximate)=> PFCOUNT art:{1009854}:unq

The reason that the article id is appended in {} is so that if you have a cluster, all your stats of an article will belong to a single node, and you can use lua script or PFCOUNT commands easily.

Also, setting expiry time of keys with date in them will make sure that for an article, a maximum of 8+1+1=10 keys are present for an article at a particular time.

Tuco

[Simone Mosciatti]

This is one of the use cases for which I have developed rediSQL: https://github.com/RedBeardLab/rediSQL

You may use a table for articles and another one for visits, in the visit table just save the time when the page is visited and what article the user is reading.

Then you can just use standard SQL.

[Okanis]

Hello Tuco,

First of all thank you very much for answer. I tried your solition and it work good for me. I only changed PFADD to SADD for store ip address. Now I can get "Uniq Total View Count", "Total View Count", "Today Till Now View Count" and specific HHMM.

But I don't understand what should I do for last 5 minutes. Can I get last five minutes with one command ? or I should exec that command "hmget art:{1024}:19122016 1525",  "hmget art:{1024}:19122016 1526" and then count on Node.JS app

Time: 15:25 =>

HINCRBY art:{1024}:19122016 1525 1

HINCRBY art:{1024}:19122016 todayTillNow 1

SADD art:{1024}:unq 192.168.1.1

HINCRBY art:{1024}:stats totalViews 1

Time: 15:26 =>

HINCRBY art:{1024}:19122016 1526 1

HINCRBY art:{1024}:19122016 todayTillNow 1

SADD art:{1024}:unq 192.168.1.2

HINCRBY art:{1024}:stats totalViews 1

Get uniq total view count

SCARD art:{1024}:unq

(integer) 2

Get total view count

hget art:{1024}:stats totalViews

"2"

Get till today now view count

hget art:{1024}:19122016 todayTillNow

"2"

Get view count for 15:25

hget art:{1024}:19122016 1525

"1"

Best Regards

[Tuco]

Hello Okanis, 

You can use the below commands.

hmget art:{1024}:19122016 1525 1526 1527 1528 1529

This will give the data for all the HHMM (if they exist, otherwise you can assume 0 for it).

After that you can add in node.js and return.

A better way will be to use lua script to get execute multiple redis commands and do some operations. Lua script is to redis as stored procedure is to a relational database. Depending on your use case, you can pass articleId, Date and HHMM to lua script and it should return all the processed info for that date and time(like last 5 minutes count, total count, last day count, last day 5 min count etc). If you are unaware of it, you should definitely try it, as it will really work for your use case.

Also, you can choose SADD over PFADD, but SADD will increase the size of your database because you will store the user ids of all the users who visited the article. Imagine you have 1 million different user Id browsing thousands of different articles, it can consume a lot of memory.

Each key in which the data is added using PFADD will consume a maximum of 12 kB memory, which is less(but PFADD only approximates the count).

You can choose your command depending on your specific infra and requirements.

Tuco

[Okanis]

Hello Tuco,

You are right. I will choose PFADD. I am new to Redis :))

I research lua script. And I think it is good idea. And I tried a few commands with Lua script for get all article of last 5 minutes. But I don't know my code is good or bad. it is my first lua script. And it can only get all article of hhMM

hello.lua Script

  

local hhMM = KEYS[2]
local allArticleKeys = redis.call("KEYS", KEYS[1])
local table = {}
for i, v in ipairs(allArticleKeys) do
local art = redis.call("HMGET", v, hhMM)

for j, k in ipairs(art) do
table[i] = {v,k}
end

end
return table

Exec Com.

C:\Windows\system32>redis-cli --eval "C:\Program Files\Redis\hello.lua" art:*:19122016 1526

Result

1) 1) "art:{1024}:19122016"

    2) "1"

2) 1) "art:{1025}:19122016"

    2) "1" 

Okanis

[Tuco]

Hi Okanis, 

If it satisfies your use case, it should be fine. Normally redis is very fast, so even without lua, things will be good..

Regards

[Okanis]

Hi Tuco,

You teached me logic of Redis. Thank you so much. 

Regards

[Rahul Babbar]

:)

--
You received this message because you are subscribed to a topic in the Google Groups "Redis DB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/redis-db/Bt3qZKAjVf8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.