reg: Building Django REST API to show number of matching pattern from log file

18 views

Skip to first unread message

Amitesh Sahay

unread,

Jan 13, 2020, 5:59:19 AM1/13/20

to Django Users

Hello Users,

I have a scenario for which I need to develop a Web API through Django RESTFrameWork.

1) We have a multiple linux machines on which a log file is generated as soon as there is some change in hash value of a license file . The log file generation and the license file monitoring has been automated through Python script on the servers itself. So far so good.

2) Now, I need develop an API that will read the matching pattern from the log file, and push it to the DB.

3) I also need to develop a dashboard where we can pull data on Daily, Weekly, monthly, and Yearly basis and show graph.

So far, I could do some testing with the Python RegEx to catch the matching pattern and its fine. But the problem is how to use DRF to pull those matching pattern directly from the log file to the DB each time the patter is generated in real time? I have been brainstorming for 3 days now, but reached nowhere.

Note:

1) There is a log rotation happening I believe every 24 hours, and the log files are kept only for seven days. Prior to that the logs get deleted.

I am not sure even if its possible, or what would be my approach.Please pour in with your suggestions.

Regards,

Amitesh

Derek

unread,

Jan 14, 2020, 2:21:58 AM1/14/20

to Django users

Some ideas (untested of course)

"I need develop an API that will read the matching pattern from the log file, and push it to the DB." <== this is NOT an API - but have look at how a different script could access your log:

https://stackoverflow.com/questions/12523044/how-can-i-tail-a-log-file-in-python

Alternately:

"The log file generation and the license file monitoring has been automated through Python script on the servers itself."

Why not simply extend those scripts to also write selected outputs to your database?

For the dashboard side, there are many existing tools that will allow you to create this; just one example is Grafana - https://grafana.com/ (Bonus: you can also use this tool to do the metrics on the server itself.)