Storing log events data

[kiran]

Hello

I am trying to push log events for lambda functions in Prometheus.

I am trying to see if we can even save this kind of data and if so any recommended structure in Prometheus. E.g whenever a lambda function is invoked, AWS puts log events data and each run will have multiple events and associated data for each event. Any suggestions on how to design the data? Is a event considered a metric in this case. 

Here is a generic structure of the data from AWS where you see ‘LogEvents’ is an array of events. 

https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchlogs.html

[Stuart Clark]

Prometheus is a metrics system and not an event store. As a result it
isn't suitable for recording events such as logs. Event stores include
applications such as Elasticsearch, Cloudwatch Logs or various databases.

It is possible to create metrics out of events. For example you might
create a metric of the number of events. You wouldn't be able to
discover any details about what those evens were, but you would be able
to produce a graph over time of the events occurring, or alert if too
many or too few events happened.

[Christian Hoffmann]

Hi,

On 11/23/20 10:41 PM, kiran wrote:> I am trying to push log events for

https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchlogs.html <https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchlogs.html>
This FAQ entry might be relevant:
https://prometheus.io/docs/introduction/faq/#how-to-feed-logs-into-prometheus

In short, if you want to extract metrics from logs, there are tools to
do that. If you want to store plain logs, use something which aims to
support that such as ELK or Loki. :)

Kind regards,
Christian