Dongyu Shen
unread,Sep 5, 2023, 9:34:43 PM9/5/23Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to lokiproject
I am using regular expressions to process my logs.
My promtail config is:
```yaml
- job_name: kubernetes-pods
pipeline_stages:
- docker: {}
- regex:
expression: '^(?P<timestamp>[0-9 :.-]+?) \| \\u001b\[0;00;[0-9]+m(?P<logLevel>[DIWEF])\\u001b\[0m \| (?P<traceID>.*) \| (?P<location>[^\s]+:\d+) \\u001b\[0;00;[0-9]+m(?P<message>.+)\\u001b\[0m.*$'
- labels:
timestamp: timestamp
logLevel: logLevel
traceID: traceID
location: location
message: message
```
My log is like:
```
09-04 18:22:07.415 | \u001b[0;00;32mI\u001b[0m | | handler.go:383 \u001b[0;00;32mget all robot stop plan info\u001b[0m\n
```
I got the error message from promtail debug log:
```
level=debug ts=2023-09-06T01:27:27.313448004Z caller=regex.go:121 component=file_pipeline component=stage type=regex msg="regex did not match" input="09-04 18:22:07.415 | \u001b[0;00;32mI\u001b[0m | | handler.go:383 \u001b[0;00;32mget all robot stop plan info\u001b[0m\n" regex="^(?P<timestamp>[0-9 :.-]+?) \\| \\\\u001b\\[0;00;[0-9]+m(?P<logLevel>[DIWEF])\\\\u001b\\[0m \\| (?P<traceID>.*) \\| (?P<location>[^\\s]+:\\d+) \\\\u001b\\[0;00;[0-9]+m(?P<message>.+)\\\\u001b\\[0m.*$"
```
But I tested the regex in Go and it was correct.
```go
package main
import (
"fmt"
"regexp"
)
func main() {
logLine := `09-04 18:22:07.415 | \u001b[0;00;32mI\u001b[0m | | handler.go:383 \u001b[0;00;32mget all robot stop plan info\u001b[0m\n`
re := regexp.MustCompile(`^(?P<timestamp>[0-9 :.-]+?) \| \\u001b\[0;00;[0-9]+m(?P<logLevel>[DIWEF])\\u001b\[0m \| (?P<traceID>.*) \| (?P<location>[^\s]+:\d+) \\u001b\[0;00;[0-9]+m(?P<message>.+)\\u001b\[0m.*$`)
match := re.FindStringSubmatch(logLine)
if match != nil {
groups := make(map[string]string)
for i, name := range re.SubexpNames() {
if name != "" {
groups[name] = match[i]
}
}
fmt.Printf("Timestamp: %s\n", groups["timestamp"])
fmt.Printf("Log Level: %s\n", groups["logLevel"])
fmt.Printf("Trace ID: %s\n", groups["traceID"])
fmt.Printf("Location: %s\n", groups["location"])
fmt.Printf("Message: %s\n", groups["message"])
} else {
fmt.Println("Log line does not match the expected format.")
}
}
```
output:
```
Timestamp: 09-04 18:22:07.415
Log Level: I
Trace ID:
Location: handler.go:383
Message: get all robot stop plan info
```
Who can tell me where the problem is? THX!