Logstash Request Parser

The request_parser plugin parses the request field from the access log or F5 ASM or LTM log. It extracts the path, query and parameters and decodes it. With this deeper parsing, it can be analyzed further, and is easy to read for humans.

It is fully free and fully open source. The license is Apache 2.0, meaning you are free to use it however you want.

This is a Request Parser plugin for Logstash.

Sample

input { stdin { } }

# '1.1.1.1 - - [09/Jul/2019:11:41:32 +0200] "GET /api/v4/projects/4/merge_requests?page=1&per_page=100&state=opened HTTP/2.0" 304 0 "" "Mozilla/5.0"'

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

filter {
  request_parser {
    request => "request"
    target_path => "url.path"
    target_query => "url.query"
    target_query_parameters => "url.parameters"
    parse_query_parameters => true # requires ES mapping as non-indexed object
  }
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => json }
}

#  {
#    "request": "/api/v4/projects/4/merge_requests?page=1&per_page=100&state=opened",
#    "url.path": "/api/v4/projects/4/merge_requests",
#    "url.query": "page=1&per_page=100&state=opened",
#    "prameters": {
#      "page": "1",
#      "per_page": "100",
#      "state": "opened"
#    }
#  }

Options

Setting	Input type	Required	Default
request	string	No	request
separate_query_field	boolean	No	false
query	string	No	query
target_path	string	No	path
target_query	string	No	query
target_query_parameters	string	No	parameters
parse_query_parameters	boolean	No	true