[ASIM] Web Session AWS WAF#14496
Conversation
…bsession/awswaf to be up to date before creating a pull request.
|
@microsoft-github-policy-service agree company="BlueVoyant" |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds AWS WAF support to the ASIM WebSession normalization layer by introducing dedicated AWS WAF parsers and wiring them into the top-level WebSession aggregators, with accompanying ARM deployment assets and test artifacts.
Changes:
- Added new AWS WAF WebSession parsers (
ASimWebSessionAWSAWSWAF,vimWebSessionAWSAWSWAF) and associated ARM templates/README/changelogs. - Updated top-level
ASimWebSession/imWebSessionparsers to include the new AWS WAF implementations and bumped versions. - Added sample schema and parser test result CSVs for AWS WAF.
Reviewed changes
Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Sample Data/ASIM/AWSWAF_schema.csv | Adds AWSWAF table schema sample used for validation/testing. |
| Parsers/ASimWebSession/Tests/AWS_AWS WAF_vimWebSession_SchemaTest.csv | Captures schema-test output for the filtering parser. |
| Parsers/ASimWebSession/Tests/AWS_AWS WAF_vimWebSession_DataTest.csv | Captures data-test output for the filtering parser. |
| Parsers/ASimWebSession/Tests/AWS_AWS WAF_ASimWebSession_SchemaTest.csv | Captures schema-test output for the normalization parser. |
| Parsers/ASimWebSession/Tests/AWS_AWS WAF_ASimWebSession_DataTest.csv | Captures data-test output for the normalization parser. |
| Parsers/ASimWebSession/Parsers/vimWebSessionAWSAWSWAF.yaml | New source-specific filtering + normalization parser for AWS WAF. |
| Parsers/ASimWebSession/Parsers/imWebSession.yaml | Wires AWS WAF filtering parser into the top-level imWebSession union and bumps version/date. |
| Parsers/ASimWebSession/Parsers/ASimWebSessionAWSAWSWAF.yaml | New source-specific normalization parser for AWS WAF. |
| Parsers/ASimWebSession/Parsers/ASimWebSession.yaml | Wires AWS WAF normalization parser into the top-level ASimWebSession union and bumps version/date. |
| Parsers/ASimWebSession/CHANGELOG/vimWebSessionAWSAWSWAF.md | New changelog for the AWS WAF filtering parser. |
| Parsers/ASimWebSession/CHANGELOG/imWebSession.md | Records new AWS WAF parser addition for imWebSession. |
| Parsers/ASimWebSession/CHANGELOG/ASimWebSessionAWSAWSWAF.md | New changelog for the AWS WAF normalization parser. |
| Parsers/ASimWebSession/CHANGELOG/ASimWebSession.md | Records new AWS WAF parser addition for ASimWebSession. |
| Parsers/ASimWebSession/ARM/vimWebSessionAWSAWSWAF/vimWebSessionAWSAWSWAF.json | Adds ARM template to deploy the filtering parser function. |
| Parsers/ASimWebSession/ARM/vimWebSessionAWSAWSWAF/README.md | Adds deployment documentation for the filtering parser. |
| Parsers/ASimWebSession/ARM/imWebSession/imWebSession.json | Updates deployed imWebSession function to union in the AWS WAF filtering parser. |
| Parsers/ASimWebSession/ARM/FullDeploymentWebSession.json | Adds linked deployments for the new AWS WAF parsers in the full deployment template. |
| Parsers/ASimWebSession/ARM/ASimWebSessionAWSAWSWAF/README.md | Adds deployment documentation for the normalization parser. |
| Parsers/ASimWebSession/ARM/ASimWebSessionAWSAWSWAF/ASimWebSessionAWSAWSWAF.json | Adds ARM template to deploy the normalization parser function. |
| Parsers/ASimWebSession/ARM/ASimWebSession/ASimWebSession.json | Updates deployed ASimWebSession function to union in the AWS WAF normalization parser. |
| | extend hostIsClean = host matches regex @'^[A-Za-z0-9\.\-\:\[\]_]+$' | ||
| | extend hostIsIp = hostIsClean and (host matches regex @'^\d{1,3}(\.\d{1,3}){3}$' or host contains ':') | ||
| | extend hostHasDomain = hostIsClean and not(hostIsIp) and array_length(split(host, '.')) > 1 | ||
| | extend | ||
| DstIpAddr = iff(isnotnull(parse_ipv4(host)), host, ''), | ||
| DstHostname = case( | ||
| isnotnull(parse_ipv4(host)), host, | ||
| hostIsIp, '', | ||
| hostIsClean, tostring(split(host, '.')[0]), | ||
| '' | ||
| ), | ||
| DstFQDN = iff(hostHasDomain, host, ''), | ||
| DstDomain = iff(hostHasDomain, strcat_array(array_slice(split(host, '.'), 1, -1), '.'), '') |
| | extend hostIsClean = host matches regex @'^[A-Za-z0-9\.\-\:\[\]_]+$' | ||
| | extend hostIsIp = hostIsClean and (host matches regex @'^\d{1,3}(\.\d{1,3}){3}$' or host contains ':') | ||
| | extend hostHasDomain = hostIsClean and not(hostIsIp) and array_length(split(host, '.')) > 1 | ||
| | extend | ||
| DstIpAddr = iff(isnotnull(parse_ipv4(host)), host, ''), | ||
| DstHostname = case( | ||
| isnotnull(parse_ipv4(host)), host, | ||
| hostIsIp, '', | ||
| hostIsClean, tostring(split(host, '.')[0]), | ||
| '' | ||
| ), | ||
| DstFQDN = iff(hostHasDomain, host, ''), | ||
| DstDomain = iff(hostHasDomain, strcat_array(array_slice(split(host, '.'), 1, -1), '.'), '') |
| # AWS WAF ASIM WebSession Normalization Parser | ||
|
|
||
| ARM template for ASIM WebSession schema parser for AWS WAF. | ||
|
|
||
| This ASIM parser supports filtering and normalizing AWS Web Application Firewall (WAF) web session logs from the AWSWAF table to the ASIM Web Session normalized schema. |
| @@ -0,0 +1,35 @@ | |||
| ColumnName,ColumnOrdinal,DataType,ColumnType | |||
|
Hi @manuelhauch, |
| This ASIM parser supports normalizing AWS Web Application Firewall (WAF) web session logs from the AWSWAF table to the ASIM Web Session normalized schema. | ||
| ParserName: ASimWebSessionAWSAWSWAF | ||
| EquivalentBuiltInParser: _ASim_WebSession_AWSAWSWAF | ||
| Exceptions: |
There was a problem hiding this comment.
Can remove this Exceptions block
| Link: https://docs.aws.amazon.com/waf/latest/developerguide/logging-fields.html | ||
| Description: | | ||
| This ASIM parser supports normalizing AWS Web Application Firewall (WAF) web session logs from the AWSWAF table to the ASIM Web Session normalized schema. | ||
| ParserName: ASimWebSessionAWSAWSWAF |
There was a problem hiding this comment.
Having AWSAWSWAF seems redundant to have AWS called twice. In EventProduct, we can have it as just WAF instead of AWS WAF
| HttpStatusCode, | ||
| DstHostname, | ||
| SrcIpAddr, | ||
| ASimMatchingIpAddr = "SrcIpAddr", |
There was a problem hiding this comment.
Can remove ASimMatchingIpAddr and ASimMAtchingHostname from this parser as it's not used in parameter-less parser
| Src = tostring(parsedRequest.clientIp), | ||
| SrcGeoCountry = tostring(parsedRequest.country) | ||
| | where (array_length(srcipaddr_has_any_prefix) == 0 or has_any_ipv4_prefix(SrcIpAddr, srcipaddr_has_any_prefix)) | ||
| | where (array_length(ipaddr_has_any_prefix) == 0 or has_any_ipv4_prefix(SrcIpAddr, ipaddr_has_any_prefix)) |
There was a problem hiding this comment.
Can move lines 188-192 to closer to the start of the parser so we do filtering early
| HttpStatusCode, | ||
| DstHostname, | ||
| SrcIpAddr, | ||
| ASimMatchingIpAddr = "SrcIpAddr", |
There was a problem hiding this comment.
Line 245+246 will be filled with SrcIpAddr or DstHostname if filtering was done. You can check other parsers for an example of how this is populated
|
Failing kql validation because the table is not available during kql checks. Please add it to script/tests/kqlvalidationtests/customTables |
Change(s):
Added WebSession ASIM parser for AWS WAF logs in the AWSWAF table.
Reason for Change(s):
Support AWS WAF logs in WebSession ASIM.
Version Updated:
Yes
Top level Web Session ASIM parsers updated to 0.5.7 (for ASimWebSession) and 0.6.5 (for imWebSession)
Testing Completed:
Yes
Checked that the validations are passing and have addressed any issues that are present:
Yes