Akamai SIEM
Akamai Security Information and Event Management (SIEM) provides a feed of security events from the Akamai platform. This guide explains how to configure Hydrolix to consume security events from Akamai's SIEM API using the Hydrolix SIEM Connector.
Prerequisites
Before you can configure the Hydrolix SIEM Connector, you'll need to configure SIEM integration. Refer to the Akamai SIEM Documentation for more information.
Next, you'll need to configure access to the Akamai SIEM endpoints from your Hydrolix cluster. Ensure that your cluster allows outbound traffic to the following domains:
*.cloudsecurity.akamaiapis.net*.luna.akamaiapis.net
Configure
The SIEM data source supports the following configuration options:
- table: the name of the table you would like to ingest SIEM data into, formatted as
projectname.tablename - name: a unique name for the data source, only used to distinguish data sources internally
- transform: a set of rules to translates ingested SIEM JSON data into rows in a table within your Hydrolix cluster
- checkpointer: the datastore table that will keep track of which SIEM data Hydrolix has already ingested by storing an offset. Default value is
zkfor the internal zookeeper cluster, but it also supports DynamoDB specified in Amazon Resource Name format and Google Cloud Storage in the formathttps://datastore.googleapis.com/{table}. The following are examples of each format:- Internal Zookeeper instance:
zk - AWS DynamoDB:
arn:aws:dynamodb:us-east-2:184958025203:table/checkpoint_table_name - Google Cloud Storage: :
https://datastore.googleapis.com/checkpoint_table_name
- Internal Zookeeper instance:
- host: URL of the Akamai SIEM API, determined when you provisioned the SIEM API
- entity ID: a unique identifier generated when you provisioned the SIEM API, also known as the Web Security Configuration ID
- client secret: a client-specific secret generated when you configured Hydrolix access in the SIEM API
- client token: a client-specific token generated when you configured Hydrolix access in the SIEM API
- access token: a token generated when you provisioned the SIEM API
- SIEM data sources also support the following scale options to fine tune your Kubernetes configuration:
- replica min: the minimum number of SIEM peers running at any given time
- replica max: the maximum number of SIEM peers running at any given times
- CPU: the amount of CPU allocated for SIEM peers in Kubernetes
- memory: the amount of memory allocated for SIEM peers in Kubernetes, measured in gigabytes (GB)
- ephemeral storage: the amount of ephemeral storage capacity allocated for SIEM peers in Kubernetes, measured in gigabytes (GB)
API
Use the following data source configuration structure to create a new Akamai SIEM data source:
{
"name": "<example name>",
"transform": "<example transform>",
"table": "<example table>",
"pool_name": "siem12-pool",
"k8s_deployment": {
"replicas": 1,
"service": "akamai-siem-peer"
},
"settings": {
"access_details": {
"host": "<example.luna.akamaiapis.net>",
"entity_id": "<example id>",
"client_secret": "<example secret>",
"client_token": "<example client token>",
"access_token": "<example access token>"
},
"checkpointer": {
"name": "<example arn>"
}
}
}
For more information about the values you'll provide for the configuration, see the definitions above.
UI
- Navigate to the Data page in the Hydrolix portal.
- Select the Table Source option from the list.
- The UI will display a collection of configuration options. Enter details for your configuration.
- Click the Add source button to add the data source configuration to your cluster.
For more information about the values you'll provide for the configuration, see the definitions above.
Transform
SIEM provides data in JSON format. Each line is a separate JSON object that represents a single request.
We recommend the following configuration to ingest SIEM data:
-
Add the following custom SQL Data Functions:
akamai_siem_extract:
string -> arrayPopBack(arrayMap ( x -> base64Decode (x), splitByChar (';', decodeURLComponent (assumeNotNull (string)))) )akamai_extract_key_pair:
(string, pair_delimiters, key_value_pair_delimiter) -> CAST( ( arrayMap(y -> (y[1]), arrayMap(v -> splitByChar(key_value_pair_delimiter, v), flatten(extractAllGroups(assumeNotNull(string), concat('([^', pair_delimiters, ']+)')))) ), arrayMap(y -> (y[2]), arrayMap(v -> splitByChar(key_value_pair_delimiter, v), flatten(extractAllGroups(assumeNotNull(string), concat('([^', pair_delimiters, ']+)'))))) ), 'Map(String, String)' ) -
Use the following Transform SQL to preprocess incoming SIEM data:
SELECT
NULL as rulesStr,
NULL as ruleDataStr,
NULL as ruleTagsStr,
NULL as ruleActionsStr,
NULL as ruleMessagesStr,
NULL as ruleVersionsStr,
NULL as ruleSelectorsStr,
NULL as responseHeadersStr,
NULL as requestHeadersStr,
EXCEPT
(
rulesStr,
ruleDataStr,
ruleTagsStr,
ruleActionsStr,
ruleMessagesStr,
ruleVersionsStr,
ruleSelectorsStr
)
FROM(
SELECT
clientIP as clientIP,
configId as configId,
policyId as policyId,
geoASN as geoASN,
geoCity as geoCity,
geoContinent as geoContinent,
geoCountry as geoCountry,
geoRegionCode as geoRegionCode,
rulesStr as rulesStr,
ruleDataStr as ruleDataStr,
ruleTagsStr as ruleTagsStr,
ruleActionsStr as ruleActionsStr,
ruleMessagesStr as ruleMessagesStr,
ruleVersionsStr as ruleVersionsStr,
ruleSelectorsStr as ruleSelectorsStr,
bytes as bytes,
host as host,
method as method,
path as path,
port as port,
protocol as protocol,
decodeURLComponent(requestHeadersStr) as requestHeadersStr,
requestId as requestId,
decodeURLComponent(responseHeadersStr) as responseHeadersStr,
start as start,
status as status,
tls as tls,
akamai_siem_extract(rulesStr) as rules,
akamai_siem_extract(ruleDataStr) as ruleData,
akamai_siem_extract(ruleTagsStr) as ruleTags,
akamai_siem_extract(ruleActionsStr) as ruleActions,
akamai_siem_extract(ruleMessagesStr) as ruleMessages,
akamai_siem_extract(ruleVersionsStr) as ruleVersions,
akamai_siem_extract(ruleSelectorsStr) as ruleSelectors,
akamai_extract_key_pair(requestHeadersStr,'\\r\\n', ':') as requestHeaders,
akamai_extract_key_pair(responseHeadersStr,'\\r\\n', ':') as responseHeaders
FROM { STREAM }
)
- Use the following transform output columns to ingest the preprocessed SIEM data into a Hydrolix table:
"output_columns": [
{
"name": "file",
"datatype": {
"type": "string",
"index": true,
"source": null,
"ignore": true,
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "level",
"datatype": {
"type": "string",
"index": true,
"source": null,
"ignore": true,
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "message",
"datatype": {
"type": "string",
"index": true,
"source": null,
"ignore": true,
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "timestamp",
"datatype": {
"type": "string",
"index": true,
"source": null,
"ignore": true,
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "clientIP",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/clientIP"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "configId",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/configId"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "policyId",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/policyId"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "rules",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleData",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleTags",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleActions",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleMessages",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleVersions",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "ruleSelectors",
"datatype": {
"type": "array",
"index": false,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
}
]
}
},
{
"name": "geoASN",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/geo/asn"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "geoCity",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/geo/city"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "geoContinent",
"datatype": {
"type": "string",
"index": false,
"source": {
"from_json_pointers": [
"/data/geo/continent"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "geoCountry",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/geo/country"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "geoRegionCode",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/geo/regionCode"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "bytes",
"datatype": {
"type": "int64",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/bytes"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "host",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/host"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "method",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/method"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "path",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/path"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "port",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/port"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "protocol",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/protocol"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "requestHeaders",
"datatype": {
"type": "map",
"index": true,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
},
{
"type": "string",
"index": true
}
]
}
},
{
"name": "requestId",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/requestId"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "responseHeaders",
"datatype": {
"type": "map",
"index": true,
"source": {
"from_input_field": "sql_transform"
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null,
"elements": [
{
"type": "string",
"index": true
},
{
"type": "string",
"index": true
}
]
}
},
{
"name": "start",
"datatype": {
"type": "epoch",
"index": false,
"primary": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/start"
]
},
"format": "s",
"resolution": "milliseconds",
"default": null,
"script": null
}
},
{
"name": "status",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/status"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "tls",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/tls"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "rulesStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/rules"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleDataStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleData"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleTagsStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleTags"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleActionsStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleActions"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleMessagesStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleMessages"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleVersionsStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleVersions"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "ruleSelectorsStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/attackData/ruleSelectors"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "requestHeadersStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/requestHeaders"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
},
{
"name": "responseHeadersStr",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_json_pointers": [
"/data/httpMessage/responseHeaders"
]
},
"format": null,
"resolution": "seconds",
"default": null,
"script": null
}
}
],
Updated about 2 months ago