Network Addresses
Functions in this sections deal with parsing and operating on IP addresses, URLs, and MAC addresses.
IP Addresses
cutIPv6
cutIPv6
zeros out a number of bytes from an IPv6 formatted address.
Addresses that were originally IPv6 or IPv4 are treated differently, even though the representation is IPv6. For example, zeroing two bytes from the following IPv6 formatted addresses would have different results:
IPv6 formatted | result |
---|---|
2020:5588:89ac:bb88:cafe:babe:cab1:72ba | 2020:5588:89ac:bb88:cafe:babe:cab1:7200 |
::ffff:81.82.83.84 | ::ffff:81.82.0.0 |
Parameters:
ipv6_address
- IPv6 formatted address.ipv6_to_cut
- the number of IPv6 bytes to zero. Accepts values 0-16.ipv4_to_cut
- the number IPv4, formatted to IPv6, bytes to zero. Accepts values between 0-16, but 6-16 will all result in an empty address:::
.
Usage:
cutIPv6(ipv6_address, ipv6_to_cut, ipv4_to_cut)
Returns:
- a
String
with the appropriate characters zeroed.
IPv4CIDRToRange
IPv4CIDRToRange
returns the lower and upper values of IP addresses given an IPv4 value and a CIDR.
Parameters:
ipv4
- the base address in IPv4 formatcidr
- a UInt representing the CIDR
Usage:
IPv4CIDRToRange(ipv4, cidr)
Returns:
- an array of two Strings, representing the lower and upper IPv4 values of the range. For example:
["8.8.8.0","8.8.8.255"]
IPv4NumToString
IPv4NumToString
returns an IPv4 formatted String from a Big Endian UInt representation of an IP address.
Parameters:
ipv4_num
- the UInt to translate
Usage:
- function call:
IPv4NumToString(ipv4_num)
Returns:
- a
String
representation of the dot notation of the IPv4 address.
IPv4NumToStringClassC
IPv4NumToStringClassC
returns an IPv4 formatted String from a Big Endian UInt representation of an IP address, obfuscating the last octect with xxx
Parameters:
ipv4_num
- the UInt to translate
Usage:
IPv4NumToStringClassC(ipv4_num)
Returns:
- a
String
representation of the dot notation of the IPv4 address, with xxx as the last octect. For example:"7.57.254.xxx"
IPv4StringToNum, toIPv4
IPv4StringToNum
and toIPv4
return an IPv4 data type representation of a IPv4 address in String
format. This is useful when a function needs an IPv4 data type rather than a String
representation.
Parameters:
ipv4_string
- theString
to translate
Usage:
IPv4StringToNum(ipv4_string)
toIPv4(ipv4_string)
Returns:
- an IPv4 representation for use in IPv4 functions.
IPv4ToIPv6
IPv4ToIPv6
translates an IPv4 address data type to an IPv6 address data type.
Parameters:
ipv4_num
- theUint64
to translate
Usage:
IPv4ToIPv6(ipv4_num)
Returns:
String
IPv6 representation for use in IPv6 functions.
IPv6CIDRToRange
IPv6CIDRToRange
returns the lower and upper values of IP addresses given an IPv6 value and a CIDR.
Parameters:
ipv6
- the base address in IPv6 formatcidr
- a UInt representing the CIDR
Usage:
IPv6CIDRToRange(ipv6, cidr)
Returns:
- an array of two Strings, representing the lower and upper IPv6 values of the range. For example:
["2020:5500::","2020:55ff:ffff:ffff:ffff:ffff:ffff:ffff"]
IPv6NumToString
IPv6NumToString
returns an IPv6 formatted String from a Big Endian UInt representation of an IP address.
Parameters:
ipv6_num
- the UInt to translate
Usage:
IPv6NumToString(ipv6_num)
Returns:
- a
String
representation of the IPv6 address.
IPv6StringToNum, toIPv6
IPv6StringToNum
and toIPv6
return an IPv6 data type representation of a IPv6 address in String
format. This is useful when a function needs an IPv6 data type rather than a String
representation.
Parameters:
ipv6_string
- theString
to translate
Usage:
IPv6StringToNum(ipv6_string)
toIPv6(ipv6_string)
Returns:
IPv6
representation for use in IPv6 functions.
MAC Addresses
MACNumToString
MACNumToString
returns a String
that is the representation of the MAC address given as a UInt64
in Big Endian format.
Parameters:
mac_num
- theUInt
to translate
Usage:
MACNumToString(mac_num)
Returns:
String
representation of the MAC address.
MACStringToNum
MACStringToNum
returns a UInt64
that is the representation of the MAC address given as a String
.
Parameters:
- mac_string - the
String
to translate
Usage:
MACStringToNum(mac_string)
Returns:
UInt64
representation of the MAC address.
MACStringToOUI
MACStringToOUI
returns a UInt64
that is the OUI for the MAC address passed in as a String
. The OUI is the first 3 octets of the MAC address.
Parameters:
mac_string
- theString
to extract the OUI from
Usage:
MACStringToOUI(mac_string)
Returns:
- a
UInt64
representation of the OUI.
URLs
cutFragment
cutFragment
returns a String
that removes any fragments referring to anchors (like myurl#theAnchor
), returning the URL portion only
Parameters:
url
- theString
representing the full URL
Usage:
cutFragment(url)
Returns:
String
that is the URL without any anchor references
cutQueryString
cutQueryString
returns a String
that removes any query strings, returning the URL portion only
Parameters:
url
- theString
representing the full URL
Usage:
cutQueryString(url)
Returns:
String
that is the URL without any query string
cutQueryStringAndFragment
cutQueryStringAndFragment
returns a String
that removes any query strings and anchor references, returning the URL portion only. It is the equivalent of calling cutFragment and cutQueryString on the same URL.
Parameters:
url
- theString
representing the full URL
Usage:
cutQueryStringAndFragment(url)
Returns:
String
that is the URL without any query string or anchor references
cutToFirstSignificantSubdomain
cutToFirstSignificantSubdomain
is a Clickhouse specific function that trims off the prefix of a URL before the firstSignificantSubdomain. For example http://www.foo.com
would return foo.com
where foo
is the first significant subdomain.
Parameters:
url
- theString
representing the full URL
Usage:
cutToFirstSignificantSubdomain(url)
Returns:
String
that is the URL with everthing to the left of the first significant subdomain removed.
cutURLParameter
cutURLParameter
removes a named parameter from a URL
Parameters:
url
- theString
representing the full URLparam
- aString
representing the name of the parameter to remove from the URL.
Usage:
cutURLParameter(url, param)
Returns:
String
that is the URL with everything in tact, except the named parameter has been removed.
cutWWW
cutWWW
removes the www
portion of a URL if it exists. If www
appears more than once, the first one is removed.
Parameters:
url
- theString
representing the full URL
Usage:
cutWWW(url)
Returns:
String
that is the URL with the first instance ofwww
removed.
decodeURLComponent
decodeURLComponent
translates URL encoded characters to their normal characters, so %20
would become a space.
Parameters:
url
- theString
to decode.
Usage:
decodeURLComponent(url)
Returns:
String
where all URL encoded characters are their normal text characters.
domain
domain
extracts just the domain name from a URL String
.
Parameters:
url
- theString
to extract the domain from.
Usage:
domain(url)
Returns:
String
with just the domain name.
domainWithoutWWW
domainWithoutWWW
extracts just the domain name from a URL String
, excluding one www
if www
is part of the domain in the URL.
Parameters:
url
- theString
to extract the domain from.
Usage:
domainWithoutWWW(url)
Returns:
String
that is the domain name without the firstwww
if it was part of the original URL.
extractURLParameter
extractURLParameter
extracts the value of a named parameter from a URL
Parameters:
url
- theString
to extract the parameter from.param
- the name of the parameter
Usage:
extractURLParameter(url, param)
Returns:
String
that is the value of the named parameter.
extractURLParameters
extractURLParameters
extracts all parameter name/value pairs from the URL.
Parameters:
url
- theString
to extract the parameters from.
Usage:
extractURLParameters(url)
Returns:
- an array that containing
String
structured as <parameter name>=<parameter value>
extractURLParameterNames
extractURLParameterNames
extracts all parameter names from the URL.
Parameters:
url
- theString
to extract the parameters from.
Usage:
extractURLParameters(url)
Returns:
- an array that containing all parameter names in the URL as
String
firstSignificantSubdomain
firstSignificantSubdomain
extracts the first "significant" part of the domain name from the URL that is encountered. This is a function specific to Clickhouse
Parameters:
url
- theString
to extract first significant subdomain from
Usage:
firstSignificantSubdomain(url)
Returns:
String
that is the first significant subdomain that is found in the URL.
fragment
fragment
extracts the fragment identifier and anything after it from the URL.
Parameters:
url
- theString
to extract fragment from
Usage:
fragment(url)
Returns:
String
that is the fragment identifier for any anchor references, and anything following it, not including the#
. If no fragment identifier is part of the URL, an emptyString
is returned.
path
path
extracts the specific path from the URL, not including the domain name, fragment identifiers, or query parameters.
Parameters:
url
- theString
to extract path from
Usage:
path(url)
Returns:
String
that is the path of the URL, without the domain name, fragment identifier, or query parameters
pathFull
pathFull
returns the path, along with any following anchor fagment identifiers and query query string.
Parameters:
url
- theString
to extract path from
Usage:
pathFull(url)
Returns:
String
that is the path of the URL, without the domain name, but including any following fragment identifiers or query strings.
protocol
protocol
returns the protocol in the URL, meaning everything preceeding ://
Parameters:
url
- theString
to extract the protocol from
Usage:
protocol(url)
Returns:
String
that is the protocol of the URL.
queryString
queryString
returns just the query string from the URL.
Parameters:
url
- theString
to extract the query string from
Usage:
queryString(url)
Returns:
String
that is the queryString of the URL.
queryStringAndFragment
queryStringAndFragment
returns the query string and any following anchor fragment identifiers. If the fragment identifier precedes the query string, it will not be included.
Parameters:
url
- theString
to extract the query string and following fragment from
Usage:
queryStringAndFragment(url)
Returns:
String
that is the queryString of the URL and any fragement identifiers the follow the query string.
topLevelDomain
topLevelDomain
returns the top level domain from a URL, such as com, edu, and org.
Parameters:
url
- theString
to extract the top level domain from
Usage:
topLevelDomain(url)
Returns:
String
that is the top level domain of the URL.
URLHierarchy
URLHierarchy
returns an array of Strings that are the Hierarchy of the URL, starting with the domain and ending with the original URL.
Parameters:
url
- theString
to operate on
Usage:
URLHierarchy(url)
Returns:
- an array of
String
that are the Hierarchy of the URL, starting with the domain and ending with the original URL. Each/
in the path will add an additionalpath
to the array. The last element in the array will be the original URL.
URLPathHierarchy
URLPathHierarchy
is the same as URLHierarchy, but does not include protocol and domain in the results.
Parameters:
url
- theString
to operate on
Usage:
URLPathHierarchy(url)
Returns:
- an array of
String
that are the Hierarchy of the URL, excluding protocol and domain. The first path is the first one following the domain. The last path will be the full string of the URL, not including protocol and domain in any of the results.
Updated 9 months ago