setDiff - Returns difference between two arrays

setDiff returns the set difference of two arrays. The resulting array includes elements from array1 that are not in array2.

Syntax

Like many functions in DataPrime, setDiff supports two notations, function and method notation. These interchangeable forms allow flexibility in how you structure expressions.

Function notation

setDiff(array1: array<T>, array2: array<T>): array<T>

Method notation

array1: array<T>.setDiff(array2: array<T>): array<T>

Arguments

Name	Type	Required	Description
array1	array of type T	true	T must be either `string`, `bool`, `number`, `interval`, `timestamp`, `regexp` or `enum`
array2	array of type T	true	T must be either `string`, `bool`, `number`, `interval`, `timestamp`, `regexp` or `enum`

Example - Compare IP addresses & Allow Lists

Consider the following documents:

{
    "ip_address": "156.76.87.4",
    "path": "/home"
},
{
    "ip_address": "156.76.87.4",
    "path": "/checkout"
},
{
    "ip_address": "156.76.12.4",
    "path": "/home"
},
{
    "ip_address": "156.76.1.4",
    "path": "/home"
}

We want to see if all of the IP addresses listed in the above traffic, are in an allow list. First, we'll use a collect aggregation to create a new array.

groupby path collect(ip_address) as ip_addresses

This results in the following documents:

{
    "path": "/home",
    "ip_addresses": ["156.76.87.4", "156.76.12.4", "156.74.1.4"]
},
{
    "path": "/checkout",
    "ip_addresses": ["156.76.87.4"]
}

We now know which IP addresses accessed which paths. We can now compare them against our known allow list:

Function notation

create unauthorized_ip_addresses from setDiff(ip_addresses, ["156.76.12.4", "156.76.87.4"])

Method notation

create unauthorized_ip_addresses from ip_addresses.setDiff(["156.76.12.4", "156.76.87.4"])

This results in the following documents:

{
    "path": "/home",
    "ip_addresses": ["156.76.87.4", "156.76.12.4", "156.74.1.4"],
    "unauthorized_ip_addresses": ["156.74.1.4"]
},
{
    "path": "/checkout",
    "ip_addresses": ["156.76.87.4"],
    "unauthorized_ip_addresses": []
}

As we can see, the path value /home has been accesed by one IP address that is not part of our allow list.

isSuperset

Next setDiffSymmetric