From 7a7e4ee69f272900d6869b8fa07800653c4298a1 Mon Sep 17 00:00:00 2001
From: Ivan Nardi <12729895+IvanNardi@users.noreply.github.com>
Date: Mon, 28 Feb 2022 15:25:58 +0100
Subject: Add a new flow risk `NDPI_ANONYMOUS_SUBSCRIBER` (#1462)

The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.

The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.

That usually happens for the vast majority of the traffic.

However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.

Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.

A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.

TODO: IPv6
---
 .../icloud_private_relay_ip_addresses_download.sh  | 30 ++++++++++++++++++++++
 utils/ipaddr2list.py                               |  2 ++
 2 files changed, 32 insertions(+)
 create mode 100755 utils/icloud_private_relay_ip_addresses_download.sh

(limited to 'utils')

diff --git a/utils/icloud_private_relay_ip_addresses_download.sh b/utils/icloud_private_relay_ip_addresses_download.sh
new file mode 100755
index 000000000..149185d83
--- /dev/null
+++ b/utils/icloud_private_relay_ip_addresses_download.sh
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+cd "$(dirname "${0}")" || return
+
+DEST=../src/lib/ndpi_icloud_private_relay_match.c.inc
+TMP=/tmp/icloud.csv
+LIST=/tmp/icloud.list
+LIST_MERGED=/tmp/icloud.list_m
+ORIGIN="https://mask-api.icloud.com/egress-ip-ranges.csv"
+
+
+echo "(1) Downloading file..."
+http_response=$(curl -s -o "$TMP" -w "%{http_code}" ${ORIGIN})
+if [ "$http_response" != "200" ]; then
+    echo "Error $http_response: you probably need to update the list url!"
+    return
+fi
+
+echo "(2) Processing IP addresses..."
+
+# Note: the "grep -v :" is used to skip IPv6 addresses
+cut -d ',' -f 1 $TMP | grep -v ':' > $LIST
+./mergeipaddrlist.py $LIST > $LIST_MERGED
+./ipaddr2list.py $LIST_MERGED NDPI_ANONYMOUS_SUBSCRIBER > $DEST
+rm -f $TMP $LIST $LIST_MERGED
+
+echo "(3) iCloud Private Relay IPs are available in $DEST"
+
+
+
diff --git a/utils/ipaddr2list.py b/utils/ipaddr2list.py
index f43ec3ca1..5b2b10bf3 100755
--- a/utils/ipaddr2list.py
+++ b/utils/ipaddr2list.py
@@ -3,6 +3,8 @@
 import sys
 import socket, struct
 
+# This scripts is mainly used to create "ip -> protocols" lists.
+# However it is also used to create "ip -> risk" lists
 proto = "NDPI_PROTOCOL_XYX"
 if len (sys.argv) < 2 :
     print("Usage: ipaddr2list.py <file> <protocol>")
-- 
cgit v1.2.3