Remove classification "by-ip" from protocol stack (#1743)

Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687

1 files changed, 16 insertions, 12 deletions

diff --git a/tests/result/quic.pcap.out b/tests/result/quic.pcap.out
index 23cb2fce9..b32989262 100644
--- a/tests/result/quic.pcap.out
+++ b/tests/result/quic.pcap.out
@@ -1,7 +1,7 @@
 Guessed flow protos:	1
 
 DPI Packets (UDP):	12	(1.20 pkts/flow)
-Confidence Match by IP      : 1 (flows)
+Confidence Unknown          : 1 (flows)
 Confidence DPI              : 9 (flows)
 Num dissector calls: 213 (21.30 diss/flow)
 LRU cache ookla:      0/0/0 (insert/search/found)
@@ -20,18 +20,22 @@ Patricia risk mask:   20/0 (search/found)
 Patricia risk:        0/0 (search/found)
 Patricia protocols:   13/10 (search/found)
 
+Unknown	3	364	1
 GMail	413	254874	1
 YouTube	85	76193	5
-Google	14	10427	3
+Google	11	10063	2
 QUIC	6	7072	1
 
-	1	UDP 192.168.1.109:57833 <-> 216.58.212.101:443 [proto: 188.122/QUIC.GMail][Encrypted][Confidence: DPI][cat: Email/3][161 pkts/23930 bytes <-> 252 pkts/230944 bytes][Goodput ratio: 72/95][37.93 sec][Hostname/SNI: mail.google.com][bytes ratio: -0.812 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 303/161 21144/21225 1960/1485][Pkt Len c2s/s2c min/avg/max/stddev: 67/61 149/916 1392/1392 207/581][User-Agent: beta Chrome/43.0.2357.45][PLAIN TEXT (mail.google.com)][Plen Bins: 4,37,1,5,3,0,3,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,40,0,0,0,0,0]
-	2	UDP 192.168.1.109:35236 <-> 216.58.210.206:443 [proto: 188.124/QUIC.YouTube][Encrypted][Confidence: DPI][cat: Media/1][25 pkts/5276 bytes <-> 44 pkts/53157 bytes][Goodput ratio: 80/97][1.00 sec][Hostname/SNI: www.youtube.com][bytes ratio: -0.819 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 52/26 803/828 183/134][Pkt Len c2s/s2c min/avg/max/stddev: 79/61 211/1208 1392/1392 358/430][User-Agent: Chrome/50.0.2661.102 Linux x86_64][PLAIN TEXT (www.youtube.com)][Plen Bins: 1,35,1,0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,57,0,0,0,0,0]
-	3	UDP 10.0.0.4:40134 -> 10.0.0.3:6121 [proto: 188/QUIC][Encrypted][Confidence: DPI][cat: Web/5][6 pkts/7072 bytes -> 0 pkts/0 bytes][Goodput ratio: 96/0][4.00 sec][bytes ratio: 1.000 (Upload)][IAT c2s/s2c min/avg/max/stddev: 150/0 800/0 1749/0 595/0][Pkt Len c2s/s2c min/avg/max/stddev: 112/0 1179/0 1392/0 477/0][Risk: ** Known Proto on Non Std Port **** Missing SNI TLS Extn **** Unidirectional Traffic **][Risk Score: 110][Risk Info: No server to client traffic][Plen Bins: 0,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,83,0,0,0,0,0]
-	4	UDP 192.168.1.105:34438 <-> 216.58.210.238:443 [proto: 188.124/QUIC.YouTube][Encrypted][Confidence: DPI][cat: Media/1][4 pkts/3682 bytes <-> 3 pkts/2863 bytes][Goodput ratio: 95/96][0.10 sec][Hostname/SNI: www.youtube.com][bytes ratio: 0.125 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 22/20 33/10 52/20 13/10][Pkt Len c2s/s2c min/avg/max/stddev: 82/79 920/954 1392/1392 538/619][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (www.youtube.com)][Plen Bins: 0,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,0,0,0,0,0]
-	5	UDP 192.168.1.105:40030 <-> 216.58.201.227:443 [proto: 188.126/QUIC.Google][Encrypted][Confidence: DPI][cat: Web/5][3 pkts/2866 bytes <-> 3 pkts/2863 bytes][Goodput ratio: 96/96][0.10 sec][Hostname/SNI: fonts.gstatic.com][bytes ratio: 0.001 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 25/21 50/10 74/21 24/10][Pkt Len c2s/s2c min/avg/max/stddev: 82/79 955/954 1392/1392 618/619][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (fonts.gstatic.com)][Plen Bins: 0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,0,0,0,0,0]
-	6	UDP 192.168.1.105:55934 <-> 216.58.201.238:443 [proto: 188.124/QUIC.YouTube][Encrypted][Confidence: DPI][cat: Media/1][2 pkts/2784 bytes <-> 2 pkts/2784 bytes][Goodput ratio: 97/97][0.09 sec][Hostname/SNI: s.ytimg.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (s.ytimg.com)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0]
-	7	UDP 192.168.1.105:45669 <-> 172.217.16.4:443 [proto: 188.126/QUIC.Google][Encrypted][Confidence: DPI][cat: Web/5][3 pkts/1550 bytes <-> 2 pkts/2784 bytes][Goodput ratio: 92/97][0.16 sec][Hostname/SNI: www.google.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (www.google.comO)][Plen Bins: 0,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60,0,0,0,0,0]
-	8	UDP 192.168.1.105:48445 <-> 216.58.214.110:443 [proto: 188.124/QUIC.YouTube][Encrypted][Confidence: DPI][cat: Media/1][2 pkts/1471 bytes <-> 1 pkts/1392 bytes][Goodput ratio: 94/97][0.10 sec][Hostname/SNI: i.ytimg.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (i.ytimg.com)][Plen Bins: 0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,0,0,0,0,0]
-	9	UDP 192.168.1.105:53817 <-> 216.58.210.225:443 [proto: 188.124/QUIC.YouTube][Encrypted][Confidence: DPI][cat: Media/1][1 pkts/1392 bytes <-> 1 pkts/1392 bytes][Goodput ratio: 97/97][0.08 sec][Hostname/SNI: yt3.ggpht.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (yt3.ggpht.com)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0]
-	10	UDP 192.168.1.105:40461 <-> 172.217.16.3:443 [proto: 126/Google][Encrypted][Confidence: Match by IP][cat: Web/5][2 pkts/241 bytes <-> 1 pkts/123 bytes][Goodput ratio: 65/65][0.09 sec][Plen Bins: 0,33,33,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
+	1	UDP 192.168.1.109:57833 <-> 216.58.212.101:443 [proto: 188.122/QUIC.GMail][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Email/3][161 pkts/23930 bytes <-> 252 pkts/230944 bytes][Goodput ratio: 72/95][37.93 sec][Hostname/SNI: mail.google.com][bytes ratio: -0.812 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 303/161 21144/21225 1960/1485][Pkt Len c2s/s2c min/avg/max/stddev: 67/61 149/916 1392/1392 207/581][User-Agent: beta Chrome/43.0.2357.45][PLAIN TEXT (mail.google.com)][Plen Bins: 4,37,1,5,3,0,3,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,40,0,0,0,0,0]
+	2	UDP 192.168.1.109:35236 <-> 216.58.210.206:443 [proto: 188.124/QUIC.YouTube][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Media/1][25 pkts/5276 bytes <-> 44 pkts/53157 bytes][Goodput ratio: 80/97][1.00 sec][Hostname/SNI: www.youtube.com][bytes ratio: -0.819 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 52/26 803/828 183/134][Pkt Len c2s/s2c min/avg/max/stddev: 79/61 211/1208 1392/1392 358/430][User-Agent: Chrome/50.0.2661.102 Linux x86_64][PLAIN TEXT (www.youtube.com)][Plen Bins: 1,35,1,0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,57,0,0,0,0,0]
+	3	UDP 10.0.0.4:40134 -> 10.0.0.3:6121 [proto: 188/QUIC][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Web/5][6 pkts/7072 bytes -> 0 pkts/0 bytes][Goodput ratio: 96/0][4.00 sec][bytes ratio: 1.000 (Upload)][IAT c2s/s2c min/avg/max/stddev: 150/0 800/0 1749/0 595/0][Pkt Len c2s/s2c min/avg/max/stddev: 112/0 1179/0 1392/0 477/0][Risk: ** Known Proto on Non Std Port **** Missing SNI TLS Extn **** Unidirectional Traffic **][Risk Score: 110][Risk Info: No server to client traffic][Plen Bins: 0,0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,83,0,0,0,0,0]
+	4	UDP 192.168.1.105:34438 <-> 216.58.210.238:443 [proto: 188.124/QUIC.YouTube][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Media/1][4 pkts/3682 bytes <-> 3 pkts/2863 bytes][Goodput ratio: 95/96][0.10 sec][Hostname/SNI: www.youtube.com][bytes ratio: 0.125 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 22/20 33/10 52/20 13/10][Pkt Len c2s/s2c min/avg/max/stddev: 82/79 920/954 1392/1392 538/619][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (www.youtube.com)][Plen Bins: 0,28,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,14,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,0,0,0,0,0]
+	5	UDP 192.168.1.105:40030 <-> 216.58.201.227:443 [proto: 188.126/QUIC.Google][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Web/5][3 pkts/2866 bytes <-> 3 pkts/2863 bytes][Goodput ratio: 96/96][0.10 sec][Hostname/SNI: fonts.gstatic.com][bytes ratio: 0.001 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 25/21 50/10 74/21 24/10][Pkt Len c2s/s2c min/avg/max/stddev: 82/79 955/954 1392/1392 618/619][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (fonts.gstatic.com)][Plen Bins: 0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,0,0,0,0,0]
+	6	UDP 192.168.1.105:55934 <-> 216.58.201.238:443 [proto: 188.124/QUIC.YouTube][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Media/1][2 pkts/2784 bytes <-> 2 pkts/2784 bytes][Goodput ratio: 97/97][0.09 sec][Hostname/SNI: s.ytimg.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (s.ytimg.com)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0]
+	7	UDP 192.168.1.105:45669 <-> 172.217.16.4:443 [proto: 188.126/QUIC.Google][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Web/5][3 pkts/1550 bytes <-> 2 pkts/2784 bytes][Goodput ratio: 92/97][0.16 sec][Hostname/SNI: www.google.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (www.google.comO)][Plen Bins: 0,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60,0,0,0,0,0]
+	8	UDP 192.168.1.105:48445 <-> 216.58.214.110:443 [proto: 188.124/QUIC.YouTube][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Media/1][2 pkts/1471 bytes <-> 1 pkts/1392 bytes][Goodput ratio: 94/97][0.10 sec][Hostname/SNI: i.ytimg.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (i.ytimg.com)][Plen Bins: 0,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,0,0,0,0,0]
+	9	UDP 192.168.1.105:53817 <-> 216.58.210.225:443 [proto: 188.124/QUIC.YouTube][IP: 0/Unknown][Encrypted][Confidence: DPI][cat: Media/1][1 pkts/1392 bytes <-> 1 pkts/1392 bytes][Goodput ratio: 97/97][0.08 sec][Hostname/SNI: yt3.ggpht.com][User-Agent: Chrome/49.0.2623.87 Linux x86_64][PLAIN TEXT (yt3.ggpht.com)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0]
+
+
+Undetected flows:
+	1	UDP 192.168.1.105:40461 <-> 172.217.16.3:443 [proto: 0/Unknown][IP: 126/Google][ClearText][Confidence: Unknown][2 pkts/241 bytes <-> 1 pkts/123 bytes][Goodput ratio: 65/65][0.09 sec][Plen Bins: 0,33,33,33,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]