Add a "confidence" field about the reliability of the classification. (#1395)

As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.

1 files changed, 6 insertions, 4 deletions

diff --git a/tests/result/simple-dnscrypt.pcap.out b/tests/result/simple-dnscrypt.pcap.out
index 395dd8f8e..b8411ed51 100644
--- a/tests/result/simple-dnscrypt.pcap.out
+++ b/tests/result/simple-dnscrypt.pcap.out
@@ -1,6 +1,8 @@
 Guessed flow protos:	0
 
 DPI Packets (TCP):	47	(11.75 pkts/flow)
+Confidence DPI (cache)      : 3 (flows)
+Confidence DPI              : 1 (flows)
 
 DNScrypt	111	44676	4
 
@@ -9,7 +11,7 @@ JA3 Host Stats:
 	1	 192.168.43.167           	 2      
 
 
-	1	TCP 192.168.43.167:50233 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][cat: Network/14][18 pkts/1788 bytes <-> 21 pkts/14580 bytes][Goodput ratio: 45/92][0.71 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.782 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 36/20 114/119 43/34][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 99/694 272/1364 68/594][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: b8f81673c0e1d29908346f3bab892b9b][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,25,8,4,0,0,8,0,4,4,0,4,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,33,0,0,0,0,0,0,0]
-	2	TCP 192.168.43.167:50259 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][cat: Network/14][18 pkts/1988 bytes <-> 18 pkts/9290 bytes][Goodput ratio: 50/89][0.52 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.647 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/25 105/106 34/35][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 110/516 334/1364 76/542][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,28,13,4,0,0,4,0,9,4,0,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,0,0,0,0,0,0,0]
-	3	TCP 192.168.43.167:50253 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][cat: Network/14][8 pkts/780 bytes <-> 10 pkts/7735 bytes][Goodput ratio: 43/93][0.44 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.817 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 72/32 188/124 74/51][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 98/774 264/1364 75/597][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,0,10,10,0,0,10,0,10,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0]
-	4	TCP 192.168.43.167:50258 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][cat: Network/14][8 pkts/780 bytes <-> 10 pkts/7735 bytes][Goodput ratio: 43/93][0.36 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.817 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 60/32 136/140 59/53][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 98/774 264/1364 75/597][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,0,10,10,0,0,10,0,10,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0]
+	1	TCP 192.168.43.167:50233 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][Confidence: DPI][cat: Network/14][18 pkts/1788 bytes <-> 21 pkts/14580 bytes][Goodput ratio: 45/92][0.71 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.782 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 36/20 114/119 43/34][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 99/694 272/1364 68/594][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: b8f81673c0e1d29908346f3bab892b9b][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,25,8,4,0,0,8,0,4,4,0,4,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,33,0,0,0,0,0,0,0]
+	2	TCP 192.168.43.167:50259 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][Confidence: DPI (cache)][cat: Network/14][18 pkts/1988 bytes <-> 18 pkts/9290 bytes][Goodput ratio: 50/89][0.52 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.647 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 25/25 105/106 34/35][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 110/516 334/1364 76/542][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,28,13,4,0,0,4,0,9,4,0,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,0,0,0,0,0,0,0]
+	3	TCP 192.168.43.167:50253 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][Confidence: DPI (cache)][cat: Network/14][8 pkts/780 bytes <-> 10 pkts/7735 bytes][Goodput ratio: 43/93][0.44 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.817 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 72/32 188/124 74/51][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 98/774 264/1364 75/597][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,0,10,10,0,0,10,0,10,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0]
+	4	TCP 192.168.43.167:50258 <-> 134.119.26.24:443 [proto: 91.208/TLS.DNScrypt][Encrypted][Confidence: DPI (cache)][cat: Network/14][8 pkts/780 bytes <-> 10 pkts/7735 bytes][Goodput ratio: 43/93][0.36 sec][Hostname/SNI: simplednscrypt.org][ALPN: h2;http/1.1][bytes ratio: -0.817 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 60/32 136/140 59/53][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 98/774 264/1364 75/597][Risk: ** Possibly Malicious JA3 Fingerprint **][Risk Score: 50][TLSv1.2][JA3C: 83e04bc58d402f9633983cbf22724b02][ServerNames: simplednscrypt.org,www.simplednscrypt.org][JA3S: 76cc3e2d3028143b23ec18e27dbd7ca9][Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA][Subject: OU=Domain Control Validated, OU=PositiveSSL, CN=simplednscrypt.org][Certificate SHA-1: 3E:20:0F:BF:AD:D8:5C:A1:A1:1B:E5:B2:A7:D4:68:E2:6A:DB:01:41][Validity: 2015-09-21 00:00:00 - 2017-09-20 23:59:59][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,0,10,10,0,0,10,0,10,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0]