Improved TLS application data detection.improved/tls-appdata-detection-the-second

 * #1532 did fx TLS appdata detection only partially
 * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload

Signed-off-by: lns <matzeton@googlemail.com>

1 files changed, 6 insertions, 6 deletions

diff --git a/tests/result/skype_no_unknown.pcap.out b/tests/result/skype_no_unknown.pcap.out
index b788f0d91..b99fd6bb3 100644
--- a/tests/result/skype_no_unknown.pcap.out
+++ b/tests/result/skype_no_unknown.pcap.out
@@ -4,8 +4,8 @@ DPI Packets (TCP):	1240	(16.32 pkts/flow)
 DPI Packets (UDP):	310	(1.67 pkts/flow)
 DPI Packets (other):	5	(1.00 pkts/flow)
 Confidence Unknown          : 45 (flows)
-Confidence Match by port    : 26 (flows)
-Confidence DPI              : 196 (flows)
+Confidence Match by port    : 22 (flows)
+Confidence DPI              : 200 (flows)
 
 Unknown	850	152468	45
 DNS	2	267	1
@@ -28,11 +28,11 @@ JA3 Host Stats:
 
 	1	TCP 192.168.1.34:51230 <-> 157.56.126.211:443 [proto: 91.125/TLS.Skype_Teams][Encrypted][Confidence: DPI][cat: VoIP/10][166 pkts/39042 bytes <-> 182 pkts/142645 bytes][Goodput ratio: 72/92][51.22 sec][bytes ratio: -0.570 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 370/331 45360/45460 3946/3736][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 235/784 1506/1506 433/565][Risk: ** Obsolete TLS (v1.1 or older) **][Risk Score: 100][TLSv1][JA3C: 06207a1730b5deeb207b0556e102ded2][ServerNames: *.gateway.messenger.live.com,*.beta.gateway.edge.messenger.live.com,*.by2.gateway.edge.messenger.live.com,*.sn1.gateway.edge.messenger.live.com][JA3S: 5e4e5596180ebd0ac0317125ee490707][Issuer: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, OU=Microsoft IT, CN=Microsoft IT SSL SHA2][Subject: CN=*.gateway.messenger.live.com][Certificate SHA-1: 95:C4:07:41:85:D4:EF:AA:D9:1F:0F:1F:3C:08:BF:8E:8B:D0:90:51][Validity: 2014-10-27 22:51:07 - 2016-10-26 22:51:07][Cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA][Plen Bins: 19,2,1,5,0,1,2,0,0,3,0,0,0,1,0,0,0,1,1,0,0,1,1,0,1,0,1,10,1,1,0,0,0,0,0,0,2,0,0,0,3,5,0,0,0,30,0,0]
 	2	TCP 192.168.1.34:51227 <-> 17.172.100.36:443 [proto: 91.140/TLS.Apple][Encrypted][Confidence: DPI][cat: Web/5][38 pkts/9082 bytes <-> 38 pkts/10499 bytes][Goodput ratio: 77/80][68.36 sec][bytes ratio: -0.072 (Mixed)][IAT c2s/s2c min/avg/max/stddev: 0/0 2273/323 55625/8255 10014/1510][Pkt Len c2s/s2c min/avg/max/stddev: 54/60 239/276 680/1494 273/358][Plen Bins: 16,16,0,0,0,0,0,0,0,0,16,0,0,0,0,5,2,5,13,16,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,2,0,0]
-	3	TCP 192.168.1.34:51307 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: Match by port][cat: Web/5][19 pkts/16968 bytes <-> 7 pkts/531 bytes][Goodput ratio: 93/13][10.40 sec][bytes ratio: 0.939 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 625/19 4127/44 1113/18][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 893/76 1506/123 670/20][Plen Bins: 27,5,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,5,0,0,0,0,0,0,51,0,0]
-	4	TCP 192.168.1.34:51312 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: Match by port][cat: Web/5][18 pkts/15111 bytes <-> 7 pkts/531 bytes][Goodput ratio: 92/13][6.05 sec][bytes ratio: 0.932 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 377/19 2072/42 642/17][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 840/76 1506/123 681/20][Plen Bins: 23,5,11,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,48,0,0]
+	3	TCP 192.168.1.34:51307 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: DPI][cat: Web/5][19 pkts/16968 bytes <-> 7 pkts/531 bytes][Goodput ratio: 93/13][10.40 sec][bytes ratio: 0.939 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 625/19 4127/44 1113/18][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 893/76 1506/123 670/20][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Plen Bins: 27,5,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,5,0,0,0,0,0,0,51,0,0]
+	4	TCP 192.168.1.34:51312 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: DPI][cat: Web/5][18 pkts/15111 bytes <-> 7 pkts/531 bytes][Goodput ratio: 92/13][6.05 sec][bytes ratio: 0.932 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 377/19 2072/42 642/17][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 840/76 1506/123 681/20][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Plen Bins: 23,5,11,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,0,0,0,0,0,0,0,0,0,0,5,0,0,0,48,0,0]
 	5	UDP 192.168.0.254:1025 -> 239.255.255.250:1900 [proto: 12/SSDP][ClearText][Confidence: DPI][cat: System/18][36 pkts/13402 bytes -> 0 pkts/0 bytes][Goodput ratio: 89/0][60.04 sec][bytes ratio: 1.000 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 1254/0 19850/0 4801/0][Pkt Len c2s/s2c min/avg/max/stddev: 327/0 372/0 405/0 29/0][PLAIN TEXT (NOTIFY )][Plen Bins: 0,0,0,0,0,0,0,0,11,27,22,38,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
-	6	TCP 192.168.1.34:51315 <-> 212.161.8.36:13392 [proto: 91/TLS][Encrypted][Confidence: Match by port][cat: Web/5][16 pkts/11797 bytes <-> 7 pkts/493 bytes][Goodput ratio: 91/6][3.34 sec][bytes ratio: 0.920 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 218/30 1428/74 413/32][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 737/70 1506/85 681/7][Plen Bins: 33,0,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,40,0,0]
-	7	TCP 192.168.1.34:51317 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: Match by port][cat: Web/5][12 pkts/5655 bytes <-> 8 pkts/553 bytes][Goodput ratio: 86/5][0.16 sec][bytes ratio: 0.822 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 15/19 43/43 19/18][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 471/69 1506/85 596/8][Plen Bins: 45,9,0,9,0,0,0,0,0,0,0,0,0,0,9,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,0,0,18,0,0]
+	6	TCP 192.168.1.34:51315 <-> 212.161.8.36:13392 [proto: 91/TLS][Encrypted][Confidence: DPI][cat: Web/5][16 pkts/11797 bytes <-> 7 pkts/493 bytes][Goodput ratio: 91/6][3.34 sec][bytes ratio: 0.920 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 218/30 1428/74 413/32][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 737/70 1506/85 681/7][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Plen Bins: 33,0,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,0,0,40,0,0]
+	7	TCP 192.168.1.34:51317 <-> 149.13.32.15:13392 [proto: 91/TLS][Encrypted][Confidence: DPI][cat: Web/5][12 pkts/5655 bytes <-> 8 pkts/553 bytes][Goodput ratio: 86/5][0.16 sec][bytes ratio: 0.822 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 15/19 43/43 19/18][Pkt Len c2s/s2c min/avg/max/stddev: 66/60 471/69 1506/85 596/8][Risk: ** Known Proto on Non Std Port **][Risk Score: 50][Plen Bins: 45,9,0,9,0,0,0,0,0,0,0,0,0,0,9,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,0,0,18,0,0]
 	8	TCP 192.168.1.34:51231 <-> 23.206.33.166:443 [proto: 91.125/TLS.Skype_Teams][Encrypted][Confidence: DPI][cat: VoIP/10][16 pkts/3461 bytes <-> 1 pkts/74 bytes][Goodput ratio: 69/0][54.57 sec][Hostname/SNI: apps.skype.com][bytes ratio: 0.958 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 3429/0 6616/0 2851/0][Pkt Len c2s/s2c min/avg/max/stddev: 54/74 216/74 251/74 72/0][Risk: ** TLS (probably) Not Carrying HTTPS **][Risk Score: 10][TLSv1.2][JA3C: 799135475da362592a4be9199d258726][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
 	9	TCP 108.160.163.108:443 <-> 192.168.1.34:51222 [proto: 91.121/TLS.Dropbox][Encrypted][Confidence: DPI][cat: Cloud/13][4 pkts/818 bytes <-> 4 pkts/2172 bytes][Goodput ratio: 68/88][30.64 sec][bytes ratio: -0.453 (Download)][IAT c2s/s2c min/avg/max/stddev: 222/2 10212/10139 30193/30413 14128/14336][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 204/543 343/1020 138/477][Plen Bins: 0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
 	10	TCP 192.168.1.34:51295 <-> 23.206.33.166:443 [proto: 91.125/TLS.Skype_Teams][Encrypted][Confidence: DPI][cat: VoIP/10][11 pkts/2074 bytes <-> 1 pkts/74 bytes][Goodput ratio: 64/0][14.82 sec][Hostname/SNI: apps.skype.com][bytes ratio: 0.931 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 1393/0 6406/0 1894/0][Pkt Len c2s/s2c min/avg/max/stddev: 66/74 189/74 233/74 73/0][Risk: ** Obsolete TLS (v1.1 or older) **][Risk Score: 100][TLSv1][JA3C: 3d49c0a7161d6636fcb6973f14e05046][Plen Bins: 0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]