aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols/tls.c
Commit message (Collapse)AuthorAge
* Improved nDPI JSON serialization. (#1689)Toni2022-08-02
| | | | | | | | * fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors * introduced portable version of gmtime_r aka ndpi_gmtime_r * do as most as possible of the serialization work in ndpi_utils.c * use flow2json in ndpiReader Signed-off-by: lns <matzeton@googlemail.com>
* TLS: avoid useless extra dissection (#1690)Ivan Nardi2022-07-31
| | | | | | | If we have seen Application Data blocks in both directions, it means we are after the handshake. At that point, extra dissection is useless. Useful feature with TLS mid-sessions, i.e. sessions without initial packets.
* SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)Ivan Nardi2022-07-30
| | | | | | The big change in TLS code is to allow "master" protocols other than TLS/DTLS, like SMTPS, POPS and IMAPS. This change will allow, in a future, a proper and complete TLS dissection for all these protocols with "STARTTLS"-like messages.
* First step in simplify `ndpi_process_extra_packet()` (#1680)Ivan Nardi2022-07-29
| | | | | | | Move the prottocol specific logic into the proper dissector code, where it belongs. Next step: remove that list of protocols. Long goal: remove this function altogether...
* TLS: improve reassembler (#1669)Ivan Nardi2022-07-22
| | | | | | | | * TLS: cosmetic changes * TLS: improve reassembler We might need to contemporary re-order messages from both directions: use one buffer per direction.
* DTLS: fix exclusion of DTLS protocolNardi Ivan2022-07-20
| | | | Add an helper to exclude a generic protocol
* Cosmetic changeLuca2022-07-12
|
* TLS: ignore invalid Content Type valuesNardi Ivan2022-07-07
|
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. (#1639)Toni2022-07-06
| | | | | | | | | | | | | | | | | * Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Revert "SMTP with STARTTLS is now identified as SMTPS" This reverts commit 52d987b603f49d996b4060f43265d1cf43c3c482. * Revert "Compilation fix" This reverts commit c019946f601bf3b55f64f78841a0d696e6c0bfc5. * Sync unit tests. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* SMTP with STARTTLS is now identified as SMTPSLuca Deri2022-07-05
|
* Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes ↵Toni2022-07-05
| | | | | | | #1630. (#1637) * FTP needs to get updated as well as it has similiar STARTTLS semantics -> follow-up Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Cleaned-up issuer DN check code addingLuca Deri2022-07-04
| | | | | | | | | u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN); Added new API function for checking nDPI-configured exceptions u_int8_t ndpi_check_flow_risk_exception(struct ndpi_detection_module_struct *ndpi_str, u_int num_params, ndpi_risk_params **params);
* Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.Toni Uhlig2022-07-04
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: add support for old DTLS versions and for detection of mid-sessions (#1619)Ivan Nardi2022-07-03
|
* Enhanced TLS risk info reported to usersLuca Deri2022-06-28
|
* Fixed syslog false positives. (#1577)Toni2022-06-03
| | | | | | | * syslog: removed unnecessary/unreliable printable string check * added `ndpi_isalnum()` * splitted `ndpi_is_printable_string()` into `ndpi_is_printable_buffer()` and `ndpi_normalize_printable_string()` Signed-off-by: lns <matzeton@googlemail.com>
* TLS: fix use-of-uninitialized-value error (#1573)Ivan Nardi2022-06-02
| | | | | | | | | | | | | | | | | | | | | | Proper fix for the error already reported in 9040bc74 ``` Uninitialized bytes in __interceptor_strlen at offset 3 inside [0x7ffc7a147390, 4) ==111876==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55e3e4f32e5b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13 #1 0x55e3e4ef7391 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12 #2 0x55e3e5022fdf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1523:8 #3 0x55e3e503af44 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5 #4 0x55e3e50397cd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2 #5 0x55e3e503570c in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2453:5 #6 0x55e3e4f84a6a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6 #7 0x55e3e4f85778 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12 #8 0x55e3e4f851e1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12 #9 0x55e3e4f96c7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15 #10 0x55e3e4ed91ef in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #11 0x55e3e4eda27e in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #12 0x7f5cb3146082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #13 0x55e3e4e5140d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d) (BuildId: 0c02c433e039970dd13a60382b94dd5a8e19f625) ```
* TLS: fix stack-buffer-overflow error (#1567)Ivan Nardi2022-05-31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==300852==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f108951f060 at pc 0x5641db0ee78c bp 0x7fff3b10b910 sp 0x7fff3b10b0d0 WRITE of size 116 at 0x7f108951f060 thread T0 #0 0x5641db0ee78b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) #1 0x5641db28efad in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1224:7 #2 0x5641db27ec76 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1533:6 #3 0x5641db295677 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5 #4 0x5641db2935bb in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2 #5 0x5641db28f692 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2446:5 #6 0x5641db1d87ce in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6 #7 0x5641db1d91e4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12 #8 0x5641db1d8e87 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12 #9 0x5641db1eb4a7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15 #10 0x5641db140b45 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #11 0x5641db14130b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #12 0x7f108bcab082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #13 0x5641db07f46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5df46d) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) Address 0x7f108951f060 is located in stack of thread T0 at offset 96 in frame #0 0x5641db28ec4f in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1204 This frame has 1 object(s): [32, 96) 'str' (line 1218) <== Memory access at offset 96 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) in __interceptor_strncpy Shadow bytes around the buggy address: ``` Avoid zeroing the entire string. Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47730
* TLS: fix use-of-uninitialized-value error (#1566)Ivan Nardi2022-05-30
| | | | | | | | | | | | | | | | | | | | | | ``` Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x7ffc987443a0, 7) ==282918==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x5651277e001b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13 #1 0x5651277a45c1 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12 #2 0x5651278cfaa6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1515:8 #3 0x5651278e7834 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5 #4 0x5651278e60bd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2 #5 0x5651278e1ffc in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2445:5 #6 0x565127831c2a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6 #7 0x565127832938 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12 #8 0x5651278323a1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12 #9 0x565127843e3a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15 #10 0x56512778641f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #11 0x5651277874ae in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #12 0x7f4c46794082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #13 0x5651276fe40d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d) ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47728
* Certificate timestamps should be printed in UTC (#1563)Ivan Nardi2022-05-30
| | | This way, the CI results are consistent
* Updated tests resultsLuca Deri2022-05-30
| | | | Code cleanup
* Added RiskInfo stringLuca Deri2022-05-30
|
* Fix dissection of IPv4 header (#1561)Ivan Nardi2022-05-29
| | | | | See: https://github.com/ntop/nDPI/runs/6643914510?check_suite_focus=true Convert al the `MIN(a,b)` calls to `ndpi_min(a,b)`
* Compilation fixesLuca Deri2022-05-27
|
* Improved detection of invalid SNI and hostnames in TLS, HTTPLuca Deri2022-05-24
|
* Improved TLS alert detection. (#1542)Toni2022-05-08
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Improved TLS application data detection. (#1541)Toni2022-05-08
| | | | | | * #1532 did fx TLS appdata detection only partially * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload Signed-off-by: lns <matzeton@googlemail.com>
* Improved TLS application data detection. (#1532)Toni2022-04-27
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Added ability to store custom category file in patricia treeLuca Deri2022-04-26
|
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* DTLS: fix access to certificate cache (#1450)Ivan Nardi2022-02-21
| | | | | | | | | | | | | | | | ``` protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short') SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in AddressSanitizer:DEADLYSIGNAL ================================================================= ==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1) ==47401==The signal is caused by a READ memory access. ==47401==Hint: address points to the zero page. #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41 #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7 #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13 ```
* TLS: fix parsing of certificate elements (#1435)Ivan Nardi2022-02-07
| | | | | | | | | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280 ``` ==263603==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9 #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7 #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7 #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13 #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2 #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5 #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6 ```
* Make some protocols more "big-endian" friendly (#1402)Ivan Nardi2022-01-29
| | | See #1312
* Kerberos, TLS, example: fix some memory errors (#1419)Ivan Nardi2022-01-27
| | | | | | Detected by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
* Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow riskLuca Deri2022-01-26
| | | | Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
* Fix Grease values parsing (#1416)havsah2022-01-21
| | | | | | | | | | | The check for grease was too broad and filtered some valid values. In particular, the value 257 was skipped because it matched the previous check. This has been discovered while parsing tests/pcap/443-firefox.pcap expected ja3: 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0 previously generated ja3: 771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0 Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
* Fixed certificate mismatch checkLuca Deri2022-01-19
|
* TLS, H323, examples: fix some memory errors (#1414)Ivan Nardi2022-01-18
| | | | | | | Detected by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782 https://oss-fuzz.com/testcase-detail/6334089358082048
* TLS: fix a use-of-uninitialized-value error (#1411)Ivan Nardi2022-01-16
| | | | Detected by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
* TLS: fix heap-buffer-overflow error (#1408)Ivan Nardi2022-01-15
| | | | Detected by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
* Added the ability to specify trusted issueDN often used in companies to ↵Luca Deri2022-01-13
| | | | | | | | | | | self-signed certificates This allows to avoid triggering alerts for trusted albeit private certificate issuers. Extended the example/protos.txt with the new syntax for specifying trusted issueDN. Example: trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
* Fix two use-of-uninitialized-value errors (#1398)Ivan Nardi2022-01-12
| | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432 Fix fuzz compilation (follow-up of f5545a80)
* Add a "confidence" field about the reliability of the classification. (#1395)Ivan Nardi2022-01-11
| | | | | | | | | | | | | As a general rule, the higher the confidence value, the higher the "reliability/precision" of the classification. In other words, this new field provides an hint about "how" the flow classification has been obtained. For example, the application may want to ignore classification "by-port" (they are not real DPI classifications, after all) or give a second glance at flows classified via LRU caches (because of false positives). Setting only one value for the confidence field is a bit tricky: more work is probably needed in the next future to tweak/fix/improve the logic.
* Update copyrightAlfredo Cardigliano2022-01-03
|
* A final(?) effort to reduce memory usage per flow (#1389)Ivan Nardi2021-12-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove some unused fields and re-organize other ones. In particular: * Update the parameters of `ndpi_ssl_version2str()` function * Zattoo, Thunder: these timestamps aren't really used. * Ftp/mail: these protocols are dissected only over TCP. * Attention must be paid to TLS.Bittorrent flows to avoid invalid read/write to `flow->protos.bittorrent.hash` field. This is the last(?) commit of a long series (see 22241a1d, 227e586e, 730c2360, a8ffcd8b) aiming to reduce library memory consumption. Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 231.71 KB Flow Memory (per flow): 1008 B <--------- ``` i.e. memory usage per flow has been reduced by 66%, dropping below the psychological threshold of 1 KB. To further reduce this value, we probably need to look into #1279: let's fight this battle another day.
* Improve/add several protocols (#1383)Ivan Nardi2021-12-18
| | | | | | | | | | | | | | | | | | | Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok detection. Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection. Try to differentiate between Messenger/Signal standard flows (i.e chat) and their VOIP (video)calls (like we already do for Whatsapp and Snapchat). Add a partial list of some ADS/Tracking stuff. Fix Cassandra, Radius and GTP false positives. Fix DNS, Syslog and SIP false negatives. Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and GTP_PRIME. Fix 3 LGTM warnings.
* TLS: add support for IPV6 in Subject Alt Names field (#1385)Ivan Nardi2021-12-18
|
* TLS: fix usage of certificate cache (#1384)Ivan Nardi2021-12-18
|
* Fixed issue that prevented alt certificate names to be fully detected when ↵Luca Deri2021-12-07
| | | | ipAddress and rfc822Name were specified in certificates
* Rework how hostname/SNI info is saved (#1330)Ivan Nardi2021-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Looking at `struct ndpi_flow_struct` the two bigger fields are `host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and `protos.tls_quic.client_requested_server_name[256]` (for TLS/QUIC SNIs). This commit aims to reduce `struct ndpi_flow_struct` size, according to two simple observations: 1) maximum one of these two fields is used for each flow. So it seems safe to merge them; 2) even if hostnames/SNIs might be very long, in practice they are rarely longer than a fews tens of bytes. So, using a (single) large buffer is a waste of memory for all kinds of flows. If we need to truncate the name, we keep the *last* characters, easing domain matching. Analyzing some real traffic, it seems safe to assume that the vast majority of hostnames/SNIs is shorter than 80 bytes. Hostnames/SNIs are always converted to lowercase. Attention was given so as to be sure that unit-tests outputs are not affected by this change. Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the *first* 64 ones): as a consequence, there were some "Suspicious DGA domain name" and "TLS Certificate Mismatch" false positives.
Powered by cgit, Themed by Ted Yin.