aboutsummaryrefslogtreecommitdiff
path: root/tests/pcap
Commit message (Collapse)AuthorAge
* Improved Zoom protocol detectionLuca Deri2022-01-23
|
* Added EthernetIP dissectorLuca Deri2022-01-12
|
* GTP: fix some false positives (#1394)Ivan Nardi2022-01-08
|
* Added support for Log4J/Log4Shell detection in nDPI via a new flow risk ↵Luca Deri2021-12-23
| | | | named NDPI_POSSIBLE_EXPLOIT
* A final(?) effort to reduce memory usage per flow (#1389)Ivan Nardi2021-12-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove some unused fields and re-organize other ones. In particular: * Update the parameters of `ndpi_ssl_version2str()` function * Zattoo, Thunder: these timestamps aren't really used. * Ftp/mail: these protocols are dissected only over TCP. * Attention must be paid to TLS.Bittorrent flows to avoid invalid read/write to `flow->protos.bittorrent.hash` field. This is the last(?) commit of a long series (see 22241a1d, 227e586e, 730c2360, a8ffcd8b) aiming to reduce library memory consumption. Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 231.71 KB Flow Memory (per flow): 1008 B <--------- ``` i.e. memory usage per flow has been reduced by 66%, dropping below the psychological threshold of 1 KB. To further reduce this value, we probably need to look into #1279: let's fight this battle another day.
* Improve/add several protocols (#1383)Ivan Nardi2021-12-18
| | | | | | | | | | | | | | | | | | | Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok detection. Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection. Try to differentiate between Messenger/Signal standard flows (i.e chat) and their VOIP (video)calls (like we already do for Whatsapp and Snapchat). Add a partial list of some ADS/Tracking stuff. Fix Cassandra, Radius and GTP false positives. Fix DNS, Syslog and SIP false negatives. Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and GTP_PRIME. Fix 3 LGTM warnings.
* QUIC: add support for QUICv2 (draft 00) (#1379)Ivan Nardi2021-12-04
| | | | It is already time to start looking at the new QUIC version. See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-v2-00
* Added Salesforce detectionLuca Deri2021-11-26
|
* Fix writes to `flow->protos` union fields (#1354)Ivan Nardi2021-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | We can write to `flow->protos` only after a proper classification. This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP, SMTP, IMAP and POP code. There are two kinds of fixes: * write to `flow->protos` only if a final protocol has been detected * move protocol state out of `flow->protos` The hard part is to find, for each protocol, the right tradeoff between memory usage and code complexity. Handle Kerberos like DNS: if we find a request, we set the protocol and an extra callback to further parsing the reply. For all the other protocols, move the state out of `flow->protos`. This is an issue only for the FTP/MAIL stuff. Add DHCP Class Identification value to the output of ndpiReader and to the Jason serialization. Extend code coverage of fuzz tests. Close #1343 Close #1342
* Add detection of OCSP (#1370)Ivan Nardi2021-11-11
| | | | | | | | | | This protocol is detected via HTTP Content-Type header. Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to classify a HTTP flow according to this header. Since then, this automa has been useless because it is always empty. Re-enable it to match only a string seems overkilling. Remove all `content_automa` leftovers.
* IMAP, POP3, SMTP: improve dissection (#1368)Ivan Nardi2021-11-11
| | | Avoid NATS false positives
* Avoid overwriting valid protocol in `ndpi_detection_giveup` (#1360)Ivan Nardi2021-10-27
| | | | | | | | | | | | | | | | | We should avoid updating any valid protocol in `ndpi_detection_giveup`; we should try to find a proper classification only if the flow is still completely unclassified. For example in the attached pcap there is a valid TLS session, recognized as such by TLS dissector. However, the `ndpi_detection_giveup`function updates it to "HTTP/TLS" (!?) simply because the server port is 80. Note that the real issue is not the wrong classification, but the wrong access to `flow->protos` union. If we already set some fields of `flow->protos` and we change the protocol in `ndpi_detection_giveup`, we might end up freeing some invalid pointers in `ndpi_free_flow_data` (no wonder this issue has been found while fuzzing #1354) Fix GIT and TLS dissectors (issues found by CI fuzzer)
* Detect invalid characters in text and set a risk. Fixes #1347. (#1363)Toni2021-10-26
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: fix a heap-buffer-overflow (#1356)Ivan Nardi2021-10-22
| | | | Revert of c3d1c697 Error reproducible with the attached pcap and valgrind
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* QUIC: fix an integer overflow (#1337)Ivan Nardi2021-10-11
| | | | Long standing bug: credits to @lnslbrty for digging into it and to @aouinizied for the CI improvements
* WHOIS: enhance detection, avoiding false positives (#1320)Ivan Nardi2021-10-05
| | | We are interested only in the domain name required, not in the long reply.
* FTP: fix support for START-TLS sessionsNardi Ivan2021-09-21
| | | | | | When TLS-over-FTP is used, the credentials are encrypted. So we must not wait for the username and the password commands, otherwise we elaborate a lot of packets for nothing.
* STUN: fix extraction of Realm attributeNardi Ivan2021-09-20
| | | | While at it, improve detection of Facebook Messenger
* Added DNS fragmented test pcapLuca Deri2021-09-17
|
* Added new risk for clear text credentialsLuca Deri2021-09-10
|
* Add Cassandra protocol dissector (#1285)lucasbaile2021-09-09
| | | Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
* Added extraction of hostname in SMTPLuca Deri2021-08-11
| | | | Fixed mail incalid subprotocol calculation
* Skip whitespaces between HTTP method and URL. (#1271)Toni2021-08-08
| | | | | * be less case-restrictive, RFC2616 wants it that way Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added testing pcap for TLS fatal alertLuca Deri2021-08-07
|
* Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)Toni2021-07-31
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved TFTP detection. Fixes #1242, #1256 (#1262)Toni2021-07-25
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS Risks - Certificate Validity Too Long (#1239)pacant2021-07-14
| | | | | | | | | | | * Added flow risk: TLS certificate too long * Added flow risk: TLS certificate too long * Date for TLS limit added * TLS certificate check fixed Co-authored-by: pacant <a.pace97@outlook.com>
* Added AVAST SecureDNS protocol. (#1244)Toni2021-07-14
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved Steam detection (Steam Datagram Relay - SDR). (#1243)Toni2021-07-14
| | | | | * improved DNSCrypt midstream detection again (sufficient for all tested use-cases) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved dnscrypt midstream detection. (#1241)Toni2021-07-13
| | | | | * fixed skype false-positive detection of dnscrypt traffic Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* ahoсorasick. Code review. Part 2. (#1236)Vitaly Lavrov2021-07-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplified the process of adding lines to AC_AUTOMATA_t. Use the ndpi_string_to_automa() function to add patterns with domain names. For other cases can use ndpi_add_string_value_to_automa(). ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding and compare data in a case insensitive manner. For mandatory pattern comparison from the end of the line, the "ac_pattern.rep.at_end=1" flag is used. This eliminated unnecessary conversions to lowercase and adding "$" for end-of-line matching in domain name patterns. ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified. ac_domain_match_handler() looks for the template with the highest domain level. For special cases it is possible to manually specify the domain level. Added test for checking ambiguous domain names like: - short.weixin.qq.com is QQ, not Wechat - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook If you specify a NULL handler when creating the AC_AUTOMATA_t structure, then a pattern with the maximum length that satisfies the search conditions will be found (exact match, from the beginning of the string, from the end of the string, or a substring). Added debugging for ac_automata_search. To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG). The search will display "name" and a list of matching patterns. Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates and which templates were found. The ac_automata_dump() prototype has been changed. Now it outputs data to a file. If it is specified as NULL, then the output will be directed to stdout. If you need to get data as a string, then use open_memstream(). Added the ability to run individual tests via the do.sh script
* Added test pcapLuca Deri2021-07-06
|
* Improved RTSP via HTTP detection. (#1232)Toni2021-07-06
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved Z39.50 detection. (#1225)Toni2021-07-05
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added Z39.50 protocol. (#1219)Toni2021-06-29
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* QUIC: add basic support for fragmented Client Hello (#1216)Ivan Nardi2021-06-24
| | | | Only in-order and non overlapping fragments are handled See #1195
* New testing pcap with syn scan attackLuca2021-06-08
|
* Added TLS certifiacate cachingLuca Deri2021-05-15
| | | | Added Fortigate protocol
* Converted some test .pcapng files to pcap formatLuca Deri2021-05-13
|
* Added browser TLS heuristicLuca Deri2021-05-13
|
* Improved SSL certificate name wildcard handling and risk. #1182 (#1183)Toni2021-05-11
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)Ivan Nardi2021-05-09
| | | | | | | | | | | | Error detected with valgrind. ==13127== Conditional jump or move depends on uninitialised value(s) ==13127== at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==13127== by 0x1A93B6: ndpi_strdup (ndpi_main.c:159) ==13127== by 0x1C07CC: processClientServerHello (tls.c:1678) ==13127== by 0x1C0C4C: processTLSBlock (tls.c:712) ==13127== by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849) See also 8c3674e9
* Add Genshin Impact protocol. (#1173)Toni2021-04-25
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)Toni2021-04-20
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)Ivan Nardi2021-04-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Error detected with valgrind. ==125883== Conditional jump or move depends on uninitialised value(s) ==125883== at 0x438F57: processClientServerHello (tls.c:1421) ==125883== by 0x43B35A: processTLSBlock (tls.c:712) ==125883== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849) ==125883== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426) ==125883== by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301) ==125916== Conditional jump or move depends on uninitialised value(s) ==125916== at 0x438D7D: processClientServerHello (tls.c:1379) ==125916== by 0x43B35A: processTLSBlock (tls.c:712) ==125916== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849) ==125916== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426) ==125932== Conditional jump or move depends on uninitialised value(s) ==125932== at 0x438C1D: processClientServerHello (tls.c:1298) ==125932== by 0x43B35A: processTLSBlock (tls.c:712) ==125932== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849) ==125932== by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426) ==125950== Conditional jump or move depends on uninitialised value(s) ==125950== at 0x438D4F: processClientServerHello (tls.c:1371) ==125950== by 0x43B35A: processTLSBlock (tls.c:712) ==125950== by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849) ==125950== by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
* Fix detunneling of GTP-U traffic (#1168)Ivan Nardi2021-04-18
| | | | | Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U detunneling code. (see https://github.com/ntop/nDPI/actions/runs/719882047)
* Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)Toni2021-03-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Try to detect one subprotocol if a detected protocol can have one. * This adds a performance overhead due to much more protocol detection routine calls. See #1148 for more information. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (1/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (2/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactored aimini HTTP detection parts (somehow related to #1148). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Added aimini client/server test pcap. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto * Adjusted Python code to conform to the changes made during the refactoring process. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add support for Snapchat voip calls (#1147)Ivan Nardi2021-03-06
| | | | | | | | | * Add support for Snapchat voip calls Snapchat multiplexes some of its audio/video real time traffic with QUIC sessions. The peculiarity of these sessions is that they are Q046 and don't have any SNI. * Fix tests with libgcrypt disabled
* DTLS: improve support (#1146)Ivan Nardi2021-03-02
| | | | | | | * DTLS: add some pcap tests * DTLS: fix parsing of Client/Server Helllo message * DTLS: add parsing of server certificates
Powered by cgit, Themed by Ted Yin.