aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* ndpi_memmem: optimized, fixed bug, added tests (#2499)Petr2024-07-15
|
* Optimize performance of ndpi_strnstr() and possible bugfix (#2494)Petr2024-07-15
|
* fuzzing: improve coverage (#2495)Ivan Nardi2024-07-12
| | | | | | | | | | | | | | Fix detection of WebDAV and Gnutella (over HTTP) Fix detection of z3950 Add two fuzzers to test `ndpi_memmem()` and `ndpi_strnstr()` Remove some dead code: * RTP: the same exact check is performed at the very beginning of the function * MQTT: use a better helper to exclude the protocol * Colletd: `ndpi_hostname_sni_set()` never fails Update pl7m code (fix a Use-of-uninitialized-value error)
* Improve detection of Cloudflare WARP traffic (#2491)Ivan Nardi2024-07-04
| | | See: #2484
* Add infrastructure for explicit support of Fist Packet Classification (#2488)Ivan Nardi2024-07-03
| | | | | Let's start with some basic helpers and with FPC based on flow addresses. See: #2322
* Add detection of Twitter bot (#2487)Ivan Nardi2024-07-03
| | | Update the global list of crawlers ips
* tunnelbear: improve detection over wireguard (#2485)Ivan Nardi2024-07-01
| | | See #2484
* Improve detection of Twitter/X (#2482)Ivan Nardi2024-07-01
|
* Add detection of OpenAI ChatGPT bots (#2481)Ivan Nardi2024-07-01
|
* fuzz: improve fuzzers using pl7m (#2486)Ivan Nardi2024-06-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For some unclear reasons, fuzzers using pl7m create huge corpus, triggering OOM in oss-fuzz runs (where the memory RSS limit is set to 2560Mb). Example: ``` ==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb) To change the out-of-memory limit use -rss_limit_mb=<N> Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts) 1285841683 byte(s) (54%) in 2956 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 #6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7 #7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3 #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 1031350683 byte(s) (43%) in 2468 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 #6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19 #7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5 #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 ``` See: https://oss-fuzz.com/testcase-detail/4717811415449600 See: https://oss-fuzz.com/testcase-detail/6164130982068224 Let's *try* the following workaround: set the parameter `-max-len` to 512K, to force the engine to not genereate inputs (i.e. pcap files...) larger than 512K. Right now the value used is 1MB, i.e the default, because we have file larger than 1MB in the initial seeds (i.e. `/tests/pcaps/*`). Let's hope than having smaller files lead to smaller corpus... Update pl7m code (fix a Use-of-uninitialized-value error)
* fuzz: pl7m: add a custom mutator for better fuzzing of pcap files (#2483)Ivan Nardi2024-06-27
| | | | | | | | | | | Pl7m is a custom mutator (used for structure aware fuzzing) for network traffic packet captures (i.e. pcap files). The output of the mutator is always a valid pcap file, containing the same flows/sessions of the input file. That's it: the mutator only changes the packet payload after the TCP/UDP header, keeping all the original L2/L3 information (IP addresses and L4 ports). See: https://github.com/IvanNardi/pl7m
* Remove duplicate API for `ndpi_serialize_uint32_double` in ndpi_api.h (#2480)Anshul Thakur2024-06-26
| | | Co-authored-by: Anshul Thakur <athakur@cdot.in>
* Improved logic for checking invalid DNS queriesLuca Deri2024-06-23
|
* Added default port mappings to ndpiReader help -H (#2477)Mark Jeffery2024-06-19
| | | Close #2125
* Zoom: fix detection of screen sharing (#2476)Ivan Nardi2024-06-17
|
* fuzz: improve fuzzing coverage (#2474)Ivan Nardi2024-06-17
| | | | | | Remove some code never triggered AFP: the removed check is included in the following one MQTT: fix flags extraction
* Add Ripe Atlas probe protocol. (#2473)Toni2024-06-17
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Zoom: harden RTP/RTCP detectionNardi Ivan2024-06-17
|
* Zoom: remove "stun_zoom" LRU cacheNardi Ivan2024-06-17
| | | | | Since 070a0908b we are able to detect P2P calls directly from the packet content, without any correlation among flows
* Update main Readme and remove commented out code (#2471)Ivan Nardi2024-06-17
| | | | Avoid that useless code is copy-and-pasted into new dissector, as in https://github.com/ntop/nDPI/pull/2470#discussion_r1639384434
* Added protocol - JRMI - Java Remote Method Invocation (#2470)Mark Jeffery2024-06-15
|
* Improved detection of Android connectiity checksLuca2024-06-12
|
* Zoom: fix integer overflow (#2469)Ivan Nardi2024-06-10
| | | | | | | | | | | | | | ``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==29508==ERROR: AddressSanitizer: SEGV on unknown address 0x50710145d51d (pc 0x55cb788f25fe bp 0x7ffcfefa15f0 sp 0x7ffcfefa1240 T0) ==29508==The signal is caused by a READ memory access. #0 0x55cb788f25fe in ndpi_search_zoom /home/ivan/svnrepos/nDPI/src/lib/protocols/zoom.c:210:24 #1 0x55cb787e9418 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7174:6 #2 0x55cb7883f753 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7209:10 #3 0x55cb7883bc9d in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7240:12 ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=69520
* RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)Ivan Nardi2024-06-07
| | | | | | | | | | After a flow has been classified as RTP or RTCP, nDPI might analyse more packets to look for STUN/DTLS packets, i.e. to try to tell if this flow is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with STUN/DTLS. Useful for proper (sub)classification when the beginning of the flows are not captured or if there are lost packets in the the captured traffic. Disabled by default
* Zoom: faster detection of P2P flows (#2467)Ivan Nardi2024-06-07
|
* STUN: add support for Microsoft Multiplexed TURN channels (#2464)Ivan Nardi2024-06-05
|
* TLS: add support for DTLS (over STUN) over TCP (#2463)Ivan Nardi2024-06-05
| | | | | TODO: TCP reassembler on top of UDP reassembler See: #2414
* Fixes accounting of number of processed packets in ↵Luca Deri2024-06-04
| | | | ndpi_internal_detection_process_packet()
* RTP: fix detection over TCP (#2462)Ivan Nardi2024-05-29
| | | | | | RFC4571 is not the only way to wrap RTP messages in TCP streams. For example, when RTP is encapsulated over TURN flows (i.e. via DATA attribute) there is no additional framing. See also 6127e0490
* support rtp/rtcp over tcp (#2422) (#2457)Maatuq2024-05-28
| | | | | Support rtp/rtcp over tcp as per rfc4571. Signed-off-by: mmaatuq <mahmoudmatook.mm@gmail.com>
* Add ZUG consensus protocol dissector. (#2458)Toni2024-05-28
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* CiscoVPN: we detect it only over UDP (#2454)Ivan Nardi2024-05-28
| | | The original code handled also TCP/TLS, but it was removed in 6fc29b3ae
* Improved Kafka dissector. (#2456)Toni2024-05-27
| | | | | | | | | * detect more Kafka request packet's * requires less flow memory * same detection behavior as before e.g. no asym detection implemented (can be done by dissecting responses, requires more effort) Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
* Rename Messenger to FacebookMessenger (#2453)Vladimir Gavrilov2024-05-23
|
* More NDPI_PROBING_ATTEMPT changesLuca2024-05-22
|
* Replace ndpi_strnstr() implementation with an optimal one (#2447)Vladimir Gavrilov2024-05-22
|
* Enlarged bufferLuca Deri2024-05-22
|
* DTLS: fix JA4 fingerprint (#2446)Ivan Nardi2024-05-21
|
* Remove unused code. (#2450)Toni2024-05-21
| | | | | | | | | * some `#ifdef`ed code dates back to 2019, 2020 and 2021 * some function signatures were still present in `ndpi_main.h` which may cause linker errors for libnDPI dependee's * return an error while trying to serialize a double instead of `fprintf(stderr, ...)` Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* DTLS: add support for DTLS 1.3 (#2445)Ivan Nardi2024-05-21
|
* Follow-up of 2093ac5bf (#2451)Ivan Nardi2024-05-21
|
* Small fixes after API cleanup done in c63446e59 (#2449)Ivan Nardi2024-05-20
|
* Minor dissector optimizationsLuca Deri2024-05-20
|
* Added dpi.compute_entropy configuration parameterLuca Deri2024-05-18
|
* Cleaned up APILuca Deri2024-05-17
| | | | | | | | | Removed - int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id); - int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id); Added (it supports both IPv4 and v6) + int ndpi_load_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
* Add Call of Duty Mobile support (#2438)Vladimir Gavrilov2024-05-15
|
* H323: improve detection and avoid false positives (#2432)Ivan Nardi2024-05-11
|
* Revert `ndpi_strnstr()` optimization introduced in a813121e0 (#2439)Ivan Nardi2024-05-11
| | | | | New implementation fails tests 11b, 12 and 13. Revert to the original (BSD) implementation (with also some basic parameters check)
* Add Ethernet Global Data support (#2437)Vladimir Gavrilov2024-05-11
|
* IRC: simplify detection (#2423)Ivan Nardi2024-05-11
| | | | | | | | | | | | | | | | IRC has its best times well behind, but there are still some servers using it. We should try to simplify the detection logic, still based on OpenDPI logic. Let's start with some easy changes: * try to detect TLS connection via standard hostname/SNI matching, removing an old heuristic (we have never had any trace matching it); * add some basic server names; * once we detect that the flow is IRC, we don't have to perform anything else; * remove HTTP stuff; real HTTP flows never trigger that data path * use `ndpi_memmem()` when possible
Powered by cgit, Themed by Ted Yin.