aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Fix a use-of-uninitialized-value error on PCRE codeNardi Ivan2022-10-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is likely a false positive, triggered by the fact that libpcre is usually compiled without MASAN support. It it was a real error, ASAN would complain loudly with a invalid-free error at the end of the same function. ``` ==83793==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x562296111174 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1631:3 #1 0x5622960e3e4a in ndpi_is_rce_injection /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1636:5 #2 0x5622960de7cd in ndpi_validate_url /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1741:12 #3 0x5622960dae45 in ndpi_dpi2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1362:29 #4 0x5622960e2751 in ndpi_flow2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1512:10 #5 0x562296033b0f in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1310:9 #6 0x5622960501f9 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1659:2 #7 0x562296045aef in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2202:10 #8 0x562295e85374 in ndpi_process_packet /home/ivan/svnrepos/nDPI/example/ndpiReader.c:3937:7 #9 0x7f1235053466 (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x23466) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61) #10 0x7f1235041f67 in pcap_loop (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x11f67) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61) #11 0x562295e53139 in runPcapLoop /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4060:15 #12 0x562295e51e7f in processing_thread /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4130:3 #13 0x7f1234e53608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 #14 0x7f1234d2f132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Uninitialized value was created by an allocation of 'pcreErrorStr' in the stack frame #0 0x5622961108a6 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1603:3 ```
* Add some popular streaming applicationsNardi Ivan2022-10-17
| | | | | | | A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142; full credits to @ChrisNelson-CyberReef. Close #1716
* Updated protocol dissector serialisation code. (#1767)Toni2022-10-15
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* HTTP: fix stack-buffer-overflow (#1768)Ivan Nardi2022-10-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==24879==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fa085b31e60 at pc 0x55cc63f203e2 bp 0x7ffc9ec91b10 sp 0x7ffc9ec91298 READ of size 17 at 0x7fa085b31e60 thread T0 #0 0x55cc63f203e1 in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55cc63f20769 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x50e769) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) #2 0x55cc63f22210 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x510210) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) #3 0x55cc6420fc76 in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:668:4 #4 0x55cc6420344b in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:742:5 #5 0x55cc642031ce in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:737:7 #6 0x55cc641fac9f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1352:4 #7 0x55cc641f2fd5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1461:3 #8 0x55cc64085275 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5580:6 #9 0x55cc64085c87 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5628:12 #10 0x55cc64085927 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5647:12 #11 0x55cc64095fcb in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6458:15 #12 0x55cc63fd08b4 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #13 0x55cc63fd09f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #14 0x7fa0880fb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #15 0x55cc63efb45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e945d) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) Address 0x7fa085b31e60 is located in stack of thread T0 at offset 96 in frame #0 0x55cc6420f1bf in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:644 This frame has 5 object(s): [32, 36) 'a' (line 653) [48, 52) 'b' (line 653) [64, 68) 'c' (line 653) [80, 96) 'buf' (line 654) [112, 176) 'msg' (line 662) <== Memory access at offset 96 partially underflows this variable ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52229
* ndpiReader: fix help message. There isn't a 'J' option (#1770)Ivan Nardi2022-10-14
|
* Sync utests results (#1774)Ivan Nardi2022-10-14
|
* Removed gettimeofday as win files has located in windows/src/Luca Deri2022-10-14
|
* Added DNS error code mapping number -> stringLuca Deri2022-10-14
|
* Added detection of URLs starting with a dot (.)Luca Deri2022-10-10
|
* LINE_CALL: add detection of LINE voip calls (#1761)Ivan Nardi2022-10-06
| | | | | | | | These flows are classifed as `LINE_CALL`; another option was `RTP/LINE_CALL`. No sure about the best solution... Extend LINE domains list. Remove RTP dead code.
* Windows changesLuca Deri2022-10-06
|
* irc: fix heap-buffer-overflow (#1765)Ivan Nardi2022-10-06
|
* Fixes Stack-buffer-overflow READ {*} · check_content_type_and_change_protocolLuca Deri2022-10-06
|
* Updated IRC resultLuca Deri2022-10-05
|
* Improved credentials check in IRCLuca Deri2022-10-05
|
* Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx ↵Luca2022-10-04
| | | | are supported
* Fix compilation and sync utests resultsNardi Ivan2022-10-04
|
* Improved HTTP payload handlingLuca2022-10-04
|
* Improved detection of WebShell and PHP code in HTTP URLs that is reported ↵Luca2022-10-04
| | | | via flow risk
* Added the ability to track the payload via -E and via the new option ↵Luca2022-10-04
| | | | 'ndpi_track_flow_payload'
* Fix ndpi_timeval_to_milliseconds/microseconds for platforms with tv_usec is ↵Toni2022-10-02
| | | | | | | | | an unsigned int. (#1762) * implicit conversion to an 32 bit unsigned int leads to an overflow Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Restore Confidence enum constants to ascending values. (#1760)Toni2022-10-02
| | | | | | | | | | * NDPI_CONFIDENCE_MAX is useless otherwise since it is supposed the be the highest value w/o any gaps * for client applications, it makes sense to use that the numbers directly w/o any additional mapping Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Line app support (#1759)sharonenoch2022-10-01
| | | | | | | | | | | | | | | | | | | * Standard support for LINE app * Added test pcap for LINE app * make check result for LINE app * Make check success as 1kxun has LINE packets * Added the ASN inc file for LINE * Removed extra lines as its effecting make check * Editing the SNI required a new pcap output file for TLS.Line format * Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out Co-authored-by: Sharon Enoch <sharone@amzetta.com>
* TLS/DTLS: we process certificate for UDP flows, tooNardi Ivan2022-09-30
| | | | | | Note that current code access `certificate_processed` state even before setting the protocol classification, so this piece of information can't be saved in `flow->protos` union.
* Fixed invalid unidirectional traffic alert for unidirectional protocols ↵Luca2022-09-29
| | | | (e.g. sFlow)
* Remove unused codeNardi Ivan2022-09-28
| | | | | | | | | LRU callbacks have been added in 460ff3c7a, but they have never been used and they have never been extended to the other LRU caches. `ndpi_search_tcp_or_udp()` basically returns the classification by port/ip of the flow; calling it from the dissector is useless. The same for TOR detection: ips are checked in the generic code
* Fix value of `ndpi_protocol->protocol_by_ip`Nardi Ivan2022-09-25
| | | | Fix: a7c2734b
* DNS: change category of DNS flowsNardi Ivan2022-09-25
| | | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any). Follow-up of 83de3e47
* Changed some protocol categories to align with others.Toni Uhlig2022-09-23
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* DNS: change category of DNS flowsNardi Ivan2022-09-22
| | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any).
* Extend content match listsNardi Ivan2022-09-22
|
* NAT-PMP: fix metadata extractionNardi Ivan2022-09-21
|
* Improved NATPMP dissection. (#1745)Toni2022-09-21
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Build ndpiReader and run regression tests.Toni Uhlig2022-09-21
| | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed VS2019 build.segfault2022-09-21
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Provide extra `ndpi_define.h` for Visual Studio.Toni Uhlig2022-09-21
| | | | | | * Caution: file needs to keep in sync with `src/include/ndpi_define.h.in` manually Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Moved `NDPI_API_VERSION` to `src/include/ndpi_define.h.in` where it belongs.Toni Uhlig2022-09-21
| | | | | | | That way, we can make `src/include/ndpi_api.h` independent of autoconf. MSBuild will like that. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Retargeted to Windows 10.segfault2022-09-21
| | | | | * disable Spectre Mitigation (may change) * remove autoconf generated header files
* Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.mark andrews2022-09-21
| | | | | | * add CI support via MSBuild Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Copilation fixLuca Deri2022-09-21
|
* Added missing libraryLuca Deri2022-09-21
|
* Finalized nBPF support. You can now define custom protocols such asLuca Deri2022-09-21
| | | | | | | | | | (see exaple/protos.txt) nbpf:"host 192.168.1.1 and port 80"@HomeRouter In order to have nBPF support, you need to compile nDPI with it. Just download https://github.com/ntop/PF_RING in the same directory where you have downloaded nDPI and compile PF_RING/userland/nbpf
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Fix warnings on OpenWrt CCs, caused by the memcpy builtin. (#1741)Toni2022-09-18
| | | | | | | | | | | | | | | | | | | In file included from ../include/ndpi_includes.h:31, from ../include/ndpi_main.h:27, from ../include/ndpi_api.h:28, from protocols/quic.c:27: In function 'memcpy', inlined from 'tls13_hkdf_expand_label_context' at protocols/quic.c:473:5, inlined from 'tls13_hkdf_expand_label' at protocols/quic.c:498:10, inlined from 'quic_hkdf_expand_label.constprop' at protocols/quic.c:512:6: /home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: error: argument 2 null where non-null expected [-Werror=nonnull] 53 | return __builtin_memcpy(__od, __os, __n); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ protocols/quic.c: In function 'quic_hkdf_expand_label.constprop': /home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: note: in a call to built-in function '__builtin_memcpy' Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added MAX_NBPF_CUSTOM_PROTO defineLuca Deri2022-09-17
|
* Initial work towards custom BPF protocol definitionLuca2022-09-16
|
* Add CryNetwork dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Syncthing dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.