aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_main.c
Commit message (Collapse)AuthorAge
...
* Remove `struct ndpi_packet_struct` from the public API (#2129)Ivan Nardi2023-11-07
| | | | | | | | | | | | | | | | | Right now, the only instance of `struct ndpi_packet_struct` is embedded into `struct ndpi_detection_module_struct`. Since the latter is a private structure (because of `NDPI_LIB_COMPILATION` ) there is no way for the application to get a pointer to `ndpi_struct->packet`. Bottom line: the application can't use any API functions having `struct ndpi_packet_struct *` as parameter. Remove them all (since they are completly unused and unusable). There are no public helper functions to initialize/populate/deinit a `struct ndpi_packet_struct` object, so the application can't neither create its own instance of this object. Protect `struct ndpi_packet_struct` via the same define `NDPI_LIB_COMPILATION`.
* Added malicious sites from the polish cert. (#2121)Toni2023-11-02
| | | | | * added handling of parsing errors Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Custom rules: add missing check (#2132)Ivan Nardi2023-10-31
| | | | | | | | | | | | | | | | | Fix an use-of-uninitialized-value error ``` ==28646==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8 #1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10 #2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12 #3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11 #4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8 #5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 [...] ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754
* Custom rules: fixes for Windows (#2130)Ivan Nardi2023-10-31
| | | | | | | Even after adding `WSAStartup()` call, the behavior with IPv6 addresses was different on Windows; it is the same issue reported on: https://stackoverflow.com/questions/66755681/getaddrinfo-c-on-windows-not-handling-ipv6-correctly-returning-error-code-1 Fix bitmasks.
* Added missing freeaddrinfo() callLuca Deri2023-10-30
|
* Replaced gethostbyname2 with getaddrinfo (portable code)Luca Deri2023-10-30
|
* STUN: major code rework (#2116)Ivan Nardi2023-10-30
| | | | | | | | | | | | Try to have a faster classification, on first packet; use standard extra dissection data path for sub-classification, metadata extraction and monitoring. STUN caches: * use the proper confidence value * lookup into the caches only once per flow, after having found a proper STUN classification Add identification of Telegram VoIP calls.
* Custom rules: fix a stack overflow (#2128)Ivan Nardi2023-10-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0 WRITE of size 58 at 0x7f515bb3bf80 thread T0 #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13 #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16 #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8 #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16 #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703 This frame has 4 object(s): [32, 36) 'pin' (line 2705) [48, 64) 'pin6' (line 2706) [80, 96) 'd' (line 2769) [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*) Shadow bytes around the buggy address: ```
* FreeBSD compilation fix (missingcvg #include <netdb.h>)Luca Deri2023-10-29
|
* Implements support for symbolic host names (#2123)Luca Deri2023-10-29
|
* Rename some functions with more useful/clear names (#2127)Ivan Nardi2023-10-29
|
* IPv6: add support for custom categories (#2126)Ivan Nardi2023-10-29
|
* IPv6: add support for IPv6 risk exceptions (#2122)Ivan Nardi2023-10-29
|
* IPv6: add support for custom rules (#2120)Ivan Nardi2023-10-29
|
* IPv6: add support for IPv6 risk tree (#2118)Ivan Nardi2023-10-27
| | | Fix the script to download crawler addressess
* ipv6: add support for ipv6 addresses lists (#2113)Ivan Nardi2023-10-26
|
* add ethereum protocol dissector. (#2111)Maatuq2023-10-25
| | | | | | | as explained here for bitcoin https://www.ntop.org/guides/nDPI/protocols.html#ndpi-protocol-bitcoin the same is applicable for ethereum. ethereum detection was removed from mining protocol and is now handled separately. Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
* Added generic Google Protobuf dissector. (#2109)Toni2023-10-24
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved detection as non DGA for hostnames belnging to a CDN (#2068)Luca Deri2023-10-23
|
* Add CAN over Ethernet dissector.Toni Uhlig2023-10-23
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Remote Management Control Protocol (RMCP).Toni Uhlig2023-10-19
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fuzz: extend fuzzing coverageNardi Ivan2023-10-15
|
* Added NDPI_MALWARE_HOST_CONTACTED flow riskLuca Deri2023-10-13
|
* fuzzing: extend fuzzing coverageNardi Ivan2023-10-09
| | | | | Try fuzzing some functions which write to file/file descriptor; to avoid slowing the fuzzer, close its stdout
* fuzz: extend fuzzing coverageNardi Ivan2023-10-07
|
* Windows code reworkLuca Deri2023-10-05
|
* Windows compilation fixesLuca Deri2023-10-05
|
* Win include changeLuca Deri2023-10-05
|
* Added HAProxy protocol. (#2088)Toni2023-10-02
| | | | | | * fixed tests/do.sh.in failure print Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added printf/fprintf replacement for some internal modules. (#1974)Toni2023-09-26
| | | | | | * logging is instead redirected to `ndpi_debug_printf` Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications ↵Luca Deri2023-09-20
| | | | sitting on top of nDPI
* Add support for (un-encrypted) HTTP/2 (#2087)Ivan Nardi2023-09-18
| | | | Plaintext HTTP/2 is quite rare on the general "internet" but it is used in some private networks (example: 5G core network)
* fuzz: extend fuzzing coverageNardi Ivan2023-09-16
|
* Add `ndpi_domain_classify_finalize()` function (#2084)Ivan Nardi2023-09-12
| | | | | | | | | The "domain classify" data structure is immutable, since it uses "bitmap64". Allow to finalize it before starting to process packets (i.e. before calling `ndpi_domain_classify_contains()`) to avoid, in the data-path, all the memory allocations due to compression. Calling `ndpi_domain_classify_finalize()` is optional.
* Fix some errors found by fuzzers (#2078)Ivan Nardi2023-09-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix compilation on Windows. "dirent.h" file has been taken from https://github.com/tronkko/dirent/ Fix Python bindings Fix some warnings with x86_64-w64-mingw32-gcc: ``` protocols/dns.c: In function ‘ndpi_search_dns’: protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] 775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname; | ^ protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] 775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname; | ``` ``` In file included from ndpi_bitmap64.c:31: third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’: third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow] 160 | uint64_t hh = hash & ((1UL << 36) - 1); ``` ``` In function ‘ndpi_match_custom_category’, inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16: ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=] 3419 | strncpy(buf, name, name_len); ```
* Added OperaVPN detectionLuca Deri2023-09-09
|
* Added NDPI_TLS_ALPN_SNI_MISMATCH flow riskLuca Deri2023-09-07
|
* Disabled debug messageLuca Deri2023-09-05
|
* Merged new and old version of ndpi_domain_classify.c codeLuca Deri2023-09-02
|
* Reworked domain classification based on binary filtersLuca Deri2023-09-02
|
* Code cleanupLuca Deri2023-09-01
|
* Fixed heap-buffer-overflow issueLuca Deri2023-08-31
|
* Warning fixesLuca Deri2023-08-29
|
* Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a ↵Luca Deri2023-08-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | probabilistic approach for handling Internet domain names. For switching back to Aho-Corasick it is necessary to edit ndpi-typedefs.h and uncomment the line // #define USE_LEGACY_AHO_CORASICK [1] With Aho-Corasick $ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory nDPI Memory statistics: nDPI Memory (once): 37.34 KB Flow Memory (per flow): 960 B Actual Memory: 33.09 MB Peak Memory: 33.09 MB [2] With the new algorithm $ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory nDPI Memory statistics: nDPI Memory (once): 37.31 KB Flow Memory (per flow): 960 B Actual Memory: 7.42 MB Peak Memory: 7.42 MB In essence from ~33 MB to ~7 MB This new algorithm will enable larger lists to be loaded (e.g. top 1M domans https://s3-us-west-1.amazonaws.com/umbrella-static/index.html) In ./lists there are file names that are named as <category>_<string>.list With -G ndpiReader can load all of them at startup
* fix compilationToni Uhlig2023-08-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Changes for supporinng more efficient sub-string matchingLuca Deri2023-08-26
|
* Boundary checkLuca2023-08-21
|
* fuzz: extend coverage (#2073)Ivan Nardi2023-08-20
|
* Mullvad VPN service added (based on entry node IP addresses) (#2062)snicket21002023-08-02
|
* Add Service Location Protocol dissector. (#2036)Toni2023-08-01
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.