aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Add a new flow risk about literal IP addresses used as SNI (#1892)Ivan Nardi2023-03-02
| | | | | | | | | | RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in "HostName"." Don't set this risk if we have a valid sub-classification (example: via certificate) Since a similar risk already exists for HTTP hostnames, reuse it, with a more generic name.
* Fix insert of ip addresses into patricia tree(s) (#1895)Ivan Nardi2023-03-02
| | | Fix: d0b46dcf
* ndpiReader: print how many packets (per flow) were needed to perform full ↵Ivan Nardi2023-03-01
| | | | | | DPI (#1891) Average values are already printed, but this change should ease to identify regressions/improvements.
* Fixed missing AS_HELP_STRING in configure.ac. (#1893)Toni2023-03-01
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sync unit tests resultsNardi Ivan2023-03-01
|
* Keep master protocol in `ndpi_reconcile_protocols`Nardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | As a side effect of this change, the following memory leak has been fixed. ``` ==55523==ERROR: LeakSanitizer: detected memory leaks Direct leak of 2 byte(s) in 1 object(s) allocated from: #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52) #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25 #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13 #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46 #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5 #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
* SoftEther: fix invalid memory accessNardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | | We can't write `flow->protos` union until we are really sure about protocol classification ``` ==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0) ==28334==The signal is caused by a READ memory access. ==28334==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7 #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2 #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5 #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
* Fixed uninitialized variableLuca Deri2023-02-27
|
* Moving to bashLuca Deri2023-02-27
|
* Added missing breakLuca Deri2023-02-27
|
* Updated results after the latest changesLuca Deri2023-02-27
|
* in case of failure, failing result files are not listedLuca Deri2023-02-27
|
* Indent fixLuca Deri2023-02-27
|
* Improved protocol detection exploiting IP-based guessLuca Deri2023-02-27
| | | | Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
* Enhanced custom port definition and improved error reporting in case of ↵Luca Deri2023-02-24
| | | | duplications
* fuzz: extend fuzz coverage (#1888)Ivan Nardi2023-02-16
|
* Update GitHub runners versions (#1889)Ivan Nardi2023-02-14
| | | See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* Fix export/serialization of `flow->risk` (#1885)Ivan Nardi2023-02-14
| | | | Any risks bigger than 31 was never exported/serialized; found looking a the fuzz coverage report.
* Sync utests results (#1887)Ivan Nardi2023-02-13
|
* Fix packet counters (#1884)Ivan Nardi2023-02-13
| | | | | | We need to keep separete counters to keep tracks of packet numbers with and without any payload. Regression introduced in 5849863ef
* Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)Ivan Nardi2023-02-13
|
* Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)Ivan Nardi2023-02-13
| | | | | | | | | | | | | | | | | | | | | | | Latest RoaringBitmap version (introduced with bf413afb) triggers a new warning with GCC-7: ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s autoreconf: Entering directory `.' [...] third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] static inline int array_container_cardinality(const array_container_t *array) { ^~~~~~ third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] const array_container_t *container2) { [..] ``` The core issue is that `no_sanitize` attribute is defined only for GCC >= 8. That breaks the CI since we still use GCC-7 and `-Werror`: add a simple workaround. Fix compilation on Windows
* Centos7 fixesLuca Deri2023-02-13
|
* Update roaring bitmap codeLuca Deri2023-02-12
|
* Add Yandex services detection (#1882)0xA50C1A12023-02-09
| | | | | Add Yandex services detection Add VK and Yandex to the TLS certificate match list
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Add VK detection (#1880)0xA50C1A12023-02-02
|
* TyposLuca Deri2023-02-01
|
* Minor updateLuca Deri2023-02-01
|
* Moved to 4.7Luca Deri2023-02-01
|
* Update CHANGELOG.mdAlfredo Cardigliano2023-02-01
|
* Updated elastic_search file name in projectLuca Deri2023-01-31
|
* File rename to avoid name clashes (e.g. on Windows)Luca Deri2023-01-31
|
* Updated (C)Luca Deri2023-01-31
|
* Update changelogAlfredo Cardigliano2023-01-31
|
* Further reduction of the size of some traces used as unit test (#1879)Ivan Nardi2023-01-30
| | | See a944514d. No flow/classification/metadata have been removed.
* ndpiReader: fix packet dissection (CAPWAP and TSO) (#1878)Ivan Nardi2023-01-30
| | | | | | | | | | | | Fix decapsulation of CAPWAP; we are interested only in "real" user data tunneled via CAPWAP. When Tcp Segmentation Offload is enabled in the NIC, the received packet might have 0 as "ip length" in the IPv4 header (see https://osqa-ask.wireshark.org/questions/16279/why-are-the-bytes-00-00-but-wireshark-shows-an-ip-total-length-of-2016/) The effect of these two bugs was that some packets were discarded. Be sure that flows order is deterministic
* ndpireader: fix "Discarded bytes" statistics (#1877)Ivan Nardi2023-01-27
|
* fuzz: add a new fuzzer to test serialization/deserialization code (#1876)Ivan Nardi2023-01-27
| | | | | | | | | | Autodetecting the needed buffer size is quite complex (especially with float/double values) so it is mandatory to properly check for `ndpi_snprintf` truncation. These issues have been undetected so far probably because the default buffer is big enough for all common cases. Add an example of usage of `ndpi_deserialize_clone_all()` (taken from `ntopng`)
* Improved connection refused detectionLuca Deri2023-01-25
|
* Fix compilation and CI (#1875)Ivan Nardi2023-01-25
| | | | | | | | | | | | | | | | | | | | | ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes. Fix an error found by fuzz CI ``` ================================================================= ==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0) ==55399==The signal is caused by a READ memory access. ==55399==Hint: address points to the zero page. #0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41 #1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8 #2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5 #3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) ```
* Update every ip lists (#1872)Ivan Nardi2023-01-25
|
* CenOS compilation fixLuca Deri2023-01-25
|
* Compilation fixesLuca Deri2023-01-25
|
* Add some fuzzers to test other data structures. (#1870)Ivan Nardi2023-01-25
| | | | | | | Start using a dictionary for fuzzing (see: https://llvm.org/docs/LibFuzzer.html#dictionaries). Remove some dead code. Fuzzing with debug enabled is not usually a great idea (from performance POV). Keep the code since it might be useful while debugging.
* Some small changes (#1869)Ivan Nardi2023-01-25
| | | | | | | | All dissector callbacks should not be exported by the library; make static some other local functions. The callback logic in `ndpiReader` has never been used. With internal libgcrypt, `gcry_control()` should always return no errors. We can check `categories` length at compilation time.
* Added new risk NDPI_TCP_ISSUESLuca Deri2023-01-24
|
* fuzz: fix memory allocation failure logic (#1867)Ivan Nardi2023-01-20
| | | | | We *do* want to have some allocation errors. Fix some related bugs Fix: 29be01ef
* Bittorrent: fix detection over TCP (#1868)Ivan Nardi2023-01-19
| | | Close #1866
* Minor changes in `ndpi_detection_giveup()` (#1861)Ivan Nardi2023-01-19
| | | | | | Check the mining cache at the end of the function, like all the others LRU caches. Rewrite the STUN checks: same identical semantic but in a cleaner way, hopefully
Powered by cgit, Themed by Ted Yin.