aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)Ivan Nardi2023-02-13
|
* Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)Ivan Nardi2023-02-13
| | | | | | | | | | | | | | | | | | | | | | | Latest RoaringBitmap version (introduced with bf413afb) triggers a new warning with GCC-7: ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s autoreconf: Entering directory `.' [...] third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] static inline int array_container_cardinality(const array_container_t *array) { ^~~~~~ third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] const array_container_t *container2) { [..] ``` The core issue is that `no_sanitize` attribute is defined only for GCC >= 8. That breaks the CI since we still use GCC-7 and `-Werror`: add a simple workaround. Fix compilation on Windows
* Centos7 fixesLuca Deri2023-02-13
|
* Update roaring bitmap codeLuca Deri2023-02-12
|
* Add Yandex services detection (#1882)0xA50C1A12023-02-09
| | | | | Add Yandex services detection Add VK and Yandex to the TLS certificate match list
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Add VK detection (#1880)0xA50C1A12023-02-02
|
* TyposLuca Deri2023-02-01
|
* Minor updateLuca Deri2023-02-01
|
* Moved to 4.7Luca Deri2023-02-01
|
* Update CHANGELOG.mdAlfredo Cardigliano2023-02-01
|
* Updated elastic_search file name in projectLuca Deri2023-01-31
|
* File rename to avoid name clashes (e.g. on Windows)Luca Deri2023-01-31
|
* Updated (C)Luca Deri2023-01-31
|
* Update changelogAlfredo Cardigliano2023-01-31
|
* Further reduction of the size of some traces used as unit test (#1879)Ivan Nardi2023-01-30
| | | See a944514d. No flow/classification/metadata have been removed.
* ndpiReader: fix packet dissection (CAPWAP and TSO) (#1878)Ivan Nardi2023-01-30
| | | | | | | | | | | | Fix decapsulation of CAPWAP; we are interested only in "real" user data tunneled via CAPWAP. When Tcp Segmentation Offload is enabled in the NIC, the received packet might have 0 as "ip length" in the IPv4 header (see https://osqa-ask.wireshark.org/questions/16279/why-are-the-bytes-00-00-but-wireshark-shows-an-ip-total-length-of-2016/) The effect of these two bugs was that some packets were discarded. Be sure that flows order is deterministic
* ndpireader: fix "Discarded bytes" statistics (#1877)Ivan Nardi2023-01-27
|
* fuzz: add a new fuzzer to test serialization/deserialization code (#1876)Ivan Nardi2023-01-27
| | | | | | | | | | Autodetecting the needed buffer size is quite complex (especially with float/double values) so it is mandatory to properly check for `ndpi_snprintf` truncation. These issues have been undetected so far probably because the default buffer is big enough for all common cases. Add an example of usage of `ndpi_deserialize_clone_all()` (taken from `ntopng`)
* Improved connection refused detectionLuca Deri2023-01-25
|
* Fix compilation and CI (#1875)Ivan Nardi2023-01-25
| | | | | | | | | | | | | | | | | | | | | ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes. Fix an error found by fuzz CI ``` ================================================================= ==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0) ==55399==The signal is caused by a READ memory access. ==55399==Hint: address points to the zero page. #0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41 #1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8 #2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5 #3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) #7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9) ```
* Update every ip lists (#1872)Ivan Nardi2023-01-25
|
* CenOS compilation fixLuca Deri2023-01-25
|
* Compilation fixesLuca Deri2023-01-25
|
* Add some fuzzers to test other data structures. (#1870)Ivan Nardi2023-01-25
| | | | | | | Start using a dictionary for fuzzing (see: https://llvm.org/docs/LibFuzzer.html#dictionaries). Remove some dead code. Fuzzing with debug enabled is not usually a great idea (from performance POV). Keep the code since it might be useful while debugging.
* Some small changes (#1869)Ivan Nardi2023-01-25
| | | | | | | | All dissector callbacks should not be exported by the library; make static some other local functions. The callback logic in `ndpiReader` has never been used. With internal libgcrypt, `gcry_control()` should always return no errors. We can check `categories` length at compilation time.
* Added new risk NDPI_TCP_ISSUESLuca Deri2023-01-24
|
* fuzz: fix memory allocation failure logic (#1867)Ivan Nardi2023-01-20
| | | | | We *do* want to have some allocation errors. Fix some related bugs Fix: 29be01ef
* Bittorrent: fix detection over TCP (#1868)Ivan Nardi2023-01-19
| | | Close #1866
* Minor changes in `ndpi_detection_giveup()` (#1861)Ivan Nardi2023-01-19
| | | | | | Check the mining cache at the end of the function, like all the others LRU caches. Rewrite the STUN checks: same identical semantic but in a cleaner way, hopefully
* Fix undefined-behaviour in ahocorasick callback (#1864)Ivan Nardi2023-01-19
| | | | | | | ``` ndpi_main.c:2119:28: runtime error: left shift of 1 by 31 places cannot be represented in type 'int' ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55226
* Bittorrent: fix heap-buffer-overflow (#1863)Ivan Nardi2023-01-19
| | | | | | | | | | | | | | | | | ``` ==258287==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068ff9d at pc 0x5653a6e35def bp 0x7ffeef5aa620 sp 0x7ffeef5a9dc8 READ of size 22 at 0x60600068ff9d thread T0 #0 0x5653a6e35dee in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x4d2dee) (BuildId: 133b8c3c8ff99408109fcb9be2538bb8341f07f7) #1 0x5653a70d6624 in ndpi_search_bittorrent /home/ivan/svnrepos/nDPI/src/lib/protocols/bittorrent.c:500:71 #2 0x5653a6ff255a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5686:6 #3 0x5653a6ff331b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5722:10 #4 0x5653a6ff2cbc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5755:12 #5 0x5653a70016bf in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6578:15 #6 0x5653a6f1836d in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1678:31 #7 0x5653a6f140a1 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2256:10 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55218 Fix: 470eaa6f
* Sync unit tests results (#1865)Ivan Nardi2023-01-19
|
* Shrinked symbolic flow risks labelsLuca Deri2023-01-19
|
* Fuzz: fix compilation with nBPF (#1860)Ivan Nardi2023-01-18
| | | Close #1859
* LRU caches: add a generic (optional and configurable) expiration logic (#1855)Ivan Nardi2023-01-18
| | | Two caches already implemented a similar mechanism: make it generic.
* POP3: improve detection (#1856)Ivan Nardi2023-01-18
|
* Improve support for Snapchat voip calls (#1858)Ivan Nardi2023-01-17
| | | | Latest Snapchat versions use QUICv1 for their audio/video real time sessions. See c50a8d480
* Add detection of Tailscale (#1857)Ivan Nardi2023-01-17
| | | Extend the example of wireguard traffic
* Add Meraki Cloud protocol and improve other protocols (#1854)Ivan Nardi2023-01-17
| | | | | Avoid some LineCall and Jabber false positives. Detect Discord mid flows. Fix Bittorrent detection.
* Add some fuzzers to test algorithms and data structures (#1852)Ivan Nardi2023-01-17
| | | Fix some issues found with these new fuzzers
* Fix classification "by-port" (#1851)Ivan Nardi2023-01-17
| | | | | | | | | | | Classification "by-port" should be the last possible effort, *after* having test all the LRU caches. Remove some dead code from `ndpi_detection_giveup()`: `flow->guessed_protocol_id` is never set to any od those voip protocols and at that point in this function we never have both a master *and* a application protocols. Coverage reports (both from unit tests and from fuzzing) confirms that was dead code.
* Improved DGA detectionLuca Deri2023-01-12
|
* Reduce the size of some traces used as unit test (#1845)Ivan Nardi2023-01-05
| | | | | | | | | | | | | | | | | | | No traces and no flows has been removed; only long sessions has been reduced, keeping only their first packets. This is quite important in fuzzing systems, since these pcaps are used as initial seed. There is no value in fuzzing long sessions, because only the very first packets are really used/processed by nDPI. Before: ``` du -h tests/pcap/ 200M tests/pcap/ ``` After: ``` du -h tests/pcap/ 98M tests/pcap/ ```
* PPSTREAM: fix some heap-buffer overflows (#1846)Ivan Nardi2023-01-05
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54802
* Fix missing bracket at ppstream (#1843)Ege Çetin2023-01-03
| | | | | | | * add missing bracket * Sync unit test results Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
* Fix some warnings and two errors found while fuzzing (#1844)Ivan Nardi2023-01-03
| | | | Fix CI See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
* Added missing prototypeLuca Deri2023-01-03
|
* Added NDPI_MINOR_ISSUES risk used for storing generic/relevant information ↵Luca Deri2022-12-31
| | | | about issues found on traffic.
* Added NDPI_PERIODIC_FLOW flow risk to be used by apps based on nDPILuca Deri2022-12-30
|
Powered by cgit, Themed by Ted Yin.