aboutsummaryrefslogtreecommitdiff
path: root/fuzz
Commit message (Collapse)AuthorAge
...
* TLS: remove JA3+ fingerprints. (#2192)Ivan Nardi2023-12-05
| | | See: #2191
* STUN: parsing of DATA attribute (#2179)Ivan Nardi2023-12-01
|
* STUN: fix detection of DTLS (#2187)Ivan Nardi2023-11-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix a memory leak ``` ==97697==ERROR: LeakSanitizer: detected memory leaks Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01) #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25 #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13 #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46 #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5 #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11 #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5 #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3 #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4 #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9 #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5 #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22 #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31 #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10 #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7 [...] SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s). ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
* Keep separating public and private API (#2157)Ivan Nardi2023-11-29
| | | See: b08c787fe
* Have a clear distinction between public and private/internal API (#2137)Ivan Nardi2023-11-09
| | | | | | 1) Public API/headers in `src/include/` [as it has always been] 2) Private API/headers in `src/lib/` Try to keep the "ndpi_" prefix only for the public functions
* fuzz: improve coverage and remove dead code (#2135)Ivan Nardi2023-11-07
| | | | | | | | | | We are not able to remove custom rules: remove the empty stubs (which originate from the original OpenDPI code). `ndpi_guess_protocol_id()` is only called on the first packet of the flow, so the bitmask `flow->excluded_protocol_bitmask` is always empty, since we didn't call any dissectors yet. Move another hash function to the dedicated source file.
* STUN: major code rework (#2116)Ivan Nardi2023-10-30
| | | | | | | | | | | | Try to have a faster classification, on first packet; use standard extra dissection data path for sub-classification, metadata extraction and monitoring. STUN caches: * use the proper confidence value * lookup into the caches only once per flow, after having found a proper STUN classification Add identification of Telegram VoIP calls.
* Custom rules: fix a stack overflow (#2128)Ivan Nardi2023-10-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0 WRITE of size 58 at 0x7f515bb3bf80 thread T0 #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13 #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16 #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8 #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16 #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703 This frame has 4 object(s): [32, 36) 'pin' (line 2705) [48, 64) 'pin6' (line 2706) [80, 96) 'd' (line 2769) [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*) Shadow bytes around the buggy address: ```
* Rename some functions with more useful/clear names (#2127)Ivan Nardi2023-10-29
|
* IPv6: add support for custom categories (#2126)Ivan Nardi2023-10-29
|
* IPv6: add support for IPv6 risk exceptions (#2122)Ivan Nardi2023-10-29
|
* IPv6: add support for custom rules (#2120)Ivan Nardi2023-10-29
|
* fuzz: extend fuzzing coverageNardi Ivan2023-10-15
|
* QUIC: export QUIC version as metadataNardi Ivan2023-10-11
|
* fuzzing: extend fuzzing coverageNardi Ivan2023-10-09
| | | | | Try fuzzing some functions which write to file/file descriptor; to avoid slowing the fuzzer, close its stdout
* fuzz: extend fuzzing coverageNardi Ivan2023-10-07
|
* fuzz: extend fuzzing coverageNardi Ivan2023-09-16
|
* fuzz: extend fuzzing coverage (#2083)Ivan Nardi2023-09-10
|
* fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082)Ivan Nardi2023-09-10
|
* fuzz: add fuzzers to test reader_util code (#2080)Ivan Nardi2023-09-10
|
* Added `lists/gambling.list` to extra dist.Toni Uhlig2023-08-27
| | | | | | * make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fix compilationToni Uhlig2023-08-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fuzz: extend coverage (#2073)Ivan Nardi2023-08-20
|
* Add an heuristic to detect fully encrypted flows (#2058)Ivan Nardi2023-07-26
| | | | | | | | A fully encrypted session is a flow where every bytes of the payload is encrypted in an attempt to “look like nothing”. The heuristic needs only the very first packet of the flow. See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf A basic, but generic, inplementation of the popcpunt alg has been added
* fuzz: extend fuzzing coverage (#2052)Ivan Nardi2023-07-18
| | | | Added/merged some traces. Improved Socks identification
* fuzz: add fuzzer for DGA detection code (#2042)Ivan Nardi2023-07-13
|
* fuzz: extend fuzzing coverage (#2040)Ivan Nardi2023-07-11
| | | | | | | | | Some notes: * libinjection: according to https://github.com/libinjection/libinjection/issues/44, it seems NULL characters are valid in the input string; * RTP: `rtp_get_stream_type()` is called only for RTP packets; if you want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`; * TLS: unnecessary check; we already make the same check just above, at the beginning of the `while` loop
* STUN: keep monitoring/processing STUN flows (#2012)Ivan Nardi2023-06-21
| | | | Look for RTP packets in the STUN sessions. TODO: tell RTP from RTCP
* Fix some memory errors triggered by allocation failures (#1995)Ivan Nardi2023-05-29
| | | | | | | | | | | | Some low hanging fruits found using nallocfuzz. See: https://github.com/catenacyber/nallocfuzz See: https://github.com/google/oss-fuzz/pull/9902 Most of these errors are quite trivial to fix; the only exception is the stuff in the uthash. If the insertion fails (because of an allocation failure), we need to avoid some memory leaks. But the only way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit costly, but we don't use that code in any critical data-path.
* ndpiReader: improve printing of payload statistics (#1989)Ivan Nardi2023-05-29
| | | | | Add a basic unit test Fix an endianess issue
* Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)Ivan Nardi2023-04-25
| | | | | | | | | | | | | | | | | | | In some networks, there are some anomalous TCP flows where the smallest ACK packets have some kind of zero padding. It looks like the IP and TCP headers in those frames wrongly consider the 0x00 Ethernet padding bytes as part of the TCP payload. While this kind of packets is perfectly valid per-se, in some conditions they might be treated by the TCP reassembler logic as (partial) overlaps, deceiving the classification engine. Add an heuristic to detect these packets and to ignore them, allowing correct detection/classification. This heuristic is configurable. Default value: * in the library, it is disabled * in `ndpiReader` and in the fuzzers, it is enabled (to ease testing) Credit to @vel21ripn for the initial patch. Close #1946
* fuzz: extend fuzzers coverage (#1952)Ivan Nardi2023-04-25
|
* Improved debug output. (#1951)Toni2023-04-21
| | | | | | * try to get rid of some `printf(..)`s as they do not belong to a shared library * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)Ivan Nardi2023-04-12
| | | | | | | | | | | | | | | | | | | `ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()` is a strange function: * it is exported by the library and it is actively used by `ntopng` * it is intrinsecally ipv4-only * it returns basically something like "classification_by_ip"/"classification_by_port" (these information have already been calculated in `ndpi_do_guess()`...) * it access the bittorrent LRU caches (similarly to `ndpi_detection_giveup()` but without all the other caches...) So: * make the interface IPv4/6 agnostic * use the classifications already available This work will allow to make the Bittorrent caches IPV6-aware (see 81e1ea5). Handle Dropbox classification "by-port" in the "standard" way.
* Test multiple `ndpiReader` configurations (#1931)Ivan Nardi2023-04-06
| | | | | | | | | Extend internal unit tests to handle multiple configurations. As some examples, add tests about: * disabling some protocols * disabling Ookla aggressiveness Every configurations data is stored in a dedicated directory under `tests\cfgs`
* fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926)Ivan Nardi2023-04-04
|
* fuzz: add fuzzer to test internal gcrypt code (#1920)Ivan Nardi2023-04-04
|
* Update libinjection code (#1918)Ivan Nardi2023-04-04
| | | | | | | | | | Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39 The goal is to finally fix #1820 See: https://github.com/libinjection/libinjection/issues/33 Update the corpus of the libinjection fuzzers Close #1820
* Ookla: rework detection (#1922)Ivan Nardi2023-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | The logic of the LRU cache has been changed: once we know an ip has connected to an Ookla server, all the following (unknown) flows (for a short time interval) from the same ip to the port 8080 are treated as Ookla ones. Most of the changes in this commit are about introducing the concept of "aggressive detection". In some cases, to properly detect a protocol we might use some statistical/behavior logic that, from one side, let us to identify the protocol more often but, from the other side, might lead to some false positives. To allow the user/application to easily detect when such logic has been triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been added. It is always possible to disable/configure this kind of logic via the API. Detection of Ookla flows using plain TLS over port 8080 is the first example of aggressive detection in nDPI. Tested with: * Android 9.0 with app 4.8.3 * Ubuntu 20.04 with Firefox 110 * Win 10 with app 1.15 and 1.16 * Win 10 with Chrome 108, Edge 108 and Firefox 106
* fuzz: add a new fuzzer to test TLS certificates (#1901)Ivan Nardi2023-03-20
|
* fuzz: extend fuzz coverage (#1888)Ivan Nardi2023-02-16
|
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* fuzz: add a new fuzzer to test serialization/deserialization code (#1876)Ivan Nardi2023-01-27
| | | | | | | | | | Autodetecting the needed buffer size is quite complex (especially with float/double values) so it is mandatory to properly check for `ndpi_snprintf` truncation. These issues have been undetected so far probably because the default buffer is big enough for all common cases. Add an example of usage of `ndpi_deserialize_clone_all()` (taken from `ntopng`)
* Add some fuzzers to test other data structures. (#1870)Ivan Nardi2023-01-25
| | | | | | | Start using a dictionary for fuzzing (see: https://llvm.org/docs/LibFuzzer.html#dictionaries). Remove some dead code. Fuzzing with debug enabled is not usually a great idea (from performance POV). Keep the code since it might be useful while debugging.
* fuzz: fix memory allocation failure logic (#1867)Ivan Nardi2023-01-20
| | | | | We *do* want to have some allocation errors. Fix some related bugs Fix: 29be01ef
* Fuzz: fix compilation with nBPF (#1860)Ivan Nardi2023-01-18
| | | Close #1859
* LRU caches: add a generic (optional and configurable) expiration logic (#1855)Ivan Nardi2023-01-18
| | | Two caches already implemented a similar mechanism: make it generic.
* Add some fuzzers to test algorithms and data structures (#1852)Ivan Nardi2023-01-17
| | | Fix some issues found with these new fuzzers
* fuzz: add fuzzer testing nDPI (initial) configurations (#1830)Ivan Nardi2022-12-23
| | | | | | | | | | | | | | | | | The goal of this fuzzer is to test init and deinit of the library, with different configurations. In details: * random memory allocation failures, even during init phase * random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()` * random LRU caches sizes * random bitmask of enabled protocols * random parameters of `ndpi_set_detection_preferences()` * random initialization of opportunistic TLS * random load/don't load of configuration files This new fuzzer is a C++ file, because it uses `FuzzedDataProvider` class (see https://github.com/google/fuzzing/blob/master/docs/split-inputs.md). Note that the (existing) fuzzers need to be linked with C++ compiler anyway, so this new fuzzer doesn't add any new requirements.
* fuzz: some enhancements (#1827)Ivan Nardi2022-12-10
| | | | | | | | | | Load some custom configuration (like in the unit tests) and factorize some (fuzzing) common code. There is no way to pass file paths to the fuzzers as parameters. The safe solution seems to be to load them from the process working dir. Anyway, missing file is not a blocking error. Remove some dead code (found looking at the coverage report)
Powered by cgit, Themed by Ted Yin.