aboutsummaryrefslogtreecommitdiff
path: root/fuzz
Commit message (Collapse)AuthorAge
* Replaced ndpiReader's libjson-c support with libnDPI's internal ↵Toni2022-05-07
| | | | | | | | | | | serialization interface. (#1535) * Fixes #1528 * Serialization Interface should also fuzzed * libjson-c may only be used in the unit test to verify the internal serialization interface * Serialization Interface supports tlv(broken), csv and json * Unit test does work again and requires libjson-c Signed-off-by: lns <matzeton@googlemail.com>
* fuzz: purge old sessions (#1451)Ivan Nardi2022-02-21
| | | | | | | | At every fuzz iteration (i.e for every trace file): * keep the same ndpi context (`ndpi_init_detection_module` is very slow); * reset the flow table, otherwise it grows indefinitely. This change should fix the "out-of-memory" errors reported by oss-fuzz.
* fuzz: make fuzz_ndpi_reader faster (#1446)Ivan Nardi2022-02-20
| | | | | | | | | | | | | | | Initialize ndpi_workflow_init context only once. On a quite old notebook, before: ``` $ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/ [...] Done 3256 runs in 361 second(s) ``` after: ``` Done 5032771 runs in 361 second(s) <----------- ~1400X ``` oss-fuzz will be happy!
* Remove `struct ndpi_id_struct` (#1427)Ivan Nardi2022-01-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the last uses of `struct ndpi_id_struct`. That code is not really used and it has not been updated for a very long time: see #1279 for details. Correlation among flows is achieved via LRU caches. This change allows to further reduce memory consumption (see also 91bb77a8). At nDPI 4.0 (more precisly, at a6b10cf, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 235.27 KB Flow Memory (per flow): 688 B <-------- ``` i.e. memory usage per flow has been reduced by 77%. Close #1279
* Fix two use-of-uninitialized-value errors (#1398)Ivan Nardi2022-01-12
| | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432 Fix fuzz compilation (follow-up of f5545a80)
* Fix writes to `flow->protos` union fields (#1354)Ivan Nardi2021-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | We can write to `flow->protos` only after a proper classification. This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP, SMTP, IMAP and POP code. There are two kinds of fixes: * write to `flow->protos` only if a final protocol has been detected * move protocol state out of `flow->protos` The hard part is to find, for each protocol, the right tradeoff between memory usage and code complexity. Handle Kerberos like DNS: if we find a request, we set the protocol and an extra callback to further parsing the reply. For all the other protocols, move the state out of `flow->protos`. This is an issue only for the FTP/MAIL stuff. Add DHCP Class Identification value to the output of ndpiReader and to the Jason serialization. Extend code coverage of fuzz tests. Close #1343 Close #1342
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)Ivan Nardi2021-08-20
| | | | | | | | | | | | | | | | Fix all the warnings. Getting rid of "-Wno-unused-parameter" is quite complex because some parameters usage depends on compilation variable (i.e. `--enable-debug-messages`). The "-Werror" flag has been added only in Travis builds to avoid breaking the builds to users using uncommon/untested OS/compiler/enviroment. Tested on: * x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12 * x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag) * Raspberry 4; Debian 10.10; gcc 8.3.0
* Fixed uninitialized memory use. (#1198)Vitaly Lavrov2021-06-07
| | | The pointer "header" must be initialized before first call pcap_next_ex().
* Check datalink during fuzzing to prevent console / logfile spam. See #1175 ↵Toni2021-05-09
| | | | | for more information. (#1177) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Compilation fixLuca Deri2021-04-26
|
* Fixed mispelled API callLuca Deri2021-02-22
|
* Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)Toni2021-02-10
| | | | | | | | | | | | | | | * Fixed memory leaks caused by conditional free'ing for some TLS connections. * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Prevent reader_util.c from exit()'ing if maximum flow count reached. This confuses the fuzzer. * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS. That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix a memory leak on fuzzing code (#1036)Hosein Ghahremanzadeh2020-10-21
| | | After allocation of pcap_buffer it is necessary to free it
* Add (optional) dependency on external libraries: libgcrypt and libgpg-errorNardi Ivan2020-08-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To support QUIC payload and header decryption, it is necessary to choose an external crypto library to handle the low-level crypto stuff. Since we will use some Wireshark code, it is quite natural to choose the same library used by Wireshark itself: libgcrypt. More precisely, we will use libgcrypt and libgpg-error. Both libraries have LGPL license, so there should be no issue from this point of view. These libraries are not required to build nDPI, and their usage is optional: nDPI will keep working (and compiling) even if they are not available. However, without them, QUIC sub-classification is next to impossible. The configure flag "--disable-gcrypt" forces the build system to ignore these libraries. libgpg-error is only used for debug to have meaningful error messages and its usage is trivial. The same cannot be said for libgcrypt because its initialization is a significant issue. The rest of this commit message try explaining how libgcrypt is initialized. According to the documentation https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html https://gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html#Multi_002dThreading libgcrypt must be initialized before using it, but such initialization should be performed by the actual application and not by any library. Forcing the users to proper initialize libgcrypt in their own code seems unreasonable: most people using nDPI might be complete unaware of any crypto stuff and update each and every one application linking to nDPI with specific libgcrypt code should be out of question, anyway. Fortunately, it seems a workaround exists to initialize libgcrypt in a library https://lists.gnupg.org/pipermail/gcrypt-devel/2003-August/000458.html Therefore, we could provide a wrapper to this initialization stuff in a nDPI function. Unfortunately nDPI API lacks a global init function that must be called only once, before any other functions. We could add it, but that would be a major API break. AFAIK, ndpi_init_detection_module() might be called multiple times, for example to create multiple independent dpi engines in the same program. The proposed solution is to (optionally) initialize libgcrypt in ndpi_init_detection_module() anyway: * if the actual application doesn't directly use libgcrypt and only calls ndpi_init_detection_module() once, everything is formally correct and it should work out of the box [by far the most common user case]; * if the actual application already uses libgcrypt directly, it already performs the required initialization. In this case the ndpi_prefs.ndpi_dont_init_libgcrypt flag should be passed to ndpi_init_detection_module() to avoid further initializations. The only scenario not supported by this solution is when the application is unaware of libgcrypt and calls ndpi_init_detection_module() multiple times concurrently. But this scenario should be uncommon. A completely different option should be to switch to another crypto library, with a huge impact on the QUIC dissector code. Bottom line: crypto is hard, using libgcrypt is complex and the proposed initialization, even if not perfect, should cover the most frequent user cases and should work, for the time being. If anyone has some suggestions...
* Added memory checksLuca Deri2020-08-02
|
* Removed csv_fp as external symbol. Instead passing csv_fp through as argument.Toni Uhlig2020-07-08
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Merge pull request #940 from lnslbrty/fix/small-and-optimistic-improvmentsLuca Deri2020-07-06
|\ | | | | Fixed CodeInspector issues.
| * Fixed CodeInspector issues.Toni Uhlig2020-07-05
| | | | | | | | | | | | | | | | | | * Added compiler search list for AC_PROG_CC, AM_PROG_CC_C_O, AC_PROG_CXX, AC_PROG_CC_STDC for Mac OS X only The list rendered the AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer]) useless as it did use clang for AX_CHECK_COMPILE_FLAG but gcc during the compile process. Seems broken somehow. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* | Added fuzz_*_LDADD to give Automake the opportunity to do some dependency ↵Toni Uhlig2020-06-29
|/ | | | | | tracking. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved fuzz_ndpi_reader which supports now SMP/MT w/o race-coniditions.Toni Uhlig2020-06-29
| | | | | | | ./tests/do.sh can supports SMP/MT via environment variables. Removed -fno-sanitize=shift as well, was fixed by 317d3ffd. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Re-Added corpus zipfile generation required for google-oss fuzzerToni Uhlig2020-06-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* fixed fuzzing tests in a way that ./tests/do.sh is now able to use corpus ↵Toni Uhlig2020-06-27
| | | | | | *.pcap files from ./tests/pcap Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disabled unused codeLuca Deri2020-06-25
|
* Libfuzz fixesLuca2020-02-02
|
* Added missing additiona libsLuca2020-02-02
|
* Modifies fuzz target to fix leaksPhilippe Antoine2020-01-31
|
* Added memory boundary check in HTTP dissectorLuca Deri2020-01-24
|
* Debug compilation flag for fuzz targetsPhilippe Antoine2020-01-24
| | | | And adding include file to fuzz_ndpi_reader
* Generates automatically corpus for fuzz targetPhilippe Antoine2020-01-22
|
* New fuzz target : fuzz_ndpi_readerPhilippe Antoine2020-01-14
|
* Fixed new nDPi APILuca Deri2019-11-12
|
* Adds fuzz targetPhilippe Antoine2019-10-31
And configur option enable-fuzztargets
Powered by cgit, Themed by Ted Yin.