aboutsummaryrefslogtreecommitdiff
path: root/tests/do.sh.in
Commit message (Collapse)AuthorAge
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Add some fuzzers to test other data structures. (#1870)Ivan Nardi2023-01-25
| | | | | | | Start using a dictionary for fuzzing (see: https://llvm.org/docs/LibFuzzer.html#dictionaries). Remove some dead code. Fuzzing with debug enabled is not usually a great idea (from performance POV). Keep the code since it might be useful while debugging.
* fuzz: some enhancements (#1827)Ivan Nardi2022-12-10
| | | | | | | | | | Load some custom configuration (like in the unit tests) and factorize some (fuzzing) common code. There is no way to pass file paths to the fuzzers as parameters. The safe solution seems to be to load them from the process working dir. Anyway, missing file is not a blocking error. Remove some dead code (found looking at the coverage report)
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Fix for systems with no realpath (MacOS)Luca Deri2022-08-05
|
* Improved nDPI JSON serialization. (#1689)Toni2022-08-02
| | | | | | | | * fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors * introduced portable version of gmtime_r aka ndpi_gmtime_r * do as most as possible of the serialization work in ndpi_utils.c * use flow2json in ndpiReader Signed-off-by: lns <matzeton@googlemail.com>
* Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)Ivan Nardi2022-07-29
| | | | | | | | | | Add (basic) internal stats to the main data structures used by the library; they might be usefull to check how effective these structures are. Add an option to `ndpiReader` to dump them; enabled by default in the unit tests. This new option enables/disables dumping of "num dissectors calls" values, too (see b4cb14ec).
* Enhances gprof usage. (#1651)Toni2022-07-08
| | | | | * gprof results were incorrectly displayed Signed-off-by: lns <matzeton@googlemail.com>
* Run regression tests from different locations at the same time w/o side ↵Toni2022-07-05
| | | | | effects on the results. (#1638) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Generate profiling results as PNG.Toni Uhlig2022-07-03
| | | | | | * use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* gprof test/CI integrationlns2022-07-03
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. ↵Toni2022-07-03
| | | | | | | | Fixes #1608. (#1614) * fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization * execute serialization code while running regression tests Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Force roaring bitmap to use ndpi memory wrappers. (#1569)Toni2022-05-31
| | | | | | | | GCC analyzer won't complain about possible use-after-free (false positive). * tests/do.sh prints word diff's only once and not the same over and over again * sync unit tests Signed-off-by: lns <matzeton@googlemail.com>
* Support word diff for tests/do.sh for better readability. (#1565)Toni2022-05-30
| | | | | * Sync unit tests Signed-off-by: lns <matzeton@googlemail.com>
* Added Edgecast and Cachefly CDNs. (#1540)Toni2022-05-07
| | | | | | | | * Improved ASN update script * Ran `utils/update_every_lists.sh' * `tests/do.sh.in' prints the amount of failed pcap(s) * `utils/asn_update.sh' prints the amount of failed download(s) Signed-off-by: lns <matzeton@googlemail.com>
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* Extend tests coverage (#1476)Ivan Nardi2022-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is at least one flow under `tests/pcap` for 249 protocols out of the 284 ones supported by nDPI. The 35 protocols without any tests are: * P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK, EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET * games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA, FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU * voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE * other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE, LOTUS_NOTES, EGP, SAP Most of these protocols (expecially the P2P and games ones) have been inherited by OpenDPI and have not been updated since then: even if they are still used, the detection rules might be outdated. However code coverage (of `lib/protocols`) only increases from 65.6% to 68.9%. Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP dissection. Treat IPP as a HTTP sub protocol. Fix Cassandra false positives. Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`: these protocol ids are defined but they are never used. Remove Collectd support: its code has never been called. If someone is really interested in this protocol, we can re-add it later, updating the dissector. Add decoding of PPI (Per-Packet Information) data link type.
* Implement CI on Windows. (#1483)Zied Aouini2022-03-09
| | | | * Switch fail fast to True. * Windows CI.
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added lightweight implementation of libgcrypt. (#1444)Vitaly Lavrov2022-02-20
| | | | | | | | | | | | | | | | | | | | | Implementation borrowed from the https://github.com/ARMmbed/mbedtls.git project (v3.1.0) Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz): gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration --disable-padlock-support --disable-aesni-support \ --disable-shaext-support --disable-pclmul-support \ --disable-sse41-support --disable-drng-support \ --disable-avx-support --disable-avx2-support \ --disable-neon-support --disable-arm-crypto-support \ --disable-ppc-crypto-support --disable-amd64-as-feature-detection
* QUIC: add support for QUICv2 (draft 00) (#1379)Ivan Nardi2021-12-04
| | | | It is already time to start looking at the new QUIC version. See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-v2-00
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improve CI (#1303)Zied Aouini2021-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Improve CI pipeline * Fix branch name. * Fix branch name. * Fix libgcrypt configuration. * Update build.yml * Move to Github Actions instead of Travis CI. * Fix mingw on ubuntu bionic. * Reactivate cross compile on Ubuntu Bionic. * Switch to single line steps. * Add several compilers versions * Minor fix. * Fix build all and delete cxx * Fix RCE detection. * Fix PCRE configuration. * Add condition on PCRE test pcap. * Update WebattackRCE.pcap.out * Add missing SUBST. * Delete WebattackRCE.pcap.out * Update WebAttackRCE result. * Fix typo. * Extend jobs with pcre+msan+maxminddb. * Fix code inpector warnings. * Delete .appveyor.yml
* TLS: avoid zeroing large structures (#1300)Ivan Nardi2021-09-16
| | | | | | | | | | Zeroing large structures (i.e. size > KB) is quite costly (from a CPU point of view): we can safely avoid doing that for a couple of big structures. Standard and Valgrind tests have been diverging quite a lot: it is time to re-sync them. Use the same script and enable Valgrind via an enviroment variable: NDPI_TESTS_VALGRIND=1 ./tests/do.sh
* Fix unit tests when "--disable-gcrypt" flag is used (#1255)Ivan Nardi2021-07-23
|
* ahoсorasick. Code review. Part 2. (#1236)Vitaly Lavrov2021-07-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplified the process of adding lines to AC_AUTOMATA_t. Use the ndpi_string_to_automa() function to add patterns with domain names. For other cases can use ndpi_add_string_value_to_automa(). ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding and compare data in a case insensitive manner. For mandatory pattern comparison from the end of the line, the "ac_pattern.rep.at_end=1" flag is used. This eliminated unnecessary conversions to lowercase and adding "$" for end-of-line matching in domain name patterns. ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified. ac_domain_match_handler() looks for the template with the highest domain level. For special cases it is possible to manually specify the domain level. Added test for checking ambiguous domain names like: - short.weixin.qq.com is QQ, not Wechat - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook If you specify a NULL handler when creating the AC_AUTOMATA_t structure, then a pattern with the maximum length that satisfies the search conditions will be found (exact match, from the beginning of the string, from the end of the string, or a substring). Added debugging for ac_automata_search. To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG). The search will display "name" and a list of matching patterns. Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates and which templates were found. The ac_automata_dump() prototype has been changed. Now it outputs data to a file. If it is specified as NULL, then the output will be directed to stdout. If you need to get data as a string, then use open_memstream(). Added the ability to run individual tests via the do.sh script
* Add support for Snapchat voip calls (#1147)Ivan Nardi2021-03-06
| | | | | | | | | * Add support for Snapchat voip calls Snapchat multiplexes some of its audio/video real time traffic with QUIC sessions. The peculiarity of these sessions is that they are Q046 and don't have any SNI. * Fix tests with libgcrypt disabled
* Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)Toni2021-02-26
| | | | | | * An external file which contains known malicious SSL certificate SHA-1 hashes can be loaded via ndpi_load_malicious_sha1_file(...) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added NDPI_MALICIOUS_JA3 flow riskLuca Deri2021-02-22
| | | | Added ndpi_load_malicious_ja3_file() API call
* Added new data for risky domains testLuca Deri2021-02-21
|
* Fix utests when "--disable-gcrypt" flag is used (#1128)Ivan Nardi2021-02-04
| | | Fix: d6684f4b
* Added fuzzy targets conditional in tests/do.sh.in which prevents the fuzzer ↵Toni2021-02-04
| | | | | | | | | from running if nDPI was configured previously --enable-fuzztargets but not for the current config (may produce invalid results). (#1126) * fixed possible NULL pointer dereference for memcpy(), src pointer should never be NULL Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Disable tests that require libgcrypt if --disable-gcrypt set. (#1121)Toni2021-01-21
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.