aboutsummaryrefslogtreecommitdiff
path: root/.github
Commit message (Collapse)AuthorAge
* Move from PCRE to PCRE2 (#2134)Christian Marangi2023-11-01
| | | | | | | | | | Move from PCRE to PCRE2. PCRE is EOL and won't receive any security updates anymore. Convert to PCRE2 by converting any function PCRE2 new API. Also update every entry in github workflows and README to point to the new configure flag. (--with-pcre2) Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
* STUN: major code rework (#2116)Ivan Nardi2023-10-30
| | | | | | | | | | | | Try to have a faster classification, on first packet; use standard extra dissection data path for sub-classification, metadata extraction and monitoring. STUN caches: * use the proper confidence value * lookup into the caches only once per flow, after having found a proper STUN classification Add identification of Telegram VoIP calls.
* PR templateAlfredo Cardigliano2023-10-19
|
* Included Gambling website data from the Polish `hazard.mf.gov.pl` list (#2041)snicket21002023-07-14
| | | | | | | | | | | | | * Refreshed the Belgium Gambling Site list data Unfortunately some hostnames have been removed from that list, which means they are disappearing from the `ndpi_gambling_match.c.inc` file as well. * build: added `libxml2-utils` (for `xmllint`) * Included Gambling website data from the Polish `hazard.mf.gov.pl` list The list contains over 30k gambling website hostnames as of today.
* Added coverage targets to `Makefile.am` for convenience. (#2039)Toni2023-07-07
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)Toni2023-05-20
| | | | | * added CI check Signed-off-by: lns <matzeton@googlemail.com>
* Improve detection of crawlers/bots (#1968)Ivan Nardi2023-05-09
| | | Add support for Facebook crawler
* CI: fix `Performance` job (#1936)Ivan Nardi2023-04-09
| | | Fix: 7714507f
* configure: add an option to enable debug build, i.e `-g` (#1929)Ivan Nardi2023-04-06
| | | | | Add this new flag to one CI job, to test it Close #1925
* Rework CI jobs to try reducing CI duration (#1903)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | CI duration is quite long: the longest jobs is the "Performance" one. Try to reduce the overall duration: that job (and some others) will not be triggered for each PR/commit anymore, but asynchronously, once a day (this scheduling seems right since the frequency of the PR/commits in the project). It should be possibly to trigger them manually, via GUI, anyway. Remove two identical jobs; we already tests ASAN with 4 different compilers. After 9eff0754 it is safe to reduce fuzzing time. Bottom line: try to have as upper-time of CI tests the duration of the fuzzing jobs
* Update GitHub runners versions (#1889)Ivan Nardi2023-02-14
| | | See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* fuzz: add a new fuzzer to test serialization/deserialization code (#1876)Ivan Nardi2023-01-27
| | | | | | | | | | Autodetecting the needed buffer size is quite complex (especially with float/double values) so it is mandatory to properly check for `ndpi_snprintf` truncation. These issues have been undetected so far probably because the default buffer is big enough for all common cases. Add an example of usage of `ndpi_deserialize_clone_all()` (taken from `ntopng`)
* Add some fuzzers to test algorithms and data structures (#1852)Ivan Nardi2023-01-17
| | | Fix some issues found with these new fuzzers
* fuzz: add fuzzer testing nDPI (initial) configurations (#1830)Ivan Nardi2022-12-23
| | | | | | | | | | | | | | | | | The goal of this fuzzer is to test init and deinit of the library, with different configurations. In details: * random memory allocation failures, even during init phase * random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()` * random LRU caches sizes * random bitmask of enabled protocols * random parameters of `ndpi_set_detection_preferences()` * random initialization of opportunistic TLS * random load/don't load of configuration files This new fuzzer is a C++ file, because it uses `FuzzedDataProvider` class (see https://github.com/google/fuzzing/blob/master/docs/split-inputs.md). Note that the (existing) fuzzers need to be linked with C++ compiler anyway, so this new fuzzer doesn't add any new requirements.
* Add support for LTO and Gold linker (#1812)Ivan Nardi2022-12-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit add (optional) support for Link-Time-Optimization and Gold linker. This is the first, mandatory step needed to make nDPI compliant with "introspector" sanitizer requirements in OSS-Fuzz: see https://github.com/google/oss-fuzz/issues/8939 Gold linker is not supported by Windows and by macOS, so this feature is disabled by default. It has been enable in CI in two linux targets ("latest" gcc and clang). Fix some warnings triggered by LTO. The changes in `src/lib/ndpi_serializer.c` seams reasonable. However, the change in `tests/unit/unit.c` is due to the following warning, which seems to be a false positive. ``` unit.c: In function ‘serializerUnitTest’: ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here 67 | ndpi_serializer serializer, deserializer; | ^ ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here 67 | ndpi_serializer serializer, deserializer; ``` Since this warning is triggered only with an old version of gcc and `tests/unit/unit.c` is used only during the tests, the easiest fix has been applied. Some (unknown to me) combinations of OS and compiler trigger the following warnings at linker time (with sanitizer and gold linker) ``` /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm' [..] ``` I have not found any references to this kind of message, with the only exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975 which seems to suggest that these messages can be safely ignored. In any case, the compilation results are sound. Fix `clean` target in the Makefile in the `example` directory. In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error (as always, when the gold linker is involved...). It's come out that the culprit was the `tempnam` function: the code has been changed to use `tmpfile` instead. No sure why... :( Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a` anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds as workaround for the "missing dependencies errors" described in https://github.com/google/oss-fuzz/issues/8939
* CI: GitHub is updating Ubuntu runners (#1817)Ivan Nardi2022-12-03
| | | | | | | | | | | GitHub is moving `ubuntu-latest` to `ubuntu-22.04`: update our dependencies. See: https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/ This is the reason of the recent random failures in CI. Update "newest" tested gcc to gcc-12. Fix a memory error introduced in 557bbcfc5a5165c9eb43bbdd78435796239cd3c9
* Extend CodeQL configuration and fix two warnings (#1816)Ivan Nardi2022-12-02
| | | | | | | | | ``` Call to `memset` may be deleted Comparison of narrow type with wide type in loop condition ``` See https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-yang.pdf for the new portable function `secure_memzero`.
* Add CodeQL workflow (#1815)Alvaro Muñoz2022-12-02
|
* Add a new fuzzer for QUIC (#1800)Ivan Nardi2022-12-01
| | | | | | | | | | | | QUIC packets are encrypted/obfuscated; that means that we need to decrypt them before parsing the real (TLS) message. Fuzzing is not effective here, since a random buffer is hardly a valid encrypted QUIC packet. Add a new fuzzer, testing *decrypted* QUIC packets. Add a basic corpus. Fix a few bugs already found by this fuzzer.
* GitHub Actions: fix a warning (#1807)Ivan Nardi2022-11-22
| | | | | | | ``` The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ ```
* GitHub Actions: update to Node.js 16 actions (#1801)Ivan Nardi2022-11-13
| | | | | | Fix warnings on recent CI results; example: https://github.com/ntop/nDPI/actions/runs/3455588082 See: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
* Build ndpiReader and run regression tests.Toni Uhlig2022-09-21
| | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed VS2019 build.segfault2022-09-21
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.mark andrews2022-09-21
| | | | | | * add CI support via MSBuild Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Update Github CI OS and compilers (#1733)Ivan Nardi2022-09-12
| | | | | | | | | | ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04). macos-latest points to macos-11, so it makes sense to test macos-12, too. About the compilers, the general idea it to test the oldest and the newest versions easily available: switch to gcc-11 and clang-14. See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* Add TiVoConnect dissector. Fixes #1697. (#1699)Toni2022-08-08
| | | | | * added static assert if supported, to complain if the flow struct changes Signed-off-by: lns <matzeton@googlemail.com>
* Add support for Memory sanitizer (#1652)Ivan Nardi2022-07-24
|
* Do not interfere with CFLAGS/LDFLAGS env anymore. (#1659)Toni2022-07-13
| | | | | | | * CI fixes * some build systems do not like that (e.g. OpenWrt) * fixed some rrdtool related build warnings/errors Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Generate profiling results as PNG.Toni Uhlig2022-07-03
| | | | | | * use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* gprof test/CI integrationlns2022-07-03
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added TSAN support. (#1613)Toni2022-07-03
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Force roaring bitmap to use ndpi memory wrappers. (#1569)Toni2022-05-31
| | | | | | | | GCC analyzer won't complain about possible use-after-free (false positive). * tests/do.sh prints word diff's only once and not the same over and over again * sync unit tests Signed-off-by: lns <matzeton@googlemail.com>
* Updated README.md (#1562)Toni2022-05-30
| | | | | | | * make check great again (not so much) * make doc/doc-view * CI updates Signed-off-by: lns <matzeton@googlemail.com>
* Support word diff for tests/do.sh for better readability. (#1565)Toni2022-05-30
| | | | | * Sync unit tests Signed-off-by: lns <matzeton@googlemail.com>
* Use Doxygen to generate the API documentation. (#1558)Toni2022-05-29
| | | | | * Integrated Doxygen documentation into Sphinx Signed-off-by: lns <matzeton@googlemail.com>
* Added script to compare and verify the output of `make dist'. (#1551)Toni2022-05-26
| | | | | | | This fixes some build/test issues resulting when using tarballs. * nDPI uses autotools (especially autoconf) in a wrong way, see #1163 Signed-off-by: lns <matzeton@googlemail.com>
* Fixed windows-latest build error. (#1552)Toni2022-05-26
| | | | | * The warning itself looks like a bug Signed-off-by: lns <matzeton@googlemail.com>
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* Add HOWTO Python.aouinizied2022-03-22
|
* Fix python bindings CI.aouinizied2022-03-22
|
* Complete rework of nDPI Python bindings (cffi API, automatic generation, ↵aouinizied2022-03-22
| | | | packaging and CI integration)
* Improved ASN/IP update scripts and CI integration. (#1474)Toni2022-03-09
| | | | | * CI will print a warning if ASN/IP addresses changed. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implement CI on Windows. (#1483)Zied Aouini2022-03-09
| | | | * Switch fail fast to True. * Windows CI.
* Errors fixed (#1482)Vitaly Lavrov2022-03-08
| | | | | | | | | | | | | | | Fixed errors for bigendian platforms in ndpiReader. All address and port comparisons and hash calculations are done with endian in mind. The get_ndpi_flow_info() function searched for an existing flow for the forward and reverse direction of the packet. The ndpi_workflow_node_cmp() function looked for a flow regardless of the packet's direction. This is what led to an error in determining the direction of transmission of the packet. Fixed error in "synscan" test: the number of packets in the forward and reverse direction is incorrectly defined (verified via tcpdump). Fixed bug with icmp protocol checksum check for big endian platforms.
* Added autoconf option `--enable-tls-sigs'. (#1471)Toni2022-03-03
| | | | | | * Testing more code in CI environments. * Added strict option checking for `./configure' in CI environments. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)Toni2022-02-28
| | | | | * CI job generates a coverage report Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Restore a unit test result (#1403)Ivan Nardi2022-01-13
| | | | | Deleted, probably by mistake, in 406ac7e8 Fix Makefile and add compilation of `rrdtool` in CI tests
* Run fuzzing tests also on push event (like the CI tests) (#1400)Ivan Nardi2022-01-12
|
Powered by cgit, Themed by Ted Yin.