aboutsummaryrefslogtreecommitdiff
path: root/configure.ac
Commit message (Collapse)AuthorAge
* Fix compilation on latest mac versions with external libraries (#2669)Ivan Nardi2025-01-15
| | | | | | Fix also Performance job on ubuntu-latest/24.04: see similar fix in 957a05050. Close: #2412
* Moded to 4.13Luca Deri2024-12-17
|
* Moved dev branch to 4.11Luca Deri2024-08-05
|
* Make the CI faster (#2475)Ivan Nardi2024-07-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | Without the `-fsanitize-memory-track-origins` flag, MSAN job is ~30% faster. Since this flag is useful only while debugging (and not to simply discover memory issues), avoid it on the CI. Note that, by default it is still enabled by default. Right now, MingW runs on *every* ubuntu builds: limit it only to the standard matrix (i.e. ubuntu 20.04, 22.04, 24.04 with default configuration), without any sanitizers (note that MingW doesn't support *san anyway). armhf job is by far the longest job in the CI: remove asan configuration to make it faster. Note that we already have a lot of different jobs (on x86_64) with some sanitizers, and that the other 2 jobs on arm/s390x don't have asan support anyway. If we really, really want a job with arm + asan we can add it as a async/scheduled job. Remove an old workaround for ubuntu jobs Avoid installing packages needed only for the documentation About `check_symbols.sh` script: even if uses the compiled library/objects, it basicaly only checks if we are using, in the source code, same functions that we shoudn't. We don't need to perform the same kind of check so many times..
* Domain Classification Improvements (#2396)Luca Deri2024-04-18
| | | | | | | | | | | | | | | | | | | * Added size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize); size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize); used to compress short strings such as domain names. This code is based on https://github.com/Ed-von-Schleck/shoco * Major code rewrite for ndpi_hash and ndpi_domain_classify * Improvements to make sure custom categories are loaded and enabled * Fixed string encoding * Extended SalesForce/Cloudflare domains list
* Limit git commit hash to 7 characters. (#2368)Toni2024-04-05
| | | | | * ensure consistency across all platforms Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disable `-Wno-unused-parameter -Wno-unused-function`. (#2358)Toni2024-04-03
| | | | | * unused parameters and functions pollute the code and decrease readability Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed gcc version checking. (#2360)Vitaly Lavrov2024-04-02
| | | Using the "test" utility instead of "[".
* Added support for roaring bitmap v3 (#2355)Luca Deri2024-03-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Integrated RoaringBitmap v3 * Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse * Fixes to ndpi_bitmap for new roaring library * Fixes for bitmap serialization * Fixed format * Warning fix * Conversion fix * Warning fix * Added check for roaring v3 support * Updated file name * Updated path * Uses clang-9 (instead of clang-7) for builds * Fixed fuzz_ds_bitmap64_fuse * Fixes nDPI printf handling * Disabled printf * Yet another printf fix * Cleaup * Fx for compiling on older platforms * Fixes for old compilers * Initialization changes * Added compiler check * Fixes for old compilers * Inline function is not static inline * Added missing include
* Enable `USE_GLOBAL_CONTEXT` via `CFLAGS` passed to the CC. (#2348)Toni2024-03-15
| | | | | | | * `ndpi_typedefs.h`: requires to include `ndpi_config.h` for the `HAVE_STRUCT_TIMESPEC` check That will never happen, because `USE_GLOBAL_CONTEXT` is defined inside `ndpi_config.h`. It's better to use `CFLAGS` to achieve the same. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disable `AX_PTHREAD` for MingW/MSYS builds. (#2338)Toni2024-03-07
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Allow multiple `struct ndpi_detection_module_struct` to share some state (#2271)Ivan Nardi2024-02-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the concept of "global context". Right now every instance of `struct ndpi_detection_module_struct` (we will call it "local context" in this description) is completely independent from each other. This provide optimal performances in multithreaded environment, where we pin each local context to a thread, and each thread to a specific CPU core: we don't have any data shared across the cores. Each local context has, internally, also some information correlating **different** flows; something like: ``` if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then flow2 (PeerC <-> PeerD) will be PROTOCOL_Y ``` To get optimal classification results, both flow1 and flow2 must be processed by the same local context. This is not an issue at all in the far most common scenario where there is only one local context, but it might be impractical in some more complex scenarios. Create the concept of "global context": multiple local contexts can use the same global context and share some data (structures) using it. This way the data correlating multiple flows can be read/write from different local contexts. This is an optional feature, disabled by default. Obviously data structures shared in a global context must be thread safe. This PR updates the code of the LRU implementation to be, optionally, thread safe. Right now, only the LRU caches can be shared; the other main structures (trees and automas) are basically read-only: there is little sense in sharing them. Furthermore, these structures don't have any information correlating multiple flows. Every LRU cache can be shared, independently from the others, via `ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`. It's up to the user to find the right trade-off between performances (i.e. without shared data) and classification results (i.e. with some shared data among the local contexts), depending on the specific traffic patterns and on the algorithms used to balance the flows across the threads/cores/local contexts. Add some basic examples of library initialization in `doc/library_initialization.md`. This code needs libpthread as external dependency. It shouldn't be a big issue; however a configure flag has been added to disable global context support. A new CI job has been added to test it. TODO: we should need to find a proper way to add some tests on multithreaded enviroment... not an easy task... *** API changes *** If you are not interested in this feature, simply add a NULL parameter to any `ndpi_init_detection_module()` calls.
* CI: update list of compilers (#2223)Ivan Nardi2023-12-20
| | | | | Try using latest gcc and clang versions. We still care about RHEL7: since handling a RHEL7 runner on GitHub is quite complex, let try to use a similar version of gcc, at least
* Move from PCRE to PCRE2 (#2134)Christian Marangi2023-11-01
| | | | | | | | | | Move from PCRE to PCRE2. PCRE is EOL and won't receive any security updates anymore. Convert to PCRE2 by converting any function PCRE2 new API. Also update every entry in github workflows and README to point to the new configure flag. (--with-pcre2) Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
* Moved to 4.9Luca Deri2023-10-23
|
* Minor warning fixesLuca Deri2023-09-05
|
* Added ndpi_domain_classify_XXX(0 APILuca Deri2023-08-26
|
* Removed -fno-color-diagnostics not available on all platformsLuca Deri2023-07-13
|
* Implemented Count-Min Sketch [count how many times a value has been observed]Luca Deri2023-07-13
| | | | | | | - ndpi_cm_sketch_init() - ndpi_cm_sketch_add() - ndpi_cm_sketch_count() - ndpi_cm_sketch_destroy()
* Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)Toni2023-06-12
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved helper scripts. (#1986)Toni2023-05-28
| | | | | * added additional (more restrictive) checks Signed-off-by: lns <matzeton@googlemail.com>
* configure: add an option to enable debug build, i.e `-g` (#1929)Ivan Nardi2023-04-06
| | | | | Add this new flag to one CI job, to test it Close #1925
* Fixed missing AS_HELP_STRING in configure.ac. (#1893)Toni2023-03-01
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)Ivan Nardi2023-02-13
| | | | | | | | | | | | | | | | | | | | | | | Latest RoaringBitmap version (introduced with bf413afb) triggers a new warning with GCC-7: ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s autoreconf: Entering directory `.' [...] third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] static inline int array_container_cardinality(const array_container_t *array) { ^~~~~~ third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] const array_container_t *container2) { [..] ``` The core issue is that `no_sanitize` attribute is defined only for GCC >= 8. That breaks the CI since we still use GCC-7 and `-Werror`: add a simple workaround. Fix compilation on Windows
* Moved to 4.7Luca Deri2023-02-01
|
* fuzz: add fuzzer testing nDPI (initial) configurations (#1830)Ivan Nardi2022-12-23
| | | | | | | | | | | | | | | | | The goal of this fuzzer is to test init and deinit of the library, with different configurations. In details: * random memory allocation failures, even during init phase * random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()` * random LRU caches sizes * random bitmask of enabled protocols * random parameters of `ndpi_set_detection_preferences()` * random initialization of opportunistic TLS * random load/don't load of configuration files This new fuzzer is a C++ file, because it uses `FuzzedDataProvider` class (see https://github.com/google/fuzzing/blob/master/docs/split-inputs.md). Note that the (existing) fuzzers need to be linked with C++ compiler anyway, so this new fuzzer doesn't add any new requirements.
* Add support for LTO and Gold linker (#1812)Ivan Nardi2022-12-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit add (optional) support for Link-Time-Optimization and Gold linker. This is the first, mandatory step needed to make nDPI compliant with "introspector" sanitizer requirements in OSS-Fuzz: see https://github.com/google/oss-fuzz/issues/8939 Gold linker is not supported by Windows and by macOS, so this feature is disabled by default. It has been enable in CI in two linux targets ("latest" gcc and clang). Fix some warnings triggered by LTO. The changes in `src/lib/ndpi_serializer.c` seams reasonable. However, the change in `tests/unit/unit.c` is due to the following warning, which seems to be a false positive. ``` unit.c: In function ‘serializerUnitTest’: ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here 67 | ndpi_serializer serializer, deserializer; | ^ ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here 67 | ndpi_serializer serializer, deserializer; ``` Since this warning is triggered only with an old version of gcc and `tests/unit/unit.c` is used only during the tests, the easiest fix has been applied. Some (unknown to me) combinations of OS and compiler trigger the following warnings at linker time (with sanitizer and gold linker) ``` /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm' [..] ``` I have not found any references to this kind of message, with the only exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975 which seems to suggest that these messages can be safely ignored. In any case, the compilation results are sound. Fix `clean` target in the Makefile in the `example` directory. In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error (as always, when the gold linker is involved...). It's come out that the culprit was the `tempnam` function: the code has been changed to use `tmpfile` instead. No sure why... :( Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a` anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds as workaround for the "missing dependencies errors" described in https://github.com/google/oss-fuzz/issues/8939
* Moved `NDPI_API_VERSION` to `src/include/ndpi_define.h.in` where it belongs.Toni Uhlig2022-09-21
| | | | | | | That way, we can make `src/include/ndpi_api.h` independent of autoconf. MSBuild will like that. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Initial work towards custom BPF protocol definitionLuca2022-09-16
|
* Removed maxmind checkLuca Deri2022-08-31
|
* Improved nDPI JSON serialization. (#1689)Toni2022-08-02
| | | | | | | | * fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors * introduced portable version of gmtime_r aka ndpi_gmtime_r * do as most as possible of the serialization work in ndpi_utils.c * use flow2json in ndpiReader Signed-off-by: lns <matzeton@googlemail.com>
* Add support for Memory sanitizer (#1652)Ivan Nardi2022-07-24
|
* Restored -O2 in default buildsLuca2022-07-14
|
* Do not interfere with CFLAGS/LDFLAGS env anymore. (#1659)Toni2022-07-13
| | | | | | | * CI fixes * some build systems do not like that (e.g. OpenWrt) * fixed some rrdtool related build warnings/errors Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Enhances gprof usage. (#1651)Toni2022-07-08
| | | | | * gprof results were incorrectly displayed Signed-off-by: lns <matzeton@googlemail.com>
* Moved to 4.5Luca Deri2022-07-06
|
* Generate profiling results as PNG.Toni Uhlig2022-07-03
| | | | | | * use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* gprof test/CI integrationlns2022-07-03
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added TSAN support. (#1613)Toni2022-07-03
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added gprof CPU/HEAP profiling support. (#1592)Toni2022-06-12
| | | | | * Some small auto{conf,make} improvements Signed-off-by: lns <matzeton@googlemail.com>
* Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is ↵Toni2022-06-06
| | | | | | | empty. (#1584) - This is not a fix for #1494, just an improvment. Signed-off-by: lns <matzeton@googlemail.com>
* Updated README.md (#1562)Toni2022-05-30
| | | | | | | * make check great again (not so much) * make doc/doc-view * CI updates Signed-off-by: lns <matzeton@googlemail.com>
* Use Doxygen to generate the API documentation. (#1558)Toni2022-05-29
| | | | | * Integrated Doxygen documentation into Sphinx Signed-off-by: lns <matzeton@googlemail.com>
* Fixes compilation issues on RedHat systemsLuca Deri2022-05-26
|
* Replaced obsolete autoconf macros. (#1553)Toni2022-05-26
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).Toni Uhlig2022-05-16
| | | | | | * Related to #1545, #1494 and #1189 as well Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Replaced ndpiReader's libjson-c support with libnDPI's internal ↵Toni2022-05-07
| | | | | | | | | | | serialization interface. (#1535) * Fixes #1528 * Serialization Interface should also fuzzed * libjson-c may only be used in the unit test to verify the internal serialization interface * Serialization Interface supports tlv(broken), csv and json * Unit test does work again and requires libjson-c Signed-off-by: lns <matzeton@googlemail.com>
* Updated RRD dependenciesLuca Deri2022-05-03
|
* Minor fix.Zied Aouini2022-04-15
|
Powered by cgit, Themed by Ted Yin.