aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_serializer.c
Commit message (Collapse)AuthorAge
* Update buff_diffAlfredo Cardigliano2024-10-31
|
* Fix blocks with inner-json modeAlfredo Cardigliano2024-10-31
|
* Add new json serialization type ndpi_serialization_format_inner_jsonAlfredo Cardigliano2024-10-31
|
* Remove unused code. (#2450)Toni2024-05-21
| | | | | | | | | * some `#ifdef`ed code dates back to 2019, 2020 and 2021 * some function signatures were still present in `ndpi_main.h` which may cause linker errors for libnDPI dependee's * return an error while trying to serialize a double instead of `fprintf(stderr, ...)` Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disable `-Wno-unused-parameter -Wno-unused-function`. (#2358)Toni2024-04-03
| | | | | * unused parameters and functions pollute the code and decrease readability Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix some warnings reported by CODESonar (#2227)Ivan Nardi2024-01-12
| | | | | | | | | | | | | | | | | | | Remove some unreached/duplicated code. Add error checking for `atoi()` calls. About `isdigit()` and similar functions. The warning reported is: ``` Negative Character Value help isdigit() is invoked here with an argument of signed type char, but only has defined behavior for int arguments that are either representable as unsigned char or equal to the value of macro EOF(-1). Casting the argument to unsigned char will avoid the undefined behavior. In a number of libc implementations, isdigit() is implemented using lookup tables (arrays): passing in a negative value can result in a read underrun. ``` Switching to our macros fix that. Add a check to `check_symbols.sh` to avoid using the original functions from libc.
* Fix for buffer overflow in serializationLuca2023-10-11
|
* Serialization fixLuca Deri2023-10-11
|
* Add support for multiline jsonAlfredo Cardigliano2023-07-12
|
* Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)Toni2023-05-20
| | | | | * added CI check Signed-off-by: lns <matzeton@googlemail.com>
* Updated (C)Luca Deri2023-04-28
|
* fuzz: extend fuzz coverage (#1888)Ivan Nardi2023-02-16
|
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* fuzz: add a new fuzzer to test serialization/deserialization code (#1876)Ivan Nardi2023-01-27
| | | | | | | | | | Autodetecting the needed buffer size is quite complex (especially with float/double values) so it is mandatory to properly check for `ndpi_snprintf` truncation. These issues have been undetected so far probably because the default buffer is big enough for all common cases. Add an example of usage of `ndpi_deserialize_clone_all()` (taken from `ntopng`)
* fuzz: add a new fuzzer testing memory allocation failures (#1818)Ivan Nardi2022-12-06
| | | | | | | | | Try to fuzz error paths triggered by allocation errors. Fix some errors already found by this new fuzzer. Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to (not)compile specific code in fuzzing builds. See: https://llvm.org/docs/LibFuzzer.html
* Add support for LTO and Gold linker (#1812)Ivan Nardi2022-12-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit add (optional) support for Link-Time-Optimization and Gold linker. This is the first, mandatory step needed to make nDPI compliant with "introspector" sanitizer requirements in OSS-Fuzz: see https://github.com/google/oss-fuzz/issues/8939 Gold linker is not supported by Windows and by macOS, so this feature is disabled by default. It has been enable in CI in two linux targets ("latest" gcc and clang). Fix some warnings triggered by LTO. The changes in `src/lib/ndpi_serializer.c` seams reasonable. However, the change in `tests/unit/unit.c` is due to the following warning, which seems to be a false positive. ``` unit.c: In function ‘serializerUnitTest’: ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here 67 | ndpi_serializer serializer, deserializer; | ^ ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here 67 | ndpi_serializer serializer, deserializer; ``` Since this warning is triggered only with an old version of gcc and `tests/unit/unit.c` is used only during the tests, the easiest fix has been applied. Some (unknown to me) combinations of OS and compiler trigger the following warnings at linker time (with sanitizer and gold linker) ``` /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm' [..] ``` I have not found any references to this kind of message, with the only exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975 which seems to suggest that these messages can be safely ignored. In any case, the compilation results are sound. Fix `clean` target in the Makefile in the `example` directory. In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error (as always, when the gold linker is involved...). It's come out that the culprit was the `tempnam` function: the code has been changed to use `tmpfile` instead. No sure why... :( Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a` anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds as workaround for the "missing dependencies errors" described in https://github.com/google/oss-fuzz/issues/8939
* Serializer supports double for JSON and CSV only for the time being (TLV ↵Alfredo Cardigliano2022-09-01
| | | | needs to be extended)
* Support serialization of double-precision floating-point numbers. Fixes #1702.lns2022-08-24
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added ability to return risk info in JSON format in ndpi_get_flow_risk_info()Luca2022-05-30
|
* Add ndpi_json_string_escape to the APIAlfredo Cardigliano2022-04-26
|
* Removed superfluous ifdef'd includes. (#1519)Toni2022-04-19
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* Provide some API functions for convenience. (#1456)Toni2022-02-25
| | | | | * Extended JSON serializsation: risk, risk score, confidence Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added ndpi_serialize_string_string_len() APi callLuca Deri2022-02-08
| | | | Fixed CSV string serialization
* Fix ndpi_serialize_string_int64Alfredo Cardigliano2022-01-21
|
* Update copyrightAlfredo Cardigliano2022-01-03
|
* Optimized ndpi_serialize_string_int32Luca Deri2021-12-07
|
* Remove printf when not requiredAlfredo Cardigliano2021-12-01
|
* Improve JSON serializationAlfredo Cardigliano2021-12-01
|
* Improved CSV serializationLuca Deri2021-11-30
|
* Fixed seriealizer unitLuca Deri2021-11-30
|
* Added ndpi_serializer_skip_header() serialization APILuca Deri2021-11-26
|
* Return 0 on ndpi_init_serializer_ll success for consistencyAlfredo Cardigliano2021-10-12
|
* Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)Ivan Nardi2021-08-20
| | | | | | | | | | | | | | | | Fix all the warnings. Getting rid of "-Wno-unused-parameter" is quite complex because some parameters usage depends on compilation variable (i.e. `--enable-debug-messages`). The "-Werror" flag has been added only in Travis builds to avoid breaking the builds to users using uncommon/untested OS/compiler/enviroment. Tested on: * x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12 * x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag) * Raspberry 4; Debian 10.10; gcc 8.3.0
* Add ndpi_serialize_binary_boolean for consistency. Fix comments.Alfredo Cardigliano2021-03-04
|
* (C) UpdateLuca Deri2021-01-07
|
* Handle list items in ndpi_deserialize_get_single_sizeAlfredo Cardigliano2020-10-15
|
* Add serialization of values list in TLVAlfredo Cardigliano2020-10-15
|
* Merge pull request #1017 from lnslbrty/fix/mingw-xcompileLuca Deri2020-09-20
|\ | | | | Added support for mingw xcompile.
| * Using NDPI_I64_FORMAT, NDPI_U64_FORMAT format string to differentiate ↵Toni Uhlig2020-09-18
| | | | | | | | | | | | between Windows and non-Windows. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
| * Fixed shlib xcompile for x86_64-w64-mingw32Toni Uhlig2020-09-08
| | | | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* | Commented unused functionLuca2020-09-13
| |
* | Temporarily keep using snprintfAlfredo Cardigliano2020-09-11
| |
* | Replace snprintf with ndpi_snappendAlfredo Cardigliano2020-09-11
| |
* | Add missing low-level serializer calls to the APIAlfredo Cardigliano2020-09-09
|/
* Add start_of_block/end_of_block support to TLVAlfredo Cardigliano2020-09-04
|
* Major rework of QUIC dissectorNardi Ivan2020-08-21
| | | | | Improve support for GQUIC (up to Q046) and add support for Q050 and (IETF-)QUIC Still no sub-classification for Q050 and QUIC
* Add API ndpi_serializer_get_formatAlfredo Cardigliano2020-06-16
|
* Handle EOR in TLV testAlfredo Cardigliano2020-06-06
|
* Support for multiple records in CSV serializationAlfredo Cardigliano2020-06-05
|
Powered by cgit, Themed by Ted Yin.