aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_analyze.c
Commit message (Collapse)AuthorAge
* Add some heuristics to detect encrypted/obfuscated/proxied TLS flows (#2553)Ivan Nardi2024-09-24
| | | | | | | | | | | | Based on the paper: "Fingerprinting Obfuscated Proxy Traffic with Encapsulated TLS Handshakes". See: https://www.usenix.org/conference/usenixsecurity24/presentation/xue-fingerprinting Basic idea: * the packets/bytes distribution of a TLS handshake is quite unique * this fingerprint is still detectable if the handshake is encrypted/proxied/obfuscated All heuristics are disabled by default.
* Implemented algorithms for K-Nearest Neighbor Search (KNN) (#2554)Luca Deri2024-09-10
| | | | | | | | | | | | | | | | * Extended API with functions for vector similarity based on KD-trees https://en.wikipedia.org/wiki/K-d_tree ndpi_kd_tree* ndpi_kd_create(u_int num_dimensions); void ndpi_kd_free(ndpi_kd_tree *tree); void ndpi_kd_clear(ndpi_kd_tree *tree); bool ndpi_kd_insert(ndpi_kd_tree *tree, const double *data_vector, void *user_data); ndpi_kd_tree_result *ndpi_kd_nearest(ndpi_kd_tree *tree, const double *data_vector); u_int32_t ndpi_kd_num_results(ndpi_kd_tree_result *res); bool ndpi_kd_result_end(ndpi_kd_tree_result *res); double* ndpi_kd_result_get_item(ndpi_kd_tree_result *res, double **user_data); bool ndpi_kd_result_next(ndpi_kd_tree_result *res); void ndpi_kd_result_free(ndpi_kd_tree_result *res); double ndpi_kd_distance(double *a1, double *b2, u_int num_dimensions);
* bins: fix `ndpi_set_bin`, `ndpi_inc_bin` and `ndpi_get_bin_value` (#2536)Ivan Nardi2024-09-03
| | | | When the required slot is too big, use the latest/bigger available bin, not in the first one.
* Endian-independent implementation of IEEE 802.3 CRC32 (#2529)Vladimir Gavrilov2024-08-25
|
* Fix some warnings reported by CODESonar (#2227)Ivan Nardi2024-01-12
| | | | | | | | | | | | | | | | | | | Remove some unreached/duplicated code. Add error checking for `atoi()` calls. About `isdigit()` and similar functions. The warning reported is: ``` Negative Character Value help isdigit() is invoked here with an argument of signed type char, but only has defined behavior for int arguments that are either representable as unsigned char or equal to the value of macro EOF(-1). Casting the argument to unsigned char will avoid the undefined behavior. In a number of libc implementations, isdigit() is implemented using lookup tables (arrays): passing in a negative value can result in a read underrun. ``` Switching to our macros fix that. Add a check to `check_symbols.sh` to avoid using the original functions from libc.
* Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)Vladimir Gavrilov2024-01-02
| | | | | | | | | | | | | | | | | | | * Add IEC62056 (DLMS/COSEM) protocol dissector * Fix detection on big endian architectures * Update protocols.rst * Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c * Update pcap sample * Remove empty .out file * iec62056: add some documentation --------- Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
* fuzz: improve fuzzing coverage (#2239)Ivan Nardi2024-01-02
|
* Implemented ndpi_is_outlier() for detecting outliers using z-scoreLuca2023-12-28
|
* Implements ndpi_pearson_correlation for measuring how correlated are two seriesLuca Deri2023-12-27
|
* fuzz: extend fuzzing coverage (#2205)Ivan Nardi2023-12-11
|
* Add some fast CRC16 algorithms implementation (#2195)Vladimir Gavrilov2023-12-05
| | | | | | | | | * Add some fast CRC16 algorithms implementation * Update ndpi_crc.c * Move crc16 stuff to ndpi_analyze.c * IEEE C37.118: use new fast CRC-16/CCITT-FALSE implementation
* Fixed implicit u32 cast in `ndpi_data_min()` / `ndpi_data_max()`. (#2139)Toni2023-11-09
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added printf/fprintf replacement for some internal modules. (#1974)Toni2023-09-26
| | | | | | * logging is instead redirected to `ndpi_debug_printf` Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix compilation on Windows (#2072)Ivan Nardi2023-08-14
|
* Reworked ndpi_filter_xxx implementation using compressed bitmapsLuca Deri2023-08-14
|
* Add an heuristic to detect fully encrypted flows (#2058)Ivan Nardi2023-07-26
| | | | | | | | A fully encrypted session is a flow where every bytes of the payload is encrypted in an attempt to “look like nothing”. The heuristic needs only the very first packet of the flow. See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf A basic, but generic, inplementation of the popcpunt alg has been added
* fuzz: extend fuzzing coverage (#2052)Ivan Nardi2023-07-18
| | | | Added/merged some traces. Improved Socks identification
* Implemented Count-Min Sketch [count how many times a value has been observed]Luca Deri2023-07-13
| | | | | | | - ndpi_cm_sketch_init() - ndpi_cm_sketch_add() - ndpi_cm_sketch_count() - ndpi_cm_sketch_destroy()
* Numeric truncation at `ndpi_analyze.c` at lines 101, 104, 107, 110 (#1999)headshog2023-05-30
| | | | | | | | | | | * fixed numeric truncation error in ndpi_analyze.c * fixed numeric truncation error in ndpi_analyze.c x2 * fixed numeric truncation error in ndpi_analyze.c x3 * fixed numeric truncation error in ndpi_analyze.c and printf format * fixed tests
* fixed numeric truncation errorheadshog2023-05-20
|
* Implemented ndpi_predict_linear() for predicting a timeseries value overtimeLuca Deri2023-05-19
|
* Added missing checkLuca Deri2023-04-28
|
* fuzz: extend fuzzers coverage (#1952)Ivan Nardi2023-04-25
|
* Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hwLuca Deri2023-04-08
|
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Add some fuzzers to test other data structures. (#1870)Ivan Nardi2023-01-25
| | | | | | | Start using a dictionary for fuzzing (see: https://llvm.org/docs/LibFuzzer.html#dictionaries). Remove some dead code. Fuzzing with debug enabled is not usually a great idea (from performance POV). Keep the code since it might be useful while debugging.
* fuzz: fix memory allocation failure logic (#1867)Ivan Nardi2023-01-20
| | | | | We *do* want to have some allocation errors. Fix some related bugs Fix: 29be01ef
* Add some fuzzers to test algorithms and data structures (#1852)Ivan Nardi2023-01-17
| | | Fix some issues found with these new fuzzers
* fuzz: add a new fuzzer testing memory allocation failures (#1818)Ivan Nardi2022-12-06
| | | | | | | | | Try to fuzz error paths triggered by allocation errors. Fix some errors already found by this new fuzzer. Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to (not)compile specific code in fuzzing builds. See: https://llvm.org/docs/LibFuzzer.html
* Fixes on hw calculationLuca Deri2022-09-01
|
* Modified definition of hll_addLuca Deri2022-08-17
|
* bins: add support for 64bit bins (#1626)Ivan Nardi2022-07-03
|
* Removed superfluous ifdef'd includes. (#1519)Toni2022-04-19
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* Added ndpi_find_outliers() API call using Z-ScoreLuca Deri2022-04-04
|
* Update copyrightAlfredo Cardigliano2022-01-03
|
* Added example for finding similarities in RRDs using nDPI statistical APIsLuca Deri2021-12-04
|
* Demo tool for finding anomalies in RRD filesLuca Deri2021-12-03
|
* Implemented ndpi_ses_fitting() and ndpi_des_fitting()Luca Deri2021-10-12
| | | | for comuting the best alpha/beta values for exponential smoothing
* Fixed DES initialisationLuca Deri2021-10-12
|
* Fix some ndpi_malloc/ndpi_free pairs (#1298)Ivan Nardi2021-09-13
|
* Fix possible segfault/leakAlfredo Cardigliano2021-07-05
|
* Upgraded exponential smoothing to 64 bit valuesLuca Deri2021-06-18
|
* Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)Toni2021-03-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Try to detect one subprotocol if a detected protocol can have one. * This adds a performance overhead due to much more protocol detection routine calls. See #1148 for more information. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (1/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (2/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactored aimini HTTP detection parts (somehow related to #1148). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Added aimini client/server test pcap. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto * Adjusted Python code to conform to the changes made during the refactoring process. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implemented square erro rollup to avoid overflowLuca Deri2021-03-14
|
* Added double exponential smoothing implementationLuca2021-03-11
|
* Added single exponential smoothing APILuca Deri2021-03-11
| | | | | int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance); int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
* Added HW checksLuca Deri2021-03-01
|
* Added further HW checksLuca Deri2021-03-01
|
* Holt-Winters calculation improvementLuca Deri2021-02-27
|
Powered by cgit, Themed by Ted Yin.