aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_main.c
Commit message (Collapse)AuthorAge
* Compilation fixLuca Deri2021-10-05
|
* Numeric IPs are not considered for DGA checksLuca Deri2021-10-05
|
* Improved DGA detection for skipping potential DGAs of known/popular domain namesLuca Deri2021-10-05
|
* Remove `struct ndpi_packet_struct` from `struct ndpi_flow_struct` (#1319)Ivan Nardi2021-10-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet") in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid saving dissection information of "current packet" into the "flow" state, i.e. in the flow management table. The nDPI detection module processes only one packet at the time, so it is safe to save packet dissection information in `struct ndpi_detection_module_struct`, reusing always the same "packet" instance and saving a huge amount of memory. Bottom line: we need only one copy of "packet" (for detection module), not one for each "flow". It is not clear how/why "packet" ended up in "flow" in the first place. It has been there since the beginning of the GIT history, but in the original OpenDPI code `struct ipoque_packet_struct` was embedded in `struct ipoque_detection_module_struct`, i.e. there was the same exact situation this commit wants to achieve. Most of the changes in this PR are some boilerplate to update something like "flow->packet" into something like "module->packet" throughout the code. Some attention has been paid to update `ndpi_init_packet()` since we need to reset some "packet" fields before starting to process another packet. There has been one important change, though, in ndpi_detection_giveup(). Nothing changed for the applications/users, but this function can't access "packet" anymore. The reason is that this function can be called "asynchronously" with respect to the data processing, i.e in context where there is no valid notion of "current packet"; for example ndpiReader calls it after having processed all the traffic, iterating the entire session table. Mining LRU stuff seems a bit odd (even before this patch): probably we need to rethink it, as a follow-up.
* Added -a <num> to ndpiReader for generating OPNsense configurationLuca Deri2021-10-04
| | | | See https://github.com/ntop/opnsense
* Remove `detected_protocol_stack` field from `ndpi_packet_struct` (#1317)Ivan Nardi2021-09-29
| | | | | | | | | | | | | This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`: * at the very beginning of packet dissection, the value saved in `flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack` (via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`) * every time we update `flow->detected_protocol_stack` we update `packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()` -> `ndpi_int_change_packet_protocol()`) These two fields are always in sync: keeping the same value in two different places is useless.
* FTP: fix support for START-TLS sessionsNardi Ivan2021-09-21
| | | | | | When TLS-over-FTP is used, the credentials are encrypted. So we must not wait for the username and the password commands, otherwise we elaborate a lot of packets for nothing.
* Fix parsing of ipv6 packets with extension headersNardi Ivan2021-09-19
| | | | | | | | Decoding of ipv6 traffic with extension headers was completely broken, since the beginning of the L4 header was always set to a wrong value. Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first one and drop the others.
* Small fixes after latest commits (#1308)Ivan Nardi2021-09-18
|
* A (tiny) effort to reduce memory usage per flow (#1304)Ivan Nardi2021-09-18
| | | | | | | Follow-up of 22241a1d Only trivial changes: * remove completely unused fields * remove fields only written (but never read) * CSGO protocol only handles UDP traffic
* DNS dissection fixesLuca Deri2021-09-17
|
* Progetto esame Gestione di Reti - Debora Cerretini (#1290)deboracerretini2021-09-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Hangout: fix usage of STUN function (#1299)Ivan Nardi2021-09-13
|
* Update the list of default ports for QUIC protocol (#1297)Ivan Nardi2021-09-13
| | | There are no reasons to register UDP/80 as a default port for QUIC
* Added new risk for clear text credentialsLuca Deri2021-09-10
|
* Add Cassandra protocol dissector (#1285)lucasbaile2021-09-09
| | | Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
* Fix access to some TLS fields in flow structure (#1277)Ivan Nardi2021-08-20
| | | | Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by QUIC (i.e UDP...), too.
* Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)Ivan Nardi2021-08-20
| | | | | | | | | | | | | | | | Fix all the warnings. Getting rid of "-Wno-unused-parameter" is quite complex because some parameters usage depends on compilation variable (i.e. `--enable-debug-messages`). The "-Werror" flag has been added only in Travis builds to avoid breaking the builds to users using uncommon/untested OS/compiler/enviroment. Tested on: * x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12 * x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag) * Raspberry 4; Debian 10.10; gcc 8.3.0
* Fixed Mingw64 build, SonerCloud-CI and more. (#1273)Toni2021-08-18
| | | | | | | | | | | | | * Added ARM build and unit test run for SonarCloud-CI. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Fixed Mingw64 build. * adapted to SonarCloud-CI workflow * removed broken and incomplete Windows example (tested on VS2017/VS2019) * removed unnecessary include (e.g. pthread.h for the library which does not make use of it) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed some invalid TLS guessesLuca Deri2021-08-17
|
* Added extraction of hostname in SMTPLuca Deri2021-08-11
| | | | Fixed mail incalid subprotocol calculation
* Clode cleanup (after last merge)Luca Deri2021-08-08
|
* Added entropy calculation to check for suspicious (encrypted) payload. (#1270)Toni2021-08-08
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Added TLS fatal alert flow riskLuca Deri2021-08-07
|
* Added ability to report whether a protocol is encryptedLuca Deri2021-08-07
| | | | | | | | | Added new API calls - u_int8_t ndpi_is_valid_protoId(u_int16_t protoId); Cleaned up code and organized in a function. - u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto); Check whether a protocol is encrypted or not based on master and app protocols
* Implemented ALPN automa for checking uncommon ALPNsLuca Deri2021-07-24
|
* Reworked flow risk implementationLuca Deri2021-07-23
|
* Fixed risk mask implementationLuca Deri2021-07-23
|
* Implementation of flow risk eception (work in progress)Luca Deri2021-07-22
|
* Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)Toni2021-07-19
| | | | | | * validates client/server hello TLS extensions * inspects content for some extensions Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated download category nameLuca Deri2021-07-16
|
* Bug fix. (#1250)Vitaly Lavrov2021-07-16
| | | Restore part of the fields flow->packet after cleaning flow structure.
* TLS Risks - Certificate Validity Too Long (#1239)pacant2021-07-14
| | | | | | | | | | | * Added flow risk: TLS certificate too long * Added flow risk: TLS certificate too long * Date for TLS limit added * TLS certificate check fixed Co-authored-by: pacant <a.pace97@outlook.com>
* Added AVAST SecureDNS protocol. (#1244)Toni2021-07-14
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* ahoсorasick. Code review. Part 2. (#1236)Vitaly Lavrov2021-07-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplified the process of adding lines to AC_AUTOMATA_t. Use the ndpi_string_to_automa() function to add patterns with domain names. For other cases can use ndpi_add_string_value_to_automa(). ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding and compare data in a case insensitive manner. For mandatory pattern comparison from the end of the line, the "ac_pattern.rep.at_end=1" flag is used. This eliminated unnecessary conversions to lowercase and adding "$" for end-of-line matching in domain name patterns. ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified. ac_domain_match_handler() looks for the template with the highest domain level. For special cases it is possible to manually specify the domain level. Added test for checking ambiguous domain names like: - short.weixin.qq.com is QQ, not Wechat - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook If you specify a NULL handler when creating the AC_AUTOMATA_t structure, then a pattern with the maximum length that satisfies the search conditions will be found (exact match, from the beginning of the string, from the end of the string, or a substring). Added debugging for ac_automata_search. To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG). The search will display "name" and a list of matching patterns. Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates and which templates were found. The ac_automata_dump() prototype has been changed. Now it outputs data to a file. If it is specified as NULL, then the output will be directed to stdout. If you need to get data as a string, then use open_memstream(). Added the ability to run individual tests via the do.sh script
* Fixed false positives in Z39.50Luca Deri2021-07-09
|
* Updated category labelsLuca Deri2021-07-08
|
* Fix for #1230 (#1235)Vitaly Lavrov2021-07-07
| | | | | | | | | | | | | | | * Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)" This reverts commit 58665e93a98d014b53d131b2481ccab074efc9ff. * Checking the return code after calling ndpi_match_string_subprotocol() ndpi_api.h: Description of the returned error codes for the ndpi_match_string_subprotocol() function. If the ndpi_match_string_subprotocol() function returned an error, then return NDPI_PROTOCOL_UNKNOWN. http: The "Content-type" header is only checked if it is not empty.
* Improved RTSP detection the second. (#1232) (#1233)Toni2021-07-07
| | | | | | | * RTSP is no subprotocol of HTTP (most of the time) * detection patterns should stay in rtsp.c * set detected HTTP protocol only if at least a valid HTTP method detected Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved RTSP via HTTP detection. (#1232)Toni2021-07-06
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix return value of ndpi_match_string_subprotocol() (#1230)Ivan Nardi2021-07-06
|
* Added protocols to HTTP as subprotocols. (#1228)Toni2021-07-05
| | | | | | * if HTTP was detected, all of those defined subprotocol detection routines should called Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improve content match lists (#1226)Ivan Nardi2021-07-05
| | | | | | Add support for Likee app (https://likee.video/) and GitLab Improve detection of Google domains ("gtv1.com" added in 6dd42d19f was likely a typo) Improve Snapchat and Tiktok detection
* Removed ht_hash as it is not used anymore. (#1220)Toni2021-06-29
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added Z39.50 protocol. (#1219)Toni2021-06-29
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* reconcile PPSTREAM protocol category with content_match declarations (#1218)vpiserchia2021-06-28
| | | | | | | * reconcile PPSTREAM protocol category with content_match declarations * remove duplicate declaration for PPSTREAM protocol Co-authored-by: vpiserchia <vito.piserchia@dreamlab.net>
* QUIC: add basic support for fragmented Client Hello (#1216)Ivan Nardi2021-06-24
| | | | Only in-order and non overlapping fragments are handled See #1195
* Speed and memory size optimisation (#1214)Vitaly Lavrov2021-06-23
| | | | | | | Removed bigram_automata, impossible_bigram_automata, trigram_automata. The ahocorasick structure is replaced with a bitmap. The bitmap size for ndpi_en_bigram is 176 bytes. The bitmap size for ndpi_en_trigram is 2201 bytes. On the test machine, the test execution time was reduced from 27.3 seconds to 24.7 (9%).
* Addesses partial application matching (#1207)Luca Deri2021-06-21
|
* Restored 32 bit value in ndpi_match_string_value()Luca Deri2021-06-21
|
Powered by cgit, Themed by Ted Yin.