aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_main.c
Commit message (Collapse)AuthorAge
...
* Add detection of OCSP (#1370)Ivan Nardi2021-11-11
| | | | | | | | | | This protocol is detected via HTTP Content-Type header. Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to classify a HTTP flow according to this header. Since then, this automa has been useless because it is always empty. Re-enable it to match only a string seems overkilling. Remove all `content_automa` leftovers.
* IMAP, POP3, SMTP: improve dissection (#1368)Ivan Nardi2021-11-11
| | | Avoid NATS false positives
* Improved BitTorrent detectionLuca Deri2021-11-04
|
* Fix use-of-uninitialized-value (#1366)Ivan Nardi2021-11-03
| | | Close #1346
* Avoid overwriting valid protocol in `ndpi_detection_giveup` (#1360)Ivan Nardi2021-10-27
| | | | | | | | | | | | | | | | | We should avoid updating any valid protocol in `ndpi_detection_giveup`; we should try to find a proper classification only if the flow is still completely unclassified. For example in the attached pcap there is a valid TLS session, recognized as such by TLS dissector. However, the `ndpi_detection_giveup`function updates it to "HTTP/TLS" (!?) simply because the server port is 80. Note that the real issue is not the wrong classification, but the wrong access to `flow->protos` union. If we already set some fields of `flow->protos` and we change the protocol in `ndpi_detection_giveup`, we might end up freeing some invalid pointers in `ndpi_free_flow_data` (no wonder this issue has been found while fuzzing #1354) Fix GIT and TLS dissectors (issues found by CI fuzzer)
* Improved STUN and RTP detectionLuca Deri2021-10-27
|
* Detect invalid characters in text and set a risk. Fixes #1347. (#1363)Toni2021-10-26
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix some invalid memory reads (#1350)Ivan Nardi2021-10-19
| | | | | | | | `ndpi_detection_giveup()` (and any functions called by it) can't access `ndpi_detection_module_struct->packet` anymore since 730c236. Sync unit tests results Close #1348
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked flow risks asignmentLuca Deri2021-10-16
| | | | Added esceptions for windows update and binary application transfer risk
* Removed outdated (and broken) soulseek dissectorLuca Deri2021-10-15
|
* Added missing call to ndpi_reconcile_protocols() with protocol guessLuca Deri2021-10-14
|
* Fixed issue on DGA numeric IP detectionLuca Deri2021-10-13
|
* IndemtLuca Deri2021-10-13
|
* Compilation fixLuca Deri2021-10-05
|
* Numeric IPs are not considered for DGA checksLuca Deri2021-10-05
|
* Improved DGA detection for skipping potential DGAs of known/popular domain namesLuca Deri2021-10-05
|
* Remove `struct ndpi_packet_struct` from `struct ndpi_flow_struct` (#1319)Ivan Nardi2021-10-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet") in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid saving dissection information of "current packet" into the "flow" state, i.e. in the flow management table. The nDPI detection module processes only one packet at the time, so it is safe to save packet dissection information in `struct ndpi_detection_module_struct`, reusing always the same "packet" instance and saving a huge amount of memory. Bottom line: we need only one copy of "packet" (for detection module), not one for each "flow". It is not clear how/why "packet" ended up in "flow" in the first place. It has been there since the beginning of the GIT history, but in the original OpenDPI code `struct ipoque_packet_struct` was embedded in `struct ipoque_detection_module_struct`, i.e. there was the same exact situation this commit wants to achieve. Most of the changes in this PR are some boilerplate to update something like "flow->packet" into something like "module->packet" throughout the code. Some attention has been paid to update `ndpi_init_packet()` since we need to reset some "packet" fields before starting to process another packet. There has been one important change, though, in ndpi_detection_giveup(). Nothing changed for the applications/users, but this function can't access "packet" anymore. The reason is that this function can be called "asynchronously" with respect to the data processing, i.e in context where there is no valid notion of "current packet"; for example ndpiReader calls it after having processed all the traffic, iterating the entire session table. Mining LRU stuff seems a bit odd (even before this patch): probably we need to rethink it, as a follow-up.
* Added -a <num> to ndpiReader for generating OPNsense configurationLuca Deri2021-10-04
| | | | See https://github.com/ntop/opnsense
* Remove `detected_protocol_stack` field from `ndpi_packet_struct` (#1317)Ivan Nardi2021-09-29
| | | | | | | | | | | | | This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`: * at the very beginning of packet dissection, the value saved in `flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack` (via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`) * every time we update `flow->detected_protocol_stack` we update `packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()` -> `ndpi_int_change_packet_protocol()`) These two fields are always in sync: keeping the same value in two different places is useless.
* FTP: fix support for START-TLS sessionsNardi Ivan2021-09-21
| | | | | | When TLS-over-FTP is used, the credentials are encrypted. So we must not wait for the username and the password commands, otherwise we elaborate a lot of packets for nothing.
* Fix parsing of ipv6 packets with extension headersNardi Ivan2021-09-19
| | | | | | | | Decoding of ipv6 traffic with extension headers was completely broken, since the beginning of the L4 header was always set to a wrong value. Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first one and drop the others.
* Small fixes after latest commits (#1308)Ivan Nardi2021-09-18
|
* A (tiny) effort to reduce memory usage per flow (#1304)Ivan Nardi2021-09-18
| | | | | | | Follow-up of 22241a1d Only trivial changes: * remove completely unused fields * remove fields only written (but never read) * CSGO protocol only handles UDP traffic
* DNS dissection fixesLuca Deri2021-09-17
|
* Progetto esame Gestione di Reti - Debora Cerretini (#1290)deboracerretini2021-09-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Hangout: fix usage of STUN function (#1299)Ivan Nardi2021-09-13
|
* Update the list of default ports for QUIC protocol (#1297)Ivan Nardi2021-09-13
| | | There are no reasons to register UDP/80 as a default port for QUIC
* Added new risk for clear text credentialsLuca Deri2021-09-10
|
* Add Cassandra protocol dissector (#1285)lucasbaile2021-09-09
| | | Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
* Fix access to some TLS fields in flow structure (#1277)Ivan Nardi2021-08-20
| | | | Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by QUIC (i.e UDP...), too.
* Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)Ivan Nardi2021-08-20
| | | | | | | | | | | | | | | | Fix all the warnings. Getting rid of "-Wno-unused-parameter" is quite complex because some parameters usage depends on compilation variable (i.e. `--enable-debug-messages`). The "-Werror" flag has been added only in Travis builds to avoid breaking the builds to users using uncommon/untested OS/compiler/enviroment. Tested on: * x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12 * x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag) * Raspberry 4; Debian 10.10; gcc 8.3.0
* Fixed Mingw64 build, SonerCloud-CI and more. (#1273)Toni2021-08-18
| | | | | | | | | | | | | * Added ARM build and unit test run for SonarCloud-CI. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Fixed Mingw64 build. * adapted to SonarCloud-CI workflow * removed broken and incomplete Windows example (tested on VS2017/VS2019) * removed unnecessary include (e.g. pthread.h for the library which does not make use of it) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed some invalid TLS guessesLuca Deri2021-08-17
|
* Added extraction of hostname in SMTPLuca Deri2021-08-11
| | | | Fixed mail incalid subprotocol calculation
* Clode cleanup (after last merge)Luca Deri2021-08-08
|
* Added entropy calculation to check for suspicious (encrypted) payload. (#1270)Toni2021-08-08
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Added TLS fatal alert flow riskLuca Deri2021-08-07
|
* Added ability to report whether a protocol is encryptedLuca Deri2021-08-07
| | | | | | | | | Added new API calls - u_int8_t ndpi_is_valid_protoId(u_int16_t protoId); Cleaned up code and organized in a function. - u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto); Check whether a protocol is encrypted or not based on master and app protocols
* Implemented ALPN automa for checking uncommon ALPNsLuca Deri2021-07-24
|
* Reworked flow risk implementationLuca Deri2021-07-23
|
* Fixed risk mask implementationLuca Deri2021-07-23
|
* Implementation of flow risk eception (work in progress)Luca Deri2021-07-22
|
* Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)Toni2021-07-19
| | | | | | * validates client/server hello TLS extensions * inspects content for some extensions Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated download category nameLuca Deri2021-07-16
|
* Bug fix. (#1250)Vitaly Lavrov2021-07-16
| | | Restore part of the fields flow->packet after cleaning flow structure.
* TLS Risks - Certificate Validity Too Long (#1239)pacant2021-07-14
| | | | | | | | | | | * Added flow risk: TLS certificate too long * Added flow risk: TLS certificate too long * Date for TLS limit added * TLS certificate check fixed Co-authored-by: pacant <a.pace97@outlook.com>
* Added AVAST SecureDNS protocol. (#1244)Toni2021-07-14
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* ahoсorasick. Code review. Part 2. (#1236)Vitaly Lavrov2021-07-12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Simplified the process of adding lines to AC_AUTOMATA_t. Use the ndpi_string_to_automa() function to add patterns with domain names. For other cases can use ndpi_add_string_value_to_automa(). ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding and compare data in a case insensitive manner. For mandatory pattern comparison from the end of the line, the "ac_pattern.rep.at_end=1" flag is used. This eliminated unnecessary conversions to lowercase and adding "$" for end-of-line matching in domain name patterns. ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified. ac_domain_match_handler() looks for the template with the highest domain level. For special cases it is possible to manually specify the domain level. Added test for checking ambiguous domain names like: - short.weixin.qq.com is QQ, not Wechat - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook If you specify a NULL handler when creating the AC_AUTOMATA_t structure, then a pattern with the maximum length that satisfies the search conditions will be found (exact match, from the beginning of the string, from the end of the string, or a substring). Added debugging for ac_automata_search. To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG). The search will display "name" and a list of matching patterns. Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates and which templates were found. The ac_automata_dump() prototype has been changed. Now it outputs data to a file. If it is specified as NULL, then the output will be directed to stdout. If you need to get data as a string, then use open_memstream(). Added the ability to run individual tests via the do.sh script
* Fixed false positives in Z39.50Luca Deri2021-07-09
|
Powered by cgit, Themed by Ted Yin.