aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* Improved HTTP payload handlingLuca2022-10-04
|
* Improved detection of WebShell and PHP code in HTTP URLs that is reported ↵Luca2022-10-04
| | | | via flow risk
* Added the ability to track the payload via -E and via the new option ↵Luca2022-10-04
| | | | 'ndpi_track_flow_payload'
* Fix ndpi_timeval_to_milliseconds/microseconds for platforms with tv_usec is ↵Toni2022-10-02
| | | | | | | | | an unsigned int. (#1762) * implicit conversion to an 32 bit unsigned int leads to an overflow Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Restore Confidence enum constants to ascending values. (#1760)Toni2022-10-02
| | | | | | | | | | * NDPI_CONFIDENCE_MAX is useless otherwise since it is supposed the be the highest value w/o any gaps * for client applications, it makes sense to use that the numbers directly w/o any additional mapping Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Line app support (#1759)sharonenoch2022-10-01
| | | | | | | | | | | | | | | | | | | * Standard support for LINE app * Added test pcap for LINE app * make check result for LINE app * Make check success as 1kxun has LINE packets * Added the ASN inc file for LINE * Removed extra lines as its effecting make check * Editing the SNI required a new pcap output file for TLS.Line format * Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out Co-authored-by: Sharon Enoch <sharone@amzetta.com>
* TLS/DTLS: we process certificate for UDP flows, tooNardi Ivan2022-09-30
| | | | | | Note that current code access `certificate_processed` state even before setting the protocol classification, so this piece of information can't be saved in `flow->protos` union.
* Fixed invalid unidirectional traffic alert for unidirectional protocols ↵Luca2022-09-29
| | | | (e.g. sFlow)
* Remove unused codeNardi Ivan2022-09-28
| | | | | | | | | LRU callbacks have been added in 460ff3c7a, but they have never been used and they have never been extended to the other LRU caches. `ndpi_search_tcp_or_udp()` basically returns the classification by port/ip of the flow; calling it from the dissector is useless. The same for TOR detection: ips are checked in the generic code
* Fix value of `ndpi_protocol->protocol_by_ip`Nardi Ivan2022-09-25
| | | | Fix: a7c2734b
* DNS: change category of DNS flowsNardi Ivan2022-09-25
| | | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any). Follow-up of 83de3e47
* Changed some protocol categories to align with others.Toni Uhlig2022-09-23
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* DNS: change category of DNS flowsNardi Ivan2022-09-22
| | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any).
* Extend content match listsNardi Ivan2022-09-22
|
* NAT-PMP: fix metadata extractionNardi Ivan2022-09-21
|
* Improved NATPMP dissection. (#1745)Toni2022-09-21
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Build ndpiReader and run regression tests.Toni Uhlig2022-09-21
| | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed VS2019 build.segfault2022-09-21
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Provide extra `ndpi_define.h` for Visual Studio.Toni Uhlig2022-09-21
| | | | | | * Caution: file needs to keep in sync with `src/include/ndpi_define.h.in` manually Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Moved `NDPI_API_VERSION` to `src/include/ndpi_define.h.in` where it belongs.Toni Uhlig2022-09-21
| | | | | | | That way, we can make `src/include/ndpi_api.h` independent of autoconf. MSBuild will like that. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Retargeted to Windows 10.segfault2022-09-21
| | | | | * disable Spectre Mitigation (may change) * remove autoconf generated header files
* Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.mark andrews2022-09-21
| | | | | | * add CI support via MSBuild Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Copilation fixLuca Deri2022-09-21
|
* Added missing libraryLuca Deri2022-09-21
|
* Finalized nBPF support. You can now define custom protocols such asLuca Deri2022-09-21
| | | | | | | | | | (see exaple/protos.txt) nbpf:"host 192.168.1.1 and port 80"@HomeRouter In order to have nBPF support, you need to compile nDPI with it. Just download https://github.com/ntop/PF_RING in the same directory where you have downloaded nDPI and compile PF_RING/userland/nbpf
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Fix warnings on OpenWrt CCs, caused by the memcpy builtin. (#1741)Toni2022-09-18
| | | | | | | | | | | | | | | | | | | In file included from ../include/ndpi_includes.h:31, from ../include/ndpi_main.h:27, from ../include/ndpi_api.h:28, from protocols/quic.c:27: In function 'memcpy', inlined from 'tls13_hkdf_expand_label_context' at protocols/quic.c:473:5, inlined from 'tls13_hkdf_expand_label' at protocols/quic.c:498:10, inlined from 'quic_hkdf_expand_label.constprop' at protocols/quic.c:512:6: /home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: error: argument 2 null where non-null expected [-Werror=nonnull] 53 | return __builtin_memcpy(__od, __os, __n); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ protocols/quic.c: In function 'quic_hkdf_expand_label.constprop': /home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: note: in a call to built-in function '__builtin_memcpy' Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added MAX_NBPF_CUSTOM_PROTO defineLuca Deri2022-09-17
|
* Initial work towards custom BPF protocol definitionLuca2022-09-16
|
* Add CryNetwork dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Syncthing dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add NATPMP dissector. (#1738)Toni2022-09-16
| | | | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Remove a case of guessed sub-classificationNardi Ivan2022-09-14
| | | | | | | | | | | | | | | | | | This code is triggered only for "unknown" flows with a valid sni/hostname. Why in that case the guessed classification should be something like `DNS/Subprotocol_depending_on_hostname`? Why DNS as master and not HTTP or TLS or QUIC? Furthermore, I have not been able to trigger a positive match from that lookup. I strongly think that if we had a valid subprotocol, we would have a valid master in the first place. In doubt, remove it completely. As a follow up, we should investigate why some dissectors (the HTTP one, at least) set the sni/hostname field without setting a valid protocol, in the first place. This behaviour seems quite suspicious, if not plainly buggy.
* Dissectors shouldn't update `flow->guessed_host_protocol_id`Nardi Ivan2022-09-14
| | | | | | | | | | | | | | | | The field `flow->guessed_host_protocol_id` is set at the beginning of the flow analysis and it represents the "classification by ip" of the flow itself. This field should never be changed. Dissectors which want to provide an "hint" about the classification, should update `flow->guessed_protocol_id` instead. Such "hint" is useless if the dissector set the "extra-dissection" data-path. Rename such field to `guessed_protocol_id_by_ip` to better describe its role. Preliminary work necessary for #1687
* Anydesk: improve detection (#1735)Ivan Nardi2022-09-13
|
* HTTP: remove some wrong codeNardi Ivan2022-09-13
| | | | | | | | | | This code seems wrong or in the wrong place, at least: * "classification by port" and "classification by ip" protocols (i.e "guessed" protocols) should be used to set the protocol stack only after trying all the dissectors, and only by the generic code * there are no reason (for a dissector) to update the "guessed" information using the protocol stack values: it is usually the other way around (see previous point)
* Replace obsolete linux macroAlfredo Cardigliano2022-09-13
|
* Fix `ndpi_do_guess()` (#1731)Ivan Nardi2022-09-12
| | | | | Avoid a double call of `ndpi_guess_host_protocol_id()`. Some code paths work for ipv4/6 both Remove some never used code.
* Update Github CI OS and compilers (#1733)Ivan Nardi2022-09-12
| | | | | | | | | | ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04). macos-latest points to macos-11, so it makes sense to test macos-12, too. About the compilers, the general idea it to test the oldest and the newest versions easily available: switch to gcc-11 and clang-14. See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* STUN: several improvementsNardi Ivan2022-09-11
| | | | | | | | | Add detection over TCP and fix detection over IPv6. Rename some variables since Stun dissector is no more "udp-centric". Stun dissector should always classified the flow as `STUN` or `STUN/Something`. Don't touch `flow->guessed_host_protocol_id` field, which should be always be related to "ip-classification" only.
* Update Changelog file (version 4.4)Nardi Ivan2022-09-10
| | | | Notes shameless taken from https://github.com/ntop/nDPI/releases
* HTTP: correctly set the classification protocolsNardi Ivan2022-09-06
| | | | | | | | | | | | Classification should always be set via `ndpi_set_detected_protocol()` to be sure to set a correct `confidence` value, too. Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as confidence, is not valid. This code in HTTP dissector likely needs some more thoughts (the classification itself of the attached example doesn't make a lot of sense), but the goal of this commit is only to always have a valid `confidence` value.
* Serialize nDPI detection/dissection data as object. (#1725)Toni2022-09-06
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix some warningsNardi Ivan2022-09-05
| | | | | | | | | | | | | | | | | | | | | Ubuntu-20.04, clang-16 (nightly build) ``` Making all in src/lib protocols/smpp.c:70:17: warning: variable 'pdu_c' set but not used [-Wunused-but-set-variable] u_int16_t pdu_c = 1; ^ 1 warning generated. third_party/src/ahocorasick.c:173:20: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion] thiz->root->root = 1; ^ ~ third_party/src/ahocorasick.c:336:15: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion] n->ff = 1; ^ ~ third_party/src/ahocorasick.c:716:21: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion] node->final = 1; [...] ```
* Avoid useless host automa lookup (#1724)Ivan Nardi2022-09-05
| | | | | | | | | | | | | | | | | | | The host automa is used for two tasks: * protocol sub-classification (obviously); * DGA evaluation: the idea is that if a domain is present in this automa, it can't be a DGA, regardless of its format/name. In most dissectors both checks are executed, i.e. the code is something like: ``` ndpi_match_host_subprotocol(..., flow->host_server_name, ...); ndpi_check_dga_name(..., flow->host_server_name,...); ``` In that common case, we can perform only one automa lookup: if we check the sub-classification before the DGA, we can avoid the second lookup in the DGA function itself.
* Fix string copy truncation warning. (#1723)Toni2022-09-05
| | | | | | | | | | protocols/ubntac2.c: In function ‘ndpi_search_ubntac2’: protocols/ubntac2.c:69:4: warning: ‘strncpy’ output may be truncated copying between 0 and 31 bytes from a string of length 255 [-Wstringop-truncation] 69 | strncpy(flow->protos.ubntac2.version, (const char *)version, len); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed FastCGI memory issue (was not using nDPI's malloc wrapper). (#1722)Toni2022-09-05
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix endianess in IP-Port lookupNardi Ivan2022-09-05
| | | | | Port parameter in `ndpi_network_port_ptree_match()` must be in network order.
Powered by cgit, Themed by Ted Yin.