aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols/tls.c
Commit message (Collapse)AuthorAge
* TLS: fix heap-buffer-overflow (#2235)Ivan Nardi2024-01-02
| | | | | | | | | | | | | | | | | | | | ``` ==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988 READ of size 2 at 0x511000007e04 thread T0 #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50 #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5 #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11 #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5 #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6 #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10 #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12 #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15 #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22 #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31 #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1 ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362
* New ndpi_sha256() nDPI API call (#2230)Luca Deri2023-12-23
| | | | | * Added ndpi_sha256.c to the Windows project * Added ndpi_sha256() nDPI API call
* Changes to avoid type redefinition on windowsLuca Deri2023-12-23
|
* Buffer overflow fixLuca Deri2023-12-23
|
* Warning fixLuca Deri2023-12-22
|
* Implements JA4 Support (#2191)Luca Deri2023-12-22
|
* Changes to make room for JA4Luca Deri2023-12-22
|
* TLS: be sure to always set a category value (#2206)Ivan Nardi2023-12-14
| | | | | We probably need to check everywhere we set the category in the "extra dissection" data path: a flow with a proper classification can't have NDPI_PROTOCOL_CATEGORY_UNSPECIFIED as category.
* fuzz: extend fuzzing coverage (#2205)Ivan Nardi2023-12-11
|
* TLS: remove JA3+ fingerprints. (#2192)Ivan Nardi2023-12-05
| | | See: #2191
* STUN: improve demultiplexing of DTLS packets (#2153)Ivan Nardi2023-11-27
| | | | | | Keep demultiplexing STUN/RTP/RTCP packets after DTLS ones. We might end up processing the session a little longer, because we will process the STUN/RTP/RTCP packets after the DTLS handshake.
* Have a clear distinction between public and private/internal API (#2137)Ivan Nardi2023-11-09
| | | | | | 1) Public API/headers in `src/include/` [as it has always been] 2) Private API/headers in `src/lib/` Try to keep the "ndpi_" prefix only for the public functions
* Added HAProxy protocol. (#2088)Toni2023-10-02
| | | | | | * fixed tests/do.sh.in failure print Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added NDPI_TLS_ALPN_SNI_MISMATCH flow riskLuca Deri2023-09-07
|
* TLS: add basic, basic, detection of Encrypted ClientHello (#2053)Ivan Nardi2023-07-21
|
* fuzz: extend fuzzing coverage (#2040)Ivan Nardi2023-07-11
| | | | | | | | | Some notes: * libinjection: according to https://github.com/libinjection/libinjection/issues/44, it seems NULL characters are valid in the input string; * RTP: `rtp_get_stream_type()` is called only for RTP packets; if you want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`; * TLS: unnecessary check; we already make the same check just above, at the beginning of the `while` loop
* Fixed heap-overflow if compiled with `--enable-tls-sigs`. (#2038)Toni2023-07-07
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* STUN: add dissection of DTLS handshake (#2018)Ivan Nardi2023-06-26
|
* Compilation fixLuca Deri2023-06-22
|
* Waring fixLuca Deri2023-06-21
|
* Numeric truncation at `tls.c:1010` (#2005)headshog2023-06-08
| | | | | * fixed numtrunc error in protocols/tls.c * fixed build error for tls.c
* Ookla: rework detection (#1922)Ivan Nardi2023-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | The logic of the LRU cache has been changed: once we know an ip has connected to an Ookla server, all the following (unknown) flows (for a short time interval) from the same ip to the port 8080 are treated as Ookla ones. Most of the changes in this commit are about introducing the concept of "aggressive detection". In some cases, to properly detect a protocol we might use some statistical/behavior logic that, from one side, let us to identify the protocol more often but, from the other side, might lead to some false positives. To allow the user/application to easily detect when such logic has been triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been added. It is always possible to disable/configure this kind of logic via the API. Detection of Ookla flows using plain TLS over port 8080 is the first example of aggressive detection in nDPI. Tested with: * Android 9.0 with app 4.8.3 * Ubuntu 20.04 with Firefox 110 * Win 10 with app 1.15 and 1.16 * Win 10 with Chrome 108, Edge 108 and Firefox 106
* TLS: fix another interger overflow in certificate processing (#1915)Ivan Nardi2023-03-30
|
* TLS: fix parsing of certificate elements (#1910)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | | ``` ==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0) ==1228==The signal is caused by a WRITE memory access. #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79 #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3 #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15 #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6 #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9 #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 #7 0x41f78d in _start ``` Found by oss-fuzz. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
* fuzz: add a new fuzzer to test TLS certificates (#1901)Ivan Nardi2023-03-20
|
* Add a new flow risk about literal IP addresses used as SNI (#1892)Ivan Nardi2023-03-02
| | | | | | | | | | RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in "HostName"." Don't set this risk if we have a valid sub-classification (example: via certificate) Since a similar risk already exists for HTTP hostnames, reuse it, with a more generic name.
* LRU caches: add a generic (optional and configurable) expiration logic (#1855)Ivan Nardi2023-01-18
| | | Two caches already implemented a similar mechanism: make it generic.
* Add protocol disabling feature (#1808)Ivan Nardi2022-12-18
| | | | | | | | | | | | | | | | | | | | | | The application may enable only some protocols. Disabling a protocol means: *) don't register/use the protocol dissector code (if any) *) disable classification by-port for such a protocol *) disable string matchings for domains/certificates involving this protocol *) disable subprotocol registration (if any) This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`. Custom protocols are always enabled. Technically speaking, this commit doesn't introduce any API/ABI incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()` is now mandatory, just after having called `ndpi_init_detection_module()`. Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to the removing of some function parameters. Fix the low level macro `NDPI_LOG`. This issue hasn't been detected sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
* DTLS: handle (certificate) fragments (#1811)Ivan Nardi2022-12-10
| | | | | Keep using the existing function to handle reassembling buffer: rename it from `ndpi_search_tls_tcp_memory` to `ndpi_search_tls_memory` and make it "transport" agnostic
* fuzz: add a new fuzzer testing memory allocation failures (#1818)Ivan Nardi2022-12-06
| | | | | | | | | Try to fuzz error paths triggered by allocation errors. Fix some errors already found by this new fuzzer. Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to (not)compile specific code in fuzzing builds. See: https://llvm.org/docs/LibFuzzer.html
* Make LRU caches ipv6 aware (#1810)Ivan Nardi2022-12-03
| | | | | | | | | | | | | Simplest solution, keeping the existing cache data structure TLS certificate cache is used for DTLS traffic, too. Note that Ookla cache already works with ipv6 flows. TODO: * make the key/hashing more robust (extending the key size?) * update bittorrent cache too. That task is quite difficult because ntopng uses a public function (`ndpi_guess_undetected_protocol()`) intrinsically ipv4 only...
* TLS: be sure to always set `ssl_version` field (#1806)Ivan Nardi2022-11-22
| | | Useful with asymmetric traffic with (D)TLS <= 1.2
* TLS: improve handling of ALPN(s) (#1784)Ivan Nardi2022-10-25
| | | | | | | | Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is extracted from the CH, the latter from the SH. Add some entries to the known ALPN list. Fix printing of "TLS Supported Versions" field.
* TLS: allow sub-classification via ALPNNardi Ivan2022-10-20
| | | | | | | | In some rare cases, it is possible to sub-classify the flow via ALPN matching. This is particularly usefull for asymmetric traffic where the Client Hello doens't have the SNI. For the time being there is only one rule, about ANYDESK.
* TLS: explicit ignore client certificate (#1776)Ivan Nardi2022-10-18
| | | | | | | | | | | | | TLS classification usually stops after processing *server* certificates (if any). That means, that *client* certificate, if present, is usually ignored. However in some corner cases (i.e. unidirectional traffic) we might end up processing client certificate and exposing its metadata: the issue is that the application will think that this metadata are about the server and not about the client. So, for the time being, always ignore client certificate processing. As a future work, we might find an efficient way to process and export both certificates.
* TLS/DTLS: we process certificate for UDP flows, tooNardi Ivan2022-09-30
| | | | | | Note that current code access `certificate_processed` state even before setting the protocol classification, so this piece of information can't be saved in `flow->protos` union.
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Dissectors shouldn't update `flow->guessed_host_protocol_id`Nardi Ivan2022-09-14
| | | | | | | | | | | | | | | | The field `flow->guessed_host_protocol_id` is set at the beginning of the flow analysis and it represents the "classification by ip" of the flow itself. This field should never be changed. Dissectors which want to provide an "hint" about the classification, should update `flow->guessed_protocol_id` instead. Such "hint" is useless if the dissector set the "extra-dissection" data-path. Rename such field to `guessed_protocol_id_by_ip` to better describe its role. Preliminary work necessary for #1687
* Avoid useless host automa lookup (#1724)Ivan Nardi2022-09-05
| | | | | | | | | | | | | | | | | | | The host automa is used for two tasks: * protocol sub-classification (obviously); * DGA evaluation: the idea is that if a domain is present in this automa, it can't be a DGA, regardless of its format/name. In most dissectors both checks are executed, i.e. the code is something like: ``` ndpi_match_host_subprotocol(..., flow->host_server_name, ...); ndpi_check_dga_name(..., flow->host_server_name,...); ``` In that common case, we can perform only one automa lookup: if we check the sub-classification before the DGA, we can avoid the second lookup in the DGA function itself.
* Add support to opportunistic TLSNardi Ivan2022-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A lot of protocols provide the feature to upgrade their plain text connections to an encrypted one, via some kind of "STARTTLS" command. Add generic code to support this extension, and allow dissection of the entire TLS handshake. As examples, SMTP, POP, IMAP and FTP dissectors have been updated. Since this feature requires to process more packets per flow, add the possibility to disable it. Fix some log messages. Slight improvement on TCP sequence number tracking. As a side effect, this commit fix also a memory leak found by oss-fuzzer ``` ==108966==ERROR: LeakSanitizer: detected memory leaks Direct leak of 22 byte(s) in 1 object(s) allocated from: #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063) #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10 #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25 #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13 #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34 #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5 #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9 #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9 #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5 #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31 #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10 #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
* Further simplification of `ndpi_process_extra_packet()` (#1698)Ivan Nardi2022-08-05
| | | | | | | | See 95e16872. After c0732eda, we can safely remove the protocol list from `ndpi_process_extra_packet()`. The field `flow->check_extra_packets` is redundant; remove it.
* Improved nDPI JSON serialization. (#1689)Toni2022-08-02
| | | | | | | | * fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors * introduced portable version of gmtime_r aka ndpi_gmtime_r * do as most as possible of the serialization work in ndpi_utils.c * use flow2json in ndpiReader Signed-off-by: lns <matzeton@googlemail.com>
* TLS: avoid useless extra dissection (#1690)Ivan Nardi2022-07-31
| | | | | | | If we have seen Application Data blocks in both directions, it means we are after the handshake. At that point, extra dissection is useless. Useful feature with TLS mid-sessions, i.e. sessions without initial packets.
* SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)Ivan Nardi2022-07-30
| | | | | | The big change in TLS code is to allow "master" protocols other than TLS/DTLS, like SMTPS, POPS and IMAPS. This change will allow, in a future, a proper and complete TLS dissection for all these protocols with "STARTTLS"-like messages.
* First step in simplify `ndpi_process_extra_packet()` (#1680)Ivan Nardi2022-07-29
| | | | | | | Move the prottocol specific logic into the proper dissector code, where it belongs. Next step: remove that list of protocols. Long goal: remove this function altogether...
* TLS: improve reassembler (#1669)Ivan Nardi2022-07-22
| | | | | | | | * TLS: cosmetic changes * TLS: improve reassembler We might need to contemporary re-order messages from both directions: use one buffer per direction.
* DTLS: fix exclusion of DTLS protocolNardi Ivan2022-07-20
| | | | Add an helper to exclude a generic protocol
* Cosmetic changeLuca2022-07-12
|
* TLS: ignore invalid Content Type valuesNardi Ivan2022-07-07
|
Powered by cgit, Themed by Ted Yin.