aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols/tls.c
Commit message (Collapse)AuthorAge
* A new interface for dissectors registration (#2843)Ivan Nardi2025-05-24
| | | | | | | | | | | | | | | | | | | | | We use `registr_dissector()` instead of `ndpi_set_bitmask_protocol_detection()`. Every file in `src/lib/protocols/*.c` is a dissector. Every dissector can handle multiple protocols. The real goal is this small change: ``` struct call_function_struct { - NDPI_PROTOCOL_BITMASK detection_bitmask; ``` i.e. getting rid of another protocol bitmask: this is mandatory to try to fix #2136 (see also e845e8205b68752c997d05224d8b2fd45acde714) As a nice side effect, we remove a bitmask comparison in the hot function `check_ndpi_detection_func()` TODO: change logging configuration from per-protocol to per-dissector
* Minor simplification on protocol/dissector registration (#2833)Ivan Nardi2025-05-20
|
* Remove duplicate ALPS extension (#2821)FS2025-05-19
| | | | | * Removed dupe extension * Removed ECH from suspicious extensions
* TLS: register TLS dissector only once (#2825)Ivan Nardi2025-05-19
| | | | This is the first, tiny, step into a better separation between "protocols" and "dissectors"
* Added check while dissecting non-TCP packets ndpi_search_tls_tcpLuca Deri2025-04-04
| | | | Renamed ndpi_search_tls_udp to ndpi_search_dtls
* Implemented detection of the latest Signal video/audio calls leveraging on ↵Luca Deri2025-04-02
| | | | Cloudflare CDN
* TLS: avoid exporting TLS heuristic fingerprint as metadata (#2783)Ivan Nardi2025-03-27
|
* Improved configuration to enable/disable export of flow risk info (#2780)Ivan Nardi2025-03-25
| | | | Follow-up of f56831336334dddcff00eaf2132e5e0f226f0e32: now the configuration is for flow-risk, not global
* Remove `NDPI_TLS_SUSPICIOUS_ESNI_USAGE` flow risk (#2778)Ivan Nardi2025-03-25
| | | | | | That flow risk was introduced in 79b89d286605635f15edfe3c21297aaa3b5f3acf but we can now use the generic `NDPI_TLS_SUSPICIOUS_EXTENSION` instead: ESNI is quite suspicious nowadays in itself (i.e. even without SNI). Note that ESNI support has been removed in cae9fb9989838f213eeb857b8fc4bbeac6940049
* TLS: avoid sub-classification for RDP flows (#2769)Ivan Nardi2025-03-14
| | | | | | | | | | | | | | | | These flows are already classified as TLS.RDP. This change also fix a memory leak ``` Direct leak of 62 byte(s) in 1 object(s) allocated from: #0 0x5883d762429f in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:68:3 #1 0x5883d76fe46a in ndpi_malloc ndpi/src/lib/ndpi_memory.c:57:46 #2 0x5883d76fe46a in ndpi_strdup ndpi/src/lib/ndpi_memory.c:110:13 #3 0x5883d77adcd6 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:2298:46 #4 0x5883d77ab2ec in processClientServerHello ndpi/src/lib/protocols/tls.c:3314:10 #5 0x5883d77a4c51 in processTLSBlock ndpi/src/lib/protocols/tls.c:1319:5 ``` Found by oss-fuzz. See: https://oss-fuzz.com/testcase-detail/5244512192757760
* Add configuration parameter to enable/disable export of flow risk info (#2761)Ivan Nardi2025-03-05
| | | | For the most common protocols, avoid creating the string message if we are not going to use it
* Added valid TLS extensions that used to trigger invalid risksLuca Deri2025-02-27
|
* Improved Tor detectionLuca Deri2025-02-24
|
* DNS: fix check for DGA domain (#2716)Ivan Nardi2025-02-11
| | | | If we have a (potential) valid sub-classification, we shoudn't check for DGA, even if the subclassification itself is disabled!
* Create a specific configuration for classification only (#2689)Ivan Nardi2025-01-31
| | | | | | | | | | | | | | | | | | | | | | | | In some scenarios, you might not be interested in flow metadata or flow-risks at all, but you might want only flow (sub-)classification. Examples: you only want to forward the traffic according to the classification or you are only interested in some protocol statistics. Create a new configuration file (for `ndpiReader`, but you can trivially adapt it for the library itself) allowing exactly that. You can use it via: `ndpiReader --conf=example/only_classification.conf ...` Note that this way, the nDPI overhead is lower because it might need less packets per flow: * TLS: nDPI processes only the CH (in most cases) and not also the SH and certificates * DNS: only the request is processed (instead of both request and response) We might extend the same "shortcut-logic" (stop processing the flow immediately when there is a final sub-classification) for others protocols. Add the configuration options to enable/disable the extraction of some TLS metadata.
* JA4: Fix SSL 2 version and remove fictional SSL 1 version along with ↵Daniel Roethlisberger2025-01-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mis-mapping to s3 (#2684) * JA4: Fix SSL 2 version constant to 0x0002 SSL 2 uses a version field of 0x0002, not 0x0200. This is confirmed not only in the original Netscape spec [1] and RFC draft of the time [2], but also in major implementations such as OpenSSL [3] and Wireshark [4]. An earlier version of the JA4 spec [5] also mistakenly used 0x0200 for SSL 2 and 0x0100 for SSL 1. This was fixed in [6] in August 2024. [1] https://www-archive.mozilla.org/projects/security/pki/nss/ssl/draft02.html [2] https://datatracker.ietf.org/doc/html/draft-hickman-netscape-ssl-00 [3] https://github.com/openssl/openssl/blob/OpenSSL_0_9_6m/ssl/ssl2.h#L66-L71 [4] https://github.com/wireshark/wireshark/blob/release-4.4/epan/dissectors/packet-tls-utils.h#L266-L277 [5] https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md#tls-and-dtls-version [6] FoxIO-LLC/ja4#150 * JA4: Remove fictional (and mis-mapped to "s3") SSL 1 SSL 1 was never actually deployed, the design was iterated upon to become SSL 2 before it was released by Netscape [1] [2] [3] [4]. I don't think it's public knowledge what the version field for SSL 1 would have looked like, or if it even was two bytes large or at the same offset on the wire; given that SSL 2 used 0x0002 it seems more likely to have been 0x0001 than 0x0100. Version field 0x0100, that is currently misattributed to SSL 1, was used by an early pre-RFC4347 implementation of DTLS in OpenSSL before 0.9.8f [5], when OpenSSL switched to the version field specified by RFC4347. This use of 0x0100 is also reflected in Wireshark's TLS dissector [4] (`DTLSV1DOT0_OPENSSL_VERSION`). For these reasons, it seems to make sense to remove the fictional SSL 1 code entirely. This also removes an issue where the resulting JA4 string would be "s3" instead of the intended "s1". An earlier version of the JA4 spec [6] also mistakenly used 0x0200 for SSL 2 and 0x0100 for SSL 1. This was fixed in [7] in August 2024. [1] https://www-archive.mozilla.org/projects/security/pki/nss/ssl/draft02.html [2] https://datatracker.ietf.org/doc/html/draft-hickman-netscape-ssl-00 [3] https://github.com/openssl/openssl/blob/OpenSSL_0_9_6m/ssl/ssl2.h#L66-L71 [4] https://github.com/wireshark/wireshark/blob/release-4.4/epan/dissectors/packet-tls-utils.h#L266-L277 [5] https://github.com/openssl/openssl/compare/OpenSSL_0_9_8e...OpenSSL_0_9_8f [6] https://github.com/FoxIO-LLC/ja4/blob/main/technical_details/JA4.md#tls-and-dtls-version [7] FoxIO-LLC/ja4#150 * Fix tests where old DTLS (0x0100) was mis-identified as SSL 3.0 These two tests contain DTLS flows using a version field of 0x0100 as used by OpenSSL pre 0.9.8f, before OpenSSL switched to the standardised version code points for its DTLS implementation. The correct JA4 mapping is "d00", not "ds3".
* TLS: remove JA3C (#2679)Ivan Nardi2025-01-14
| | | | | | | | Last step of removing JA3C fingerprint Remove some duplicate tests: testing with ja4c/ja3s disabled is already performed by `disable_metadata_and_flowrisks` configuration. Close:#2551
* Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)Ivan Nardi2025-01-14
| | | | | | | | | It might be usefull to be able to match traffic against a list of suspicious JA4C fingerprints Use the same code/logic/infrastructure used for JA3C (note that we are going to remove JA3C...) See: #2551
* QUIC: remove extraction of user-agent (#2650)Ivan Nardi2025-01-07
| | | | | In very old (G)QUIC versions by Google, the user agent was available on plain text. That is not true anymore, since about end of 2021. See: https://github.com/google/quiche/commit/f282c934f4731a9f4be93409c9f3e8687f0566a7
* Classifications "by-port"/"by-ip" should never change (#2656)Ivan Nardi2025-01-06
| | | Add a new variable to keep track of internal partial classification
* Add the ability to enable/disable every specific flow risks (#2653)Ivan Nardi2025-01-06
|
* QUIC: extract "max idle timeout" parameter (#2649)Ivan Nardi2025-01-06
| | | | | Even if it is only the proposed value by the client (and not the negotiated one), it might be use as hint for timeout by the (external) flows manager
* TLS: fix `NDPI_TLS_WEAK_CIPHER` flow risk (#2647)Ivan Nardi2025-01-06
| | | | We should set it also for "obsolete"/"insecure" ciphers, not only for the "weak" ones.
* TLS: remove ESNI support (#2648)Ivan Nardi2025-01-06
| | | | | ESNI has been superseded by ECH for years, now. See: https://blog.cloudflare.com/encrypted-client-hello/ Set the existing flow risk if we still found this extension.
* TLS: export heuristic fingerprint as metadata (#2609)Ivan Nardi2024-10-28
|
* Added support for RDP over TLSLuca Deri2024-10-19
|
* Fixed JA4 invalid computation due to code bug and uninitialized valuesLuca Deri2024-10-13
|
* Added support for printing JA4r when enabledLuca Deri2024-10-11
|
* TLS: detect abnormal padding usage (#2579)Ivan Nardi2024-10-01
| | | | Padding is usually some hundreds byte long. Longer padding might be used as obfuscation technique to force unusual CH fragmentation
* TLS: heuristics: fix memory allocations (#2577)Ivan Nardi2024-09-30
| | | | Allocate heuristics state only if really needed. Fix memory leak (it happened with WebSocket traffic on port 443)
* Add some heuristics to detect encrypted/obfuscated/proxied TLS flows (#2553)Ivan Nardi2024-09-24
| | | | | | | | | | | | Based on the paper: "Fingerprinting Obfuscated Proxy Traffic with Encapsulated TLS Handshakes". See: https://www.usenix.org/conference/usenixsecurity24/presentation/xue-fingerprinting Basic idea: * the packets/bytes distribution of a TLS handshake is quite unique * this fingerprint is still detectable if the handshake is encrypted/proxied/obfuscated All heuristics are disabled by default.
* Added Sonos protocol detectionLuca Deri2024-09-24
|
* TLS: improve handling of Change Cipher message (#2564)Ivan Nardi2024-09-23
|
* Tls out of order (#2561)Ivan Nardi2024-09-18
| | | | | | | | | | | | * Revert "Added fix for handling Server Hello before CLient Hello" This reverts commit eb15b22e7757cb70894fdcde440e62bc40f22df1. * TLS: add some tests with unidirectional traffic * TLS: another attempt to process CH received after the SH Obviously, we will process unidirectional traffic longer, because we are now waiting for messages in both directions
* Added fix for handling Server Hello before CLient HelloLuca2024-09-17
|
* TLS: fix stack-buffer-overflowNardi Ivan2024-09-16
| | | | | | | | | | | | | | | | | | | | | ``` SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1812:22 ================================================================= ==97754==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ba835bde8e5 at pc 0x557ebb644241 bp 0x7ffec04b0ea0 sp 0x7ffec04b0648 WRITE of size 7 at 0x7ba835bde8e5 thread T0 #0 0x557ebb644240 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6bf240) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #1 0x557ebb645b1d in snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6c0b1d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #2 0x557ebb749dbc in ndpi_compute_ja4 /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1812:12 #3 0x557ebb7445a7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2946:10 #4 0x557ebb7073c9 in process_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1397:3 #5 0x557ebb6ff815 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data.c:46:7 #6 0x557ebb602dcb in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x67ddcb) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #7 0x557ebb5ecea8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x667ea8) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #8 0x557ebb5f299a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x66d99a) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #9 0x557ebb61c482 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x697482) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #10 0x7fa837e27082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16 #11 0x557ebb5e7b5d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x662b5d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ```
* Reworked fingerprint export now in JSONLuca2024-09-16
|
* Added ability to save JA4_r as decimal valueLuca Deri2024-09-08
|
* Implemented JA4 raw (ja4_r) fingerprintLuca Deri2024-09-05
| | | | | Example: ./example/ndpiReader -i tests/pcap/safari.pcap --cfg=tls,metadata.ja4r_fingerprint,1
* TLS: better state about handshake (#2534)Ivan Nardi2024-09-03
| | | | Keep track if we received CH or/and SH messsages: usefull with unidirectional flows
* HTTP, QUIC, TLS: allow to disable sub-classification (#2533)Ivan Nardi2024-09-03
|
* Changed NDPI_MALICIOUS_JA3 to NDPI_MALICIOUS_FINGERPRINTLuca Deri2024-08-25
|
* Introduced ndpi_master_app_protocol typedefLuca Deri2024-08-24
|
* fuzz: improve fuzzing coverage (#2474)Ivan Nardi2024-06-17
| | | | | | Remove some code never triggered AFP: the removed check is included in the following one MQTT: fix flags extraction
* TLS: add support for DTLS (over STUN) over TCP (#2463)Ivan Nardi2024-06-05
| | | | | TODO: TCP reassembler on top of UDP reassembler See: #2414
* DTLS: fix JA4 fingerprint (#2446)Ivan Nardi2024-05-21
|
* DTLS: add support for DTLS 1.3 (#2445)Ivan Nardi2024-05-21
|
* TLS: avoid setting `NDPI_TLS_SELFSIGNED_CERTIFICATE` for webrtc traffic (#2417)Ivan Nardi2024-05-06
| | | | | | See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use self-signed certificates Follow-up of b287d6ec8
* TLS: fix Ja4 fingerprint computation (#2419)Ivan Nardi2024-05-05
| | | | | | | | | | | | | | | | | The new values has been checked against the ones reported by Wireshark. Found while fixing a Use-of-uninitialized-value error reported by oss-fuzz ``` ==7582==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10 #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10 #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5 #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2 #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id
* Fixes JA4 computation adding a better GREASE detect funzionLuca Deri2024-05-02
|
Powered by cgit, Themed by Ted Yin.