diff options
| author | Ivan Nardi <12729895+IvanNardi@users.noreply.github.com> | 2021-11-24 10:46:48 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-11-24 10:46:48 +0100 |
| commit | a8ffcd8bb0273d59600c6310a80b81206096c113 (patch) | |
| tree | 2a62911824363509ea5e7c69afa189e98556e495 /python | |
| parent | fd02e1b3043eecc5711eb8254aadaa3f43ca7503 (diff) | |
Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).
This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.
Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.
Hostnames/SNIs are always converted to lowercase.
Attention was given so as to be sure that unit-tests outputs are not
affected by this change.
Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
Diffstat (limited to 'python')
| -rw-r--r-- | python/ndpi.py | 7 | ||||
| -rw-r--r-- | python/ndpi_typestruct.py | 2 |
2 files changed, 4 insertions, 5 deletions
diff --git a/python/ndpi.py b/python/ndpi.py index a31a41a17..ba081acb9 100644 --- a/python/ndpi.py +++ b/python/ndpi.py @@ -986,8 +986,8 @@ struct ndpi_flow_struct { /* Place textual flow info here */ char flow_extra_info[16]; - /* HTTP host or DNS query */ - uint8_t host_server_name[240]; + char host_server_name[80]; + uint8_t initial_binary_bytes[8], initial_binary_bytes_len; uint8_t risk_checked; ndpi_risk risk; /* Issues found with this flow [bitmask of ndpi_risk] */ @@ -1051,8 +1051,7 @@ struct ndpi_flow_struct { struct { char ssl_version_str[12]; uint16_t ssl_version, server_names_len; - char client_requested_server_name[64], *server_names, - *alpn, *tls_supported_versions, *issuerDN, *subjectDN; + char *server_names, *alpn, *tls_supported_versions, *issuerDN, *subjectDN; uint32_t notBefore, notAfter; char ja3_client[33], ja3_server[33]; uint16_t server_cipher; diff --git a/python/ndpi_typestruct.py b/python/ndpi_typestruct.py index 09c1f7d7e..3909897ee 100644 --- a/python/ndpi_typestruct.py +++ b/python/ndpi_typestruct.py @@ -686,7 +686,7 @@ NDPIFlowStruct._fields_ = [ ("num_processed_pkts", c_uint8), ("extra_packets_func", CFUNCTYPE(c_int, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))), ("l4", L4), - ("host_server_name", c_ubyte * 256), + ("host_server_name", c_char * 80), ("http", Http), ("stun", Stun), ("ftp_imap_pop_smtp", FtpImapPopSmtp), |