| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
* added static assert if supported, to complain if the flow struct changes
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
| |
* fixed RiotGames false positive
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.
Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
|
|
|
|
|
|
|
|
|
|
|
| |
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.
Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
|
|
|
|
|
|
|
| |
Tcp retransmissions should be ignored.
Remove some unused protocol bitmasks.
Update script to download Whatsapp IP list.
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.
Add basic logic to autodetect this kind of information.
nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.
These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
Added ability to identify application and network protocols
|
|
|
|
| |
Code cleanup
|
| |
|
| |
|
|
|
| |
Fix: 7a3aa41a
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.
In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).
Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.
Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).
Add a partial list of some ADS/Tracking stuff.
Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.
Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.
Fix 3 LGTM warnings.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).
This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.
Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.
Hostnames/SNIs are always converted to lowercase.
Attention was given so as to be sure that unit-tests outputs are not
affected by this change.
Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We can write to `flow->protos` only after a proper classification.
This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
* write to `flow->protos` only if a final protocol has been detected
* move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.
Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.
For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.
Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.
Extend code coverage of fuzz tests.
Close #1343
Close #1342
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.
Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Before
Accuracy 66%, Precision 86%, Recall 38%
After
Accuracy 71%, Precision 89%, Recall 49%
|
|
|
|
|
| |
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
|
| |
|
| |
|
| |
|
|
|
|
| |
not condidered safe/secure
|
|
|
|
| |
Optimized stddev calculation
|
| |
|
| |
|
| |
|
|
|
|
| |
Packet bins are not printed wehn empty
|
|
|
|
| |
Added packet lenght distribution bins
|