| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.
There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN dissectors.
In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.
Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.
See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.
Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.
Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.
Close #1687
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
| |
Avoid a double call of `ndpi_guess_host_protocol_id()`.
Some code paths work for ipv4/6 both
Remove some never used code.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A lot of protocols provide the feature to upgrade their plain text
connections to an encrypted one, via some kind of "STARTTLS" command.
Add generic code to support this extension, and allow dissection of the
entire TLS handshake.
As examples, SMTP, POP, IMAP and FTP dissectors have been updated.
Since this feature requires to process more packets per flow, add the
possibility to disable it.
Fix some log messages.
Slight improvement on TCP sequence number tracking.
As a side effect, this commit fix also a memory leak found by
oss-fuzzer
```
==108966==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 22 byte(s) in 1 object(s) allocated from:
#0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063)
#1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10
#2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25
#3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
#4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
#5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
#6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9
#7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9
#8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5
#9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31
#10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10
#11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
```
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
|
|
|
|
|
|
|
|
| |
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
* CQL: increased required successful dissected packets to prevent false-positives
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
| |
* added static assert if supported, to complain if the flow struct changes
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.
Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
|
|
|
|
|
|
|
|
|
|
|
| |
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.
Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
|
|
|
|
|
|
|
| |
Tcp retransmissions should be ignored.
Remove some unused protocol bitmasks.
Update script to download Whatsapp IP list.
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
| |
* TLSv1.3 UltraSurf flows are not detected by now
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
| |
* The traces are not up to date, but this is the best we got so far.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
| |
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
Close #1598
|
| |
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
| |
- added syslog false-positive pcap that was missing in 09fbe0a64a11b08a35435f516e9a19f7e0c20d7c
- added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
There is some overlap with Citrix protocol.
|
|
|
|
| |
Code cleanup
|
|
|
| |
Update .gitignore file
|
|
|
|
|
|
| |
* Improved HTTP POST detection
* Refactored subprotocol detection
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
| |
* Improved ASN update script
* Ran `utils/update_every_lists.sh'
* `tests/do.sh.in' prints the amount of failed pcap(s)
* `utils/asn_update.sh' prints the amount of failed download(s)
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
| |
Most of the credits should go to @utoni (see #1521)
|
|
|
|
|
| |
* Frame Set PDU's do not get fully dissected for the sake of simplicity
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
| |
Follow-up of 8b062295
Add a new protocol id for generic Tencent/Wechat flows
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.
Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).
Fixed bug with icmp protocol checksum check for big endian platforms.
|
|
|
|
| |
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
|
|
|
|
| |
leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
|
|
|
|
| |
Removed attic directory now obsolete
|
|
|
|
|
|
|
| |
* Sync utest results
* Fix read-heap-buffer-overflow error reported by CI
See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.
Fix some RX false positives.
Fix some "Uncommon TLS ALPN"-risk false positives.
Fix "confidence" value for some Zoom/Torrent classifications.
Minor fix in Lua script for Wireshark extcap.
Update .gitignore file.
Let GitHub correctly detect the language type of *.inc files.
Zattoo example has been provided by @subhajit-cdot in #1148.
|