| Commit message (Collapse) | Author | Age |
... | |
|
|
|
|
|
|
| |
Some QUIC flows are not properly decoded while using internal crypto
code: the authentication buffer is too small.
The new value (like the old one) is arbitrary.
Close #1463
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* The current behaviour ignores any user preferences
and was also incorrectly implemented, because the
flow->num_processed_pkts wraps every 65535 and nDPI
will process packets again until
NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.
The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.
That usually happens for the vast majority of the traffic.
However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.
Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF
This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.
A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.
TODO: IPv6
|
| |
|
|
|
|
|
| |
The '--enable-debug-messages' option works again.
Fixed warning in ahocorasick.c
Fixed integer overflow in ndpiReader.c for 32bit systems.
|
|
|
|
|
|
|
|
|
|
|
| |
While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the
companies themselves (or by some interested parties), these new lists
are directly extracted from BGP information, via AS prefixes.
*Usually*, these new lists are far more stable than the previous ones.
TODO:
* add some other ASNs (see `src/lib/ndpi_content_match.c.inc`)
* IPv6, as usual :-(
|
|
|
|
|
| |
* Extended JSON serializsation: risk, risk score, confidence
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
|
|
|
| |
TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be
ignored.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
```
protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1)
==47401==The signal is caused by a READ memory access.
==47401==Hint: address points to the zero page.
#0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41
#1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7
#2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13
```
|
|
|
| |
Fix:1e1cfb89
|
|
|
|
| |
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)
Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):
gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms
gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms
gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms
gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms
gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
--disable-padlock-support --disable-aesni-support \
--disable-shaext-support --disable-pclmul-support \
--disable-sse41-support --disable-drng-support \
--disable-avx-support --disable-avx2-support \
--disable-neon-support --disable-arm-crypto-support \
--disable-ppc-crypto-support
--disable-amd64-as-feature-detection
|
|
|
|
| |
'strcasestr' is not defined in all enviroments: quicker fix is to use
'ndpi_strncasestr' instead.
|
| |
|
|
|
|
|
|
|
|
|
| |
NDPI_SUSPICIOUS_DGA_DOMAIN,
NDPI_BINARY_APPLICATION_TRANSFER,
NDPI_HTTP_NUMERIC_IP_HOST,
NDPI_MALICIOUS_JA3,
for predefined connectivity check and cybersec categories
|
| |
|
| |
|
| |
|
| |
|
|
|
| |
Handle all message types.
|
| |
|
|
|
|
| |
leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
|
| |
|
|
|
| |
* RFC 7348
|
|
|
|
|
|
|
| |
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482
It should be the same error reported (and only partially fixed) in
79968f32
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add few scripts to easily update some IPs lists
Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.
Add the possibility to don't load a specific list.
Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.
TODO: ipv6
Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.
* Added quotes
* Added quotes
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
|
|
|
|
| |
Fixed CSV string serialization
|
|
|
|
| |
Removed attic directory now obsolete
|
| |
|
|
|
|
|
|
| |
* Checking for port 5353/5355 is not enough.
* Added additional multicast address and header checks.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
#1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
#2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
#3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
#4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
#5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
#6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
|
|
|
|
|
|
|
| |
* Sync utest results
* Fix read-heap-buffer-overflow error reported by CI
See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
|
| |
|
| |
|
|
|
|
|
|
|
| |
types of RPCs (not limited to DCE)
Extended HTTP plugin to support RPC
Improved HTTP crear text detection to limit it to Basic and Digest
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.
Correlation among flows is achieved via LRU caches.
This change allows to further reduce memory consumption (see also
91bb77a8).
At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
nDPI Memory (once): 221.15 KB
Flow Memory (per flow): 2.94 KB
```
Now:
```
nDPI Memory statistics:
nDPI Memory (once): 235.27 KB
Flow Memory (per flow): 688 B <--------
```
i.e. memory usage per flow has been reduced by 77%.
Close #1279
|
|
|
|
| |
PS VUE service has been discontinued on January 30, 2020
https://en.wikipedia.org/wiki/PlayStation_Vue
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.
Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.
This is quite important since we have lots of addresses from CDN
operators.
The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.
Fix some RX false positives.
Fix some "Uncommon TLS ALPN"-risk false positives.
Fix "confidence" value for some Zoom/Torrent classifications.
Minor fix in Lua script for Wireshark extcap.
Update .gitignore file.
Let GitHub correctly detect the language type of *.inc files.
Zattoo example has been provided by @subhajit-cdot in #1148.
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
| |
See #1312
|
| |
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|