aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Internal crypto: increase size of authentication buffer (#1468)Ivan Nardi2022-03-02
| | | | | | | Some QUIC flows are not properly decoded while using internal crypto code: the authentication buffer is too small. The new value (like the old one) is arbitrary. Close #1463
* reader_util: fix parsing of MPLS packets (#1467)Ivan Nardi2022-03-02
| | | | | | | | | | | | | | ``` ==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8 READ of size 4 at 0x60300000578c thread T0 #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16 #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37) 0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d) ```
* Add ICMP checksum check and set risk if mismatch detected. (#1464)Toni2022-03-02
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TypoLuca Deri2022-03-02
|
* Added configureable ndpi packet processing limit. (#1466)Toni2022-03-02
| | | | | | | | | * The current behaviour ignores any user preferences and was also incorrectly implemented, because the flow->num_processed_pkts wraps every 65535 and nDPI will process packets again until NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix libgcrypt(-light/-internal) compile error. (#1465)Toni2022-03-02
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add a new flow risk `NDPI_ANONYMOUS_SUBSCRIBER` (#1462)Ivan Nardi2022-02-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of a DPI engine is usually to determine "what", i.e. which types of traffic flow on the network. However the applications using DPI are often interested also in "who", i.e. which "user/subscriber" generated that traffic. The association between a flow and a subscriber is usually done via some kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element of the flow used to identify the user is the source ip address. That usually happens for the vast majority of the traffic. However, depending on the protocols involved and on the position on the net where the traffic is captured, the source ip address might have been changed/anonymized. In that case, that address is useless for any flow-username association. Example: iCloud Private Relay traffic captured between the exit relay and the server. See the picture at page 5 on: https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting that the ip addresses shouldn't be used to identify the user associated with the flow. As a first example of this new feature, the entire list of the relay ip addresses used by Private Relay is added. A key point to note is that list is NOT used for flow classification (unlike all the other ip lists present in nDPI) but only for setting this new flow risk. TODO: IPv6
* Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt ↵Luca Deri2022-02-28
| | | | implementation
* Removed some unused fields (#1461)Ivan Nardi2022-02-28
|
* Bug fixing. (#1459)Vitaly Lavrov2022-02-28
| | | | | The '--enable-debug-messages' option works again. Fixed warning in ahocorasick.c Fixed integer overflow in ndpiReader.c for 32bit systems.
* Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)Toni2022-02-28
| | | | | * CI job generates a coverage report Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* reader_util: fix TZSP decapsulation (#1460)Ivan Nardi2022-02-27
| | | | | | | | | | | | | | | | | ``` ==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48 READ of size 1 at 0x60400008957f thread T0 #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18 #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98) 0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f) allocated by thread T0 here: ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
* Add some scripts to easily update some IPs lists (#1449)Ivan Nardi2022-02-25
| | | | | | | | | | | While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the companies themselves (or by some interested parties), these new lists are directly extracted from BGP information, via AS prefixes. *Usually*, these new lists are far more stable than the previous ones. TODO: * add some other ASNs (see `src/lib/ndpi_content_match.c.inc`) * IPv6, as usual :-(
* Provide some API functions for convenience. (#1456)Toni2022-02-25
| | | | | * Extended JSON serializsation: risk, risk score, confidence Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Win fixesLuca Deri2022-02-23
|
* Replaced strdup with ndpi_strupLuca Deri2022-02-23
|
* Directly drop malformed packets (#1455)Ivan Nardi2022-02-22
| | | | TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be ignored.
* reader_util: fix parsing of IPv6 extension headers (#1453)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
* reader_util: fix infinite loop in packet dissection (#1454)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
* fuzz: purge old sessions (#1451)Ivan Nardi2022-02-21
| | | | | | | | At every fuzz iteration (i.e for every trace file): * keep the same ndpi context (`ndpi_init_detection_module` is very slow); * reset the flow table, otherwise it grows indefinitely. This change should fix the "out-of-memory" errors reported by oss-fuzz.
* DTLS: fix access to certificate cache (#1450)Ivan Nardi2022-02-21
| | | | | | | | | | | | | | | | ``` protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short') SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in AddressSanitizer:DEADLYSIGNAL ================================================================= ==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1) ==47401==The signal is caused by a READ memory access. ==47401==Hint: address points to the zero page. #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41 #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7 #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13 ```
* EthernetIP: add missing initialization (#1448)Ivan Nardi2022-02-20
| | | Fix:1e1cfb89
* Add support for Google Cloud (#1447)Ivan Nardi2022-02-20
| | | | Differentiate between Google its own apps/services and Google Cloud. We already do something similar for Amazon vs AWS and Microsoft vs Azure.
* fuzz: make fuzz_ndpi_reader faster (#1446)Ivan Nardi2022-02-20
| | | | | | | | | | | | | | | Initialize ndpi_workflow_init context only once. On a quite old notebook, before: ``` $ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/ [...] Done 3256 runs in 361 second(s) ``` after: ``` Done 5032771 runs in 361 second(s) <----------- ~1400X ``` oss-fuzz will be happy!
* Added lightweight implementation of libgcrypt. (#1444)Vitaly Lavrov2022-02-20
| | | | | | | | | | | | | | | | | | | | | Implementation borrowed from the https://github.com/ARMmbed/mbedtls.git project (v3.1.0) Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz): gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration --disable-padlock-support --disable-aesni-support \ --disable-shaext-support --disable-pclmul-support \ --disable-sse41-support --disable-drng-support \ --disable-avx-support --disable-avx2-support \ --disable-neon-support --disable-arm-crypto-support \ --disable-ppc-crypto-support --disable-amd64-as-feature-detection
* Fix compilation and sync unit tests results (#1445)Ivan Nardi2022-02-19
| | | | 'strcasestr' is not defined in all enviroments: quicker fix is to use 'ndpi_strncasestr' instead.
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* SilencedLuca Deri2022-02-14
| | | | | | | | | NDPI_SUSPICIOUS_DGA_DOMAIN, NDPI_BINARY_APPLICATION_TRANSFER, NDPI_HTTP_NUMERIC_IP_HOST, NDPI_MALICIOUS_JA3, for predefined connectivity check and cybersec categories
* Extended cybersecurity protocol dissectionLuca Deri2022-02-14
|
* Added SNMP error code checkLuca Deri2022-02-14
|
* Exteended cybersecurity listLuca Deri2022-02-10
|
* Invalid prototupe fixLuca Deri2022-02-10
|
* HSRP: fix dissection over IPv6 (#1443)Ivan Nardi2022-02-10
| | | Handle all message types.
* Added cybersecurity category mapping to stringLuca Deri2022-02-10
|
* Added cybersecurity protocol and category that groups traffic towards ↵Luca Deri2022-02-10
| | | | leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
* HSRP: add support for IPv6 (#1440)Ivan Nardi2022-02-09
|
* Added VXLAN dissector (#1439)Dmytrii Vitman2022-02-09
| | | * RFC 7348
* Fix memory access in ndpi_strnstr() (#1438)Ivan Nardi2022-02-09
| | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482 It should be the same error reported (and only partially fixed) in 79968f32
* Add few scripts to easily update some IPs lists (#1436)Ivan Nardi2022-02-09
| | | | | | | | | | | | | | | | | | | | | | | | * Add few scripts to easily update some IPs lists Some IPs lists should be updated frequently: try to easy the process. The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure addresses): one specific .c.inc file and one script for each protocol. Add the possibility to don't load a specific list. Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK, to identify Hotmail/Outlook/Exchange flows. TODO: ipv6 Remove the 9 addresses associated to BitTorrent: they have been added in e2f21116 but it is not clear why all the traffic to/from these ips should be classified as BitTorrent. * Added quotes * Added quotes Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Increment current/total number of active flows on successful flow insertion ↵ol-andreyizrailev2022-02-09
| | | | | | | | (#1434) Memory allocation or ndpi_tsearch might fail, so the two values should be incremented only when insertion actually happened. Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
* Added ndpi_serialize_string_string_len() APi callLuca Deri2022-02-08
| | | | Fixed CSV string serialization
* Added HSRP protocol detectionLuca Deri2022-02-08
| | | | Removed attic directory now obsolete
* Added check to ignore multicast packets marking the as SkypeLuca Deri2022-02-08
|
* Improved MDNS/LLMNR detection. (#1437)Toni2022-02-07
| | | | | | * Checking for port 5353/5355 is not enough. * Added additional multicast address and header checks. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: fix parsing of certificate elements (#1435)Ivan Nardi2022-02-07
| | | | | | | | | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280 ``` ==263603==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9 #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7 #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7 #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13 #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2 #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5 #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6 ```
* Sync utests (#1433)Ivan Nardi2022-02-04
| | | | | | | * Sync utest results * Fix read-heap-buffer-overflow error reported by CI See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
* Add commentAlfredo Cardigliano2022-02-03
|
* Updated test resultsLuca Deri2022-02-03
|
* Added NDPI_ERROR_CODE_DETECTED riskLuca Deri2022-02-03
|
Powered by cgit, Themed by Ted Yin.