aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* reader_util: fix TZSP decapsulation (#1460)Ivan Nardi2022-02-27
| | | | | | | | | | | | | | | | | ``` ==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48 READ of size 1 at 0x60400008957f thread T0 #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18 #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98) 0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f) allocated by thread T0 here: ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
* Add some scripts to easily update some IPs lists (#1449)Ivan Nardi2022-02-25
| | | | | | | | | | | While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the companies themselves (or by some interested parties), these new lists are directly extracted from BGP information, via AS prefixes. *Usually*, these new lists are far more stable than the previous ones. TODO: * add some other ASNs (see `src/lib/ndpi_content_match.c.inc`) * IPv6, as usual :-(
* Provide some API functions for convenience. (#1456)Toni2022-02-25
| | | | | * Extended JSON serializsation: risk, risk score, confidence Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Win fixesLuca Deri2022-02-23
|
* Replaced strdup with ndpi_strupLuca Deri2022-02-23
|
* Directly drop malformed packets (#1455)Ivan Nardi2022-02-22
| | | | TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be ignored.
* reader_util: fix parsing of IPv6 extension headers (#1453)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
* reader_util: fix infinite loop in packet dissection (#1454)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
* fuzz: purge old sessions (#1451)Ivan Nardi2022-02-21
| | | | | | | | At every fuzz iteration (i.e for every trace file): * keep the same ndpi context (`ndpi_init_detection_module` is very slow); * reset the flow table, otherwise it grows indefinitely. This change should fix the "out-of-memory" errors reported by oss-fuzz.
* DTLS: fix access to certificate cache (#1450)Ivan Nardi2022-02-21
| | | | | | | | | | | | | | | | ``` protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short') SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in AddressSanitizer:DEADLYSIGNAL ================================================================= ==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1) ==47401==The signal is caused by a READ memory access. ==47401==Hint: address points to the zero page. #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41 #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7 #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13 ```
* EthernetIP: add missing initialization (#1448)Ivan Nardi2022-02-20
| | | Fix:1e1cfb89
* Add support for Google Cloud (#1447)Ivan Nardi2022-02-20
| | | | Differentiate between Google its own apps/services and Google Cloud. We already do something similar for Amazon vs AWS and Microsoft vs Azure.
* fuzz: make fuzz_ndpi_reader faster (#1446)Ivan Nardi2022-02-20
| | | | | | | | | | | | | | | Initialize ndpi_workflow_init context only once. On a quite old notebook, before: ``` $ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/ [...] Done 3256 runs in 361 second(s) ``` after: ``` Done 5032771 runs in 361 second(s) <----------- ~1400X ``` oss-fuzz will be happy!
* Added lightweight implementation of libgcrypt. (#1444)Vitaly Lavrov2022-02-20
| | | | | | | | | | | | | | | | | | | | | Implementation borrowed from the https://github.com/ARMmbed/mbedtls.git project (v3.1.0) Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz): gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration --disable-padlock-support --disable-aesni-support \ --disable-shaext-support --disable-pclmul-support \ --disable-sse41-support --disable-drng-support \ --disable-avx-support --disable-avx2-support \ --disable-neon-support --disable-arm-crypto-support \ --disable-ppc-crypto-support --disable-amd64-as-feature-detection
* Fix compilation and sync unit tests results (#1445)Ivan Nardi2022-02-19
| | | | 'strcasestr' is not defined in all enviroments: quicker fix is to use 'ndpi_strncasestr' instead.
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* SilencedLuca Deri2022-02-14
| | | | | | | | | NDPI_SUSPICIOUS_DGA_DOMAIN, NDPI_BINARY_APPLICATION_TRANSFER, NDPI_HTTP_NUMERIC_IP_HOST, NDPI_MALICIOUS_JA3, for predefined connectivity check and cybersec categories
* Extended cybersecurity protocol dissectionLuca Deri2022-02-14
|
* Added SNMP error code checkLuca Deri2022-02-14
|
* Exteended cybersecurity listLuca Deri2022-02-10
|
* Invalid prototupe fixLuca Deri2022-02-10
|
* HSRP: fix dissection over IPv6 (#1443)Ivan Nardi2022-02-10
| | | Handle all message types.
* Added cybersecurity category mapping to stringLuca Deri2022-02-10
|
* Added cybersecurity protocol and category that groups traffic towards ↵Luca Deri2022-02-10
| | | | leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
* HSRP: add support for IPv6 (#1440)Ivan Nardi2022-02-09
|
* Added VXLAN dissector (#1439)Dmytrii Vitman2022-02-09
| | | * RFC 7348
* Fix memory access in ndpi_strnstr() (#1438)Ivan Nardi2022-02-09
| | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482 It should be the same error reported (and only partially fixed) in 79968f32
* Add few scripts to easily update some IPs lists (#1436)Ivan Nardi2022-02-09
| | | | | | | | | | | | | | | | | | | | | | | | * Add few scripts to easily update some IPs lists Some IPs lists should be updated frequently: try to easy the process. The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure addresses): one specific .c.inc file and one script for each protocol. Add the possibility to don't load a specific list. Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK, to identify Hotmail/Outlook/Exchange flows. TODO: ipv6 Remove the 9 addresses associated to BitTorrent: they have been added in e2f21116 but it is not clear why all the traffic to/from these ips should be classified as BitTorrent. * Added quotes * Added quotes Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
* Increment current/total number of active flows on successful flow insertion ↵ol-andreyizrailev2022-02-09
| | | | | | | | (#1434) Memory allocation or ndpi_tsearch might fail, so the two values should be incremented only when insertion actually happened. Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
* Added ndpi_serialize_string_string_len() APi callLuca Deri2022-02-08
| | | | Fixed CSV string serialization
* Added HSRP protocol detectionLuca Deri2022-02-08
| | | | Removed attic directory now obsolete
* Added check to ignore multicast packets marking the as SkypeLuca Deri2022-02-08
|
* Improved MDNS/LLMNR detection. (#1437)Toni2022-02-07
| | | | | | * Checking for port 5353/5355 is not enough. * Added additional multicast address and header checks. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: fix parsing of certificate elements (#1435)Ivan Nardi2022-02-07
| | | | | | | | | | | | | | Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280 ``` ==263603==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9 #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7 #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7 #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13 #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2 #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5 #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6 ```
* Sync utests (#1433)Ivan Nardi2022-02-04
| | | | | | | * Sync utest results * Fix read-heap-buffer-overflow error reported by CI See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
* Add commentAlfredo Cardigliano2022-02-03
|
* Updated test resultsLuca Deri2022-02-03
|
* Added NDPI_ERROR_CODE_DETECTED riskLuca Deri2022-02-03
|
* Renamed DCERPC to more generic RPC protocol so we can use also for other ↵Luca Deri2022-02-03
| | | | | | | types of RPCs (not limited to DCE) Extended HTTP plugin to support RPC Improved HTTP crear text detection to limit it to Basic and Digest
* TypoLuca Deri2022-02-03
|
* Improved risks descriptionLuca Deri2022-02-03
|
* Updated risk documentationLuca Deri2022-02-03
|
* Added new IDN/Punycode risk for spotting internationalized domain namesLuca2022-02-03
|
* Added missing __sync_fetch_and_add() definition in WindowsLuca Deri2022-02-02
|
* Moved to 4.3Luca2022-02-01
|
* Update changelogAlfredo Cardigliano2022-01-31
|
* Remove `struct ndpi_id_struct` (#1427)Ivan Nardi2022-01-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the last uses of `struct ndpi_id_struct`. That code is not really used and it has not been updated for a very long time: see #1279 for details. Correlation among flows is achieved via LRU caches. This change allows to further reduce memory consumption (see also 91bb77a8). At nDPI 4.0 (more precisly, at a6b10cf, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 235.27 KB Flow Memory (per flow): 688 B <-------- ``` i.e. memory usage per flow has been reduced by 77%. Close #1279
* Remove Playstation VUE protocol (#1426)Ivan Nardi2022-01-30
| | | | PS VUE service has been discontinued on January 30, 2020 https://en.wikipedia.org/wiki/PlayStation_Vue
* Commented old code (see https://github.com/ntop/nDPI/pull/1425)Luca Deri2022-01-30
|
* Improve protocol stacks (#1425)Ivan Nardi2022-01-30
| | | | | | | | | | | | | | | | | We should have two protocols in classification results only when the "master" protocol allows some sub-protocols. Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`, `DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading. For example, `ndpiReader`shows `BitTorrent/Azure` flows under `Azure` statistics; that seems to be wrong or, at least, very misleading. This is quite important since we have lots of addresses from CDN operators. The only drawback of this solution is that right now ICMP traffic is classified simply as `ICMP`; if we are really interested in ICMP stuff we can restore the old behaviour later.
Powered by cgit, Themed by Ted Yin.