aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* Updated (C)Luca Deri2023-04-28
|
* Added missing checkLuca Deri2023-04-28
|
* Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)Ivan Nardi2023-04-25
| | | | | | | | | | | | | | | | | | | In some networks, there are some anomalous TCP flows where the smallest ACK packets have some kind of zero padding. It looks like the IP and TCP headers in those frames wrongly consider the 0x00 Ethernet padding bytes as part of the TCP payload. While this kind of packets is perfectly valid per-se, in some conditions they might be treated by the TCP reassembler logic as (partial) overlaps, deceiving the classification engine. Add an heuristic to detect these packets and to ignore them, allowing correct detection/classification. This heuristic is configurable. Default value: * in the library, it is disabled * in `ndpiReader` and in the fuzzers, it is enabled (to ease testing) Credit to @vel21ripn for the initial patch. Close #1946
* fuzz: extend fuzzers coverage (#1952)Ivan Nardi2023-04-25
|
* RTP: remove dead-code (#1953)Ivan Nardi2023-04-25
| | | | | | | | | | | | The checks `isValidMSRTPType(..) == 1` is a subset of `is_valid_rtp_payload_type()` so this if-branch is never reached. More importantly, the article describing how to detect Microsoft Lync and Skype for Business is from 2014. These payload types are static or they are in the dynamic range: in both cases, these values might be used (and they are used indeed) pretty much by every application. Bottom line: we can't use PT alone to identify a specific protocol. Keep the list, since it is used to tell audio streams from video ones.
* Compilation fix on RH7Luca Deri2023-04-24
|
* Add "Heroes of the Storm" video game signature detection. (#1949)nikitamishagin2023-04-22
|
* Improved debug output. (#1951)Toni2023-04-21
| | | | | | * try to get rid of some `printf(..)`s as they do not belong to a shared library * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added OICQ dissector. (#1950)Toni2023-04-21
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)Ivan Nardi2023-04-12
| | | | | | | | | | | | | | | | | | | `ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()` is a strange function: * it is exported by the library and it is actively used by `ntopng` * it is intrinsecally ipv4-only * it returns basically something like "classification_by_ip"/"classification_by_port" (these information have already been calculated in `ndpi_do_guess()`...) * it access the bittorrent LRU caches (similarly to `ndpi_detection_giveup()` but without all the other caches...) So: * make the interface IPv4/6 agnostic * use the classifications already available This work will allow to make the Bittorrent caches IPV6-aware (see 81e1ea5). Handle Dropbox classification "by-port" in the "standard" way.
* Added BACnet dissector. (#1940)Toni2023-04-11
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved debug logging.lns2023-04-11
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved invalid logging via printf().lns2023-04-11
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added Source Engine dissector. (#1937)Toni2023-04-11
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* RiotGames: add detection of flows (#1935)Jacopo2023-04-11
| | | | | * RiotGames: add detection of flows * remove of akamai domains
* Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hwLuca Deri2023-04-08
|
* Extend content list of Microsoft protocols (#1930)Ivan Nardi2023-04-06
|
* MySql: improve detection (#1928)Ivan Nardi2023-04-06
| | | | | Add support for flows with "caching_sha2_password" authentication plugin. See #1924
* H323: fix false positives (#1916)Ivan Nardi2023-04-06
|
* add support for gre decapsulation (#1442) (#1921)Maatuq2023-04-04
|
* fuzz: add fuzzer to test internal gcrypt code (#1920)Ivan Nardi2023-04-04
|
* Update libinjection code (#1918)Ivan Nardi2023-04-04
| | | | | | | | | | Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39 The goal is to finally fix #1820 See: https://github.com/libinjection/libinjection/issues/33 Update the corpus of the libinjection fuzzers Close #1820
* Ookla: rework detection (#1922)Ivan Nardi2023-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | The logic of the LRU cache has been changed: once we know an ip has connected to an Ookla server, all the following (unknown) flows (for a short time interval) from the same ip to the port 8080 are treated as Ookla ones. Most of the changes in this commit are about introducing the concept of "aggressive detection". In some cases, to properly detect a protocol we might use some statistical/behavior logic that, from one side, let us to identify the protocol more often but, from the other side, might lead to some false positives. To allow the user/application to easily detect when such logic has been triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been added. It is always possible to disable/configure this kind of logic via the API. Detection of Ookla flows using plain TLS over port 8080 is the first example of aggressive detection in nDPI. Tested with: * Android 9.0 with app 4.8.3 * Ubuntu 20.04 with Firefox 110 * Win 10 with app 1.15 and 1.16 * Win 10 with Chrome 108, Edge 108 and Firefox 106
* TLS: fix another interger overflow in certificate processing (#1915)Ivan Nardi2023-03-30
|
* remove redefinition to vxlanhdr struct in vxlan dissector (#1911)Maatuq2023-03-25
|
* TLS: fix parsing of certificate elements (#1910)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | | ``` ==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0) ==1228==The signal is caused by a WRITE memory access. #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79 #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3 #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15 #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6 #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9 #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 #7 0x41f78d in _start ``` Found by oss-fuzz. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
* Added the ability to define custom protocols with arbitrary Ids in proto.txtLuca Deri2023-03-22
| | | | | | | | | | Example - ip:213.75.170.11/32:443@CustomProtocol nDPI assigns an is that can change based on protos.txt content - ip:213.75.170.11/32:443@CustomProtocol=9999 nDPI assigns 9999 as protocolId to CustomProtocol and won't change when protos.txt content will chaneg
* Improved checks for duplicated entries in protocols fileLuca Deri2023-03-21
|
* DisneyPlus/Hulu ip lists should be auto-generated (#1905)Ivan Nardi2023-03-20
| | | | | | Remove two stale ip lists: 1) these 3 ips are in the Amazon ranges (now)... 2) the Instagram list originated from AS32934, which is now a Facebook AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
* Add a new protocol id for generic Adult Content traffic (#1906)Ivan Nardi2023-03-20
| | | | The list has been taken from https://www.similarweb.com/top-websites/adult/ Fix a GoTo false positive.
* Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)Ivan Nardi2023-03-20
|
* fuzz: add a new fuzzer to test TLS certificates (#1901)Ivan Nardi2023-03-20
|
* FreeBSD compilation fixLuca Deri2023-03-14
| | | | (C) update
* fuzz: simplify fuzzers dependencies in CIFuzz (#1896)Ivan Nardi2023-03-14
| | | | | | | | | | | | | | | | | | | | | | CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz targets on pull requests. It only runs the fuzzers affected by a pull request/commit. Otherwise it will divide up the allotted fuzzing time among all fuzzers in the project. Since: * we have more than 20 fuzzers and most of them use the custom memory allocation functions (to force allocation failures) even if they are not strictly about DPI stuff; * we need to keep fuzzing time relatively small (to avoid waiting the CI results for a long time) it is important that fuzzers dependencies (which are based on *files* changed by the single commit/PR) are as small as possible. Bottom line: move all the low-level allocation callbacks to a dedicated file; this way most of the fuzzers don't depend anymore on `ndpi_main.c` file (which is touched by ever commit/PR). The goal is to have only the "most important" fuzzers running during (most of) the CI.
* Added ability to define an unlimited number of custom rules IP:port for the ↵Luca Deri2023-03-13
| | | | same IP (it used tobe limited to 2)
* Minor cleanupLuca Deri2023-03-13
|
* Added fix for better handling exceptions rollback in case of later matchLuca Deri2023-03-06
|
* Add a new flow risk about literal IP addresses used as SNI (#1892)Ivan Nardi2023-03-02
| | | | | | | | | | RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in "HostName"." Don't set this risk if we have a valid sub-classification (example: via certificate) Since a similar risk already exists for HTTP hostnames, reuse it, with a more generic name.
* Fix insert of ip addresses into patricia tree(s) (#1895)Ivan Nardi2023-03-02
| | | Fix: d0b46dcf
* Keep master protocol in `ndpi_reconcile_protocols`Nardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | As a side effect of this change, the following memory leak has been fixed. ``` ==55523==ERROR: LeakSanitizer: detected memory leaks Direct leak of 2 byte(s) in 1 object(s) allocated from: #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52) #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25 #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13 #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46 #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5 #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
* SoftEther: fix invalid memory accessNardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | | We can't write `flow->protos` union until we are really sure about protocol classification ``` ==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0) ==28334==The signal is caused by a READ memory access. ==28334==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7 #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2 #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5 #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
* Fixed uninitialized variableLuca Deri2023-02-27
|
* Added missing breakLuca Deri2023-02-27
|
* Improved protocol detection exploiting IP-based guessLuca Deri2023-02-27
| | | | Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
* Enhanced custom port definition and improved error reporting in case of ↵Luca Deri2023-02-24
| | | | duplications
* fuzz: extend fuzz coverage (#1888)Ivan Nardi2023-02-16
|
* Fix export/serialization of `flow->risk` (#1885)Ivan Nardi2023-02-14
| | | | Any risks bigger than 31 was never exported/serialized; found looking a the fuzz coverage report.
* Fix packet counters (#1884)Ivan Nardi2023-02-13
| | | | | | We need to keep separete counters to keep tracks of packet numbers with and without any payload. Regression introduced in 5849863ef
* Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)Ivan Nardi2023-02-13
|
* Centos7 fixesLuca Deri2023-02-13
|
Powered by cgit, Themed by Ted Yin.