aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols
Commit message (Collapse)AuthorAge
...
* Compilation fix on RH7Luca Deri2023-04-24
|
* Add "Heroes of the Storm" video game signature detection. (#1949)nikitamishagin2023-04-22
|
* Added OICQ dissector. (#1950)Toni2023-04-21
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)Ivan Nardi2023-04-12
| | | | | | | | | | | | | | | | | | | `ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()` is a strange function: * it is exported by the library and it is actively used by `ntopng` * it is intrinsecally ipv4-only * it returns basically something like "classification_by_ip"/"classification_by_port" (these information have already been calculated in `ndpi_do_guess()`...) * it access the bittorrent LRU caches (similarly to `ndpi_detection_giveup()` but without all the other caches...) So: * make the interface IPv4/6 agnostic * use the classifications already available This work will allow to make the Bittorrent caches IPV6-aware (see 81e1ea5). Handle Dropbox classification "by-port" in the "standard" way.
* Added BACnet dissector. (#1940)Toni2023-04-11
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added Source Engine dissector. (#1937)Toni2023-04-11
| | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* MySql: improve detection (#1928)Ivan Nardi2023-04-06
| | | | | Add support for flows with "caching_sha2_password" authentication plugin. See #1924
* H323: fix false positives (#1916)Ivan Nardi2023-04-06
|
* Ookla: rework detection (#1922)Ivan Nardi2023-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | The logic of the LRU cache has been changed: once we know an ip has connected to an Ookla server, all the following (unknown) flows (for a short time interval) from the same ip to the port 8080 are treated as Ookla ones. Most of the changes in this commit are about introducing the concept of "aggressive detection". In some cases, to properly detect a protocol we might use some statistical/behavior logic that, from one side, let us to identify the protocol more often but, from the other side, might lead to some false positives. To allow the user/application to easily detect when such logic has been triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been added. It is always possible to disable/configure this kind of logic via the API. Detection of Ookla flows using plain TLS over port 8080 is the first example of aggressive detection in nDPI. Tested with: * Android 9.0 with app 4.8.3 * Ubuntu 20.04 with Firefox 110 * Win 10 with app 1.15 and 1.16 * Win 10 with Chrome 108, Edge 108 and Firefox 106
* TLS: fix another interger overflow in certificate processing (#1915)Ivan Nardi2023-03-30
|
* remove redefinition to vxlanhdr struct in vxlan dissector (#1911)Maatuq2023-03-25
|
* TLS: fix parsing of certificate elements (#1910)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | | ``` ==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0) ==1228==The signal is caused by a WRITE memory access. #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79 #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3 #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15 #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6 #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9 #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 #7 0x41f78d in _start ``` Found by oss-fuzz. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
* Add a new protocol id for generic Adult Content traffic (#1906)Ivan Nardi2023-03-20
| | | | The list has been taken from https://www.similarweb.com/top-websites/adult/ Fix a GoTo false positive.
* fuzz: add a new fuzzer to test TLS certificates (#1901)Ivan Nardi2023-03-20
|
* Add a new flow risk about literal IP addresses used as SNI (#1892)Ivan Nardi2023-03-02
| | | | | | | | | | RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in "HostName"." Don't set this risk if we have a valid sub-classification (example: via certificate) Since a similar risk already exists for HTTP hostnames, reuse it, with a more generic name.
* SoftEther: fix invalid memory accessNardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | | We can't write `flow->protos` union until we are really sure about protocol classification ``` ==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0) ==28334==The signal is caused by a READ memory access. ==28334==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7 #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2 #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5 #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* File rename to avoid name clashes (e.g. on Windows)Luca Deri2023-01-31
|
* Updated (C)Luca Deri2023-01-31
|
* Some small changes (#1869)Ivan Nardi2023-01-25
| | | | | | | | All dissector callbacks should not be exported by the library; make static some other local functions. The callback logic in `ndpiReader` has never been used. With internal libgcrypt, `gcry_control()` should always return no errors. We can check `categories` length at compilation time.
* Bittorrent: fix detection over TCP (#1868)Ivan Nardi2023-01-19
| | | Close #1866
* Bittorrent: fix heap-buffer-overflow (#1863)Ivan Nardi2023-01-19
| | | | | | | | | | | | | | | | | ``` ==258287==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068ff9d at pc 0x5653a6e35def bp 0x7ffeef5aa620 sp 0x7ffeef5a9dc8 READ of size 22 at 0x60600068ff9d thread T0 #0 0x5653a6e35dee in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x4d2dee) (BuildId: 133b8c3c8ff99408109fcb9be2538bb8341f07f7) #1 0x5653a70d6624 in ndpi_search_bittorrent /home/ivan/svnrepos/nDPI/src/lib/protocols/bittorrent.c:500:71 #2 0x5653a6ff255a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5686:6 #3 0x5653a6ff331b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5722:10 #4 0x5653a6ff2cbc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5755:12 #5 0x5653a70016bf in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6578:15 #6 0x5653a6f1836d in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1678:31 #7 0x5653a6f140a1 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2256:10 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55218 Fix: 470eaa6f
* LRU caches: add a generic (optional and configurable) expiration logic (#1855)Ivan Nardi2023-01-18
| | | Two caches already implemented a similar mechanism: make it generic.
* POP3: improve detection (#1856)Ivan Nardi2023-01-18
|
* Improve support for Snapchat voip calls (#1858)Ivan Nardi2023-01-17
| | | | Latest Snapchat versions use QUICv1 for their audio/video real time sessions. See c50a8d480
* Add detection of Tailscale (#1857)Ivan Nardi2023-01-17
| | | Extend the example of wireguard traffic
* Add Meraki Cloud protocol and improve other protocols (#1854)Ivan Nardi2023-01-17
| | | | | Avoid some LineCall and Jabber false positives. Detect Discord mid flows. Fix Bittorrent detection.
* PPSTREAM: fix some heap-buffer overflows (#1846)Ivan Nardi2023-01-05
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54802
* Fix missing bracket at ppstream (#1843)Ege Çetin2023-01-03
| | | | | | | * add missing bracket * Sync unit test results Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
* Fix some warnings and two errors found while fuzzing (#1844)Ivan Nardi2023-01-03
| | | | Fix CI See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
* Added NDPI_MINOR_ISSUES risk used for storing generic/relevant information ↵Luca Deri2022-12-31
| | | | about issues found on traffic.
* Implemented EDNS(0) support in DNS dissectorLuca Deri2022-12-29
| | | | Improved DNS dissection
* Remove some old protocols (#1832)Ivan Nardi2022-12-23
| | | | | | | | | | | | | | | | | | | | | These protocols: * have been addeded in the OpenDPI era * have never been updated since then * we don't have any pcap examples [*] If (and it is a big if...) some of these protocols are still somehow used and if someone is still interested in them, we can probably re-add them starting from scratch (because the current detection rules are probably outdated) Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT, OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA, FLORENSIA, AIMINI, SOPCAST PPSTREAM dissector works (...) only on UDP. [*]: with do have an AIMINI test pcap but it was some trivial http traffic detected only by hostname matching, on domains no more available...
* postgres: improve detection (#1831)Ivan Nardi2022-12-22
| | | Remove some dead code (found via coverage report)
* Added TP-LINK Smart Home Protocol dissector. (#1841)Darryl Sokoloski2022-12-20
| | | | | Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca> Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
* Added TUYA LAN Protocol dissector. (#1838)Darryl Sokoloski2022-12-19
| | | | | | | | Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n. Used by many "smart" devices such as LED light strips, bulbs, etc. Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca> Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
* Add protocol disabling feature (#1808)Ivan Nardi2022-12-18
| | | | | | | | | | | | | | | | | | | | | | The application may enable only some protocols. Disabling a protocol means: *) don't register/use the protocol dissector code (if any) *) disable classification by-port for such a protocol *) disable string matchings for domains/certificates involving this protocol *) disable subprotocol registration (if any) This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`. Custom protocols are always enabled. Technically speaking, this commit doesn't introduce any API/ABI incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()` is now mandatory, just after having called `ndpi_init_detection_module()`. Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to the removing of some function parameters. Fix the low level macro `NDPI_LOG`. This issue hasn't been detected sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
* Added ability to decode DNS PTR records used for reverse address resolutionLuca Deri2022-12-15
|
* Minor RTP fixLuca Deri2022-12-15
|
* STUN: add detection of ZOOM peer-to-peer flows (#1825)Ivan Nardi2022-12-11
| | | | See: "Enabling Passive Measurement of Zoom Performance in Production Networks" https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
* fuzz: some enhancements (#1827)Ivan Nardi2022-12-10
| | | | | | | | | | Load some custom configuration (like in the unit tests) and factorize some (fuzzing) common code. There is no way to pass file paths to the fuzzers as parameters. The safe solution seems to be to load them from the process working dir. Anyway, missing file is not a blocking error. Remove some dead code (found looking at the coverage report)
* DTLS: handle (certificate) fragments (#1811)Ivan Nardi2022-12-10
| | | | | Keep using the existing function to handle reassembling buffer: rename it from `ndpi_search_tls_tcp_memory` to `ndpi_search_tls_memory` and make it "transport" agnostic
* Added Zoom screen share detectionLuca Deri2022-12-09
|
* Added RTP stream type in flow metadataLuca Deri2022-12-09
|
* Improved Zoom protocol detectionLuca Deri2022-12-08
|
* fuzz: add a new fuzzer testing memory allocation failures (#1818)Ivan Nardi2022-12-06
| | | | | | | | | Try to fuzz error paths triggered by allocation errors. Fix some errors already found by this new fuzzer. Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to (not)compile specific code in fuzzing builds. See: https://llvm.org/docs/LibFuzzer.html
* Exported HTTP server in metadataLuca Deri2022-12-05
|
* Make LRU caches ipv6 aware (#1810)Ivan Nardi2022-12-03
| | | | | | | | | | | | | Simplest solution, keeping the existing cache data structure TLS certificate cache is used for DTLS traffic, too. Note that Ookla cache already works with ipv6 flows. TODO: * make the key/hashing more robust (extending the key size?) * update bittorrent cache too. That task is quite difficult because ntopng uses a public function (`ndpi_guess_undetected_protocol()`) intrinsically ipv4 only...
* Added check for empty/missing user-agent in HTTPLuca Deri2022-12-03
|
* Add a new fuzzer for QUIC (#1800)Ivan Nardi2022-12-01
| | | | | | | | | | | | QUIC packets are encrypted/obfuscated; that means that we need to decrypt them before parsing the real (TLS) message. Fuzzing is not effective here, since a random buffer is hardly a valid encrypted QUIC packet. Add a new fuzzer, testing *decrypted* QUIC packets. Add a basic corpus. Fix a few bugs already found by this fuzzer.
Powered by cgit, Themed by Ted Yin.