aboutsummaryrefslogtreecommitdiff
path: root/src/lib/ndpi_main.c
Commit message (Collapse)AuthorAge
* Updated SkypeCall -> Skype_TeamsCallLuca Deri2022-05-26
|
* Moved RTSP http patterns to the protocol source file.lns2022-05-19
| | | | Signed-off-by: lns <matzeton@googlemail.com>
* DNS-over-QUIC: update default port (#1548)Ivan Nardi2022-05-12
| | | | | | Final RFC (https://www.rfc-editor.org/rfc/rfc9250.txt) changed the default port to 853 (from the 784 used on the early drafts). Since there is at least one deployed implementation still stuck at draft-02, keep also the old value, for the time being.
* Added Softether(-VPN) DDNS service detection. (#1544)Toni2022-05-09
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Added Edgecast and Cachefly CDNs. (#1540)Toni2022-05-07
| | | | | | | | * Improved ASN update script * Ran `utils/update_every_lists.sh' * `tests/do.sh.in' prints the amount of failed pcap(s) * `utils/asn_update.sh' prints the amount of failed download(s) Signed-off-by: lns <matzeton@googlemail.com>
* Fix compilation (if `--enable-debug-messages` is used) (#1539)Ivan Nardi2022-05-04
|
* Added extra check to make sure that the guessed protocol is the one we ↵Luca Deri2022-05-04
| | | | expect and not another one
* Fixes bug that prevents triggering alerts for traffic on non-standard ports ↵Luca Deri2022-05-04
| | | | | | that have been defined in the custom protocols file Fixes https://github.com/ntop/ntopng/issues/6458
* Added ndpi_get_flow_error_code() API callLuca Deri2022-05-02
| | | | Fixed typo
* Added ability to store custom category file in patricia treeLuca Deri2022-04-26
|
* Added generic user agent setter. (#1530)Toni2022-04-25
| | | | | * ndpiReader: Print user agent if one was set and not just for certain protocols. Signed-off-by: lns <matzeton@googlemail.com>
* XIAOMI: add detection of Xiaomi traffic (#1529)Ivan Nardi2022-04-25
| | | Most of the credits should go to @utoni (see #1521)
* Added RakNet protocol dissector. (#1527)Toni2022-04-24
| | | | | * Frame Set PDU's do not get fully dissected for the sake of simplicity Signed-off-by: lns <matzeton@googlemail.com>
* Code cleanup (removed redundancy)Luca Deri2022-04-22
|
* Add some scripts to easily update some IPs lists (#1522)Ivan Nardi2022-04-21
| | | | | Follow-up of 8b062295 Add a new protocol id for generic Tencent/Wechat flows
* Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol ↵Toni2022-04-20
| | | | | dissector. (#1520) Signed-off-by: lns <matzeton@googlemail.com>
* Added Toca Boca protocol dissector. (#1517)Toni2022-04-19
| | | Signed-off-by: lns <matzeton@googlemail.com>
* DGA improvementsLuca Deri2022-04-16
|
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* fix(ndpi_main):Fix memory leak about ndpi_str; (#1513)Wayne2022-04-12
| | | Co-authored-by: 林文烽 <wenfeng.lin@baishan.com>
* QUIC: handle retransmissions and overlapping fragments in reassembler ↵Vinicius Silva Nogueira2022-04-07
| | | | | | | | | | | | | | | | | | | | | (#1195) (#1498) * QUIC: handle retransmissions and overlapping fragments in reassembler * Trigger CI * minor fix: parentheses * Changing ndpi_malloc to ndpi_calloc * fix memory leak * quic_reasm_buf calloc to malloc * change order of is_ch_complete && is_reasm_buf_complete call * is_reasm_buf_complete: added handling for case where frame size is not multiple of 8 * add extra check
* DGA improvementsLuca Deri2022-04-01
|
* Moved geneated file to a separate folderLuca Deri2022-04-01
|
* Improved DGA detectionLuca Deri2022-03-31
|
* Add support for Pluralsight site (#1503)Ivan Nardi2022-03-27
|
* Reducing the size of the ndpi_detection_module_struct structure. (#1490)Vitaly Lavrov2022-03-27
| | | | | | | | | | | | | | The ndpi_detection_module_struct structure contains 5 arrays "struct ndpi_call_function_struct" size 286*144=41 kB size, which are occupied by a small number of elements. At the moment we have callback_buffer_size = 172, tcp_with_payload=114, tcp_no_payload=8, udp=93, other 8. NDPI_MAX_SUPPORTED_PROTOCOLS = 285. Size of struct ndpi_detection_module_struct is 253136 bytes. Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes. Real use memory size for struct ndpi_call_function_struct is (173+224)*144=57168 bytes.
* Improved DGA detection skipping names containign at least 3 consecutive ↵Luca Deri2022-03-26
| | | | digits in the first word
* Bug fixing. (#1487)Vitaly Lavrov2022-03-15
| | | Using the protocol_id instead of its index.
* Extend tests coverage (#1476)Ivan Nardi2022-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is at least one flow under `tests/pcap` for 249 protocols out of the 284 ones supported by nDPI. The 35 protocols without any tests are: * P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK, EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET * games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA, FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU * voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE * other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE, LOTUS_NOTES, EGP, SAP Most of these protocols (expecially the P2P and games ones) have been inherited by OpenDPI and have not been updated since then: even if they are still used, the detection rules might be outdated. However code coverage (of `lib/protocols`) only increases from 65.6% to 68.9%. Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP dissection. Treat IPP as a HTTP sub protocol. Fix Cassandra false positives. Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`: these protocol ids are defined but they are never used. Remove Collectd support: its code has never been called. If someone is really interested in this protocol, we can re-add it later, updating the dissector. Add decoding of PPI (Per-Packet Information) data link type.
* Improved ASN/IP update scripts and CI integration. (#1474)Toni2022-03-09
| | | | | * CI will print a warning if ASN/IP addresses changed. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Some small fixes (#1481)Ivan Nardi2022-03-08
| | | | | | FTP: if the authentication fails, stop analyzing the flow WSD: call the initialization routine; the dissector code has never been triggered MINING: fix dissection
* Errors fixed (#1482)Vitaly Lavrov2022-03-08
| | | | | | | | | | | | | | | Fixed errors for bigendian platforms in ndpiReader. All address and port comparisons and hash calculations are done with endian in mind. The get_ndpi_flow_info() function searched for an existing flow for the forward and reverse direction of the packet. The ndpi_workflow_node_cmp() function looked for a flow regardless of the packet's direction. This is what led to an error in determining the direction of transmission of the packet. Fixed error in "synscan" test: the number of packets in the forward and reverse direction is incorrectly defined (verified via tcpdump). Fixed bug with icmp protocol checksum check for big endian platforms.
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add ICMP checksum check and set risk if mismatch detected. (#1464)Toni2022-03-02
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added configureable ndpi packet processing limit. (#1466)Toni2022-03-02
| | | | | | | | | * The current behaviour ignores any user preferences and was also incorrectly implemented, because the flow->num_processed_pkts wraps every 65535 and nDPI will process packets again until NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add a new flow risk `NDPI_ANONYMOUS_SUBSCRIBER` (#1462)Ivan Nardi2022-02-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of a DPI engine is usually to determine "what", i.e. which types of traffic flow on the network. However the applications using DPI are often interested also in "who", i.e. which "user/subscriber" generated that traffic. The association between a flow and a subscriber is usually done via some kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element of the flow used to identify the user is the source ip address. That usually happens for the vast majority of the traffic. However, depending on the protocols involved and on the position on the net where the traffic is captured, the source ip address might have been changed/anonymized. In that case, that address is useless for any flow-username association. Example: iCloud Private Relay traffic captured between the exit relay and the server. See the picture at page 5 on: https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting that the ip addresses shouldn't be used to identify the user associated with the flow. As a first example of this new feature, the entire list of the relay ip addresses used by Private Relay is added. A key point to note is that list is NOT used for flow classification (unlike all the other ip lists present in nDPI) but only for setting this new flow risk. TODO: IPv6
* Removed some unused fields (#1461)Ivan Nardi2022-02-28
|
* Add some scripts to easily update some IPs lists (#1449)Ivan Nardi2022-02-25
| | | | | | | | | | | While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the companies themselves (or by some interested parties), these new lists are directly extracted from BGP information, via AS prefixes. *Usually*, these new lists are far more stable than the previous ones. TODO: * add some other ASNs (see `src/lib/ndpi_content_match.c.inc`) * IPv6, as usual :-(
* Provide some API functions for convenience. (#1456)Toni2022-02-25
| | | | | * Extended JSON serializsation: risk, risk score, confidence Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Replaced strdup with ndpi_strupLuca Deri2022-02-23
|
* Directly drop malformed packets (#1455)Ivan Nardi2022-02-22
| | | | TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be ignored.
* EthernetIP: add missing initialization (#1448)Ivan Nardi2022-02-20
| | | Fix:1e1cfb89
* Add support for Google Cloud (#1447)Ivan Nardi2022-02-20
| | | | Differentiate between Google its own apps/services and Google Cloud. We already do something similar for Amazon vs AWS and Microsoft vs Azure.
* Added lightweight implementation of libgcrypt. (#1444)Vitaly Lavrov2022-02-20
| | | | | | | | | | | | | | | | | | | | | Implementation borrowed from the https://github.com/ARMmbed/mbedtls.git project (v3.1.0) Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz): gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration --disable-padlock-support --disable-aesni-support \ --disable-shaext-support --disable-pclmul-support \ --disable-sse41-support --disable-drng-support \ --disable-avx-support --disable-avx2-support \ --disable-neon-support --disable-arm-crypto-support \ --disable-ppc-crypto-support --disable-amd64-as-feature-detection
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* SilencedLuca Deri2022-02-14
| | | | | | | | | NDPI_SUSPICIOUS_DGA_DOMAIN, NDPI_BINARY_APPLICATION_TRANSFER, NDPI_HTTP_NUMERIC_IP_HOST, NDPI_MALICIOUS_JA3, for predefined connectivity check and cybersec categories
* Added SNMP error code checkLuca Deri2022-02-14
|
* Added cybersecurity category mapping to stringLuca Deri2022-02-10
|
* Added cybersecurity protocol and category that groups traffic towards ↵Luca Deri2022-02-10
| | | | leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways
* Added VXLAN dissector (#1439)Dmytrii Vitman2022-02-09
| | | * RFC 7348
Powered by cgit, Themed by Ted Yin.