aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
* Moved geneated file to a separate folderLuca Deri2022-04-01
|
* Improved twitter detectionLuca Deri2022-04-01
|
* Removed SRV record from suspicious DNS trafficLuca Deri2022-03-31
|
* Improved DGA detectionLuca Deri2022-03-31
|
* Add support for Pluralsight site (#1503)Ivan Nardi2022-03-27
|
* Reducing the size of the ndpi_detection_module_struct structure. (#1490)Vitaly Lavrov2022-03-27
| | | | | | | | | | | | | | The ndpi_detection_module_struct structure contains 5 arrays "struct ndpi_call_function_struct" size 286*144=41 kB size, which are occupied by a small number of elements. At the moment we have callback_buffer_size = 172, tcp_with_payload=114, tcp_no_payload=8, udp=93, other 8. NDPI_MAX_SUPPORTED_PROTOCOLS = 285. Size of struct ndpi_detection_module_struct is 253136 bytes. Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes. Real use memory size for struct ndpi_call_function_struct is (173+224)*144=57168 bytes.
* [SSDP] Extract HTTP user-agent when available. (#1500)Darryl Sokoloski2022-03-27
| | | | | | [SSDP] Added capture file with UA header. [SSDP] Added pcap test output log file. Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
* Improved DGA detection skipping names containign at least 3 consecutive ↵Luca Deri2022-03-26
| | | | digits in the first word
* QUIC: add support for version 2 draft 01 (#1493)Ivan Nardi2022-03-25
| | | | | | Support for v2-00 has been removed (it has never been used in real networks and it is incompatible with v2-01). Chrome already supports v2-01 in latest versions in Chrome Beta channel.
* Mining: cleanup registration (#1496)Ivan Nardi2022-03-25
| | | | | Use the same pattern of all the other dissectors: one registration and one callback. Spotted by @dsokoloski
* Trying to improve QUIC reassembler (#1195) (#1489)Vinicius Silva Nogueira2022-03-24
| | | | | | | | | | | * handling QUIC out-of-order fragments * minor fix * updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out * quic test: buf_len + last_pos * QUIC: comment update in __reassemble function and minor change is_ch_complete function
* Complete rework of nDPI Python bindings (cffi API, automatic generation, ↵aouinizied2022-03-22
| | | | packaging and CI integration)
* Extended the list of cybersecurity protocolLuca Deri2022-03-21
|
* Bug fixing. (#1487)Vitaly Lavrov2022-03-15
| | | Using the protocol_id instead of its index.
* QUIC: convert logs to standard mechanism (#1485)Ivan Nardi2022-03-15
|
* QUIC: fix dissection of draft-34 (#1484)dev-1Ivan Nardi2022-03-09
| | | | QUIC-34 is probably not used in production, but fixing it is trivial and it doesn't add any noise to the already complex QUIC code.
* Extend tests coverage (#1476)Ivan Nardi2022-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is at least one flow under `tests/pcap` for 249 protocols out of the 284 ones supported by nDPI. The 35 protocols without any tests are: * P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK, EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET * games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA, FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU * voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE * other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE, LOTUS_NOTES, EGP, SAP Most of these protocols (expecially the P2P and games ones) have been inherited by OpenDPI and have not been updated since then: even if they are still used, the detection rules might be outdated. However code coverage (of `lib/protocols`) only increases from 65.6% to 68.9%. Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP dissection. Treat IPP as a HTTP sub protocol. Fix Cassandra false positives. Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`: these protocol ids are defined but they are never used. Remove Collectd support: its code has never been called. If someone is really interested in this protocol, we can re-add it later, updating the dissector. Add decoding of PPI (Per-Packet Information) data link type.
* Improved ASN/IP update scripts and CI integration. (#1474)Toni2022-03-09
| | | | | * CI will print a warning if ASN/IP addresses changed. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Some small fixes (#1481)Ivan Nardi2022-03-08
| | | | | | FTP: if the authentication fails, stop analyzing the flow WSD: call the initialization routine; the dissector code has never been triggered MINING: fix dissection
* Errors fixed (#1482)Vitaly Lavrov2022-03-08
| | | | | | | | | | | | | | | Fixed errors for bigendian platforms in ndpiReader. All address and port comparisons and hash calculations are done with endian in mind. The get_ndpi_flow_info() function searched for an existing flow for the forward and reverse direction of the packet. The ndpi_workflow_node_cmp() function looked for a flow regardless of the packet's direction. This is what led to an error in determining the direction of transmission of the packet. Fixed error in "synscan" test: the number of packets in the forward and reverse direction is incorrectly defined (verified via tcpdump). Fixed bug with icmp protocol checksum check for big endian platforms.
* EthernetIP: fix integer conversion on big-endian archs (#1477)Ivan Nardi2022-03-05
|
* Fixed a bug for BE architectures (#1478)Vitaly Lavrov2022-03-05
| | | Fixed a bug in the internal implementation of libgcrypt for bigendian architectures
* configure: fix usage of libgpg-error with `--with-local-libgcrypt` (#1472)Ivan Nardi2022-03-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now, using external libgcrypt, nDPI is not linked to libgpg-error because configure script never checks for it. ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-11 CXX=g++-11 CFLAGS="-O3 -g -Werror" ./autogen.sh --enable-debug-messages --with-pcre --with-local-libgcrypt && make -s -j [...] checking for numa_available in -lnuma... yes checking for pcap_open_live in -lpcap... yes checking for pthread_setaffinity_np in -lpthread... yes checking for gcry_cipher_checktag in -lgcrypt... yes <------- missing check for libgpg-error checking for pcre_compile in -lpcre... yes checking that generated files are newer than configure... done [...] ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ grep HAVE_LIBGPG_ERROR src/include/ndpi_config.h /* #undef HAVE_LIBGPG_ERROR */ ``` Make both libgcrypt and libgpg-error mandatory if `--with-local-libgcrypt` is used. Technically speaking, libgpg-error might be optional, because it is used only for debug messages. However having both libraries mandatory slightly simplified the logic. In most environments, libgpg-error is a dependency of libgcrypt anyway, so having both libraries should be the standard case.
* Added autoconf option `--enable-tls-sigs'. (#1471)Toni2022-03-03
| | | | | | * Testing more code in CI environments. * Added strict option checking for `./configure' in CI environments. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Internal crypto: increase size of authentication buffer (#1468)Ivan Nardi2022-03-02
| | | | | | | Some QUIC flows are not properly decoded while using internal crypto code: the authentication buffer is too small. The new value (like the old one) is arbitrary. Close #1463
* Add ICMP checksum check and set risk if mismatch detected. (#1464)Toni2022-03-02
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added configureable ndpi packet processing limit. (#1466)Toni2022-03-02
| | | | | | | | | * The current behaviour ignores any user preferences and was also incorrectly implemented, because the flow->num_processed_pkts wraps every 65535 and nDPI will process packets again until NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix libgcrypt(-light/-internal) compile error. (#1465)Toni2022-03-02
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add a new flow risk `NDPI_ANONYMOUS_SUBSCRIBER` (#1462)Ivan Nardi2022-02-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of a DPI engine is usually to determine "what", i.e. which types of traffic flow on the network. However the applications using DPI are often interested also in "who", i.e. which "user/subscriber" generated that traffic. The association between a flow and a subscriber is usually done via some kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element of the flow used to identify the user is the source ip address. That usually happens for the vast majority of the traffic. However, depending on the protocols involved and on the position on the net where the traffic is captured, the source ip address might have been changed/anonymized. In that case, that address is useless for any flow-username association. Example: iCloud Private Relay traffic captured between the exit relay and the server. See the picture at page 5 on: https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting that the ip addresses shouldn't be used to identify the user associated with the flow. As a first example of this new feature, the entire list of the relay ip addresses used by Private Relay is added. A key point to note is that list is NOT used for flow classification (unlike all the other ip lists present in nDPI) but only for setting this new flow risk. TODO: IPv6
* Removed some unused fields (#1461)Ivan Nardi2022-02-28
|
* Bug fixing. (#1459)Vitaly Lavrov2022-02-28
| | | | | The '--enable-debug-messages' option works again. Fixed warning in ahocorasick.c Fixed integer overflow in ndpiReader.c for 32bit systems.
* Add some scripts to easily update some IPs lists (#1449)Ivan Nardi2022-02-25
| | | | | | | | | | | While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the companies themselves (or by some interested parties), these new lists are directly extracted from BGP information, via AS prefixes. *Usually*, these new lists are far more stable than the previous ones. TODO: * add some other ASNs (see `src/lib/ndpi_content_match.c.inc`) * IPv6, as usual :-(
* Provide some API functions for convenience. (#1456)Toni2022-02-25
| | | | | * Extended JSON serializsation: risk, risk score, confidence Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Win fixesLuca Deri2022-02-23
|
* Replaced strdup with ndpi_strupLuca Deri2022-02-23
|
* Directly drop malformed packets (#1455)Ivan Nardi2022-02-22
| | | | TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be ignored.
* DTLS: fix access to certificate cache (#1450)Ivan Nardi2022-02-21
| | | | | | | | | | | | | | | | ``` protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short') SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in AddressSanitizer:DEADLYSIGNAL ================================================================= ==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1) ==47401==The signal is caused by a READ memory access. ==47401==Hint: address points to the zero page. #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41 #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7 #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13 ```
* EthernetIP: add missing initialization (#1448)Ivan Nardi2022-02-20
| | | Fix:1e1cfb89
* Add support for Google Cloud (#1447)Ivan Nardi2022-02-20
| | | | Differentiate between Google its own apps/services and Google Cloud. We already do something similar for Amazon vs AWS and Microsoft vs Azure.
* Added lightweight implementation of libgcrypt. (#1444)Vitaly Lavrov2022-02-20
| | | | | | | | | | | | | | | | | | | | | Implementation borrowed from the https://github.com/ARMmbed/mbedtls.git project (v3.1.0) Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz): gcrypt-gnu Test md 2897 ms enc 2777 ms dec 942 ms gcrypt-int Test md 3668 ms enc 1312 ms dec 2836 ms gcrypt-int-noaesni Test md 3652 ms enc 1916 ms dec 4458 ms gcrypt-gnu-nonopt Test md 3763 ms enc 4978 ms dec 3999 ms gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration --disable-padlock-support --disable-aesni-support \ --disable-shaext-support --disable-pclmul-support \ --disable-sse41-support --disable-drng-support \ --disable-avx-support --disable-avx2-support \ --disable-neon-support --disable-arm-crypto-support \ --disable-ppc-crypto-support --disable-amd64-as-feature-detection
* Fix compilation and sync unit tests results (#1445)Ivan Nardi2022-02-19
| | | | 'strcasestr' is not defined in all enviroments: quicker fix is to use 'ndpi_strncasestr' instead.
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* SilencedLuca Deri2022-02-14
| | | | | | | | | NDPI_SUSPICIOUS_DGA_DOMAIN, NDPI_BINARY_APPLICATION_TRANSFER, NDPI_HTTP_NUMERIC_IP_HOST, NDPI_MALICIOUS_JA3, for predefined connectivity check and cybersec categories
* Extended cybersecurity protocol dissectionLuca Deri2022-02-14
|
* Added SNMP error code checkLuca Deri2022-02-14
|
* Exteended cybersecurity listLuca Deri2022-02-10
|
* Invalid prototupe fixLuca Deri2022-02-10
|
* HSRP: fix dissection over IPv6 (#1443)Ivan Nardi2022-02-10
| | | Handle all message types.
* Added cybersecurity category mapping to stringLuca Deri2022-02-10
|
Powered by cgit, Themed by Ted Yin.