aboutsummaryrefslogtreecommitdiff
path: root/src/include
Commit message (Collapse)AuthorAge
...
* Add commentAlfredo Cardigliano2022-02-03
|
* Added NDPI_ERROR_CODE_DETECTED riskLuca Deri2022-02-03
|
* Renamed DCERPC to more generic RPC protocol so we can use also for other ↵Luca Deri2022-02-03
| | | | | | | types of RPCs (not limited to DCE) Extended HTTP plugin to support RPC Improved HTTP crear text detection to limit it to Basic and Digest
* Added new IDN/Punycode risk for spotting internationalized domain namesLuca2022-02-03
|
* Added missing __sync_fetch_and_add() definition in WindowsLuca Deri2022-02-02
|
* Remove `struct ndpi_id_struct` (#1427)Ivan Nardi2022-01-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the last uses of `struct ndpi_id_struct`. That code is not really used and it has not been updated for a very long time: see #1279 for details. Correlation among flows is achieved via LRU caches. This change allows to further reduce memory consumption (see also 91bb77a8). At nDPI 4.0 (more precisly, at a6b10cf, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 235.27 KB Flow Memory (per flow): 688 B <-------- ``` i.e. memory usage per flow has been reduced by 77%. Close #1279
* Remove Playstation VUE protocol (#1426)Ivan Nardi2022-01-30
| | | | PS VUE service has been discontinued on January 30, 2020 https://en.wikipedia.org/wiki/PlayStation_Vue
* Extend protocols support (#1422)Ivan Nardi2022-01-29
| | | | | | | | | | | | | | | | | | Add detection of AccuWeather site/app and Google Classroom. Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP. Fix some RX false positives. Fix some "Uncommon TLS ALPN"-risk false positives. Fix "confidence" value for some Zoom/Torrent classifications. Minor fix in Lua script for Wireshark extcap. Update .gitignore file. Let GitHub correctly detect the language type of *.inc files. Zattoo example has been provided by @subhajit-cdot in #1148.
* Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow riskLuca Deri2022-01-26
| | | | Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
* Improved Zoom protocol detectionLuca Deri2022-01-23
|
* Updated confidence typeLuca Deri2022-01-18
|
* H323: fix a use-after-poison error (#1412)Ivan Nardi2022-01-17
| | | | | | | Detected by oss-fuzz See: https://oss-fuzz.com/testcase-detail/6730505580576768 Fix a function prototype Update a unit test results
* Added Badoo detectionLuca Deri2022-01-17
|
* Added performance tests toolsLuca Deri2022-01-16
|
* Reduced Patricia tree bucket memory footprintLuca Deri2022-01-16
|
* FreeBSD fixesLuca Deri2022-01-13
|
* Added the ability to specify trusted issueDN often used in companies to ↵Luca Deri2022-01-13
| | | | | | | | | | | self-signed certificates This allows to avoid triggering alerts for trusted albeit private certificate issuers. Extended the example/protos.txt with the new syntax for specifying trusted issueDN. Example: trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
* Added EthernetIP dissectorLuca Deri2022-01-12
|
* Add a "confidence" field about the reliability of the classification. (#1395)Ivan Nardi2022-01-11
| | | | | | | | | | | | | As a general rule, the higher the confidence value, the higher the "reliability/precision" of the classification. In other words, this new field provides an hint about "how" the flow classification has been obtained. For example, the application may want to ignore classification "by-port" (they are not real DPI classifications, after all) or give a second glance at flows classified via LRU caches (because of false positives). Setting only one value for the confidence field is a bit tricky: more work is probably needed in the next future to tweak/fix/improve the logic.
* Remove some unused fields (#1393)Ivan Nardi2022-01-08
|
* Update copyrightAlfredo Cardigliano2022-01-03
|
* Added support for Log4J/Log4Shell detection in nDPI via a new flow risk ↵Luca Deri2021-12-23
| | | | named NDPI_POSSIBLE_EXPLOIT
* Add support for ICloud Private Relay (#1390)Ivan Nardi2021-12-22
| | | | | | | See: https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF TODO: an up-to-date list of egress IP ranges is publicly available. Can we use it somehow?
* A final(?) effort to reduce memory usage per flow (#1389)Ivan Nardi2021-12-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove some unused fields and re-organize other ones. In particular: * Update the parameters of `ndpi_ssl_version2str()` function * Zattoo, Thunder: these timestamps aren't really used. * Ftp/mail: these protocols are dissected only over TCP. * Attention must be paid to TLS.Bittorrent flows to avoid invalid read/write to `flow->protos.bittorrent.hash` field. This is the last(?) commit of a long series (see 22241a1d, 227e586e, 730c2360, a8ffcd8b) aiming to reduce library memory consumption. Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 231.71 KB Flow Memory (per flow): 1008 B <--------- ``` i.e. memory usage per flow has been reduced by 66%, dropping below the psychological threshold of 1 KB. To further reduce this value, we probably need to look into #1279: let's fight this battle another day.
* Added Microsoft Azure supportLuca Deri2021-12-19
|
* Improve/add several protocols (#1383)Ivan Nardi2021-12-18
| | | | | | | | | | | | | | | | | | | Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok detection. Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection. Try to differentiate between Messenger/Signal standard flows (i.e chat) and their VOIP (video)calls (like we already do for Whatsapp and Snapchat). Add a partial list of some ADS/Tracking stuff. Fix Cassandra, Radius and GTP false positives. Fix DNS, Syslog and SIP false negatives. Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and GTP_PRIME. Fix 3 LGTM warnings.
* Improved BitTorrent classificationLuca Deri2021-12-07
|
* Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)Ivan Nardi2021-12-04
| | | | | | | | | | | There are no valid reasons for a (generic) protocol to ignore IPv6 traffic. Note that: * I have not found the specifications of "CheckPoint High Availability Protocol", so I don't know how/if it supports IPv6 * all LRU caches are still IPv4 only Even if src_id/dst_id stuff is probably useless (see #1279), the right way to update the protocol classification is via `ndpi_set_detected_protocol()`
* Make serialize risk and proto not dependant on any flow. (#1377)Toni2021-12-04
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added example for finding similarities in RRDs using nDPI statistical APIsLuca Deri2021-12-04
|
* Added ndpi_serializer_skip_header() serialization APILuca Deri2021-11-26
|
* Added Salesforce detectionLuca Deri2021-11-26
|
* Rework how hostname/SNI info is saved (#1330)Ivan Nardi2021-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Looking at `struct ndpi_flow_struct` the two bigger fields are `host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and `protos.tls_quic.client_requested_server_name[256]` (for TLS/QUIC SNIs). This commit aims to reduce `struct ndpi_flow_struct` size, according to two simple observations: 1) maximum one of these two fields is used for each flow. So it seems safe to merge them; 2) even if hostnames/SNIs might be very long, in practice they are rarely longer than a fews tens of bytes. So, using a (single) large buffer is a waste of memory for all kinds of flows. If we need to truncate the name, we keep the *last* characters, easing domain matching. Analyzing some real traffic, it seems safe to assume that the vast majority of hostnames/SNIs is shorter than 80 bytes. Hostnames/SNIs are always converted to lowercase. Attention was given so as to be sure that unit-tests outputs are not affected by this change. Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the *first* 64 ones): as a consequence, there were some "Suspicious DGA domain name" and "TLS Certificate Mismatch" false positives.
* bittorrent old code cleanup. Enlarged BT cacheLuca Deri2021-11-16
|
* Fix writes to `flow->protos` union fields (#1354)Ivan Nardi2021-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | We can write to `flow->protos` only after a proper classification. This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP, SMTP, IMAP and POP code. There are two kinds of fixes: * write to `flow->protos` only if a final protocol has been detected * move protocol state out of `flow->protos` The hard part is to find, for each protocol, the right tradeoff between memory usage and code complexity. Handle Kerberos like DNS: if we find a request, we set the protocol and an extra callback to further parsing the reply. For all the other protocols, move the state out of `flow->protos`. This is an issue only for the FTP/MAIL stuff. Add DHCP Class Identification value to the output of ndpiReader and to the Jason serialization. Extend code coverage of fuzz tests. Close #1343 Close #1342
* Add detection of OCSP (#1370)Ivan Nardi2021-11-11
| | | | | | | | | | This protocol is detected via HTTP Content-Type header. Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to classify a HTTP flow according to this header. Since then, this automa has been useless because it is always empty. Re-enable it to match only a string seems overkilling. Remove all `content_automa` leftovers.
* IMAP, POP3, SMTP: improve dissection (#1368)Ivan Nardi2021-11-11
| | | Avoid NATS false positives
* Differentiate between standard Amazon stuff (i.e market) and AWS (#1369)Ivan Nardi2021-11-04
|
* Improved BitTorrent detectionLuca Deri2021-11-04
|
* TLS: fix two warnings (#1365)Ivan Nardi2021-11-02
| | | | | | | | Disable unit tests on CI for big-endian target. We know we have multiple issues on big-endian architectures (see #1312) and so the unit tests always fail there. Ignore this error for the time being and let the CI pass if we don't have other issues. Remove an unused automa definition
* Serialize additional information stored in the flow struct. (#1362)Toni2021-10-27
| | | | | | | * Changed function signature of ndpi_flow2json (removed unused vlan_id; API break) * Serialize NTP information. * Improved QUIC serialization. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Detect invalid characters in text and set a risk. Fixes #1347. (#1363)Toni2021-10-26
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix QUIC log and remove SoulSeek leftovers after b97dc6ba (#1351)Ivan Nardi2021-10-19
| | | | | Update .gitignore file Fix a function prototype Close #1349
* Fix some invalid memory reads (#1350)Ivan Nardi2021-10-19
| | | | | | | | `ndpi_detection_giveup()` (and any functions called by it) can't access `ndpi_detection_module_struct->packet` anymore since 730c236. Sync unit tests results Close #1348
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implemented RDP over UDP dissectionLuca Deri2021-10-18
|
* Reworked flow risks asignmentLuca Deri2021-10-16
| | | | Added esceptions for windows update and binary application transfer risk
* Removed outdated (and broken) soulseek dissectorLuca Deri2021-10-15
|
* Implemented ndpi_ses_fitting() and ndpi_des_fitting()Luca Deri2021-10-12
| | | | for comuting the best alpha/beta values for exponential smoothing
* Win fixLuca Deri2021-10-11
|
Powered by cgit, Themed by Ted Yin.