aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* Reworked HTTP protocol dissection including HTTP proxy and HTTP connectLuca Deri2021-11-25
|
* Aded default port to HTTP_CONNECTLuca Deri2021-11-25
|
* Added googletagmanager.com to the host_match listLuca Deri2021-11-25
|
* Rework how hostname/SNI info is saved (#1330)Ivan Nardi2021-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Looking at `struct ndpi_flow_struct` the two bigger fields are `host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and `protos.tls_quic.client_requested_server_name[256]` (for TLS/QUIC SNIs). This commit aims to reduce `struct ndpi_flow_struct` size, according to two simple observations: 1) maximum one of these two fields is used for each flow. So it seems safe to merge them; 2) even if hostnames/SNIs might be very long, in practice they are rarely longer than a fews tens of bytes. So, using a (single) large buffer is a waste of memory for all kinds of flows. If we need to truncate the name, we keep the *last* characters, easing domain matching. Analyzing some real traffic, it seems safe to assume that the vast majority of hostnames/SNIs is shorter than 80 bytes. Hostnames/SNIs are always converted to lowercase. Attention was given so as to be sure that unit-tests outputs are not affected by this change. Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the *first* 64 ones): as a consequence, there were some "Suspicious DGA domain name" and "TLS Certificate Mismatch" false positives.
* QUIC: fix extra dissection (#1376)Ivan Nardi2021-11-23
| | | | When we have fully reassembled the Client Hello, we need to stop extra dissection.
* BitTorrent dissector code cleanup and merge of ↵Luca Deri2021-11-17
| | | | https://github.com/ntop/nDPI/pull/1374
* BitTorrent detection improvementsLuca Deri2021-11-17
|
* Improved BT hashing to minimize false positivesLuca Deri2021-11-17
|
* bittorrent old code cleanup. Enlarged BT cacheLuca Deri2021-11-16
|
* Improved BitTorrent detectionLuca Deri2021-11-15
|
* Fix writes to `flow->protos` union fields (#1354)Ivan Nardi2021-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | We can write to `flow->protos` only after a proper classification. This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP, SMTP, IMAP and POP code. There are two kinds of fixes: * write to `flow->protos` only if a final protocol has been detected * move protocol state out of `flow->protos` The hard part is to find, for each protocol, the right tradeoff between memory usage and code complexity. Handle Kerberos like DNS: if we find a request, we set the protocol and an extra callback to further parsing the reply. For all the other protocols, move the state out of `flow->protos`. This is an issue only for the FTP/MAIL stuff. Add DHCP Class Identification value to the output of ndpiReader and to the Jason serialization. Extend code coverage of fuzz tests. Close #1343 Close #1342
* Update ndpi_main.c for HTTP line parser optimization and cleaning (#1372)ragostino2021-11-11
| | | | | | | * Update ndpi_main.c HTTP line parser: optimization and cleaning * Update ndpi_main.c
* Reduce memory used by `ndpiReader` (#1371)Ivan Nardi2021-11-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `ndpiReader` is only an example, aiming to show nDPI capabilities and integration, without any claim about performances. Nonetheless its memory usage per flow is *huge*, limiting the kinds of traces that we can test on a "normal" hardware (example: scan attacks). The key reason of that behaviour is that we preallocate all the memory needed for *all* the available features. Try to reduce memory usage simply allocating some structures only when they are really needed. Most significant example: JOY algorithms. This way we should use a lot less memory in the two most common user-cases: * `ndpiReader` invoked without any particular flag (i.e `ndpiReader -i $FILENAME_OR_IFACE`) * internal unit tests Before (on x86_64): ``` struct ndpi_flow_info { [...] /* size: 7320, cachelines: 115, members: 72 */ ``` After: ``` struct ndpi_flow_info { [...] /* size: 2128, cachelines: 34, members: 75 */ ```
* Add detection of OCSP (#1370)Ivan Nardi2021-11-11
| | | | | | | | | | This protocol is detected via HTTP Content-Type header. Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to classify a HTTP flow according to this header. Since then, this automa has been useless because it is always empty. Re-enable it to match only a string seems overkilling. Remove all `content_automa` leftovers.
* IMAP, POP3, SMTP: improve dissection (#1368)Ivan Nardi2021-11-11
| | | Avoid NATS false positives
* BitTorrent cache debugLuca Deri2021-11-08
|
* Differentiate between standard Amazon stuff (i.e market) and AWS (#1369)Ivan Nardi2021-11-04
|
* Improved BitTorrent detectionLuca Deri2021-11-04
|
* Fix use-of-uninitialized-value (#1366)Ivan Nardi2021-11-03
| | | Close #1346
* TLS: fix two warnings (#1365)Ivan Nardi2021-11-02
| | | | | | | | Disable unit tests on CI for big-endian target. We know we have multiple issues on big-endian architectures (see #1312) and so the unit tests always fail there. Ignore this error for the time being and let the CI pass if we don't have other issues. Remove an unused automa definition
* Updated test resultsLuca Deri2021-11-02
|
* Fixed certificate mismatch for TLS flows with no client hello observedLuca Deri2021-11-02
|
* BitTorrent detection improvementsLuca Deri2021-10-31
|
* Little change in TLS debug tracesLuca Deri2021-10-28
|
* Updated resultsLuca Deri2021-10-27
|
* Fixed TLS certificate dissection of subjectAltNameLuca Deri2021-10-27
|
* Minor code cleanupLuca Deri2021-10-27
|
* Avoid overwriting valid protocol in `ndpi_detection_giveup` (#1360)Ivan Nardi2021-10-27
| | | | | | | | | | | | | | | | | We should avoid updating any valid protocol in `ndpi_detection_giveup`; we should try to find a proper classification only if the flow is still completely unclassified. For example in the attached pcap there is a valid TLS session, recognized as such by TLS dissector. However, the `ndpi_detection_giveup`function updates it to "HTTP/TLS" (!?) simply because the server port is 80. Note that the real issue is not the wrong classification, but the wrong access to `flow->protos` union. If we already set some fields of `flow->protos` and we change the protocol in `ndpi_detection_giveup`, we might end up freeing some invalid pointers in `ndpi_free_flow_data` (no wonder this issue has been found while fuzzing #1354) Fix GIT and TLS dissectors (issues found by CI fuzzer)
* Serialize additional information stored in the flow struct. (#1362)Toni2021-10-27
| | | | | | | * Changed function signature of ndpi_flow2json (removed unused vlan_id; API break) * Serialize NTP information. * Improved QUIC serialization. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved STUN and RTP detectionLuca Deri2021-10-27
|
* Detect invalid characters in text and set a risk. Fixes #1347. (#1363)Toni2021-10-26
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed Git protocol dissection (#1355)Luca Deri2021-10-25
|
* Fixed cleartext protocol assignment (#1357)Ivan Nardi2021-10-25
|
* TLS: fix a heap-buffer-overflow (#1356)Ivan Nardi2021-10-22
| | | | Revert of c3d1c697 Error reproducible with the attached pcap and valgrind
* Fixed heap-buffer-overflow in TLS dissectorLuca Deri2021-10-20
|
* Fix QUIC log and remove SoulSeek leftovers after b97dc6ba (#1351)Ivan Nardi2021-10-19
| | | | | Update .gitignore file Fix a function prototype Close #1349
* Fix some invalid memory reads (#1350)Ivan Nardi2021-10-19
| | | | | | | | `ndpi_detection_giveup()` (and any functions called by it) can't access `ndpi_detection_module_struct->packet` anymore since 730c236. Sync unit tests results Close #1348
* Added clertext alert with telnetLuca Deri2021-10-19
|
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implemented RDP over UDP dissectionLuca Deri2021-10-18
|
* Refreshed results listLuca Deri2021-10-16
|
* Fixed cleartext protocol assignmentLuca Deri2021-10-16
|
* Updated test results after latest commitLuca Deri2021-10-16
|
* Reworked flow risks asignmentLuca Deri2021-10-16
| | | | Added esceptions for windows update and binary application transfer risk
* Removed outdated (and broken) soulseek dissectorLuca Deri2021-10-15
|
* Updated test resultsLuca Deri2021-10-14
|
* Added missing call to ndpi_reconcile_protocols() with protocol guessLuca Deri2021-10-14
|
* Fixed issue on DGA numeric IP detectionLuca Deri2021-10-13
|
* IndemtLuca Deri2021-10-13
|
* Add issue templates. (#1340)Zied Aouini2021-10-13
| | | | * Add issue template. * Add config log instruction.
Powered by cgit, Themed by Ted Yin.