aboutsummaryrefslogtreecommitdiff
path: root/src/include
Commit message (Collapse)AuthorAge
...
* Add protocol disabling feature (#1808)Ivan Nardi2022-12-18
| | | | | | | | | | | | | | | | | | | | | | The application may enable only some protocols. Disabling a protocol means: *) don't register/use the protocol dissector code (if any) *) disable classification by-port for such a protocol *) disable string matchings for domains/certificates involving this protocol *) disable subprotocol registration (if any) This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`. Custom protocols are always enabled. Technically speaking, this commit doesn't introduce any API/ABI incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()` is now mandatory, just after having called `ndpi_init_detection_module()`. Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to the removing of some function parameters. Fix the low level macro `NDPI_LOG`. This issue hasn't been detected sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
* Added ability to decode DNS PTR records used for reverse address resolutionLuca Deri2022-12-15
|
* STUN: add detection of ZOOM peer-to-peer flows (#1825)Ivan Nardi2022-12-11
| | | | See: "Enabling Passive Measurement of Zoom Performance in Production Networks" https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
* DTLS: handle (certificate) fragments (#1811)Ivan Nardi2022-12-10
| | | | | Keep using the existing function to handle reassembling buffer: rename it from `ndpi_search_tls_tcp_memory` to `ndpi_search_tls_memory` and make it "transport" agnostic
* Added Zoom screen share detectionLuca Deri2022-12-09
|
* Added RTP stream type in flow metadataLuca Deri2022-12-09
|
* Fix compilation (#1819)Ivan Nardi2022-12-05
|
* Exported HTTP server in metadataLuca Deri2022-12-05
|
* Improve export/print of L4 protocol information (#1799)Ivan Nardi2022-11-13
| | | Close #1797
* TLS: improve handling of ALPN(s) (#1784)Ivan Nardi2022-10-25
| | | | | | | | Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is extracted from the CH, the latter from the SH. Add some entries to the known ALPN list. Fix printing of "TLS Supported Versions" field.
* Enhanced HTTP numeric IP checkLuca Deri2022-10-22
|
* Add Elasticsearch protocol dissector. (#1782)Toni2022-10-21
| | | | | | | * all credits goes to @verzulli Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Munin protocol dissector.Toni Uhlig2022-10-20
| | | | | | * all credits goes to @verzulli Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked severities adding critical and emergency, thus shifting values downLuca Deri2022-10-19
|
* TLS: explicit ignore client certificate (#1776)Ivan Nardi2022-10-18
| | | | | | | | | | | | | TLS classification usually stops after processing *server* certificates (if any). That means, that *client* certificate, if present, is usually ignored. However in some corner cases (i.e. unidirectional traffic) we might end up processing client certificate and exposing its metadata: the issue is that the application will think that this metadata are about the server and not about the client. So, for the time being, always ignore client certificate processing. As a future work, we might find an efficient way to process and export both certificates.
* Add some popular streaming applicationsNardi Ivan2022-10-17
| | | | | | | A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142; full credits to @ChrisNelson-CyberReef. Close #1716
* Updated protocol dissector serialisation code. (#1767)Toni2022-10-15
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* LINE_CALL: add detection of LINE voip calls (#1761)Ivan Nardi2022-10-06
| | | | | | | | These flows are classifed as `LINE_CALL`; another option was `RTP/LINE_CALL`. No sure about the best solution... Extend LINE domains list. Remove RTP dead code.
* Windows changesLuca Deri2022-10-06
|
* Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx ↵Luca2022-10-04
| | | | are supported
* Fix compilation and sync utests resultsNardi Ivan2022-10-04
|
* Added the ability to track the payload via -E and via the new option ↵Luca2022-10-04
| | | | 'ndpi_track_flow_payload'
* Restore Confidence enum constants to ascending values. (#1760)Toni2022-10-02
| | | | | | | | | | * NDPI_CONFIDENCE_MAX is useless otherwise since it is supposed the be the highest value w/o any gaps * for client applications, it makes sense to use that the numbers directly w/o any additional mapping Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Line app support (#1759)sharonenoch2022-10-01
| | | | | | | | | | | | | | | | | | | * Standard support for LINE app * Added test pcap for LINE app * make check result for LINE app * Make check success as 1kxun has LINE packets * Added the ASN inc file for LINE * Removed extra lines as its effecting make check * Editing the SNI required a new pcap output file for TLS.Line format * Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out Co-authored-by: Sharon Enoch <sharone@amzetta.com>
* TLS/DTLS: we process certificate for UDP flows, tooNardi Ivan2022-09-30
| | | | | | Note that current code access `certificate_processed` state even before setting the protocol classification, so this piece of information can't be saved in `flow->protos` union.
* Remove unused codeNardi Ivan2022-09-28
| | | | | | | | | LRU callbacks have been added in 460ff3c7a, but they have never been used and they have never been extended to the other LRU caches. `ndpi_search_tcp_or_udp()` basically returns the classification by port/ip of the flow; calling it from the dissector is useless. The same for TOR detection: ips are checked in the generic code
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* Improved NATPMP dissection. (#1745)Toni2022-09-21
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Build ndpiReader and run regression tests.Toni Uhlig2022-09-21
| | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Moved `NDPI_API_VERSION` to `src/include/ndpi_define.h.in` where it belongs.Toni Uhlig2022-09-21
| | | | | | | That way, we can make `src/include/ndpi_api.h` independent of autoconf. MSBuild will like that. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Finalized nBPF support. You can now define custom protocols such asLuca Deri2022-09-21
| | | | | | | | | | (see exaple/protos.txt) nbpf:"host 192.168.1.1 and port 80"@HomeRouter In order to have nBPF support, you need to compile nDPI with it. Just download https://github.com/ntop/PF_RING in the same directory where you have downloaded nDPI and compile PF_RING/userland/nbpf
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Added MAX_NBPF_CUSTOM_PROTO defineLuca Deri2022-09-17
|
* Initial work towards custom BPF protocol definitionLuca2022-09-16
|
* Add CryNetwork dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Syncthing dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add NATPMP dissector. (#1738)Toni2022-09-16
| | | | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Dissectors shouldn't update `flow->guessed_host_protocol_id`Nardi Ivan2022-09-14
| | | | | | | | | | | | | | | | The field `flow->guessed_host_protocol_id` is set at the beginning of the flow analysis and it represents the "classification by ip" of the flow itself. This field should never be changed. Dissectors which want to provide an "hint" about the classification, should update `flow->guessed_protocol_id` instead. Such "hint" is useless if the dissector set the "extra-dissection" data-path. Rename such field to `guessed_protocol_id_by_ip` to better describe its role. Preliminary work necessary for #1687
* STUN: several improvementsNardi Ivan2022-09-11
| | | | | | | | | Add detection over TCP and fix detection over IPv6. Rename some variables since Stun dissector is no more "udp-centric". Stun dissector should always classified the flow as `STUN` or `STUN/Something`. Don't touch `flow->guessed_host_protocol_id` field, which should be always be related to "ip-classification" only.
* Avoid useless host automa lookup (#1724)Ivan Nardi2022-09-05
| | | | | | | | | | | | | | | | | | | The host automa is used for two tasks: * protocol sub-classification (obviously); * DGA evaluation: the idea is that if a domain is present in this automa, it can't be a DGA, regardless of its format/name. In most dissectors both checks are executed, i.e. the code is something like: ``` ndpi_match_host_subprotocol(..., flow->host_server_name, ...); ndpi_check_dga_name(..., flow->host_server_name,...); ``` In that common case, we can perform only one automa lookup: if we check the sub-classification before the DGA, we can avoid the second lookup in the DGA function itself.
* Fix broken ndpi_timeval_to_(milli|micro)seconds (>UINT_MAX). (#1720)Toni2022-09-04
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add support to opportunistic TLSNardi Ivan2022-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A lot of protocols provide the feature to upgrade their plain text connections to an encrypted one, via some kind of "STARTTLS" command. Add generic code to support this extension, and allow dissection of the entire TLS handshake. As examples, SMTP, POP, IMAP and FTP dissectors have been updated. Since this feature requires to process more packets per flow, add the possibility to disable it. Fix some log messages. Slight improvement on TCP sequence number tracking. As a side effect, this commit fix also a memory leak found by oss-fuzzer ``` ==108966==ERROR: LeakSanitizer: detected memory leaks Direct leak of 22 byte(s) in 1 object(s) allocated from: #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063) #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10 #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25 #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13 #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34 #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5 #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9 #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9 #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5 #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31 #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10 #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
* Serializer supports double for JSON and CSV only for the time being (TLV ↵Alfredo Cardigliano2022-09-01
| | | | needs to be extended)
* Temporarily restore ndpi_serialization_unknown = 0Alfredo Cardigliano2022-09-01
|
* Restore ndpi_serialization_type item values. Use 0 for the new ↵Alfredo Cardigliano2022-09-01
| | | | ndpi_serialization_double as there is no more key left (4 bit). Use 16 for ndpi_serialization_unknown which is not used in TLV encoded data.
* Improved MGCP dissector. (#1717)Toni2022-08-30
| | | | | | | | * typ0s fixed * dissect endpoint hostnames Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* QUIC: add support for 0-RTT packets received before the InitialNardi Ivan2022-08-24
| | | | | | | | | | | | | | | | | | | | | RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the early_data extension in the ClientHello message of a subsequent handshake; see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT packets." That means the client sends before the CH (in the Initial) and then the 0-RTT (in the same UDP datagram or not)". However, because of packet loss or out-of-order delivery, it might happens that a 0-RTT packet is received before the Initial (the original one or a retransmission). For example, Google and Facebook servers save 0-RTT packets for a small amount of time in hopes of receiving the corresponding Initial. Update the QUIC dissector to detect 0-RTT packets and keep looking for the Initial. Issue found by @utoni in #1706; the trace example has been taken from that PR.
* Add FastCGI protocol detection. (#1711)Toni2022-08-24
| | | | | | | | * CQL: fixed byte order conversion (BigEndian not LittleEndian) * CQL: increased required successful dissected packets to prevent false-positives Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Kismet protocol detection. (#1710)Toni2022-08-24
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Support serialization of double-precision floating-point numbers. Fixes #1702.lns2022-08-24
| | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.