aboutsummaryrefslogtreecommitdiff
path: root/example/reader_util.h
Commit message (Collapse)AuthorAge
* ndpiReader: add some global statistics about FPC (#2680)Ivan Nardi2025-01-17
| | | Enabled via `--dump-fpc-stats` option
* TLS: remove JA3C (#2679)Ivan Nardi2025-01-14
| | | | | | | | Last step of removing JA3C fingerprint Remove some duplicate tests: testing with ja4c/ja3s disabled is already performed by `disable_metadata_and_flowrisks` configuration. Close:#2551
* ndpiReader: update JA statistics (#2646)Ivan Nardi2025-01-06
| | | | Show JA4C and JA3S information (instead of JA3C and JA3S) See #2551 for context
* QUIC: extract "max idle timeout" parameter (#2649)Ivan Nardi2025-01-06
| | | | | Even if it is only the proposed value by the client (and not the negotiated one), it might be use as hint for timeout by the (external) flows manager
* TLS: remove ESNI support (#2648)Ivan Nardi2025-01-06
| | | | | ESNI has been superseded by ECH for years, now. See: https://blog.cloudflare.com/encrypted-client-hello/ Set the existing flow risk if we still found this extension.
* STUN/RTP: improve metadata extraction (#2641)Ivan Nardi2024-12-11
|
* STUN counter changesLuca Deri2024-11-29
|
* Enhanced STUN statsLuca Deri2024-11-28
|
* Update `flow->flow_multimedia_types` to a bitmask (#2625)Ivan Nardi2024-11-25
| | | In the same flow, we can have multiple multimedia types
* SIP: extract some basic metadataIvan Nardi2024-11-12
|
* Added HTTP credentials extractionLuca Deri2024-10-31
|
* TLS: export heuristic fingerprint as metadata (#2609)Ivan Nardi2024-10-28
|
* ndpiReader: add some statistics about monitoring (#2602)Ivan Nardi2024-10-19
|
* Implemented nDPI TCP fingerprintLuca Deri2024-10-15
|
* Add monitoring capability (#2588)Ivan Nardi2024-10-14
| | | | | | | | | | | | | Allow nDPI to process the entire flows and not only the first N packets. Usefull when the application is interested in some metadata spanning the entire life of the session. As initial step, only STUN flows can be put in monitoring. See `doc/monitoring.md` for further details. This feature is disabled by default. Close #2583
* Added support for printing JA4r when enabledLuca Deri2024-10-11
|
* Implemented (disabled by default) DNS host cache. You can set the cache size ↵Luca Deri2024-10-07
| | | | | | | | | | as follows: ndpiReader --cfg=dpi.address_cache_size,1000 -i <pcap>.pcap In the above example the cache has up to 1000 entries. In jcase ndpiReader exports data in JSON, the cache hostname (if found) is exported in the field server_hostname
* Let the library returning the packet direction calculated internally (#2572)Ivan Nardi2024-09-27
| | | wireshark, lua: add basic analysis of possible obfuscated flows
* Add infrastructure for explicit support of Fist Packet Classification (#2488)Ivan Nardi2024-07-03
| | | | | Let's start with some basic helpers and with FPC based on flow addresses. See: #2322
* wireshark: lua: export some metadataNardi Ivan2024-06-25
| | | | | | | | | | | | | | | Export some metadata (for the moment, SNI and TLS fingerprints) to Wireshark/tshark via extcap. Note that: * metadata are exported only once per flow * metadata are exported (all together) when nDPI stopped processing the flow Still room for a lot of improvements! In particular: * we need to add some boundary checks (if we are going to export other attributes) * we should try to have a variable length trailer
* Implemented STUN peer_address, relayed_address, response_origin, ↵Luca Deri2024-04-12
| | | | | | | other_address parsing Added code to ignore invalid STUN realm Extended JSON output with STUN information
* STUN: improve extraction of Mapped-Address metadata (#2370)Ivan Nardi2024-04-08
| | | | | | | | | | | | | Enable parsing of Mapped-Address attribute for all STUN flows: that means that STUN classification might require more packets. Add a configuration knob to enable/disable this feature. Note that we can have (any) STUN metadata also for flows *not* classified as STUN (because of DTLS). Add support for ipv6. Restore the correct extra dissection logic for Telegram flows.
* Added support for STUN Mapped IP addressLuca Deri2024-04-03
|
* Allow multiple `struct ndpi_detection_module_struct` to share some state (#2271)Ivan Nardi2024-02-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the concept of "global context". Right now every instance of `struct ndpi_detection_module_struct` (we will call it "local context" in this description) is completely independent from each other. This provide optimal performances in multithreaded environment, where we pin each local context to a thread, and each thread to a specific CPU core: we don't have any data shared across the cores. Each local context has, internally, also some information correlating **different** flows; something like: ``` if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then flow2 (PeerC <-> PeerD) will be PROTOCOL_Y ``` To get optimal classification results, both flow1 and flow2 must be processed by the same local context. This is not an issue at all in the far most common scenario where there is only one local context, but it might be impractical in some more complex scenarios. Create the concept of "global context": multiple local contexts can use the same global context and share some data (structures) using it. This way the data correlating multiple flows can be read/write from different local contexts. This is an optional feature, disabled by default. Obviously data structures shared in a global context must be thread safe. This PR updates the code of the LRU implementation to be, optionally, thread safe. Right now, only the LRU caches can be shared; the other main structures (trees and automas) are basically read-only: there is little sense in sharing them. Furthermore, these structures don't have any information correlating multiple flows. Every LRU cache can be shared, independently from the others, via `ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`. It's up to the user to find the right trade-off between performances (i.e. without shared data) and classification results (i.e. with some shared data among the local contexts), depending on the specific traffic patterns and on the algorithms used to balance the flows across the threads/cores/local contexts. Add some basic examples of library initialization in `doc/library_initialization.md`. This code needs libpthread as external dependency. It shouldn't be a big issue; however a configure flag has been added to disable global context support. A new CI job has been added to test it. TODO: we should need to find a proper way to add some tests on multithreaded enviroment... not an easy task... *** API changes *** If you are not interested in this feature, simply add a NULL parameter to any `ndpi_init_detection_module()` calls.
* config: move debug/log configuration to the new APINardi Ivan2024-01-18
|
* Add realtime protocol output to `ndpiReader`. (#2197)Toni2024-01-09
| | | | | | * support for using a new flow callback invoked before the flow memory is free'd * minor fixes Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implements JA4 Support (#2191)Luca Deri2023-12-22
|
* QUIC: export QUIC version as metadataNardi Ivan2023-10-11
|
* fuzz: extend fuzzing coverageNardi Ivan2023-10-07
|
* Cleaned up mining datastructureLuca2023-09-27
|
* fuzz: add fuzzers to test reader_util code (#2080)Ivan Nardi2023-09-10
|
* DNS: extract geolocation information, if available (#2065)Ivan Nardi2023-07-31
| | | | | | | The option NSID (RFC5001) is used by Google DNS to report the airport code of the metro where the DNS query is handled. This option is quite rare, but the added overhead in DNS code is pretty much zero for "normal" DNS traffic
* TLS: add basic, basic, detection of Encrypted ClientHello (#2053)Ivan Nardi2023-07-21
|
* added feature to extract filename from http attachment (#2037)Chiara Maggi2023-07-11
| | | | | | | | | | | | | | | | | | | * added feature to extract filename from http attachment * fixed some issues * added check for filename format * added check for filename format * remove an unnecessary print * changed the size from 952 to 960 * modified some test result files * small changes string size * comment removed and mallocs checked
* Simplify the report of streaming multimedia info (#2026)Ivan Nardi2023-06-26
| | | | | The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are pretty much identical: rename the former in `flow->flow_multimedia_type` and remove the latter.
* Implemented Zoom/Teams stream type detectionLuca Deri2023-06-14
|
* ndpiReader: improve printing of payload statistics (#1989)Ivan Nardi2023-05-29
| | | | | Add a basic unit test Fix an endianess issue
* HTTP: improve extraction of metadata and of flow risks (#1959)Ivan Nardi2023-05-05
|
* ndpiReader: print how many packets (per flow) were needed to perform full ↵Ivan Nardi2023-03-01
| | | | | | DPI (#1891) Average values are already printed, but this change should ease to identify regressions/improvements.
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Some small changes (#1869)Ivan Nardi2023-01-25
| | | | | | | | All dissector callbacks should not be exported by the library; make static some other local functions. The callback logic in `ndpiReader` has never been used. With internal libgcrypt, `gcry_control()` should always return no errors. We can check `categories` length at compilation time.
* Added RTP stream type in flow metadataLuca Deri2022-12-09
|
* Exported HTTP server in metadataLuca Deri2022-12-05
|
* Added support for Linux Cooked Capture v2Luca Deri2022-11-16
|
* Fix json export of ipv6 addressesNardi Ivan2022-11-07
| | | | | | | The "string" buffer was to short; better start using `INET6_ADDRSTRLEN` as reported in the man page of `inet_ntop`. Close: #1794
* TLS: improve handling of ALPN(s) (#1784)Ivan Nardi2022-10-25
| | | | | | | | Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is extracted from the CH, the latter from the SH. Add some entries to the known ALPN list. Fix printing of "TLS Supported Versions" field.
* Added the ability to track the payload via -E and via the new option ↵Luca2022-10-04
| | | | 'ndpi_track_flow_payload'
* Improved NATPMP dissection. (#1745)Toni2022-09-21
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add TiVoConnect dissector. Fixes #1697. (#1699)Toni2022-08-08
| | | | | * added static assert if supported, to complain if the flow struct changes Signed-off-by: lns <matzeton@googlemail.com>
* Improved nDPI JSON serialization. (#1689)Toni2022-08-02
| | | | | | | | * fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors * introduced portable version of gmtime_r aka ndpi_gmtime_r * do as most as possible of the serialization work in ndpi_utils.c * use flow2json in ndpiReader Signed-off-by: lns <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.