aboutsummaryrefslogtreecommitdiff
path: root/fuzz
Commit message (Collapse)AuthorAge
* Follow-up of 8e1b17215: `NDPI_UNRESOLVED_HOSTNAME` (#2933)Ivan Nardi2025-08-05
| | | Add fuzzing, documentation and unit tests
* fuzz: improve coverage (#2931)Ivan Nardi2025-08-04
| | | | Sync `pl7m` code with upstream. Add a new fuzzer to test the same flows with different L4 ports
* Add the concept of protocols stack: more than 2 protocols per flow (#2913)Ivan Nardi2025-08-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea is to remove the limitation of only two protocols ("master" and "app") in the flow classifcation. This is quite handy expecially for STUN flows and, in general, for any flows where there is some kind of transitionf from a cleartext protocol to TLS: HTTP_PROXY -> TLS/Youtube; SMTP -> SMTPS (via STARTTLS msg). In the vast majority of the cases, the protocol stack is simply Master/Application. Examples of real stacks (from the unit tests) different from the standard "master/app": * "STUN.WhatsAppCall.SRTP": a WA call * "STUN.DTLS.GoogleCall": a Meet call * "Telegram.STUN.DTLS.TelegramVoip": a Telegram call * "SMTP.SMTPS.Google": a SMTP connection to Google server started in cleartext and updated to TLS * "HTTP.Google.ntop": a HTTP connection to a Google domain (match via "Host" header) and to a ntop server (match via "Server" header) The logic to create the stack is still a bit coarse: we have a decade of code try to push everything in only ywo protocols... Therefore, the content of the stack is still **highly experimental** and might change in the next future; do you have any suggestions? It is quite likely that the legacy fields "master_protocol" and "app_protocol" will be there for a long time. Add some helper to use the stack: ``` ndpi_stack_get_upper_proto(); ndpi_stack_get_lower_proto(); bool ndpi_stack_contains(struct ndpi_proto_stack *s, u_int16_t proto_id); bool ndpi_stack_is_tls_like(struct ndpi_proto_stack *s); bool ndpi_stack_is_http_like(struct ndpi_proto_stack *s); ``` Be sure new stack logic is compatible with legacy code: ``` assert(ndpi_stack_get_upper_proto(&flow->detected_protocol.protocol_stack) == ndpi_get_upper_proto(flow->detected_protocol)); assert(ndpi_stack_get_lower_proto(&flow->detected_protocol.protocol_stack) == ndpi_get_lower_proto(flow->detected_protocol)); ```
* fuzz: extend fuzzing coverageIvan Nardi2025-06-24
| | | | Remove some unused code
* New API to enable/disable protocols. Removed ↵Ivan Nardi2025-06-23
| | | | | | | | | | | | | | | | | | | | | | | | | | `NDPI_LAST_IMPLEMENTED_PROTOCOL` (#2894) Change the API to enable/disable protocols: you can set that via the standard `ndpi_set_config()` function, as every configuration parameters. By default, all protocols are enabled. Split the (local) context initialization into two phases: * `ndpi_init_detection_module()`: generic part. It does not depend on the configuration and on the protocols being enabled or not. It also calculates the real number of internal protocols * `ndpi_finalize_initialization()`: apply the configuration. All the initialization stuff that depend on protocols being enabled or not must be put here This is the last step to have the protocols number fully calculated at runtime Remove a (now) useless fuzzer. Important API changes: * remove `NDPI_LAST_IMPLEMENTED_PROTOCOL` define * remove `ndpi_get_num_internal_protocols()`. To get the number of configured protocols (internal and custom) you must use `ndpi_get_num_protocols()` after having called `ndpi_finalize_initialization()`
* fuzz: fuzz loading of external protocols lists (#2897)Ivan Nardi2025-06-22
|
* fuzz: fix compilationIvan Nardi2025-06-18
|
* Prelimary work to remove `NDPI_LAST_IMPLEMENTED_PROTOCOL` (#2885)Ivan Nardi2025-06-16
|
* fuzz: try to improve coverage (#2883)Ivan Nardi2025-06-14
| | | Revert of 2b14b46df39e14d8c41ca1a5aa8db375bbc11ba6
* fuzz: make allocation failures a bit more unlikelyIvan Nardi2025-06-12
|
* Rework sanity checks and remove some functions from API (#2882)Ivan Nardi2025-06-12
|
* fuzz: improve coverage (#2878)Ivan Nardi2025-06-10
|
* Rename `ndpi_bitmask_dealloc` into `ndpi_bitmask_free`Ivan Nardi2025-06-09
|
* Remove `NDPI_PROTOCOL_BITMASK`; add a new generic bitmask data structure (#2871)Ivan Nardi2025-06-09
| | | | | | | | | | | | | | | | | | | The main difference is that the memory is allocated at runtime Typical usercase: ``` struct ndpi_bitmask b; ndpi_bitmask_alloc(&b, ndpi_get_num_internal_protocols()); ndpi_bitmask_set(&b, $BIT); ndpi_bitmask_is_set(&b, $BIT); [...] ndpi_bitmask_dealloc(&b); ``` See #2136
* Add category and breed support for custom rules (#2872)Vladimir Gavrilov2025-06-08
| | | Close #2594
* Add a configuration to test a huge number of custom protocols (#2865)Ivan Nardi2025-06-03
| | | File taken from #2136
* Remove `ndpi_set_proto_defaults()` from the API (#2863)Ivan Nardi2025-06-03
| | | Add an explicit field to indicate if the protocol is custom or internal
* First step into a dynamic number of protocols (#2857)Ivan Nardi2025-06-03
| | | | | | | | | | | | | We want to get rid of the defines `NDPI_MAX_SUPPORTED_PROTOCOLS` and `NDPI_MAX_NUM_CUSTOM_PROTOCOLS`. You can use: ``` ndpi_get_num_protocols() ``` See #2136 Removed some unused functions from public API
* New API to enable/disable protocols; remove ↵Ivan Nardi2025-06-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | `ndpi_set_protocol_detection_bitmask2()` (#2853) The main goal is not to have the bitmask depending on the total number of protocols anymore: `NDPI_INTERNAL_PROTOCOL_BITMASK` depends only on internal protocols, i.e. on `NDPI_MAX_INTERNAL_PROTOCOLS`, i.e. custom-defined protocols are not counted. See #2136 Keep the old data structure `NDPI_PROTOCOL_BITMASK` with the old semantic. Since we need to change the API (and all the application code...) anyway, simplify the API: by default all the protocols are enabled. If you need otherwise, please use `ndpi_init_detection_module_ext()` instead of `ndpi_init_detection_module()` (you can find an example in the `ndpiReader` code). To update the application code you likely only need to remove these 3 lines from your code: ``` - NDPI_PROTOCOL_BITMASK all; - NDPI_BITMASK_SET_ALL(all); - ndpi_set_protocol_detection_bitmask2(ndpi_str, &all); ``` Removed an unused field and struct definition.
* Follow-up of c1d372860 (TCP fingerprint format) (#2850)Ivan Nardi2025-05-26
|
* UBNTAC2,Ookla: improve detection (#2793)Ivan Nardi2025-04-10
|
* Follow-up of latest Signal call change (see: 4d41588a7)Ivan Nardi2025-04-05
|
* fuzz: extend coverage (#2786)Ivan Nardi2025-03-31
|
* fuzz: fix configurationIvan Nardi2025-03-26
|
* fuzz: fix configuration after latest updatesIvan Nardi2025-03-26
|
* Improved configuration to enable/disable export of flow risk info (#2780)Ivan Nardi2025-03-25
| | | | Follow-up of f56831336334dddcff00eaf2132e5e0f226f0e32: now the configuration is for flow-risk, not global
* fuzz: try to run one (ndpiReader-) fuzzer with a slight different cfg (#2771)Ivan Nardi2025-03-18
|
* Add Autonomous System Organization to geoip (#2763)Leonardo Teixeira Alves2025-03-06
| | | Co-authored-by: Leonardo Teixeira Alves <leonardo.alves@zerum.com>
* Add configuration parameter to enable/disable export of flow risk info (#2761)Ivan Nardi2025-03-05
| | | | For the most common protocols, avoid creating the string message if we are not going to use it
* Address cache: fix some bugs on cache traversalIvan Nardi2025-03-01
| | | | Add a new fuzzer to test it
* fuzz: extend fuzzing coverage (#2750)Ivan Nardi2025-02-28
|
* Add city as a geoip possibility (#2746)Leonardo Teixeira Alves2025-02-24
|
* RTP: payload type info should be set only for real RTP flows (#2742)Ivan Nardi2025-02-22
|
* SSDP: add configuration for disabling metadata extraction (#2736)Ivan Nardi2025-02-17
|
* DNS: fix check for DGA domain (#2716)Ivan Nardi2025-02-11
| | | | If we have a (potential) valid sub-classification, we shoudn't check for DGA, even if the subclassification itself is disabled!
* DNS: disable subclassification by default (#2715)Ivan Nardi2025-02-11
| | | | Prelimary change to start supporting multiple DNS transactions on the same flow
* Compilation fixLuca2025-02-07
|
* bittorrent: add configuration for "hash" metadata (#2706)Ivan Nardi2025-01-31
| | | Fix confidence value for same TCP flows
* HTTP: add configuration for some metadata (#2704)Ivan Nardi2025-01-31
| | | Extend file configuration for just subclassification.
* Create a specific configuration for classification only (#2689)Ivan Nardi2025-01-31
| | | | | | | | | | | | | | | | | | | | | | | | In some scenarios, you might not be interested in flow metadata or flow-risks at all, but you might want only flow (sub-)classification. Examples: you only want to forward the traffic according to the classification or you are only interested in some protocol statistics. Create a new configuration file (for `ndpiReader`, but you can trivially adapt it for the library itself) allowing exactly that. You can use it via: `ndpiReader --conf=example/only_classification.conf ...` Note that this way, the nDPI overhead is lower because it might need less packets per flow: * TLS: nDPI processes only the CH (in most cases) and not also the SH and certificates * DNS: only the request is processed (instead of both request and response) We might extend the same "shortcut-logic" (stop processing the flow immediately when there is a final sub-classification) for others protocols. Add the configuration options to enable/disable the extraction of some TLS metadata.
* fuzz: extend fuzzing coverage (#2696)Ivan Nardi2025-01-23
|
* TLS: remove JA3C (#2679)Ivan Nardi2025-01-14
| | | | | | | | Last step of removing JA3C fingerprint Remove some duplicate tests: testing with ja4c/ja3s disabled is already performed by `disable_metadata_and_flowrisks` configuration. Close:#2551
* Add (kind of) support for loading a list of JA4C malicious fingerprints (#2678)Ivan Nardi2025-01-14
| | | | | | | | | It might be usefull to be able to match traffic against a list of suspicious JA4C fingerprints Use the same code/logic/infrastructure used for JA3C (note that we are going to remove JA3C...) See: #2551
* fuzz: add 2 new fuzzers for KD-trees and Ball-trees (#2670)Ivan Nardi2025-01-13
|
* Add the ability to enable/disable every specific flow risks (#2653)Ivan Nardi2025-01-06
|
* fuzz: improve fuzzing coverage (#2642)Ivan Nardi2024-12-11
| | | Updtae pl7m code (Fix swap-direction mutation)
* Update `flow->flow_multimedia_types` to a bitmask (#2625)Ivan Nardi2024-11-25
| | | In the same flow, we can have multiple multimedia types
* fuzz: extend fuzzing coverage (#2626)Ivan Nardi2024-11-20
|
* SIP: extract some basic metadataIvan Nardi2024-11-12
|
* Unify ndpi debug logging to always use a u16 protocol id (#2613)Toni2024-11-11
| | | | | * fixes SonarCloud complaint Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.