| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The hard limit of total number of protocols (internal and custom) is ~65535,
because protocol ids are `u_int16_t`...
API changes:
1. From `NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS` to
`ndpi_get_num_protocols()` (after having called
`ndpi_finalize_initialization()`);
2. From `proto_id >= NDPI_MAX_SUPPORTED_PROTOCOLS` to
`ndpi_is_custom_protocol(proto_id)` (after having called
`ndpi_finalize_initialization()`);
Close #2136
Close #2545
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main difference is that the memory is allocated at runtime
Typical usercase:
```
struct ndpi_bitmask b;
ndpi_bitmask_alloc(&b, ndpi_get_num_internal_protocols());
ndpi_bitmask_set(&b, $BIT);
ndpi_bitmask_is_set(&b, $BIT);
[...]
ndpi_bitmask_dealloc(&b);
```
See #2136
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`ndpi_set_protocol_detection_bitmask2()` (#2853)
The main goal is not to have the bitmask depending on the total number
of protocols anymore: `NDPI_INTERNAL_PROTOCOL_BITMASK` depends only on
internal protocols, i.e. on `NDPI_MAX_INTERNAL_PROTOCOLS`, i.e.
custom-defined protocols are not counted.
See #2136
Keep the old data structure `NDPI_PROTOCOL_BITMASK` with the old
semantic.
Since we need to change the API (and all the application code...)
anyway, simplify the API: by default all the protocols are enabled.
If you need otherwise, please use `ndpi_init_detection_module_ext()`
instead of `ndpi_init_detection_module()` (you can find an example in
the `ndpiReader` code).
To update the application code you likely only need to remove these 3
lines from your code:
```
- NDPI_PROTOCOL_BITMASK all;
- NDPI_BITMASK_SET_ALL(all);
- ndpi_set_protocol_detection_bitmask2(ndpi_str, &all);
```
Removed an unused field and struct definition.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We use `registr_dissector()` instead of
`ndpi_set_bitmask_protocol_detection()`.
Every file in `src/lib/protocols/*.c` is a dissector.
Every dissector can handle multiple protocols.
The real goal is this small change:
```
struct call_function_struct {
- NDPI_PROTOCOL_BITMASK detection_bitmask;
```
i.e. getting rid of another protocol bitmask: this is mandatory to try
to fix #2136 (see also e845e8205b68752c997d05224d8b2fd45acde714)
As a nice side effect, we remove a bitmask comparison in the hot function
`check_ndpi_detection_func()`
TODO: change logging configuration from per-protocol to per-dissector
|
|
|
|
|
|
| |
In the flow, we should keep track of state of "dissectors", not
"protocols". This way, flow structure doesn't depend anymore on
the max number of protocols.
This is also the first step into fixing #2136
|
|
|
|
| |
This is the first, tiny, step into a better separation between
"protocols" and "dissectors"
|
|
|
| |
See 6899f6c17 and 9bf513b34
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Integrated RoaringBitmap v3
* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse
* Fixes to ndpi_bitmap for new roaring library
* Fixes for bitmap serialization
* Fixed format
* Warning fix
* Conversion fix
* Warning fix
* Added check for roaring v3 support
* Updated file name
* Updated path
* Uses clang-9 (instead of clang-7) for builds
* Fixed fuzz_ds_bitmap64_fuse
* Fixes nDPI printf handling
* Disabled printf
* Yet another printf fix
* Cleaup
* Fx for compiling on older platforms
* Fixes for old compilers
* Initialization changes
* Added compiler check
* Fixes for old compilers
* Inline function is not static inline
* Added missing include
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the first step into providing (more) configuration options in nDPI.
The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.
You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format
This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.
The complete list of configuration options is provided in
`doc/configuration_parameters.md`.
As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
|
|
|
|
|
|
| |
* support for using a new flow callback invoked before the flow memory is free'd
* minor fixes
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The goal of this fuzzer is to test init and deinit of the library, with
different configurations. In details:
* random memory allocation failures, even during init phase
* random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()`
* random LRU caches sizes
* random bitmask of enabled protocols
* random parameters of `ndpi_set_detection_preferences()`
* random initialization of opportunistic TLS
* random load/don't load of configuration files
This new fuzzer is a C++ file, because it uses `FuzzedDataProvider`
class (see
https://github.com/google/fuzzing/blob/master/docs/split-inputs.md).
Note that the (existing) fuzzers need to be linked with C++ compiler
anyway, so this new fuzzer doesn't add any new requirements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)
This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.
Custom protocols are always enabled.
Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.
Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.
Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
|
|
|
|
|
|
|
| |
That way, we can make `src/include/ndpi_api.h` independent of autoconf.
MSBuild will like that.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
Tcp retransmissions should be ignored.
Remove some unused protocol bitmasks.
Update script to download Whatsapp IP list.
|
|
|
|
| |
Add an helper to exclude a generic protocol
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Revert "SMTP with STARTTLS is now identified as SMTPS"
This reverts commit 52d987b603f49d996b4060f43265d1cf43c3c482.
* Revert "Compilation fix"
This reverts commit c019946f601bf3b55f64f78841a0d696e6c0bfc5.
* Sync unit tests.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
| |
Close #1598
|
| |
|
|
|
|
|
|
| |
- added syslog false-positive pcap that was missing in 09fbe0a64a11b08a35435f516e9a19f7e0c20d7c
- added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
| |
* Testing more code in CI environments.
* Added strict option checking for `./configure' in CI environments.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* The current behaviour ignores any user preferences
and was also incorrectly implemented, because the
flow->num_processed_pkts wraps every 65535 and nDPI
will process packets again until
NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
|
|
|
|
|
| |
Update .gitignore file
Fix a function prototype
Close #1349
|
| |
|
|
|
|
| |
prevents ndpi_config.h to be included everywhere in apps using nDPI that might leade to #define redefinitions after the latest changes
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix unaligned memory accesses with get_u_int64_t at armhf
see: https://bugs.debian.org/993627
* Use get_u_int64_t to avoid unaligned memory access at armhf
see: https://bugs.debian.org/993627
* Update src/include/ndpi_define.h.in
Drop const type from get_u_int64_t, from lnslbrty
Co-authored-by: Bernhard Übelacker <bernhardu@mailbox.org>
Co-authored-by: Toni <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
| |
Follow-up of 22241a1d
Only trivial changes:
* remove completely unused fields
* remove fields only written (but never read)
* CSGO protocol only handles UDP traffic
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
Since 19a29e1e (NDPI_TLS_CERT_VALIDITY_TOO_LONG is 32), unit tests are
failing on 32 bit machines (i.e Raspberry 4)
|
| |
|