| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
Integrate .md files into official documentation
See: https://stackoverflow.com/questions/46278683/include-my-markdown-readme-into-sphinx/68005314#68005314
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Allow nDPI to process the entire flows and not only the first N packets.
Usefull when the application is interested in some metadata spanning the
entire life of the session.
As initial step, only STUN flows can be put in monitoring.
See `doc/monitoring.md` for further details.
This feature is disabled by default.
Close #2583
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the concept of "global context".
Right now every instance of `struct ndpi_detection_module_struct` (we
will call it "local context" in this description) is completely
independent from each other. This provide optimal performances in
multithreaded environment, where we pin each local context to a thread,
and each thread to a specific CPU core: we don't have any data shared
across the cores.
Each local context has, internally, also some information correlating
**different** flows; something like:
```
if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then
flow2 (PeerC <-> PeerD) will be PROTOCOL_Y
```
To get optimal classification results, both flow1 and flow2 must be
processed by the same local context. This is not an issue at all in the far
most common scenario where there is only one local context, but it might
be impractical in some more complex scenarios.
Create the concept of "global context": multiple local contexts can use
the same global context and share some data (structures) using it.
This way the data correlating multiple flows can be read/write from
different local contexts.
This is an optional feature, disabled by default.
Obviously data structures shared in a global context must be thread safe.
This PR updates the code of the LRU implementation to be, optionally,
thread safe.
Right now, only the LRU caches can be shared; the other main structures
(trees and automas) are basically read-only: there is little sense in
sharing them. Furthermore, these structures don't have any information
correlating multiple flows.
Every LRU cache can be shared, independently from the others, via
`ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`.
It's up to the user to find the right trade-off between performances
(i.e. without shared data) and classification results (i.e. with some
shared data among the local contexts), depending on the specific traffic
patterns and on the algorithms used to balance the flows across the
threads/cores/local contexts.
Add some basic examples of library initialization in
`doc/library_initialization.md`.
This code needs libpthread as external dependency. It shouldn't be a big
issue; however a configure flag has been added to disable global context
support. A new CI job has been added to test it.
TODO: we should need to find a proper way to add some tests on
multithreaded enviroment... not an easy task...
*** API changes ***
If you are not interested in this feature, simply add a NULL parameter to
any `ndpi_init_detection_module()` calls.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the first step into providing (more) configuration options in nDPI.
The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.
You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format
This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.
The complete list of configuration options is provided in
`doc/configuration_parameters.md`.
As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
|
| | |
|
| | |
|
| |
|
|
|
| |
* added handling of parsing errors
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/
Fix Python bindings
Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
| ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
|
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
160 | uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
3419 | strncpy(buf, name, name_len);
```
|
| | |
|
| | |
|
| |
|
|
|
|
| |
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
| |
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".
Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
|
| | |
|
| |
|
|
|
| |
by README.md (#1572)
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
| |
* Detection is unreliable and results in too much false-positives (and false-negatives!)
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
|
|
| |
* make check great again (not so much)
* make doc/doc-view
* CI updates
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
| |
* Integrated Doxygen documentation into Sphinx
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
|
|
| |
This fixes some build/test issues resulting when using tarballs.
* nDPI uses autotools (especially autoconf) in a wrong way, see #1163
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
| |
* It is outdated (from 2018) and will most likely not work anymore
due to changed protocol files and outdated API usage in the example integration
* Removal is the only option besides fixing those issues and
provide a CI integration for future automatic checks
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
| |
packaging and CI integration)
|
| |
|
|
|
|
| |
* Tor via TLS should be detectable via DGA as a risk
* protocol limitations should be part of the official documentation in `doc/`
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
| |
* configure.seed references removed
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
| |
#1324 (#1327)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| | |
|
| |
|
|
| |
Updated Changelog
|
| |
|
|
|
| |
nDPI but still somehow essential. (#1024)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| | |
|
| |
|
|
| |
And configur option enable-fuzztargets
|
| | |
|
| |
|
|
| |
shared library.
|
| |
|
|
|
| |
Added target for install
Fixes #593
|
| | |
|
| |
|