| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We can't write to `flow->protos.dns` until we are sure this is a valid
DNS packet
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==14729==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x60e876372a86 bp 0x000000000000 sp 0x79392fdf90e0 T1)
==14729==The signal is caused by a READ memory access.
==14729==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used.
#0 0x60e876372a86 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8b0a86) (BuildId: a9c4718bcd5c3947812b6fd704e203b8bb6f633c)
#1 0x60e87640b29f in free (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x94929f) (BuildId: a9c4718bcd5c3947812b6fd704e203b8bb6f633c)
#2 0x60e87647b0ec in free_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:348:3
#3 0x60e876865454 in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:82:7
#4 0x60e8767f0d4f in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6752:2
#5 0x60e8767abd67 in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:10449:5
```
Found by oss-fuzz
|
| |
|
|
|
| |
Allow optimal FPC even if DNS subclassification is disabled
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
```
Running: /home/ivan/Downloads/clusterfuzz-testcase-minimized-fuzz_ndpi_reader_pl7m_simplest_internal-5759495480868864
protocols/dns.c:482:5: runtime error: index 4 out of bounds for type 'u_int8_t[4]' (aka 'unsigned char[4]')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/dns.c:482:5
protocols/dns.c:483:5: runtime error: index 4 out of bounds for type 'u_int32_t[4]' (aka 'unsigned int[4]')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/dns.c:483:5
protocols/dns.c:490:12: runtime error: index 4 out of bounds for type 'u_int32_t[4]' (aka 'unsigned int[4]')
```
Found by oss-fuzz
See: https://issues.oss-fuzz.com/issues/383911300?pli=1
|
|
|
|
|
|
|
|
|
|
| |
as follows:
ndpiReader --cfg=dpi.address_cache_size,1000 -i <pcap>.pcap
In the above example the cache has up to 1000 entries.
In jcase ndpiReader exports data in JSON, the cache hostname (if found) is exported in the field server_hostname
|
|
|
|
| |
Changed the default to IPv4 (used to be IPv6) in case of DNS error response
|
|
|
|
|
|
| |
"Invalid DNS Header"-risk should be set only if the flow has been
already classified as DNS. Otherwise, almost any non-DNS flows on port 53
will end up having the `NDPI_MALFORMED_PACKET` risk set, which is a little
bit confusing for non DNS traffic
|
| |
|
|
|
|
|
|
|
|
|
| |
Use DNS information to get a better First Packet Classification.
See: #2322
---------
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);
used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco
* Major code rewrite for ndpi_hash and ndpi_domain_classify
* Improvements to make sure custom categories are loaded and enabled
* Fixed string encoding
* Extended SalesForce/Cloudflare domains list
|
|
|
|
|
| |
* unused parameters and functions pollute the code and decrease readability
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
* Normalization of host_server_name
The ndpi_hostname_sni_set() function replaces all non-printable
characters with the "?" character and removing whitespace characters
at the end of the line.
* Added conditional hostname normalization.
|
|
|
|
|
| |
* Enable/disable sub-classification of DNS flows
* Enable/disable processing of DNS responses
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove some unreached/duplicated code.
Add error checking for `atoi()` calls.
About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
|
| |
|
| |
|
|
|
|
|
|
| |
1) Public API/headers in `src/include/` [as it has always been]
2) Private API/headers in `src/lib/`
Try to keep the "ndpi_" prefix only for the public functions
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/
Fix Python bindings
Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
| ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
775 | unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
|
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
160 | uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
3419 | strncpy(buf, name, name_len);
```
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
The option NSID (RFC5001) is used by Google DNS to report the
airport code of the metro where the DNS query is handled.
This option is quite rare, but the added overhead in DNS code is pretty
much zero for "normal" DNS traffic
|
|
|
|
| |
that in DNS (instead) are not permitted
|
|
|
|
| |
Fix CI
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
|
|
|
|
| |
about issues found on traffic.
|
|
|
|
| |
Improved DNS dissection
|
|
|
| |
Remove some dead code (found via coverage report)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)
This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.
Custom protocols are always enabled.
Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.
Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.
Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Load some custom configuration (like in the unit tests) and factorize some
(fuzzing) common code.
There is no way to pass file paths to the fuzzers as parameters. The safe
solution seems to be to load them from the process working dir. Anyway,
missing file is not a blocking error.
Remove some dead code (found looking at the coverage report)
|
|
|
|
| |
Found by sydr-fuzz
Close #1803
|
| |
|
|
|
|
|
|
| |
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
Follow-up of 83de3e47
|
|
|
|
|
| |
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The host automa is used for two tasks:
* protocol sub-classification (obviously);
* DGA evaluation: the idea is that if a domain is present in this
automa, it can't be a DGA, regardless of its format/name.
In most dissectors both checks are executed, i.e. the code is something
like:
```
ndpi_match_host_subprotocol(..., flow->host_server_name, ...);
ndpi_check_dga_name(..., flow->host_server_name,...);
```
In that common case, we can perform only one automa lookup: if we check the
sub-classification before the DGA, we can avoid the second lookup in
the DGA function itself.
|
|
|
|
|
|
|
|
| |
See 95e16872.
After c0732eda, we can safely remove the protocol list from
`ndpi_process_extra_packet()`.
The field `flow->check_extra_packets` is redundant; remove it.
|
|
|
|
|
|
|
| |
Move the prottocol specific logic into the proper dissector code, where
it belongs.
Next step: remove that list of protocols. Long goal: remove this
function altogether...
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
* Checking for port 5353/5355 is not enough.
* Added additional multicast address and header checks.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.
In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).
Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
|
| |
|