aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols/http.c
Commit message (Collapse)AuthorAge
...
* Added check for empty/missing user-agent in HTTPLuca Deri2022-12-03
|
* Enhanced HTTP numeric IP checkLuca Deri2022-10-22
|
* HTTP: fix stack-buffer-overflow (#1768)Ivan Nardi2022-10-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==24879==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fa085b31e60 at pc 0x55cc63f203e2 bp 0x7ffc9ec91b10 sp 0x7ffc9ec91298 READ of size 17 at 0x7fa085b31e60 thread T0 #0 0x55cc63f203e1 in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55cc63f20769 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x50e769) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) #2 0x55cc63f22210 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x510210) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) #3 0x55cc6420fc76 in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:668:4 #4 0x55cc6420344b in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:742:5 #5 0x55cc642031ce in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:737:7 #6 0x55cc641fac9f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1352:4 #7 0x55cc641f2fd5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1461:3 #8 0x55cc64085275 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5580:6 #9 0x55cc64085c87 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5628:12 #10 0x55cc64085927 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5647:12 #11 0x55cc64095fcb in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6458:15 #12 0x55cc63fd08b4 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #13 0x55cc63fd09f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #14 0x7fa0880fb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #15 0x55cc63efb45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e945d) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b) Address 0x7fa085b31e60 is located in stack of thread T0 at offset 96 in frame #0 0x55cc6420f1bf in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:644 This frame has 5 object(s): [32, 36) 'a' (line 653) [48, 52) 'b' (line 653) [64, 68) 'c' (line 653) [80, 96) 'buf' (line 654) [112, 176) 'msg' (line 662) <== Memory access at offset 96 partially underflows this variable ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52229
* Added detection of URLs starting with a dot (.)Luca Deri2022-10-10
|
* Fixes Stack-buffer-overflow READ {*} · check_content_type_and_change_protocolLuca Deri2022-10-06
|
* Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx ↵Luca2022-10-04
| | | | are supported
* Fix compilation and sync utests resultsNardi Ivan2022-10-04
|
* Improved HTTP payload handlingLuca2022-10-04
|
* Improved detection of WebShell and PHP code in HTTP URLs that is reported ↵Luca2022-10-04
| | | | via flow risk
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Dissectors shouldn't update `flow->guessed_host_protocol_id`Nardi Ivan2022-09-14
| | | | | | | | | | | | | | | | The field `flow->guessed_host_protocol_id` is set at the beginning of the flow analysis and it represents the "classification by ip" of the flow itself. This field should never be changed. Dissectors which want to provide an "hint" about the classification, should update `flow->guessed_protocol_id` instead. Such "hint" is useless if the dissector set the "extra-dissection" data-path. Rename such field to `guessed_protocol_id_by_ip` to better describe its role. Preliminary work necessary for #1687
* HTTP: remove some wrong codeNardi Ivan2022-09-13
| | | | | | | | | | This code seems wrong or in the wrong place, at least: * "classification by port" and "classification by ip" protocols (i.e "guessed" protocols) should be used to set the protocol stack only after trying all the dissectors, and only by the generic code * there are no reason (for a dissector) to update the "guessed" information using the protocol stack values: it is usually the other way around (see previous point)
* HTTP: correctly set the classification protocolsNardi Ivan2022-09-06
| | | | | | | | | | | | Classification should always be set via `ndpi_set_detected_protocol()` to be sure to set a correct `confidence` value, too. Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as confidence, is not valid. This code in HTTP dissector likely needs some more thoughts (the classification itself of the attached example doesn't make a lot of sense), but the goal of this commit is only to always have a valid `confidence` value.
* Avoid useless host automa lookup (#1724)Ivan Nardi2022-09-05
| | | | | | | | | | | | | | | | | | | The host automa is used for two tasks: * protocol sub-classification (obviously); * DGA evaluation: the idea is that if a domain is present in this automa, it can't be a DGA, regardless of its format/name. In most dissectors both checks are executed, i.e. the code is something like: ``` ndpi_match_host_subprotocol(..., flow->host_server_name, ...); ndpi_check_dga_name(..., flow->host_server_name,...); ``` In that common case, we can perform only one automa lookup: if we check the sub-classification before the DGA, we can avoid the second lookup in the DGA function itself.
* HTTP, SoftEther, Florensia: fix some memory corruptionsNardi Ivan2022-08-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==12308==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068f4b4 at pc 0x5603d2b56098 bp 0x7ffc967bd3b0 sp 0x7ffc967bd3a8 READ of size 2 at 0x60600068f4b4 thread T0 #0 0x5603d2b56097 in ndpi_search_florensia /home/ivan/svnrepos/nDPI/src/lib/protocols/florensia.c:93:40 #1 0x5603d2a2e835 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6 #2 0x5603d2a2f247 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12 #3 0x5603d2a2eee7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12 #4 0x5603d2a3ffb5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15 #5 0x5603d2955f4c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1612:31 #6 0x5603d294ee64 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10 #7 0x5603d290aa50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #8 0x5603d290ab8b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #9 0x7f972cdc9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #10 0x5603d28494fd in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x4be4fd) (BuildId: e81373a75c70e248ba9ae985fcb4134e7113121a) ``` ``` ==14164==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60d0000000c1 at pc 0x561608bada2c bp 0x7ffc40d38ed0 sp 0x7ffc40d38690 READ of size 10 at 0x60d0000000c1 thread T0 #0 0x561608bada2b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e9a2b) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b) #1 0x561608e66f2a in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:226:2 #2 0x561608e64e88 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:316:8 #3 0x561608ca7475 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6 #4 0x561608ca823b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10 #5 0x561608ca7bdc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12 #6 0x561608cb8bf5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15 #7 0x561608bff84e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 #8 0x561608bff9c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17 #9 0x7fc50662d082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #10 0x561608b3e42d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47a42d) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b) ``` Found by oss-fuzzer. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50105 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50110 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50268
* HTTP: fix classification in some corner cases (#1704)Ivan Nardi2022-08-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Initializing `low->detected_protocol_stack[1]` but not `low->detected_protocol_stack[0]` lead to *very* strange errors. Oss-fuzzer, exploiting this bug, has been able to crash the application or to leak some memory ``` ==19775==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x5624c0dd1802 bp 0x5624c197e848 sp 0x7fff4b4db690 T0) ==19775==The signal is caused by a READ memory access. ==19775==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x5624c0dd1802 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x522802) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa) #1 0x5624c0e53ea6 in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5a4ea6) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa) #2 0x5624c0ebb474 in free_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:330:3 #3 0x5624c0f75561 in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:240:7 #4 0x5624c0faa921 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4915:2 #5 0x5624c0f755ef in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8076:5 #6 0x5624c0f755c4 in ndpi_flow_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:253:5 #7 0x5624c0eba981 in ndpi_free_flow_info_half /home/ivan/svnrepos/nDPI/example/reader_util.c:305:25 #8 0x5624c0ecb4bc in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1317:5 #9 0x5624c0edca36 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1639:2 #10 0x5624c0ed5172 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10 #11 0x5624c0e90a80 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #12 0x5624c0e90bbb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #13 0x7f580731f082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #14 0x5624c0dcf52d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x52052d) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa) ``` ``` ==17492==ERROR: LeakSanitizer: detected memory leaks Direct leak of 5 byte(s) in 1 object(s) allocated from: #0 0x55b799b0f01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b) #1 0x55b799bb3bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46 #2 0x55b799bb3f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13 #3 0x55b799c8b9bf in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:683:39 #4 0x55b799c81e89 in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:835:7 #5 0x55b799c8d500 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:889:13 #6 0x55b799ca80f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2 #7 0x55b799ca3be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5 #8 0x55b799bf9345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6 #9 0x55b799bf9d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12 #10 0x55b799bf99f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12 #11 0x55b799c0b838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15 #12 0x55b799b4b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 ``` ``` ==17511==ERROR: LeakSanitizer: detected memory leaks Direct leak of 1 byte(s) in 1 object(s) allocated from: #0 0x55aed645e01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b) #1 0x55aed6502bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46 #2 0x55aed6502f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13 #3 0x55aed65ea0c6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34 #4 0x55aed65dbe77 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5 #5 0x55aed65f70f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2 #6 0x55aed65f2be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5 #7 0x55aed6548345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6 #8 0x55aed6548d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12 #9 0x55aed65489f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12 #10 0x55aed655a838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15 #11 0x55aed649a87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49844 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49842 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49906 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49915
* HTTP, Bittorrent, Kerberos: fix stopping of extra dissection (#1700)Ivan Nardi2022-08-06
| | | | | | | The return value of the extra-dissection callback indicates if the extra dissection needs to be called again. In the HTTP cose, this setting to NULL of the callabck is wrong since we stop extra dissection only if we have a hostname *and* a return code.
* Further simplification of `ndpi_process_extra_packet()` (#1698)Ivan Nardi2022-08-05
| | | | | | | | See 95e16872. After c0732eda, we can safely remove the protocol list from `ndpi_process_extra_packet()`. The field `flow->check_extra_packets` is redundant; remove it.
* HTTP: improve sub-classification (#1696)Ivan Nardi2022-08-05
| | | | | | | | | | Content-matched sub-protocols (DASH, IPP, MPEGDASH...) shouldn't ovewrite the previous master protocol (if any; usually HTTP). Furthermore. the HTTP dissector shouldn't update the classification (in the extra-dissection code path) if a content-matched sub-protocols has already been found. This commit should address the first part of the changes described in #1687.
* HTTP: fix classification (#1692)Ivan Nardi2022-07-30
| | | | | | | | | | If we have a valid HTTP sessions, we should ignore `flow->guessed_protocol_id` field (i.e. classification "by-port") altogheter. The attached trace was classified as "SIP/HTTP" only because the *client* port was 5060... As a general rule, having a classification such as "XXXX/HTTP" is *extremely* suspicious.
* Update the protocol bitmask for some protocols (#1675)Ivan Nardi2022-07-27
| | | | | | | Tcp retransmissions should be ignored. Remove some unused protocol bitmasks. Update script to download Whatsapp IP list.
* Improve handling of HTTP-Proxy and HTTP-Connect (#1673)Ivan Nardi2022-07-25
| | | | | | | | | | | | Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones: print/serialize all the attributes and allow parsing of replies. The line about "1kxun" has been removed to avoid regressions in 1KXUN classification in `tests/pcap/1kxun.pcap`. I haven't fully understod what was happening but the comment at the beginning of `static ndpi_category_match category_match[]` says that we can't have overlaps between `host_match` and `category_match` lists and that is no longer true since 938e89ca. Bottom line: removing this line seems the right thing to do, anyway.
* HTTP: improve detection of WindowsUpdate (#1658)Ivan Nardi2022-07-10
| | | | WindowsUpdate is also transported over HTTP, using a numeric IP as hostname (some kinds of CDN?)
* Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)Toni2022-05-30
| | | Signed-off-by: lns <matzeton@googlemail.com>
* HTTP: fix heap-buffer-overflow error (#1564)Ivan Nardi2022-05-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` ==222479==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b000014739 at pc 0x55af06f2364f bp 0x7ffd7b6f4bf0 sp 0x7ffd7b6f4378 READ of size 12 at 0x60b000014739 thread T0 #0 0x55af06f2364e in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55af06f24f70 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x613f70) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8) #2 0x55af0720927f in ndpi_check_http_header /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:982:2 #3 0x55af071f4797 in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1219:5 #4 0x55af071f05c5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1402:3 #5 0x55af07080d1e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6 #6 0x55af07081734 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12 #7 0x55af070813d7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12 #8 0x55af070939f7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15 #9 0x55af06fc7e1f in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1541 #10 0x55af06fc7e1f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110 #11 0x55af06fc2859 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7 #12 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17 #13 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #14 0x55af06f0055d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5ef55d) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8) 0x60b000014739 is located 0 bytes to the right of 105-byte region [0x60b0000146d0,0x60b000014739) allocated by thread T0 here: #0 0x55af06f84bae in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x673bae) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8) #1 0x55af06fc2673 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:103:31 #2 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17 #3 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 ``` Fiund by oss-fuzzer. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47724
* Updated tests resultsLuca Deri2022-05-30
| | | | Code cleanup
* Added RiskInfo stringLuca Deri2022-05-30
|
* Added detection for WordPress exploitsLuca Deri2022-05-29
| | | | Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
* Added MPEG-DASH dissector. Fixes #1223. (#1555)Toni2022-05-29
| | | | | | * Improved HTTP POST detection * Refactored subprotocol detection Signed-off-by: lns <matzeton@googlemail.com>
* Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)Toni2022-05-28
| | | | | * If HTTP was already detected, there is no need to overwrite the detection again Signed-off-by: lns <matzeton@googlemail.com>
* Improved detection of invalid SNI and hostnames in TLS, HTTPLuca Deri2022-05-24
|
* Moved RTSP http patterns to the protocol source file.lns2022-05-19
| | | | Signed-off-by: lns <matzeton@googlemail.com>
* Improved Xiaomi HTTP detection. (#1546)Toni2022-05-10
| | | | | * Merged Xiaomi pcap files Signed-off-by: lns <matzeton@googlemail.com>
* Improved suspicious http user agent detection. (#1537)Toni2022-05-02
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Added generic user agent setter. (#1530)Toni2022-04-25
| | | | | * ndpiReader: Print user agent if one was set and not just for certain protocols. Signed-off-by: lns <matzeton@googlemail.com>
* Fix compilation and sync unit tests results (#1445)Ivan Nardi2022-02-19
| | | | 'strcasestr' is not defined in all enviroments: quicker fix is to use 'ndpi_strncasestr' instead.
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* Added NDPI_ERROR_CODE_DETECTED riskLuca Deri2022-02-03
|
* Renamed DCERPC to more generic RPC protocol so we can use also for other ↵Luca Deri2022-02-03
| | | | | | | types of RPCs (not limited to DCE) Extended HTTP plugin to support RPC Improved HTTP crear text detection to limit it to Basic and Digest
* Add a "confidence" field about the reliability of the classification. (#1395)Ivan Nardi2022-01-11
| | | | | | | | | | | | | As a general rule, the higher the confidence value, the higher the "reliability/precision" of the classification. In other words, this new field provides an hint about "how" the flow classification has been obtained. For example, the application may want to ignore classification "by-port" (they are not real DPI classifications, after all) or give a second glance at flows classified via LRU caches (because of false positives). Setting only one value for the confidence field is a bit tricky: more work is probably needed in the next future to tweak/fix/improve the logic.
* Improved user agent analysisLuca Deri2022-01-09
|
* Invalid checkLuca Deri2022-01-06
|
* Update copyrightAlfredo Cardigliano2022-01-03
|
* Added support for Log4J/Log4Shell detection in nDPI via a new flow risk ↵Luca Deri2021-12-23
| | | | named NDPI_POSSIBLE_EXPLOIT
* A final(?) effort to reduce memory usage per flow (#1389)Ivan Nardi2021-12-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove some unused fields and re-organize other ones. In particular: * Update the parameters of `ndpi_ssl_version2str()` function * Zattoo, Thunder: these timestamps aren't really used. * Ftp/mail: these protocols are dissected only over TCP. * Attention must be paid to TLS.Bittorrent flows to avoid invalid read/write to `flow->protos.bittorrent.hash` field. This is the last(?) commit of a long series (see 22241a1d, 227e586e, 730c2360, a8ffcd8b) aiming to reduce library memory consumption. Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 231.71 KB Flow Memory (per flow): 1008 B <--------- ``` i.e. memory usage per flow has been reduced by 66%, dropping below the psychological threshold of 1 KB. To further reduce this value, we probably need to look into #1279: let's fight this battle another day.
* HTTP proxy improvementLuca Deri2021-11-25
|
* Reworked HTTP protocol dissection including HTTP proxy and HTTP connectLuca Deri2021-11-25
|
* Rework how hostname/SNI info is saved (#1330)Ivan Nardi2021-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Looking at `struct ndpi_flow_struct` the two bigger fields are `host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and `protos.tls_quic.client_requested_server_name[256]` (for TLS/QUIC SNIs). This commit aims to reduce `struct ndpi_flow_struct` size, according to two simple observations: 1) maximum one of these two fields is used for each flow. So it seems safe to merge them; 2) even if hostnames/SNIs might be very long, in practice they are rarely longer than a fews tens of bytes. So, using a (single) large buffer is a waste of memory for all kinds of flows. If we need to truncate the name, we keep the *last* characters, easing domain matching. Analyzing some real traffic, it seems safe to assume that the vast majority of hostnames/SNIs is shorter than 80 bytes. Hostnames/SNIs are always converted to lowercase. Attention was given so as to be sure that unit-tests outputs are not affected by this change. Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the *first* 64 ones): as a consequence, there were some "Suspicious DGA domain name" and "TLS Certificate Mismatch" false positives.
* Fix writes to `flow->protos` union fields (#1354)Ivan Nardi2021-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | We can write to `flow->protos` only after a proper classification. This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP, SMTP, IMAP and POP code. There are two kinds of fixes: * write to `flow->protos` only if a final protocol has been detected * move protocol state out of `flow->protos` The hard part is to find, for each protocol, the right tradeoff between memory usage and code complexity. Handle Kerberos like DNS: if we find a request, we set the protocol and an extra callback to further parsing the reply. For all the other protocols, move the state out of `flow->protos`. This is an issue only for the FTP/MAIL stuff. Add DHCP Class Identification value to the output of ndpiReader and to the Jason serialization. Extend code coverage of fuzz tests. Close #1343 Close #1342
Powered by cgit, Themed by Ted Yin.