aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* TLS: fix parsing of certificate elements (#1910)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | | ``` ==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0) ==1228==The signal is caused by a WRITE memory access. #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79 #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3 #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15 #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6 #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9 #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 #7 0x41f78d in _start ``` Found by oss-fuzz. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
* Add support for vxlan decapsulation (#1441) (#1900)Maatuq2023-03-22
| | | Close #1441
* Rework CI jobs to try reducing CI duration (#1903)Ivan Nardi2023-03-22
| | | | | | | | | | | | | | | | CI duration is quite long: the longest jobs is the "Performance" one. Try to reduce the overall duration: that job (and some others) will not be triggered for each PR/commit anymore, but asynchronously, once a day (this scheduling seems right since the frequency of the PR/commits in the project). It should be possibly to trigger them manually, via GUI, anyway. Remove two identical jobs; we already tests ASAN with 4 different compilers. After 9eff0754 it is safe to reduce fuzzing time. Bottom line: try to have as upper-time of CI tests the duration of the fuzzing jobs
* Added the ability to define custom protocols with arbitrary Ids in proto.txtLuca Deri2023-03-22
| | | | | | | | | | Example - ip:213.75.170.11/32:443@CustomProtocol nDPI assigns an is that can change based on protos.txt content - ip:213.75.170.11/32:443@CustomProtocol=9999 nDPI assigns 9999 as protocolId to CustomProtocol and won't change when protos.txt content will chaneg
* Removed overlapping portLuca Deri2023-03-21
|
* Improved checks for duplicated entries in protocols fileLuca Deri2023-03-21
|
* DisneyPlus/Hulu ip lists should be auto-generated (#1905)Ivan Nardi2023-03-20
| | | | | | Remove two stale ip lists: 1) these 3 ips are in the Amazon ranges (now)... 2) the Instagram list originated from AS32934, which is now a Facebook AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
* Add a new protocol id for generic Adult Content traffic (#1906)Ivan Nardi2023-03-20
| | | | The list has been taken from https://www.similarweb.com/top-websites/adult/ Fix a GoTo false positive.
* Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)Ivan Nardi2023-03-20
|
* fuzz: add a new fuzzer to test TLS certificates (#1901)Ivan Nardi2023-03-20
|
* FreeBSD compilation fixLuca Deri2023-03-14
| | | | (C) update
* fuzz: simplify fuzzers dependencies in CIFuzz (#1896)Ivan Nardi2023-03-14
| | | | | | | | | | | | | | | | | | | | | | CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz targets on pull requests. It only runs the fuzzers affected by a pull request/commit. Otherwise it will divide up the allotted fuzzing time among all fuzzers in the project. Since: * we have more than 20 fuzzers and most of them use the custom memory allocation functions (to force allocation failures) even if they are not strictly about DPI stuff; * we need to keep fuzzing time relatively small (to avoid waiting the CI results for a long time) it is important that fuzzers dependencies (which are based on *files* changed by the single commit/PR) are as small as possible. Bottom line: move all the low-level allocation callbacks to a dedicated file; this way most of the fuzzers don't depend anymore on `ndpi_main.c` file (which is touched by ever commit/PR). The goal is to have only the "most important" fuzzers running during (most of) the CI.
* Added ability to define an unlimited number of custom rules IP:port for the ↵Luca Deri2023-03-13
| | | | same IP (it used tobe limited to 2)
* Minor cleanupLuca Deri2023-03-13
|
* Added fix for better handling exceptions rollback in case of later matchLuca Deri2023-03-06
|
* Add a new flow risk about literal IP addresses used as SNI (#1892)Ivan Nardi2023-03-02
| | | | | | | | | | RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in "HostName"." Don't set this risk if we have a valid sub-classification (example: via certificate) Since a similar risk already exists for HTTP hostnames, reuse it, with a more generic name.
* Fix insert of ip addresses into patricia tree(s) (#1895)Ivan Nardi2023-03-02
| | | Fix: d0b46dcf
* ndpiReader: print how many packets (per flow) were needed to perform full ↵Ivan Nardi2023-03-01
| | | | | | DPI (#1891) Average values are already printed, but this change should ease to identify regressions/improvements.
* Fixed missing AS_HELP_STRING in configure.ac. (#1893)Toni2023-03-01
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sync unit tests resultsNardi Ivan2023-03-01
|
* Keep master protocol in `ndpi_reconcile_protocols`Nardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | As a side effect of this change, the following memory leak has been fixed. ``` ==55523==ERROR: LeakSanitizer: detected memory leaks Direct leak of 2 byte(s) in 1 object(s) allocated from: #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52) #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25 #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13 #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46 #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5 #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
* SoftEther: fix invalid memory accessNardi Ivan2023-03-01
| | | | | | | | | | | | | | | | | | | | We can't write `flow->protos` union until we are really sure about protocol classification ``` ==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0) ==28334==The signal is caused by a READ memory access. ==28334==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7) #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7 #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2 #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5 #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
* Fixed uninitialized variableLuca Deri2023-02-27
|
* Moving to bashLuca Deri2023-02-27
|
* Added missing breakLuca Deri2023-02-27
|
* Updated results after the latest changesLuca Deri2023-02-27
|
* in case of failure, failing result files are not listedLuca Deri2023-02-27
|
* Indent fixLuca Deri2023-02-27
|
* Improved protocol detection exploiting IP-based guessLuca Deri2023-02-27
| | | | Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
* Enhanced custom port definition and improved error reporting in case of ↵Luca Deri2023-02-24
| | | | duplications
* fuzz: extend fuzz coverage (#1888)Ivan Nardi2023-02-16
|
* Update GitHub runners versions (#1889)Ivan Nardi2023-02-14
| | | See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
* Fix export/serialization of `flow->risk` (#1885)Ivan Nardi2023-02-14
| | | | Any risks bigger than 31 was never exported/serialized; found looking a the fuzz coverage report.
* Sync utests results (#1887)Ivan Nardi2023-02-13
|
* Fix packet counters (#1884)Ivan Nardi2023-02-13
| | | | | | We need to keep separete counters to keep tracks of packet numbers with and without any payload. Regression introduced in 5849863ef
* Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)Ivan Nardi2023-02-13
|
* Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)Ivan Nardi2023-02-13
| | | | | | | | | | | | | | | | | | | | | | | Latest RoaringBitmap version (introduced with bf413afb) triggers a new warning with GCC-7: ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s autoreconf: Entering directory `.' [...] third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] static inline int array_container_cardinality(const array_container_t *array) { ^~~~~~ third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes] const array_container_t *container2) { [..] ``` The core issue is that `no_sanitize` attribute is defined only for GCC >= 8. That breaks the CI since we still use GCC-7 and `-Werror`: add a simple workaround. Fix compilation on Windows
* Centos7 fixesLuca Deri2023-02-13
|
* Update roaring bitmap codeLuca Deri2023-02-12
|
* Add Yandex services detection (#1882)0xA50C1A12023-02-09
| | | | | Add Yandex services detection Add VK and Yandex to the TLS certificate match list
* fuzz: some improvements and add two new fuzzers (#1881)Ivan Nardi2023-02-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from `fuzz/Makefile.am`; it is already included by the main configure script (when fuzzing). Add a knob to force disabling of AESNI optimizations: this way we can fuzz also no-aesni crypto code. Move CRC32 algorithm into the library. Add some fake traces to extend fuzzing coverage. Note that these traces are hand-made (via scapy/curl) and must not be used as "proof" that the dissectors are really able to identify this kind of traffic. Some small updates to some dissectors: CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting with "VS01" will be classified as STEAM (see steam.c around line 111). Googling it, it seems right so. XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore that condition is false. RTP, STUN: removed useless "break"s Zattoo: `flow->zattoo_stage` is never set to any values greater or equal to 5, so these checks are never true. PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it. TeamSpeak: we check for `flow->packet_counter == 3` just above, so the following check `flow->packet_counter >= 3` is always false.
* Add VK detection (#1880)0xA50C1A12023-02-02
|
* TyposLuca Deri2023-02-01
|
* Minor updateLuca Deri2023-02-01
|
* Moved to 4.7Luca Deri2023-02-01
|
* Update CHANGELOG.mdAlfredo Cardigliano2023-02-01
|
* Updated elastic_search file name in projectLuca Deri2023-01-31
|
* File rename to avoid name clashes (e.g. on Windows)Luca Deri2023-01-31
|
* Updated (C)Luca Deri2023-01-31
|
* Update changelogAlfredo Cardigliano2023-01-31
|
Powered by cgit, Themed by Ted Yin.