aboutsummaryrefslogtreecommitdiff
path: root/example
Commit message (Collapse)AuthorAge
* Updated JA3/SSL fingerprints.Toni Uhlig2022-07-04
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bins: add support for 64bit bins (#1626)Ivan Nardi2022-07-03
|
* Improved TFTP. Dissect Read/Write Request filenames. (#1617)Toni2022-07-03
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. ↵Toni2022-07-03
| | | | | | | | Fixes #1608. (#1614) * fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization * execute serialization code while running regression tests Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix compilation and sync unit tests results (#1606)Ivan Nardi2022-06-20
|
* Added unidirectional traffic flow riskLuca Deri2022-06-20
|
* Added collectd dissector (again). (#1601)Toni2022-06-17
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Replaced nDPI's internal hashmap with uthash. (#1602)Toni2022-06-17
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Add some statistics to ndpiReader (#1587)claudio_burrafato2022-06-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add some statistics to ndpiReader The purpose of this version of ndpiReader is too adding some other statistics printed by ndpiReader. In this simple version the domain names(in the flows) that are collected are: flow-> ssh_tls.server_info flow-> host_server_name and are placed in a UT_hash_table, ordering them by number of occurrences. * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update example/ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com> * Update ndpiReader.c * Update ndpiReader.c * Update ndpiReader.c * Update ndpiReader.c Co-authored-by: Toni <matzeton@googlemail.com>
* Fix invalid memory access (#1596)Ivan Nardi2022-06-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | We can access `flow->protos` union only after checking the protocol. Checking `flow->detected_protocol.master_protocol` is redundant because we already check it in `is_ndpi_proto` ``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==29739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000353820 (pc 0x7f9b64dd2717 bp 0x7fff161a52f0 sp 0x7fff161a4aa8 T0) ==29739==The signal is caused by a READ memory access. #0 0x7f9b64dd2717 /build/glibc-SzIz7B/glibc-2.31/string/../sysdeps/x86_64/multiarch/strlen-avx2.S:96 #1 0x555c65e597d8 in __interceptor_strlen (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x6407d8) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba) #2 0x555c65fd85fa in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:269:13 #3 0x555c65f3e8c6 in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1188:36 #4 0x555c65f52cab in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1567:2 #5 0x555c65f4b632 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110:10 #6 0x555c65f04d29 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7 #7 0x555c65f054bb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17 #8 0x7f9b64c6e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16 #9 0x555c65e4253d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x62953d) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba) ``` Found by oss-fuzzer. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48020
* Improved DNS traffic analysisLuca Deri2022-06-13
| | | | Added ability to identify application and network protocols
* Added gprof CPU/HEAP profiling support. (#1592)Toni2022-06-12
| | | | | * Some small auto{conf,make} improvements Signed-off-by: lns <matzeton@googlemail.com>
* Removed Makefile references to legacy code. (#1589)Toni2022-06-08
| | | | | - feature removed with f5545a80f95a90a5219bbb319b346f964b1e1548 Signed-off-by: lns <matzeton@googlemail.com>
* Updated README.md (#1562)Toni2022-05-30
| | | | | | | * make check great again (not so much) * make doc/doc-view * CI updates Signed-off-by: lns <matzeton@googlemail.com>
* Added ability to return risk info in JSON format in ndpi_get_flow_risk_info()Luca2022-05-30
|
* Fixed dispay bug for risk_infoLuca Deri2022-05-30
|
* Updated tests resultsLuca Deri2022-05-30
| | | | Code cleanup
* Added RiskInfo stringLuca Deri2022-05-30
|
* Compilation fixes for old gcc compilerLuca Deri2022-05-28
|
* Added script to compare and verify the output of `make dist'. (#1551)Toni2022-05-26
| | | | | | | This fixes some build/test issues resulting when using tarballs. * nDPI uses autotools (especially autoconf) in a wrong way, see #1163 Signed-off-by: lns <matzeton@googlemail.com>
* Removed MacOS XCode integration.Toni Uhlig2022-05-16
| | | | | | | | | * It is outdated (from 2018) and will most likely not work anymore due to changed protocol files and outdated API usage in the example integration * Removal is the only option besides fixing those issues and provide a CI integration for future automatic checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Replaced ndpiReader's libjson-c support with libnDPI's internal ↵Toni2022-05-07
| | | | | | | | | | | serialization interface. (#1535) * Fixes #1528 * Serialization Interface should also fuzzed * libjson-c may only be used in the unit test to verify the internal serialization interface * Serialization Interface supports tlv(broken), csv and json * Unit test does work again and requires libjson-c Signed-off-by: lns <matzeton@googlemail.com>
* Added BPF filtering for discarding non-IP packetsLuca Deri2022-04-27
|
* Added ability to store custom category file in patricia treeLuca Deri2022-04-26
|
* Added generic user agent setter. (#1530)Toni2022-04-25
| | | | | * ndpiReader: Print user agent if one was set and not just for certain protocols. Signed-off-by: lns <matzeton@googlemail.com>
* XIAOMI: add detection of Xiaomi traffic (#1529)Ivan Nardi2022-04-25
| | | Most of the credits should go to @utoni (see #1521)
* Reduce ndpiReader's `-h' spam. `-H' does this job now. (#1523)Toni2022-04-21
| | | Signed-off-by: lns <matzeton@googlemail.com>
* Removed superfluous ifdef'd includes. (#1519)Toni2022-04-19
| | | Signed-off-by: lns <matzeton@googlemail.com>
* DGA improvementsLuca Deri2022-04-16
|
* Fixed msys2 build warnings and re-activated CI Mingw64 build.fix/windows-msys2Toni Uhlig2022-04-14
| | | | | | | * Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com>
* Added ndpi_find_outliers() API call using Z-ScoreLuca Deri2022-04-04
|
* ndpiReader: fix compilation (#1510)Ivan Nardi2022-04-01
| | | | Not sure why Windows started complaining... anyway, the fixes has been taken from https://github.com/ntop/nDPI/pull/1491: credits to @lnslbrty
* Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509Luca Deri2022-04-01
|
* DGA improvementsLuca Deri2022-04-01
|
* Waring fixesLuca Deri2022-04-01
|
* ndpireader: add json output back. (#1509)Vitaliy Ivanov2022-04-01
| | | | | | | | | | | | | | | | - partial revert of: commit 51cfdfb0d80a7bbcc11bc3b95d1696d8dae900c2 Author: Luca Deri <deri@ntop.org> Date: Sun Nov 17 17:51:45 2019 +0100 Removed unused JSON-C code - Json option is changed from 'j' to 'k' as it's used in the new codebase. - use HAVE_LIBJSON_C instead of HAVE_JSON_C. - tabs vs spaces clean ups. Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com> Conflicts: example/ndpiReader.c
* Improved DGA detectionLuca Deri2022-03-31
|
* reader_util: add support for userAgent in SSDP (#1502)Ivan Nardi2022-03-28
| | | | Update unit tests results Follow-up of d668ab4b
* Improved DGA detection skipping names containign at least 3 consecutive ↵Luca Deri2022-03-26
| | | | digits in the first word
* Extend tests coverage (#1476)Ivan Nardi2022-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is at least one flow under `tests/pcap` for 249 protocols out of the 284 ones supported by nDPI. The 35 protocols without any tests are: * P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK, EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET * games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA, FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU * voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE * other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE, LOTUS_NOTES, EGP, SAP Most of these protocols (expecially the P2P and games ones) have been inherited by OpenDPI and have not been updated since then: even if they are still used, the detection rules might be outdated. However code coverage (of `lib/protocols`) only increases from 65.6% to 68.9%. Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP dissection. Treat IPP as a HTTP sub protocol. Fix Cassandra false positives. Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`: these protocol ids are defined but they are never used. Remove Collectd support: its code has never been called. If someone is really interested in this protocol, we can re-add it later, updating the dissector. Add decoding of PPI (Per-Packet Information) data link type.
* Errors fixed (#1482)Vitaly Lavrov2022-03-08
| | | | | | | | | | | | | | | Fixed errors for bigendian platforms in ndpiReader. All address and port comparisons and hash calculations are done with endian in mind. The get_ndpi_flow_info() function searched for an existing flow for the forward and reverse direction of the packet. The ndpi_workflow_node_cmp() function looked for a flow regardless of the packet's direction. This is what led to an error in determining the direction of transmission of the packet. Fixed error in "synscan" test: the number of packets in the forward and reverse direction is incorrectly defined (verified via tcpdump). Fixed bug with icmp protocol checksum check for big endian platforms.
* reader_util: fix parsing of MPLS packets (#1467)Ivan Nardi2022-03-02
| | | | | | | | | | | | | | ``` ==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8 READ of size 4 at 0x60300000578c thread T0 #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16 #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37) 0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d) ```
* Bug fixing. (#1459)Vitaly Lavrov2022-02-28
| | | | | The '--enable-debug-messages' option works again. Fixed warning in ahocorasick.c Fixed integer overflow in ndpiReader.c for 32bit systems.
* reader_util: fix TZSP decapsulation (#1460)Ivan Nardi2022-02-27
| | | | | | | | | | | | | | | | | ``` ==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48 READ of size 1 at 0x60400008957f thread T0 #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18 #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98) 0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f) allocated by thread T0 here: ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
* reader_util: fix parsing of IPv6 extension headers (#1453)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
* reader_util: fix infinite loop in packet dissection (#1454)Ivan Nardi2022-02-22
| | | | Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
* fuzz: purge old sessions (#1451)Ivan Nardi2022-02-21
| | | | | | | | At every fuzz iteration (i.e for every trace file): * keep the same ndpi context (`ndpi_init_detection_module` is very slow); * reset the flow table, otherwise it grows indefinitely. This change should fix the "out-of-memory" errors reported by oss-fuzz.
* Added newflow risk NDPI_HTTP_CRAWLER_BOTLuca Deri2022-02-17
|
* Increment current/total number of active flows on successful flow insertion ↵ol-andreyizrailev2022-02-09
| | | | | | | | (#1434) Memory allocation or ndpi_tsearch might fail, so the two values should be incremented only when insertion actually happened. Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
* Remove `struct ndpi_id_struct` (#1427)Ivan Nardi2022-01-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the last uses of `struct ndpi_id_struct`. That code is not really used and it has not been updated for a very long time: see #1279 for details. Correlation among flows is achieved via LRU caches. This change allows to further reduce memory consumption (see also 91bb77a8). At nDPI 4.0 (more precisly, at a6b10cf, because memory stats were wrong until that commit): ``` nDPI Memory statistics: nDPI Memory (once): 221.15 KB Flow Memory (per flow): 2.94 KB ``` Now: ``` nDPI Memory statistics: nDPI Memory (once): 235.27 KB Flow Memory (per flow): 688 B <-------- ``` i.e. memory usage per flow has been reduced by 77%. Close #1279
Powered by cgit, Themed by Ted Yin.