aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
...
* QUIC: handle retransmissions and overlapping fragments in reassembler ↵Vinicius Silva Nogueira2022-04-07
| | | | | | | | | | | | | | | | | | | | | (#1195) (#1498) * QUIC: handle retransmissions and overlapping fragments in reassembler * Trigger CI * minor fix: parentheses * Changing ndpi_malloc to ndpi_calloc * fix memory leak * quic_reasm_buf calloc to malloc * change order of is_ch_complete && is_reasm_buf_complete call * is_reasm_buf_complete: added handling for case where frame size is not multiple of 8 * add extra check
* Fix JSON-C.aouinizied2022-04-07
|
* Python bindings fix.aouinizied2022-04-07
|
* Added ndpi_find_outliers() API call using Z-ScoreLuca Deri2022-04-04
|
* Added -z flagLuca Deri2022-04-03
|
* ndpiReader: fix compilation (#1510)Ivan Nardi2022-04-01
| | | | Not sure why Windows started complaining... anyway, the fixes has been taken from https://github.com/ntop/nDPI/pull/1491: credits to @lnslbrty
* Removed un-necessary guess in miningLuca Deri2022-04-01
|
* updateLuca Deri2022-04-01
|
* Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509Luca Deri2022-04-01
|
* DGA improvementsLuca Deri2022-04-01
|
* Waring fixesLuca Deri2022-04-01
|
* ndpireader: add json output back. (#1509)Vitaliy Ivanov2022-04-01
| | | | | | | | | | | | | | | | - partial revert of: commit 51cfdfb0d80a7bbcc11bc3b95d1696d8dae900c2 Author: Luca Deri <deri@ntop.org> Date: Sun Nov 17 17:51:45 2019 +0100 Removed unused JSON-C code - Json option is changed from 'j' to 'k' as it's used in the new codebase. - use HAVE_LIBJSON_C instead of HAVE_JSON_C. - tabs vs spaces clean ups. Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com> Conflicts: example/ndpiReader.c
* Improvements for CUSTOM_NDPI_PROTOCOLSLuca Deri2022-04-01
|
* Moved geneated file to a separate folderLuca Deri2022-04-01
|
* Improved twitter detectionLuca Deri2022-04-01
|
* Removed SRV record from suspicious DNS trafficLuca Deri2022-03-31
|
* Improved DGA detectionLuca Deri2022-03-31
|
* [autoconf] Fixed .git submodule detection test. (#1507)Darryl Sokoloski2022-03-31
| | | Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
* Added code for identifiying anomalies with metrics stored in InfluxDBLuca Deri2022-03-30
|
* reader_util: add support for userAgent in SSDP (#1502)Ivan Nardi2022-03-28
| | | | Update unit tests results Follow-up of d668ab4b
* Add support for Pluralsight site (#1503)Ivan Nardi2022-03-27
|
* Fix CI tests results (#1504)Ivan Nardi2022-03-27
| | | CI integration is failing since 856d7d2.
* Reducing the size of the ndpi_detection_module_struct structure. (#1490)Vitaly Lavrov2022-03-27
| | | | | | | | | | | | | | The ndpi_detection_module_struct structure contains 5 arrays "struct ndpi_call_function_struct" size 286*144=41 kB size, which are occupied by a small number of elements. At the moment we have callback_buffer_size = 172, tcp_with_payload=114, tcp_no_payload=8, udp=93, other 8. NDPI_MAX_SUPPORTED_PROTOCOLS = 285. Size of struct ndpi_detection_module_struct is 253136 bytes. Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes. Real use memory size for struct ndpi_call_function_struct is (173+224)*144=57168 bytes.
* [SSDP] Extract HTTP user-agent when available. (#1500)Darryl Sokoloski2022-03-27
| | | | | | [SSDP] Added capture file with UA header. [SSDP] Added pcap test output log file. Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
* Improved DGA detection skipping names containign at least 3 consecutive ↵Luca Deri2022-03-26
| | | | digits in the first word
* QUIC: add support for version 2 draft 01 (#1493)Ivan Nardi2022-03-25
| | | | | | Support for v2-00 has been removed (it has never been used in real networks and it is incompatible with v2-01). Chrome already supports v2-01 in latest versions in Chrome Beta channel.
* Mining: cleanup registration (#1496)Ivan Nardi2022-03-25
| | | | | Use the same pattern of all the other dissectors: one registration and one callback. Spotted by @dsokoloski
* Trying to improve QUIC reassembler (#1195) (#1489)Vinicius Silva Nogueira2022-03-24
| | | | | | | | | | | * handling QUIC out-of-order fragments * minor fix * updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out * quic test: buf_len + last_pos * QUIC: comment update in __reassemble function and minor change is_ch_complete function
* Update Python bindings guide.aouinizied2022-03-22
|
* Fix typo.aouinizied2022-03-22
|
* Add HOWTO Python.aouinizied2022-03-22
|
* Fix python bindings CI.aouinizied2022-03-22
|
* Complete rework of nDPI Python bindings (cffi API, automatic generation, ↵aouinizied2022-03-22
| | | | packaging and CI integration)
* Extended the list of cybersecurity protocolLuca Deri2022-03-21
|
* Bug fixing. (#1487)Vitaly Lavrov2022-03-15
| | | Using the protocol_id instead of its index.
* QUIC: convert logs to standard mechanism (#1485)Ivan Nardi2022-03-15
|
* QUIC: fix dissection of draft-34 (#1484)dev-1Ivan Nardi2022-03-09
| | | | QUIC-34 is probably not used in production, but fixing it is trivial and it doesn't add any noise to the already complex QUIC code.
* Extend tests coverage (#1476)Ivan Nardi2022-03-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now there is at least one flow under `tests/pcap` for 249 protocols out of the 284 ones supported by nDPI. The 35 protocols without any tests are: * P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK, EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET * games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA, FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU * voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE * other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE, LOTUS_NOTES, EGP, SAP Most of these protocols (expecially the P2P and games ones) have been inherited by OpenDPI and have not been updated since then: even if they are still used, the detection rules might be outdated. However code coverage (of `lib/protocols`) only increases from 65.6% to 68.9%. Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP dissection. Treat IPP as a HTTP sub protocol. Fix Cassandra false positives. Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`: these protocol ids are defined but they are never used. Remove Collectd support: its code has never been called. If someone is really interested in this protocol, we can re-add it later, updating the dissector. Add decoding of PPI (Per-Packet Information) data link type.
* Improved ASN/IP update scripts and CI integration. (#1474)Toni2022-03-09
| | | | | * CI will print a warning if ASN/IP addresses changed. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Implement CI on Windows. (#1483)Zied Aouini2022-03-09
| | | | * Switch fail fast to True. * Windows CI.
* Some small fixes (#1481)Ivan Nardi2022-03-08
| | | | | | FTP: if the authentication fails, stop analyzing the flow WSD: call the initialization routine; the dissector code has never been triggered MINING: fix dissection
* Extracting the Azure Origin url from the download link (#1480)sharonenoch2022-03-08
| | | | | | | * Extracting the Azure Origin url from the download link * Response code check to address double quotes Co-authored-by: Sharon Enoch <sharone@amzetta.com>
* Errors fixed (#1482)Vitaly Lavrov2022-03-08
| | | | | | | | | | | | | | | Fixed errors for bigendian platforms in ndpiReader. All address and port comparisons and hash calculations are done with endian in mind. The get_ndpi_flow_info() function searched for an existing flow for the forward and reverse direction of the packet. The ndpi_workflow_node_cmp() function looked for a flow regardless of the packet's direction. This is what led to an error in determining the direction of transmission of the packet. Fixed error in "synscan" test: the number of packets in the forward and reverse direction is incorrectly defined (verified via tcpdump). Fixed bug with icmp protocol checksum check for big endian platforms.
* EthernetIP: fix integer conversion on big-endian archs (#1477)Ivan Nardi2022-03-05
|
* Fixed a bug for BE architectures (#1478)Vitaly Lavrov2022-03-05
| | | Fixed a bug in the internal implementation of libgcrypt for bigendian architectures
* configure: fix usage of libgpg-error with `--with-local-libgcrypt` (#1472)Ivan Nardi2022-03-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now, using external libgcrypt, nDPI is not linked to libgpg-error because configure script never checks for it. ``` ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-11 CXX=g++-11 CFLAGS="-O3 -g -Werror" ./autogen.sh --enable-debug-messages --with-pcre --with-local-libgcrypt && make -s -j [...] checking for numa_available in -lnuma... yes checking for pcap_open_live in -lpcap... yes checking for pthread_setaffinity_np in -lpthread... yes checking for gcry_cipher_checktag in -lgcrypt... yes <------- missing check for libgpg-error checking for pcre_compile in -lpcre... yes checking that generated files are newer than configure... done [...] ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ grep HAVE_LIBGPG_ERROR src/include/ndpi_config.h /* #undef HAVE_LIBGPG_ERROR */ ``` Make both libgcrypt and libgpg-error mandatory if `--with-local-libgcrypt` is used. Technically speaking, libgpg-error might be optional, because it is used only for debug messages. However having both libraries mandatory slightly simplified the logic. In most environments, libgpg-error is a dependency of libgcrypt anyway, so having both libraries should be the standard case.
* Added autoconf option `--enable-tls-sigs'. (#1471)Toni2022-03-03
| | | | | | * Testing more code in CI environments. * Added strict option checking for `./configure' in CI environments. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Drop support for non-gcrypt builds. (#1469)Toni2022-03-02
| | | | | | | | * As there is now a builtin, lightweight libgcrypt there is no need to disable tls-clho decryption. * It is still possible to use a host libgcrypt with `--with-local-libgcrypt'. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Internal crypto: increase size of authentication buffer (#1468)Ivan Nardi2022-03-02
| | | | | | | Some QUIC flows are not properly decoded while using internal crypto code: the authentication buffer is too small. The new value (like the old one) is arbitrary. Close #1463
* reader_util: fix parsing of MPLS packets (#1467)Ivan Nardi2022-03-02
| | | | | | | | | | | | | | ``` ==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8 READ of size 4 at 0x60300000578c thread T0 #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16 #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37) 0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d) ```
Powered by cgit, Themed by Ted Yin.