aboutsummaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAge
...
* Exported HTTP server in metadataLuca Deri2022-12-05
|
* Add support for LTO and Gold linker (#1812)Ivan Nardi2022-12-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit add (optional) support for Link-Time-Optimization and Gold linker. This is the first, mandatory step needed to make nDPI compliant with "introspector" sanitizer requirements in OSS-Fuzz: see https://github.com/google/oss-fuzz/issues/8939 Gold linker is not supported by Windows and by macOS, so this feature is disabled by default. It has been enable in CI in two linux targets ("latest" gcc and clang). Fix some warnings triggered by LTO. The changes in `src/lib/ndpi_serializer.c` seams reasonable. However, the change in `tests/unit/unit.c` is due to the following warning, which seems to be a false positive. ``` unit.c: In function ‘serializerUnitTest’: ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here 67 | ndpi_serializer serializer, deserializer; | ^ ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized] unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here 67 | ndpi_serializer serializer, deserializer; ``` Since this warning is triggered only with an old version of gcc and `tests/unit/unit.c` is used only during the tests, the easiest fix has been applied. Some (unknown to me) combinations of OS and compiler trigger the following warnings at linker time (with sanitizer and gold linker) ``` /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm' /usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm' [..] ``` I have not found any references to this kind of message, with the only exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975 which seems to suggest that these messages can be safely ignored. In any case, the compilation results are sound. Fix `clean` target in the Makefile in the `example` directory. In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error (as always, when the gold linker is involved...). It's come out that the culprit was the `tempnam` function: the code has been changed to use `tmpfile` instead. No sure why... :( Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a` anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds as workaround for the "missing dependencies errors" described in https://github.com/google/oss-fuzz/issues/8939
* CI: GitHub is updating Ubuntu runners (#1817)Ivan Nardi2022-12-03
| | | | | | | | | | | GitHub is moving `ubuntu-latest` to `ubuntu-22.04`: update our dependencies. See: https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/ This is the reason of the recent random failures in CI. Update "newest" tested gcc to gcc-12. Fix a memory error introduced in 557bbcfc5a5165c9eb43bbdd78435796239cd3c9
* Make LRU caches ipv6 aware (#1810)Ivan Nardi2022-12-03
| | | | | | | | | | | | | Simplest solution, keeping the existing cache data structure TLS certificate cache is used for DTLS traffic, too. Note that Ookla cache already works with ipv6 flows. TODO: * make the key/hashing more robust (extending the key size?) * update bittorrent cache too. That task is quite difficult because ntopng uses a public function (`ndpi_guess_undetected_protocol()`) intrinsically ipv4 only...
* Updated tests resultsLuca Deri2022-12-03
|
* Fix classification "by-port" of QUIC flows (#1785)Ivan Nardi2022-11-22
| | | | | | | | | nDPI is able to properly classify QUIC flows only if it elaborates the very first packets of the flow. The protocol list in `is_udp_guessable_protocol()` is basically a list of protocols which can be detected from *any* packets in the flow. Rename such function to `is_udp_not_guessable_protocol()`: the name is still quite cryptic, but at least not plainly wrong
* TLS: be sure to always set `ssl_version` field (#1806)Ivan Nardi2022-11-22
| | | Useful with asymmetric traffic with (D)TLS <= 1.2
* Improved Teamspeak3 License/Weblist detection. (#1802)Toni2022-11-15
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improve export/print of L4 protocol information (#1799)Ivan Nardi2022-11-13
| | | Close #1797
* Hangout: avoid useless lookups in the protocol tree (#1755)Ivan Nardi2022-10-25
| | | | | | | We already performed exactly these lookups in the generic code to populate `flow->guessed_protocol_id_by_ip`: use it! This code probably needs a deeper review, since it is basicaly a simple matching on ip + port.
* TLS: improve handling of ALPN(s) (#1784)Ivan Nardi2022-10-25
| | | | | | | | Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is extracted from the CH, the latter from the SH. Add some entries to the known ALPN list. Fix printing of "TLS Supported Versions" field.
* Sync unit tests results (#1783)Ivan Nardi2022-10-22
|
* Enhanced HTTP numeric IP checkLuca Deri2022-10-22
|
* Add Elasticsearch protocol dissector. (#1782)Toni2022-10-21
| | | | | | | * all credits goes to @verzulli Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Munin protocol dissector.Toni Uhlig2022-10-20
| | | | | | * all credits goes to @verzulli Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* TLS: allow sub-classification via ALPNNardi Ivan2022-10-20
| | | | | | | | In some rare cases, it is possible to sub-classify the flow via ALPN matching. This is particularly usefull for asymmetric traffic where the Client Hello doens't have the SNI. For the time being there is only one rule, about ANYDESK.
* Sync unit tests resultsNardi Ivan2022-10-19
|
* Reworked severities adding critical and emergency, thus shifting values downLuca Deri2022-10-19
|
* TLS: explicit ignore client certificate (#1776)Ivan Nardi2022-10-18
| | | | | | | | | | | | | TLS classification usually stops after processing *server* certificates (if any). That means, that *client* certificate, if present, is usually ignored. However in some corner cases (i.e. unidirectional traffic) we might end up processing client certificate and exposing its metadata: the issue is that the application will think that this metadata are about the server and not about the client. So, for the time being, always ignore client certificate processing. As a future work, we might find an efficient way to process and export both certificates.
* Add some popular streaming applicationsNardi Ivan2022-10-17
| | | | | | | A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142; full credits to @ChrisNelson-CyberReef. Close #1716
* Updated protocol dissector serialisation code. (#1767)Toni2022-10-15
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sync utests results (#1774)Ivan Nardi2022-10-14
|
* Added DNS error code mapping number -> stringLuca Deri2022-10-14
|
* LINE_CALL: add detection of LINE voip calls (#1761)Ivan Nardi2022-10-06
| | | | | | | | These flows are classifed as `LINE_CALL`; another option was `RTP/LINE_CALL`. No sure about the best solution... Extend LINE domains list. Remove RTP dead code.
* Updated IRC resultLuca Deri2022-10-05
|
* Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx ↵Luca2022-10-04
| | | | are supported
* Fix compilation and sync utests resultsNardi Ivan2022-10-04
|
* Restore Confidence enum constants to ascending values. (#1760)Toni2022-10-02
| | | | | | | | | | * NDPI_CONFIDENCE_MAX is useless otherwise since it is supposed the be the highest value w/o any gaps * for client applications, it makes sense to use that the numbers directly w/o any additional mapping Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Line app support (#1759)sharonenoch2022-10-01
| | | | | | | | | | | | | | | | | | | * Standard support for LINE app * Added test pcap for LINE app * make check result for LINE app * Make check success as 1kxun has LINE packets * Added the ASN inc file for LINE * Removed extra lines as its effecting make check * Editing the SNI required a new pcap output file for TLS.Line format * Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out Co-authored-by: Sharon Enoch <sharone@amzetta.com>
* Fixed invalid unidirectional traffic alert for unidirectional protocols ↵Luca2022-09-29
| | | | (e.g. sFlow)
* Fix value of `ndpi_protocol->protocol_by_ip`Nardi Ivan2022-09-25
| | | | Fix: a7c2734b
* DNS: change category of DNS flowsNardi Ivan2022-09-25
| | | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any). Follow-up of 83de3e47
* Sizes of LRU caches are now configurableNardi Ivan2022-09-23
| | | | | | | | 0 as size value disable the cache. The diffs in unit tests are due to the fact that some lookups are performed before the first insert: before this change these lookups weren't counted because the cache was not yet initialized, now they are.
* DNS: change category of DNS flowsNardi Ivan2022-09-22
| | | | | DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category, regardless of the subprotocol (if any).
* Extend content match listsNardi Ivan2022-09-22
|
* NAT-PMP: fix metadata extractionNardi Ivan2022-09-21
|
* Improved NATPMP dissection. (#1745)Toni2022-09-21
| | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix CI after nBPF integration (#1746)Ivan Nardi2022-09-21
| | | Add one CI job testing nBPF
* Finalized nBPF support. You can now define custom protocols such asLuca Deri2022-09-21
| | | | | | | | | | (see exaple/protos.txt) nbpf:"host 192.168.1.1 and port 80"@HomeRouter In order to have nBPF support, you need to compile nDPI with it. Just download https://github.com/ntop/PF_RING in the same directory where you have downloaded nDPI and compile PF_RING/userland/nbpf
* Remove classification "by-ip" from protocol stack (#1743)Ivan Nardi2022-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Basically: * "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is NEVER returned in the protocol stack (i.e. `flow->detected_protocol_stack[]`); * if the application is interested into such information, it can access `ndpi_protocol->protocol_by_ip` itself. There are mainly 4 points in the code that set the "classification by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/ `ndpi_detection_giveup()` functions and the HTTP/STUN dissectors. In the unit tests output, a print about `ndpi_protocol->protocol_by_ip` has been added for each flow: the huge diff of this commit is mainly due to that. Strictly speaking, this change is NOT an API/ABI breakage, but there are important differences in the classification results. For examples: * TLS flows without the initial handshake (or without a matching SNI/certificate) are simply classified as `TLS`; * similar for HTTP or QUIC flows; * DNS flows without a matching request domain are simply classified as `DNS`; we don't have `DNS/Google` anymore just because the server is 8.8.8.8 (that was an outrageous behaviour...); * flows previusoly classified only "by-ip" are now classified as `NDPI_PROTOCOL_UNKNOWN`. See #1425 for other examples of why adding the "classification by-ip" in the protocol stack is a bad idea. Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e. `ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for IPv6 flows. Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed. Close #1687
* Add CryNetwork dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add Syncthing dissector.Toni Uhlig2022-09-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add NATPMP dissector. (#1738)Toni2022-09-16
| | | | | | | Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com> Signed-off-by: lns <matzeton@googlemail.com> Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Remove a case of guessed sub-classificationNardi Ivan2022-09-14
| | | | | | | | | | | | | | | | | | This code is triggered only for "unknown" flows with a valid sni/hostname. Why in that case the guessed classification should be something like `DNS/Subprotocol_depending_on_hostname`? Why DNS as master and not HTTP or TLS or QUIC? Furthermore, I have not been able to trigger a positive match from that lookup. I strongly think that if we had a valid subprotocol, we would have a valid master in the first place. In doubt, remove it completely. As a follow up, we should investigate why some dissectors (the HTTP one, at least) set the sni/hostname field without setting a valid protocol, in the first place. This behaviour seems quite suspicious, if not plainly buggy.
* Anydesk: improve detection (#1735)Ivan Nardi2022-09-13
|
* HTTP: remove some wrong codeNardi Ivan2022-09-13
| | | | | | | | | | This code seems wrong or in the wrong place, at least: * "classification by port" and "classification by ip" protocols (i.e "guessed" protocols) should be used to set the protocol stack only after trying all the dissectors, and only by the generic code * there are no reason (for a dissector) to update the "guessed" information using the protocol stack values: it is usually the other way around (see previous point)
* Replace obsolete linux macroAlfredo Cardigliano2022-09-13
|
* Fix `ndpi_do_guess()` (#1731)Ivan Nardi2022-09-12
| | | | | Avoid a double call of `ndpi_guess_host_protocol_id()`. Some code paths work for ipv4/6 both Remove some never used code.
* STUN: several improvementsNardi Ivan2022-09-11
| | | | | | | | | Add detection over TCP and fix detection over IPv6. Rename some variables since Stun dissector is no more "udp-centric". Stun dissector should always classified the flow as `STUN` or `STUN/Something`. Don't touch `flow->guessed_host_protocol_id` field, which should be always be related to "ip-classification" only.
* HTTP: correctly set the classification protocolsNardi Ivan2022-09-06
| | | | | | | | | | | | Classification should always be set via `ndpi_set_detected_protocol()` to be sure to set a correct `confidence` value, too. Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as confidence, is not valid. This code in HTTP dissector likely needs some more thoughts (the classification itself of the attached example doesn't make a lot of sense), but the goal of this commit is only to always have a valid `confidence` value.
Powered by cgit, Themed by Ted Yin.