aboutsummaryrefslogtreecommitdiff
path: root/src/lib/protocols/http.c
Commit message (Collapse)AuthorAge
...
* Add detection of OCSP (#1370)Ivan Nardi2021-11-11
| | | | | | | | | | This protocol is detected via HTTP Content-Type header. Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to classify a HTTP flow according to this header. Since then, this automa has been useless because it is always empty. Re-enable it to match only a string seems overkilling. Remove all `content_automa` leftovers.
* Fix broken fuzz_process_packet fuzzer by adding a call to ↵Toni2021-10-18
| | | | | | | | | | | | ndpi_finalize_initialization(). (#1334) * fixed several memory errors (heap-overflow, unitialized memory, etc) * ability to build fuzz_process_packet with a main() allowing to replay crash data generated with fuzz_process_packet by LLVMs libfuzzer * temporarily disable fuzzing if `tests/do.sh` executed with env FUZZY_TESTING_ENABLED=1 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked flow risks asignmentLuca Deri2021-10-16
| | | | Added esceptions for windows update and binary application transfer risk
* Remove `struct ndpi_packet_struct` from `struct ndpi_flow_struct` (#1319)Ivan Nardi2021-10-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet") in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid saving dissection information of "current packet" into the "flow" state, i.e. in the flow management table. The nDPI detection module processes only one packet at the time, so it is safe to save packet dissection information in `struct ndpi_detection_module_struct`, reusing always the same "packet" instance and saving a huge amount of memory. Bottom line: we need only one copy of "packet" (for detection module), not one for each "flow". It is not clear how/why "packet" ended up in "flow" in the first place. It has been there since the beginning of the GIT history, but in the original OpenDPI code `struct ipoque_packet_struct` was embedded in `struct ipoque_detection_module_struct`, i.e. there was the same exact situation this commit wants to achieve. Most of the changes in this PR are some boilerplate to update something like "flow->packet" into something like "module->packet" throughout the code. Some attention has been paid to update `ndpi_init_packet()` since we need to reset some "packet" fields before starting to process another packet. There has been one important change, though, in ndpi_detection_giveup(). Nothing changed for the applications/users, but this function can't access "packet" anymore. The reason is that this function can be called "asynchronously" with respect to the data processing, i.e in context where there is no valid notion of "current packet"; for example ndpiReader calls it after having processed all the traffic, iterating the entire session table. Mining LRU stuff seems a bit odd (even before this patch): probably we need to rethink it, as a follow-up.
* Remove `detected_protocol_stack` field from `ndpi_packet_struct` (#1317)Ivan Nardi2021-09-29
| | | | | | | | | | | | | This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`: * at the very beginning of packet dissection, the value saved in `flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack` (via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`) * every time we update `flow->detected_protocol_stack` we update `packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()` -> `ndpi_int_change_packet_protocol()`) These two fields are always in sync: keeping the same value in two different places is useless.
* A (tiny) effort to reduce memory usage per flow (#1304)Ivan Nardi2021-09-18
| | | | | | | Follow-up of 22241a1d Only trivial changes: * remove completely unused fields * remove fields only written (but never read) * CSGO protocol only handles UDP traffic
* Added new risk for clear text credentialsLuca Deri2021-09-10
|
* Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)Ivan Nardi2021-08-20
| | | | | | | | | | | | | | | | Fix all the warnings. Getting rid of "-Wno-unused-parameter" is quite complex because some parameters usage depends on compilation variable (i.e. `--enable-debug-messages`). The "-Werror" flag has been added only in Travis builds to avoid breaking the builds to users using uncommon/untested OS/compiler/enviroment. Tested on: * x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12 * x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag) * Raspberry 4; Debian 10.10; gcc 8.3.0
* Skip whitespaces between HTTP method and URL. (#1271)Toni2021-08-08
| | | | | * be less case-restrictive, RFC2616 wants it that way Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)Toni2021-07-31
| | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked flow risk implementationLuca Deri2021-07-23
|
* Code cleanup. (#1246)Vitaly Lavrov2021-07-16
| | | | ndpi_utils.c: use ndpi_malloc,ndpi_calloc,ndpi_free genshin_impact.c, git.c, hpvirtgrp.c, http.c, z3950.c: removed "#include stdlib.h"
* Fix for #1230 (#1235)Vitaly Lavrov2021-07-07
| | | | | | | | | | | | | | | * Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)" This reverts commit 58665e93a98d014b53d131b2481ccab074efc9ff. * Checking the return code after calling ndpi_match_string_subprotocol() ndpi_api.h: Description of the returned error codes for the ndpi_match_string_subprotocol() function. If the ndpi_match_string_subprotocol() function returned an error, then return NDPI_PROTOCOL_UNKNOWN. http: The "Content-type" header is only checked if it is not empty.
* Improved RTSP detection the second. (#1232) (#1233)Toni2021-07-07
| | | | | | | * RTSP is no subprotocol of HTTP (most of the time) * detection patterns should stay in rtsp.c * set detected HTTP protocol only if at least a valid HTTP method detected Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)Toni2021-03-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Try to detect one subprotocol if a detected protocol can have one. * This adds a performance overhead due to much more protocol detection routine calls. See #1148 for more information. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (1/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactor subprotocol handling (2/2). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Refactored aimini HTTP detection parts (somehow related to #1148). Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Added aimini client/server test pcap. Signed-off-by: Toni Uhlig <matzeton@googlemail.com> * Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto * Adjusted Python code to conform to the changes made during the refactoring process. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151)Ivan Nardi2021-03-09
|
* Added Ookla detection over IPv6Luca Deri2021-03-09
|
* Ookla fixesLuca Deri2021-03-09
|
* Improved detection of Ookla speedtest and openspeedtest.comLuca Deri2021-03-09
|
* Improved DGA detection with trigrams. Disadvantage: slower startup timeLuca Deri2021-03-03
| | | | | Reworked Tor dissector embedded in TLS (fixes #1141) Removed false positive on HTTP User-Agent
* HTTP: fix user-agent parsing (#1124)Ivan Nardi2021-02-03
| | | | | | | | | | | User-agent information is used to try to detect the user OS; since the UA is extracted for QUIC traffic too, the "detected_os" field must be generic and not associated to HTTP flows only. Otherwise, you might overwrite some "tls_quic_stun" fields (SNI...) with random data. Strangely enough, the "detected_os" field is never used: it is never logged, or printed, or exported...
* HTTP: fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1123)Ivan Nardi2021-02-03
|
* Increased number of extra packets that is necessary since the frgament ↵Luca Deri2021-02-03
| | | | mananger introduction
* (C) UpdateLuca Deri2021-01-07
|
* HTTP: fix compilation and a memory error when NDPI_ENABLE_DEBUG_MESSAGES is ↵Ivan Nardi2021-01-07
| | | | defined (#1109)
* Split HTTP request from response Content-Type. Request Content-Type should ↵Luca Deri2021-01-06
| | | | be present with POSTs and not with other methods such as GET
* Added check for invalid HTTP contentLuca Deri2021-01-06
|
* Fixed missing symbolLuca Deri2021-01-02
|
* Added HTTP suspicious content securirty risk (useful for tracking trickbot)Luca Deri2021-01-02
|
* Added known protocol on unknown port for ntopLuca Deri2020-12-28
|
* Type change to avoid Windows compilation issuesLuca Deri2020-12-17
|
* Improved HTTP dissectionLuca Deri2020-12-16
|
* Fix for detecting numeric IPsLuca Deri2020-11-01
|
* Fixes #1033Luca Deri2020-10-21
|
* Minor UA handling improvement to avoid heap-overflowLuca Deri2020-09-22
|
* Merge pull request #1012 from IvanNardi/uaLuca Deri2020-09-17
|\ | | | | QUIC: extract User Agent information
| * http: create a common function to parse User Agent fieldNardi Ivan2020-09-08
| | | | | | | | Prepare the code to handle UA information from flows other than HTTP
* | Added extension to detect nested subdomains as used in Browsertunnel attack toolLuca Deri2020-09-09
|/ | | | https://github.com/veggiedefender/browsertunnel
* Improved ntop detection over HTTPLuca Deri2020-08-30
| | | | Added cap on number of attempts for CiscoVPN
* Fixed false positive in suspicous user agentLuca Deri2020-08-30
| | | | Optimized stddev calculation
* Passes method_len param to ndpi_http_str2methodSimone Mainardi2020-08-27
|
* Added ndpi_http_method ndpi_http_str2method(const char* method) API callLuca Deri2020-08-26
|
* Added check on payload lenght during extra packet processingLuca Deri2020-08-04
|
* User agent detection improvementsLuca Deri2020-07-21
|
* Merge pull request #932 from IvanNardi/logLuca Deri2020-07-07
|\ | | | | Log
| * Incorporated some feedbackNardi Ivan2020-07-01
| |
| * Fix compilation with --enable-debug-messages flagNardi Ivan2020-06-26
| | | | | | | | | | | | | | | | | | | | | | | | NDPI_LOG* macros dereference ndpi_detection_module_struct object which is private to ndpi library (via NDPI_LIB_COMPILATION define). So we can't use them outside the library itself, i.e. in ndpiReader code Therefore, in files in example/, convert all (rare) uses of NDPI_LOG* macros to a new very simple macro, private to ndpiReader program. If necessary, such macro may be improved. According to a comment in ndpi_define.h, each dissector must define its own NDPI_CURRENT_PROTO macro before including ndpi_api.h file
* | Improved HTTP line parsing if request splitted into multiple packets.Toni Uhlig2020-07-05
| | | | | | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* | Added malformed packet risk supportLuca Deri2020-06-26
|/
* Added checks for DGA detectionLuca Deri2020-06-17
|
Powered by cgit, Themed by Ted Yin.