summaryrefslogtreecommitdiff
path: root/nDPId.c
Commit message (Collapse)AuthorAge
* Unified IO buffer mgmt.Toni Uhlig2021-06-07
| | | | | | | * c-collectd gives the user control over collectd-exec instance name * added missing collectd type `flow_l4_icmp_count` Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked layer 4 flow length naming/calculation.Toni Uhlig2021-05-20
| | | | | | | * nDPIsrvd services usually do not care about layer4 data length, payload length is quite more essential for further processing Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Cosmetic fixes.Toni Uhlig2021-05-13
| | | | | | * daemon.sh script to simplify daemon testing Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Use layer4 specific flow timeouts.Toni Uhlig2021-05-13
| | | | | | * default values "stolen" from nf_conntrack Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added missing datalink types.Toni Uhlig2021-05-12
| | | | | | * basicially C&P from nDPI reader_utils but with some more sanity checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId semantic validation test.Toni Uhlig2021-04-14
| | | | | | | | | | | * fixed inconsistent processing of remaining flows during nDPId shutdown phase * fixed multiple `detected' flow events (instead only `detection-update' flow events can occur after a `detected' flow event) * fixed nDPIsrvd.py invalid message buffer handling * improved run_tests.sh so only valid pcap capture files are getting processed (and some more cosmetics + logging) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added IPv6 support for -I / -E.Toni Uhlig2021-04-11
| | | | | | | * added another Python search path and try-catch ModuleNotFoundError again * run_tests.sh checks for OpenBSD netcat (required for -q) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: mimic full nDPId lifecycleToni Uhlig2021-04-09
| | | | | | | * generate DAEMON_EVENT_INIT as well as DAEMON_EVENT_SHUTDOWN * process remaining flows before shutdown (and generate events) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: added JSON distribution + JSON parsing (Multithreaded design ↵Toni Uhlig2021-04-09
| | | | | | | | | | | | | | re-using most of nDPId/nDPIsrvd core) * improved Makefile.old install targets * splitted nDPIsrvd_parse into nDPIsrvd_parse_line and nDPIsrvd_parse_all for the sake of readability * minor Python script improvments (check for nDPIsrvd.py on multiple locations, may be superseeded by setuptools in the future) * some paths needs to be absolute (chdir() during daemonize) and therefor additional checks introduced * test run script checks and fails if certain files are are missing (PCAP file <=> result output file) * removed not very useful "internal format error" JSON serialization if a BUG for same exists * fixed invalid l4 type statistics counters for nDPIsrvd-collectd Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reverted internal ndpi structs to dynamic memory allocation hoping that it ↵Toni Uhlig2021-03-26
| | | | | | | | | | will help to reduce the average memory consumption. How? After the detection finished, internal ndpi structs can be free'd as they are not needed anymore. * Set the amount of max. packets to process via subopt. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId-test as all-in-one JSON dumper.Toni Uhlig2021-03-24
| | | | | | | | | * fixed invalid flow event schema type * added run_tests.sh to generate/diff JSON dumps * renamed lot's of vars/fns in nDPId.c/nDPIsrvd.c, so nDPId-test.c can include "*.c" * improved CMake dependency checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JA3 / SSL SHA1 fingerprint blacklists.Toni Uhlig2021-02-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Finalized examples/c-captured to dump packet bytes to PCAP for further analysis.Toni Uhlig2021-02-24
| | | | | | * Fixed memory holes in nDPId structs. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Extended nDPIsrvd.h with address parsing.Toni Uhlig2021-02-23
| | | | | | | | | * nDPId supports looading of custom nDPI protocol/category files * extended JSON schemas according to nDPI / nDPId JSON serializing * removed memory holes in nDPId * extended examples/c-captured Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd C API overhaul and massive simplification.Toni Uhlig2021-02-06
| | | | | | | | * nDPIsrvd.h does flow mgmt out of the box * dissect received JSON strings via callback * added new JSON key/values for packet-flows (usecTimestamp/L3/L4 info) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed braindead failure - Increase of NETWORK_BUFFER_MAX_SIZE > 9999 means ↵Toni Uhlig2021-01-27
| | | | | | | | | to also increase NETWORK_BUFFER_LENGTH_DIGITS to 5. * Fixed ARM32 xcompile warnings; Other GCC versions, other uint64_t's.. * Replaced ridiculous nDPIsrvd_JSON_BYTES with NETWORK_BUFFER_LENGTH_DIGITS. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased JSON buffer size to 12288 (libnDPI serializes more and more ↵Toni Uhlig2021-01-27
| | | | | | | | | | | information). * Making Compare&Fetch mandatory. * Added some more Compare&Fetch to prevent TSAN complaining about data races. Fixed possible but more ore less harmless data races during shutdown process. * Shrink SIGNAL handler to a minimum. SYSV Signal handling and MT-safety is awkward. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* All colors are beautiful for py-flow-info.Toni Uhlig2021-01-26
| | | | | | * instance alias is now mandatory (will be required for future use-cases) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Switched to a slightly modified base64 encoder basicially from wikibooks.Toni Uhlig2021-01-20
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added internal/external packet processing modes. #2Toni Uhlig2021-01-10
| | | | | | * fixed incorrect handling of skipped flows Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added internal/external packet processing modes.Toni Uhlig2021-01-06
| | | | | | | | * only IPv4 supported for now * refactored nDPId's internal IP address storage * use fresh ndpi_free_flow_data() to free nDPI's dynamic allocated data Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Declared nDPI id and flow structs in nDPId flow info struct.Toni Uhlig2020-12-30
| | | | | | | | Two reasons: * reduce heap memory allocations * nDPId flow info struct may be inflated in the future (more bytes to compress) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Set an optional bpf filter string for pcap based packet capture.Toni Uhlig2020-12-29
| | | | | | * serialize and send datalink layer type Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd: Distributor clients which are too slow can cause buffer bloat.Toni Uhlig2020-12-19
| | | | | | | | | Switching back to blocking mode works as a quick fix but is not sufficient. See comments. * nDPId prints more accurate error messages if command line argument validation failed Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId option validation is now less restrictiveToni Uhlig2020-12-14
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Let travis use pcap files from nDPI to produce some JSON output. (disabled, ↵Toni Uhlig2020-12-01
| | | | | | | | | | | needs further testing..) * Added pcap diff script * Added \n to JSON string end (useful for debugging and readability) * Use first host/server name character for hash calculation as well * Removed error'ing EPOLLHUP handling in nDPIsrvd (connection closing will be detected via read()) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Moved datalink layer processing into a separate function.Toni Uhlig2020-11-18
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* More detailed `nanosleep() before close()' description.Toni Uhlig2020-11-11
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved default capture device handling if none set via `-i'.Toni Uhlig2020-11-08
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Make sure that DAEMON_EVENT_SHUTDOWN gets transmitted before close() (hacky, ↵Toni Uhlig2020-11-08
| | | | | | but works). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId/nDPIsrvd: change_user_group does now chown/chmod collector/distributor ↵Toni Uhlig2020-10-07
| | | | | | socket paths Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Fixed BUG if flow was guessed but "Unknown" detected protocol serialized.Toni Uhlig2020-10-01
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added additional datalink header check for DLT_NULL/IPv6.Toni Uhlig2020-09-28
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added alias support for an nDPId instance.Toni Uhlig2020-09-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added daemon events.Toni Uhlig2020-09-27
| | | | | | * Added missing IPv4 / IPv6 datalink layer (if no layer2 protocol used). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added host_server_name length to hash to send a detection update if length ↵Toni Uhlig2020-09-26
| | | | | | changed (hacky). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added new flow event: FLOW_EVENT_DETECTION_UPDATEToni Uhlig2020-09-25
| | | | | | | * This event will be triggered when nDPI detection has some new information for us (hopefully). * Detection change is based on hashing with 32-bit murmur3 certain members of the ndpi flow struct. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Process extra packets with nDPI, still not perfect but results in a more ↵Toni Uhlig2020-09-24
| | | | | | | | | accurate detection. * set default user used for setuid() * added 2 TODOs Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Change user/group after init.Toni Uhlig2020-09-21
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Print compiled-in libgcrypt version (if libndpi was compiled with ↵Toni Uhlig2020-08-31
| | | | | | libgcrypt support). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Improved command line option parsing, app usage and subopts for ↵Toni Uhlig2020-08-26
| | | | | | (carefully) tuning some daemon options. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: subopt parsing skeletonToni Uhlig2020-08-19
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* added pkt_type and pkt_ipoffset to json serializationToni Uhlig2020-08-19
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* minor improvments regarding flow guessing on flow end/idle and other not ↵Toni Uhlig2020-08-16
| | | | | | worth to mention Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* error handling enhancementsToni Uhlig2020-08-15
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* several fixes and improvmentsToni Uhlig2020-08-14
| | | | | | | | - set errno to 0 if it is checked right after a libc call - ignore SIGPIPE as we want to avoid signal handling where possible - fixed another issue in nDPIsrvd/c-json-stdout which caused buffering errors Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: removed unused code, process remaining flows on shutdown (useful for ↵Toni Uhlig2020-08-14
| | | | | | replaying pcap files) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* set detection_completed = 1 if guessed/not-detected event thrownToni Uhlig2020-08-13
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* improved TCP-FIN/TCP-RST and TCP-keepalive/-idle timeout handlingToni Uhlig2020-08-13
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* replaced deprecated pcap_lookupdev with pcap_findalldevsToni Uhlig2020-08-12
|
Powered by cgit, Themed by Ted Yin.