aboutsummaryrefslogtreecommitdiff
path: root/test
Commit message (Collapse)AuthorAge
...
* Reworked nDPIsrvd.h C-API.Toni Uhlig2022-03-21
| | | | | | | | | | | | * nDPIsrvd.h: Provide nDPId thread storage. * nDPIsrvd.py: Fixed instance cleanup bug. * nDPIsrvd.h: Support for instance/thread user data and cleanup callback. * nDPIsrvd.h: Most recent flow time stored in thread ht instead of instance ht. * nDPId: Moved flow logger out the memory profilier into SIGUSR1 signal handling. * nDPId: Added signal fd to be usable within epoll's event handling (live-capture only!) * nDPId: Added information about ZLib compressions to daemon status/shutdown events. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Renamed basic event to error event for the sake of the logic.Toni Uhlig2022-03-13
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disconnect nDPIsrvd clients immediately instead waiting for a failed write().Toni Uhlig2022-03-13
| | | | | | | | | * nDPIsrvd: Collector/Distributor logging improved * nDPIsrvd: Command line option for max remote descriptors * nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again * nDPId: Refactored some variable names and events Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added daemon event: DAEMON_EVENT_STATUS (periodically send's daemon statistics.)Toni Uhlig2022-03-06
| | | | | | | | | | * Improved distributor timeout handling (per-thread). * flow-info.py / flow-dash.py: Distinguish between flow risk severities. * nDPId: Skip tag switch datalink packet dissection / processing. * nDPId: Fixed incorrect value for current active flows. * Improved JSON schema's. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Serialize flow risk score / confidence.Toni Uhlig2022-02-27
| | | | | | | * bump libnDPI to 8b062295cc76a60e3905c054ce37bd17669464d1 * removed ndpi_id_struct's Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved py-flow-info to provide more optional information about received ↵Toni Uhlig2022-02-05
| | | | | | | | | | timestamps. * py-flow-dashboard: Added color mapping for PieCharts/Graph that make more sense * nDPId: Renamed `flow_type' to a more precisely `flow_state' * nDPId: Changed the default setting to process only as much packets as libnDPI does Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added additional (minimalistic) detection information to flow updates.Toni Uhlig2022-02-04
| | | | | | | | | | | | | This will only affect flows with the state `FT_FINISHED' (detection done). * nDPIsrvd.py: force use of JSON schema Draft 7 validator * flow-dash.py: gather/use total processed layer4 payload size * flow-info.py: added additional event filter * flow-info.py: prettified flow events printing whose detection is in progress * py-semantic-validation.py: added validation checks for FT_FINISHED * updated flow event JSON schema Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added logging interface used by nDPId, nDPIsrvd and nDPId-test.Toni Uhlig2022-02-03
| | | | | | | | | * fixed GitLab pipeline * nDPId: added static assert (just for a test) * nDPId: memory profiling for total bytes compressed * nDPId-test: enable zLib compression if configured with ENABLE_ZLIB Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Process additional layer 3 protocols.Toni Uhlig2022-01-31
| | | | | | | | | * bump libnDPI to c53c82d4823b5a8f856d1375155ac5112b68e8af * run_tests.sh: improved execution from non-git directories e.g. via `make dist` * updated JSON schema to be more restrictive * nDPId: splitted generic get_ip_from_sockaddr into IPv4/IPv6 to prevent compiler warnings on some platforms Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Added some error messages when workflow init fails.Toni Uhlig2022-01-26
| | | | | | | * Fixed invalid array subscript typ0 (caused some trouble..) * bump libnDPI to 2cd0479204301c50c6149706fcd4df3058b2a8cc Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved Plotly/Dash example. It is now somehow informative.Toni Uhlig2022-01-25
| | | | | | | * TCP timeout after FIN/RST: switched back to the value from a35fc1d5ea8570609cc0c8cf6edadc81f8f5bb76 * py-flow-info: reset 'guessed' flag after detection/detection-update received Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Major nDPId extension. Sorry for the huge commit.Toni Uhlig2022-01-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - nDPId: fixed invalid IP4/IP6 tuple compare - nDPIsrvd: fixed caching issue (finally) - added tiny c example (can be used to check flow manager sanity) - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow` - README.md update: added example JSON sequence - nDPId: added new flow event `update` necessary for correct timeout handling (and other future use-cases) - nDPIsrvd.h and nDPIsrvd.py: switched to an instance (consists of an alias/source tuple) based flow manager - every flow related event **must** now serialize `alias`, `source`, `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout handling and verification process work correctly - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation - nDPIsrvd.py: removed PcapPacket class (unused) - py-flow-dashboard and py-flow-multiprocess: fixed race condition - py-flow-info: print statusbar with probably useful information - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`) to a generic flow event timestamp `ts_msec` - nDPId-test: added additional checks - nDPId: increased ICMP flow timeout - nDPId: using event based i/o if capturing packets from a device - nDPIsrvd: fixed memory leak on shutdown if remote descriptors were still connected Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed invalid "flow_last_seen" timestamp for the first packet.Toni Uhlig2021-10-08
| | | | | | | | | | * After the first packet was processed, "flow_last_seen" was still 0. This behaviour is invalid as the first packet may contain l4 payload data e.g. for UDP and it also breaks nDPId json consistency "flow_first_seen" > 0, but "flow_last_seen" == 0. * JSON schema: set minimum timestamp value for Epoch timestamps to 24710 for flow_*_seen and 1 for pcap packet ts. Those values are dependant on some manipulated pcap's in libnDPI/tests/pcap. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bump libnDPI to 181a03c5ad41bda533fbfa307627939c2ff30b75Toni Uhlig2021-10-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* dumb fuzzer: randpkt vs nDPId-testToni Uhlig2021-09-20
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* CI extended and fixupsToni Uhlig2021-09-19
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed some typ0's and reduced ICMP timeout to 10s.Toni Uhlig2021-09-16
| | | | | | | * nDPId: Renamed some of the misleading terms, still TODO for nDPIsrvd * CMake improvments Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd: Fixed anther bug, introduced during refactoring -_-Toni Uhlig2021-09-15
| | | | | | nDPId-test: Collect information about JSON string length's. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bump libnDPI to 0eb7a0388c4549ebbf8cd7a10d398088005cc2deToni Uhlig2021-09-14
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bump libnDPI to 6b7e5fa8d251f11c1bae16ea892a43a92b098480Toni Uhlig2021-08-03
| | | | | | | | * fixed linking issue by using CMake to check if explicit link against libm required * make nDPIsrvd collectd exit if parent pid changed, meaning that collectd died somehow * nDPId-test restores SIGPIPE to the default handler (termination), so abnormal connection drop's do now have consequences Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bump libnDPI to b95bd0358fd43d9fdfdc5266e3c8923b91e1d4dbToni Uhlig2021-07-14
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* bump libnDPI to ced6fca184a4549333c2d582e53419f66cd99ec1Toni Uhlig2021-06-29
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated CMake nDPId-test target;Toni Uhlig2021-06-18
| | | | | | | * w/o zLib * gcrypt requires to be enabled Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Enable memory profiling for nDPId-test.Toni Uhlig2021-06-18
| | | | | | * print a summary Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* libndpi updateToni Uhlig2021-06-09
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* `make dist`Toni Uhlig2021-06-08
| | | | | | | * fixed run_tests.sh file check bug, CI compat * updated results due to libnDPI submodule update Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Git tag/commit version printing for nDPId/nDPIsrvd. Reduces confusion.Toni Uhlig2021-06-08
| | | | | | * disabled subshell spawn for run_tests.sh, common pitfall while using counters Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked layer 4 flow length naming/calculation.Toni Uhlig2021-05-20
| | | | | | | * nDPIsrvd services usually do not care about layer4 data length, payload length is quite more essential for further processing Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed CMake global CFLAGS misuse which can cause xcompile errors.Toni Uhlig2021-05-19
| | | | | | | | | | nDPIsrvd-captured supports skipping flows w/o any layer 4 payload. * libndpi update * run_tests does not generate any *.out files for fuzz-*.pcap anymore and does not fail if nDPId-test exits with value 1 (most likely caused by a libpcap failure) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Use layer4 specific flow timeouts.Toni Uhlig2021-05-13
| | | | | | * default values "stolen" from nf_conntrack Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added missing datalink types.Toni Uhlig2021-05-12
| | | | | | * basicially C&P from nDPI reader_utils but with some more sanity checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Git submodule libnDPI update.Toni Uhlig2021-05-12
| | | | | | * enable ctest to run integration tests (**only** if BUILD_NDPI=ON) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* README.md updatedToni Uhlig2021-05-10
| | | | | | | * removed all those outdated Makefile.old references * added additional CMake build instructions Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Unified and improved dependency build/mgmt via CMake and travis-ci.Toni Uhlig2021-04-22
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* travis-ci build CMake projectToni Uhlig2021-04-15
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId semantic validation test.Toni Uhlig2021-04-14
| | | | | | | | | | | * fixed inconsistent processing of remaining flows during nDPId shutdown phase * fixed multiple `detected' flow events (instead only `detection-update' flow events can occur after a `detected' flow event) * fixed nDPIsrvd.py invalid message buffer handling * improved run_tests.sh so only valid pcap capture files are getting processed (and some more cosmetics + logging) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added IPv6 support for -I / -E.Toni Uhlig2021-04-11
| | | | | | | * added another Python search path and try-catch ModuleNotFoundError again * run_tests.sh checks for OpenBSD netcat (required for -q) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: mimic full nDPId lifecycleToni Uhlig2021-04-09
| | | | | | | * generate DAEMON_EVENT_INIT as well as DAEMON_EVENT_SHUTDOWN * process remaining flows before shutdown (and generate events) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JSON schema validation to run_tests.shToni Uhlig2021-04-09
| | | | | | | * Python3 scripts are now compatible with versions <3.6 * improved and prettified run_tests.sh Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: added JSON distribution + JSON parsing (Multithreaded design ↵Toni Uhlig2021-04-09
| | | | | | | | | | | | | | re-using most of nDPId/nDPIsrvd core) * improved Makefile.old install targets * splitted nDPIsrvd_parse into nDPIsrvd_parse_line and nDPIsrvd_parse_all for the sake of readability * minor Python script improvments (check for nDPIsrvd.py on multiple locations, may be superseeded by setuptools in the future) * some paths needs to be absolute (chdir() during daemonize) and therefor additional checks introduced * test run script checks and fails if certain files are are missing (PCAP file <=> result output file) * removed not very useful "internal format error" JSON serialization if a BUG for same exists * fixed invalid l4 type statistics counters for nDPIsrvd-collectd Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* run_tests.sh checks/print-stderr-on-failToni Uhlig2021-03-24
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated test outputs and pinned travis-ci's nDPI git repo sync to a specific ↵Toni Uhlig2021-03-24
| | | | | | | | commit hash. * fixed Makefile.old COpts hell Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added diff'able nDPId-test JSON dumps and travis-ci integration.Toni Uhlig2021-03-24
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId-test as all-in-one JSON dumper.Toni Uhlig2021-03-24
* fixed invalid flow event schema type * added run_tests.sh to generate/diff JSON dumps * renamed lot's of vars/fns in nDPId.c/nDPIsrvd.c, so nDPId-test.c can include "*.c" * improved CMake dependency checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.