summaryrefslogtreecommitdiff
path: root/nDPId-test.c
Commit message (Collapse)AuthorAge
* Fixed more SonarCloud complaints.Toni Uhlig2023-11-07
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed some SonarCloud complaints.Toni Uhlig2023-11-06
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Apple/BSD port (#30)Toni2023-11-06
| | | | | | | | * Add MacOS to Github CI builds. * Fixed libnDPI-4.8 CI build. * Fixed missing include for `struct sockaddr*`. * Reworked IPv4 address and netmask retrieval. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Event I/O abstraction layer. (#28)Toni2023-11-06
| | | | | | | | | | | * Finalize Event I/O abstraction layer. * Fix possible fd leakage, Gitlab-CI build and error logging. * Fixed possible uninitialized signalfd variable. * Fixed possible memory leak. * Fixed some SonarCloud complaints. * Fixed nDPId-test nDPIsrvd-arpa-mockup stuck indefinitely. * Add nDPId / nDPIsrvd command line option to use poll() on Linux instead of the default epoll(). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sonarcloud integrationToni Uhlig2023-10-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fix issues detected by SonarCloud.Toni Uhlig2023-10-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: ingore event handler failures caused by arpa mockupToni Uhlig2023-10-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId event I/O fixes.Toni Uhlig2023-10-05
| | | | | | | | * forcibly disable epoll even if available * nDPId-test event I/O selftest * CI event I/O tests Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add event I/O abstraction.Toni Uhlig2023-09-10
| | | | | | | * required to support non-Linux OS e.g. Mac OS X / BSD * see Github issue #19 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed invalid base64 encoding in some rare cases.Toni Uhlig2023-08-24
| | | | | | * nDPId-test may also verify the correct encoding/decoding Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: threads should block all unix signalsToni Uhlig2023-08-02
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: print additional startup/init log messagesToni Uhlig2023-08-02
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Correctly handle EINTR while doing I/O..Toni Uhlig2023-07-18
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: Fixed invalid error retval when epoll_wait() returns EINTR.Toni Uhlig2023-07-16
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: Reworked I/O handling to prevent some endless loop scenarios. ↵Toni Uhlig2022-12-02
| | | | | | | | Fixed a race condition in the memory wrapper as well. * nDPId: Instead of sending too long JSON strings, log an error and some parts. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Unified nDPId/nDPIsrvd command line argument storage.Toni Uhlig2022-11-21
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved the point of time when to append the raw packet base64 data to the ↵Toni Uhlig2022-11-15
| | | | | | | | | serializer. * nDPId-test: Increased the max-packets-per-flow-to-send from 3 to 5. This is quite useful for TCP as the first 3 packets are usually part of the three-way-handshake. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased network buffer size to 33792 bytes.Toni Uhlig2022-10-30
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPid-test: add buffer testToni Uhlig2022-10-30
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: Fixed thread sync/lock issue.Toni Uhlig2022-10-22
| | | | | | * rarely happens in CI Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* c-analysed: improved feature extraction from "analyse" eventsToni Uhlig2022-10-01
| | | | | | | | * c-captured: update detected risks on "detection-update" events * c-collectd: added missing flow breed * c-collectd: PUTVAL macros are more flexible now Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPIsrvd-analysed to generate CSV files from analyse events.Toni Uhlig2022-09-30
| | | | | | | * nDPIsrvd.h: iterate over JSON arrays * nDPId: calculate l3 payload packet entropies for analysis Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId-test: Force collector blocking mode.Toni Uhlig2022-09-25
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Refactored client distributor C API.Toni Uhlig2022-09-25
| | | | | | | | | | | * Still not perfect, but the code before was not even able to deal with JSON arrays. Use common "speaking" function names for all functions in nDPIsrvd.h * Provide a more or less generic and easy extendable JSON walk function. * Modified C examples to align with the changed C API. * c-collectd: Reduced lot's of code duplication by providing mapping tables. * nDPId: IAT array requires one slot less (first packet has always an IAT of 0). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd.h: Fixed bug during token parsing/hashing. Do not hash array contents.Toni Uhlig2022-09-23
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved flown analyse event:Toni Uhlig2022-09-22
| | | | | | | | * store packet directions * merged direction based IATs * merged direction based PKTLENs Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Reduced flow-updates for TCP flows to 1/4 of the timeout value.Toni Uhlig2022-09-19
| | | | | | | | | | | | * nDPId: Fixed broken validation tests. * nDPId: Removed TICK_RESOLUTION, not required anymore. * c-collectd: Improved total layer4 payload calculation/update handling. * c-collectd: Updated RRD Graph script according to total layer4 payload changes. * py-flow-info.py: Fixed several bugs and syntax errors. * Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py. * nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Provide thread sync via locking on architectures that do not support ↵Toni Uhlig2022-09-17
| | | | | | Compare&Swap. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add OpenWrt CI via Github Actions.Toni Uhlig2022-09-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* New flow event: 'analysis'.Toni Uhlig2022-09-13
| | | | | | | | | * The goal was to provide a separate event for extracted feature that are not required and only useful for a few (e.g. someone who wants do ML). * Increased network buffer size to 32kB (8192 * 4). * Switched timestamp precision from ms to us for *ALL* timestamps. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed compilation warnings on linux32 platforms.Toni Uhlig2022-09-10
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Bump libnDPI to 37f918322c0a489b5143a987c8f1a44a6f78a6f3 and updated flow ↵Toni Uhlig2022-09-06
| | | | | | | | | json schema file. * export env vars AR / CMAKE_C_COMPILER_AR and RANLIB / CMAKE_C_COMPILER_RANLIB while building libnDPI * nDPId check API version during startup (macro vs. function call) and print a warning if they are different Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId: Fixed fcntl() issue; invalid fcntl() set after a blocking-write.lns2022-08-29
| | | | | | | * nDPId: imrpvoed collector socket error messages on connect/write/etc failures * reverted `netcat` parts of the README Signed-off-by: lns <matzeton@googlemail.com>
* Split `*_l4_payload_len' into `*_src_l4_payload_len' and `*_dst_l4_payload_len'.Toni Uhlig2022-08-15
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Split `flow_packets_processed' into `flow_src_packets_processed' and ↵lns2022-08-15
| | | | | | | | | `flow_dst_packets_processed'. * no use for `flow_avg_l4_payload_len' -> removed * test/run_tests.sh does not fail if git-worktree's are used Signed-off-by: lns <matzeton@googlemail.com>
* bump libnDPI to 8f6a006e36eef0ae386f7e663d3ebecfad6a2dc9Toni Uhlig2022-07-01
| | | | | | | * try to use same wording wherever possible e.g. renamed workflow->total_l4_data_len to workflow->total_l4_payload_len Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added proper DLT_RAW dissection for IPv4 and IPv6.lns2022-04-24
| | | | | | | | | | * nDPId: Improved TCP timeout handling if FIN/RST seen which caused Midstream TCP flows when there shouldn't be any. * nDPIsrvd: Unified remote descriptor resource cleanup on disconnects/shutdown. * nDPIsrvd: Added additional error messages for remote descriptors. * py-flow-info: Better daemon status message printing. Signed-off-by: lns <matzeton@googlemail.com>
* nDPIsrvd.(h|py): Added socket read/recv timeout.lns2022-04-17
| | | | | | * nDPIsrvd.h: support for O_NONBLOCK nDPIsrvd_socket Signed-off-by: lns <matzeton@googlemail.com>
* Refactored buffer subsystem.lns2022-04-16
| | | | Signed-off-by: lns <matzeton@googlemail.com>
* Executing ./tests/run_tests.sh w/o zLib should not result in diff's anymore.Toni Uhlig2022-03-24
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reworked nDPIsrvd.h C-API.Toni Uhlig2022-03-21
| | | | | | | | | | | | * nDPIsrvd.h: Provide nDPId thread storage. * nDPIsrvd.py: Fixed instance cleanup bug. * nDPIsrvd.h: Support for instance/thread user data and cleanup callback. * nDPIsrvd.h: Most recent flow time stored in thread ht instead of instance ht. * nDPId: Moved flow logger out the memory profilier into SIGUSR1 signal handling. * nDPId: Added signal fd to be usable within epoll's event handling (live-capture only!) * nDPId: Added information about ZLib compressions to daemon status/shutdown events. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disconnect nDPIsrvd clients immediately instead waiting for a failed write().Toni Uhlig2022-03-13
| | | | | | | | | * nDPIsrvd: Collector/Distributor logging improved * nDPIsrvd: Command line option for max remote descriptors * nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again * nDPId: Refactored some variable names and events Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPIsrvd TCP/IP support for distributors.Toni Uhlig2022-03-10
| | | | | | | | * nDPIsrvd: Improved distributor client disconnect detection * nDPIsrvd: Fixed invalid usage of epoll_add instead of epoll_mod * nPDIsrvd: Improved logging for distributor clients Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added daemon event: DAEMON_EVENT_STATUS (periodically send's daemon statistics.)Toni Uhlig2022-03-06
| | | | | | | | | | * Improved distributor timeout handling (per-thread). * flow-info.py / flow-dash.py: Distinguish between flow risk severities. * nDPId: Skip tag switch datalink packet dissection / processing. * nDPId: Fixed incorrect value for current active flows. * Improved JSON schema's. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Serialize flow risk score / confidence.Toni Uhlig2022-02-27
| | | | | | | * bump libnDPI to 8b062295cc76a60e3905c054ce37bd17669464d1 * removed ndpi_id_struct's Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added logging interface used by nDPId, nDPIsrvd and nDPId-test.Toni Uhlig2022-02-03
| | | | | | | | | * fixed GitLab pipeline * nDPId: added static assert (just for a test) * nDPId: memory profiling for total bytes compressed * nDPId-test: enable zLib compression if configured with ENABLE_ZLIB Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Process additional layer 3 protocols.Toni Uhlig2022-01-31
| | | | | | | | | * bump libnDPI to c53c82d4823b5a8f856d1375155ac5112b68e8af * run_tests.sh: improved execution from non-git directories e.g. via `make dist` * updated JSON schema to be more restrictive * nDPId: splitted generic get_ip_from_sockaddr into IPv4/IPv6 to prevent compiler warnings on some platforms Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved Plotly/Dash example. It is now somehow informative.Toni Uhlig2022-01-25
| | | | | | | * TCP timeout after FIN/RST: switched back to the value from a35fc1d5ea8570609cc0c8cf6edadc81f8f5bb76 * py-flow-info: reset 'guessed' flag after detection/detection-update received Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Major nDPId extension. Sorry for the huge commit.Toni Uhlig2022-01-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - nDPId: fixed invalid IP4/IP6 tuple compare - nDPIsrvd: fixed caching issue (finally) - added tiny c example (can be used to check flow manager sanity) - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow` - README.md update: added example JSON sequence - nDPId: added new flow event `update` necessary for correct timeout handling (and other future use-cases) - nDPIsrvd.h and nDPIsrvd.py: switched to an instance (consists of an alias/source tuple) based flow manager - every flow related event **must** now serialize `alias`, `source`, `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout handling and verification process work correctly - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation - nDPIsrvd.py: removed PcapPacket class (unused) - py-flow-dashboard and py-flow-multiprocess: fixed race condition - py-flow-info: print statusbar with probably useful information - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`) to a generic flow event timestamp `ts_msec` - nDPId-test: added additional checks - nDPId: increased ICMP flow timeout - nDPId: using event based i/o if capturing packets from a device - nDPIsrvd: fixed memory leak on shutdown if remote descriptors were still connected Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* MemoryProfiling: Advanced flow usage logging.Toni Uhlig2021-11-13
| | | | | | | | * nDPId-test: disable #include <syslog.h> if NO_MAIN macro defined * nDPId-test: mock syslog flags and functions * gitlab-ci: force -Werror Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.