aboutsummaryrefslogtreecommitdiff
path: root/dependencies/nDPIsrvd.py
Commit message (Collapse)AuthorAge
* Replaced ambiguous naming of "JSON string" to more accurate "JSON message".Toni Uhlig2023-12-04
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* flow-info.py: Do not print any information if a flow is "empty" meaning no ↵Toni Uhlig2023-12-01
| | | | | | | | L4 payload seen so far. * added JsonDecodeError to provide more information if builtin JSON decoder fails Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed more SonarCloud complaints.Toni Uhlig2023-11-07
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Sonarcloud integrationToni Uhlig2023-10-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Enable custom JSON filter expressions for Python scripts.Toni Uhlig2023-08-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Small CI/CD/nDPIsrvd.py improvements.Toni Uhlig2023-01-17
| | | | | | * Updated examples/js-rt-analyzer and examples/js-rt-analyzer-frontend Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPid: Fixed base64encode bug which lead to invalid base64 strings.Toni Uhlig2023-01-09
| | | | | | | | * py-semantic-validation: Decode base64 raw packet data as well * nDPIsrvd.py: Added PACKETS_PLEN_MAX * nDPIsrvd.py: Improved JSON parse error/exception handling Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased network buffer size to 33792 bytes.Toni Uhlig2022-10-30
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* flow-info.py: Command line arguments --no-color, --no-statusbar (both useful ↵Toni Uhlig2022-09-22
| | | | | | | | for tests/CI) and --print-analyse-results. * run_tests.sh: Use flow-info.py for additional DIFF tests. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* New flow event: 'analysis'.Toni Uhlig2022-09-13
| | | | | | | | | * The goal was to provide a separate event for extracted feature that are not required and only useful for a few (e.g. someone who wants do ML). * Increased network buffer size to 32kB (8192 * 4). * Switched timestamp precision from ms to us for *ALL* timestamps. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd.py: Throw SocketTimeout Exception to catch both timeout exceptions ↵Toni Uhlig2022-06-07
| | | | | | different Python versions can throw. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd.(h|py): Added socket read/recv timeout.lns2022-04-17
| | | | | | * nDPIsrvd.h: support for O_NONBLOCK nDPIsrvd_socket Signed-off-by: lns <matzeton@googlemail.com>
* Reworked nDPIsrvd.h C-API.Toni Uhlig2022-03-21
| | | | | | | | | | | | * nDPIsrvd.h: Provide nDPId thread storage. * nDPIsrvd.py: Fixed instance cleanup bug. * nDPIsrvd.h: Support for instance/thread user data and cleanup callback. * nDPIsrvd.h: Most recent flow time stored in thread ht instead of instance ht. * nDPId: Moved flow logger out the memory profilier into SIGUSR1 signal handling. * nDPId: Added signal fd to be usable within epoll's event handling (live-capture only!) * nDPId: Added information about ZLib compressions to daemon status/shutdown events. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Renamed basic event to error event for the sake of the logic.Toni Uhlig2022-03-13
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Disconnect nDPIsrvd clients immediately instead waiting for a failed write().Toni Uhlig2022-03-13
| | | | | | | | | * nDPIsrvd: Collector/Distributor logging improved * nDPIsrvd: Command line option for max remote descriptors * nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again * nDPId: Refactored some variable names and events Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed some pyhton issues with static class members.Toni Uhlig2022-03-08
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added daemon event: DAEMON_EVENT_STATUS (periodically send's daemon statistics.)Toni Uhlig2022-03-06
| | | | | | | | | | * Improved distributor timeout handling (per-thread). * flow-info.py / flow-dash.py: Distinguish between flow risk severities. * nDPId: Skip tag switch datalink packet dissection / processing. * nDPId: Fixed incorrect value for current active flows. * Improved JSON schema's. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Serialize flow risk score / confidence.Toni Uhlig2022-02-27
| | | | | | | * bump libnDPI to 8b062295cc76a60e3905c054ce37bd17669464d1 * removed ndpi_id_struct's Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved py-flow-info to provide more optional information about received ↵Toni Uhlig2022-02-05
| | | | | | | | | | timestamps. * py-flow-dashboard: Added color mapping for PieCharts/Graph that make more sense * nDPId: Renamed `flow_type' to a more precisely `flow_state' * nDPId: Changed the default setting to process only as much packets as libnDPI does Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added additional (minimalistic) detection information to flow updates.Toni Uhlig2022-02-04
| | | | | | | | | | | | | This will only affect flows with the state `FT_FINISHED' (detection done). * nDPIsrvd.py: force use of JSON schema Draft 7 validator * flow-dash.py: gather/use total processed layer4 payload size * flow-info.py: added additional event filter * flow-info.py: prettified flow events printing whose detection is in progress * py-semantic-validation.py: added validation checks for FT_FINISHED * updated flow event JSON schema Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Major nDPId extension. Sorry for the huge commit.Toni Uhlig2022-01-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - nDPId: fixed invalid IP4/IP6 tuple compare - nDPIsrvd: fixed caching issue (finally) - added tiny c example (can be used to check flow manager sanity) - c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow` - README.md update: added example JSON sequence - nDPId: added new flow event `update` necessary for correct timeout handling (and other future use-cases) - nDPIsrvd.h and nDPIsrvd.py: switched to an instance (consists of an alias/source tuple) based flow manager - every flow related event **must** now serialize `alias`, `source`, `flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout handling and verification process work correctly - nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation - nDPIsrvd.py: removed PcapPacket class (unused) - py-flow-dashboard and py-flow-multiprocess: fixed race condition - py-flow-info: print statusbar with probably useful information - nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`) to a generic flow event timestamp `ts_msec` - nDPId-test: added additional checks - nDPId: increased ICMP flow timeout - nDPId: using event based i/o if capturing packets from a device - nDPIsrvd: fixed memory leak on shutdown if remote descriptors were still connected Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId semantic validation test.Toni Uhlig2021-04-14
| | | | | | | | | | | * fixed inconsistent processing of remaining flows during nDPId shutdown phase * fixed multiple `detected' flow events (instead only `detection-update' flow events can occur after a `detected' flow event) * fixed nDPIsrvd.py invalid message buffer handling * improved run_tests.sh so only valid pcap capture files are getting processed (and some more cosmetics + logging) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added IPv6 support for -I / -E.Toni Uhlig2021-04-11
| | | | | | | * added another Python search path and try-catch ModuleNotFoundError again * run_tests.sh checks for OpenBSD netcat (required for -q) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JSON schema validation to run_tests.shToni Uhlig2021-04-09
| | | | | | | * Python3 scripts are now compatible with versions <3.6 * improved and prettified run_tests.sh Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JSON schema files and a Python schema validator.Toni Uhlig2021-02-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Aligned nDPIsrvd.py to nDPIsrvd.h for consistency.Toni Uhlig2021-02-16
| | | | | | | | | * Simplified Python interface as well. * c-captured and flow-undetected-to-pcap.py produce similiar results * Removed overloaded nDPIsrvd.py event structures. * flow-info.py prints (with a color-hash) additional information e.g. alias/source and midstream Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed braindead failure - Increase of NETWORK_BUFFER_MAX_SIZE > 9999 means ↵Toni Uhlig2021-01-27
| | | | | | | | | to also increase NETWORK_BUFFER_LENGTH_DIGITS to 5. * Fixed ARM32 xcompile warnings; Other GCC versions, other uint64_t's.. * Replaced ridiculous nDPIsrvd_JSON_BYTES with NETWORK_BUFFER_LENGTH_DIGITS. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased JSON buffer size to 12288 (libnDPI serializes more and more ↵Toni Uhlig2021-01-27
| | | | | | | | | | | information). * Making Compare&Fetch mandatory. * Added some more Compare&Fetch to prevent TSAN complaining about data races. Fixed possible but more ore less harmless data races during shutdown process. * Shrink SIGNAL handler to a minimum. SYSV Signal handling and MT-safety is awkward. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* All colors are beautiful for py-flow-info.Toni Uhlig2021-01-26
| | | | | | * instance alias is now mandatory (will be required for future use-cases) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased network buffer size for python scripts (and raise exception if ↵Toni Uhlig2021-01-08
| | | | | | capacity reached). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Declared nDPI id and flow structs in nDPId flow info struct.Toni Uhlig2020-12-30
| | | | | | | | Two reasons: * reduce heap memory allocations * nDPId flow info struct may be inflated in the future (more bytes to compress) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Generic nDPIsrvd.h event handling integration and flow tracking with uthash.Toni Uhlig2020-12-06
| | | | | | * aims to be re-usable for many different apps and use-cases Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Renamed misleading "contrib" subfolder to "dependencies".Toni Uhlig2020-11-13
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Powered by cgit, Themed by Ted Yin.