| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
| |
* upstream changed regression test interface, needed to adapt
* improved libnDPI helper build script
* updated JSON schema
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
to `pkt_datalink`.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
serializer.
* nDPId-test: Increased the max-packets-per-flow-to-send from 3 to 5.
This is quite useful for TCP as the first 3 packets are usually part of the three-way-handshake.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
abused.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* nDPIsrvd.h: iterate over JSON arrays
* nDPId: calculate l3 payload packet entropies for analysis
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
| |
* Still not perfect, but the code before was not even able to deal with JSON arrays.
Use common "speaking" function names for all functions in nDPIsrvd.h
* Provide a more or less generic and easy extendable JSON walk function.
* Modified C examples to align with the changed C API.
* c-collectd: Reduced lot's of code duplication by providing mapping tables.
* nDPId: IAT array requires one slot less (first packet has always an IAT of 0).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
| |
* store packet directions
* merged direction based IATs
* merged direction based PKTLENs
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
* nDPId: Fixed broken validation tests.
* nDPId: Removed TICK_RESOLUTION, not required anymore.
* c-collectd: Improved total layer4 payload calculation/update handling.
* c-collectd: Updated RRD Graph script according to total layer4 payload changes.
* py-flow-info.py: Fixed several bugs and syntax errors.
* Python scripts: Added dirname(argv[0]) as search path for nDPIsrvd.py.
* nDPIsrvd&nDPId-test: Fixed missing EPOLLERR check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* ReadMe stuff
* OpenWrt Makefile: set NEED_LINKING_AGAINST_LIBM=ON
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* The goal was to provide a separate event for extracted feature that are not required
and only useful for a few (e.g. someone who wants do ML).
* Increased network buffer size to 32kB (8192 * 4).
* Switched timestamp precision from ms to us for *ALL* timestamps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
json schema file.
* export env vars AR / CMAKE_C_COMPILER_AR and RANLIB / CMAKE_C_COMPILER_RANLIB while building libnDPI
* nDPId check API version during startup (macro vs. function call) and print a warning if they are different
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
`flow_dst_packets_processed'.
* no use for `flow_avg_l4_payload_len' -> removed
* test/run_tests.sh does not fail if git-worktree's are used
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* try to use same wording wherever possible e.g.
renamed workflow->total_l4_data_len to workflow->total_l4_payload_len
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
* README/nDPId: fixed typ0
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* nDPIsrvd: Fixed buffer allocation error due to missing memset() on disconnect
* nDPIsrvd: Removed unused struct members
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
* nDPId: Improved TCP timeout handling if FIN/RST seen
which caused Midstream TCP flows when there shouldn't be any.
* nDPIsrvd: Unified remote descriptor resource cleanup on disconnects/shutdown.
* nDPIsrvd: Added additional error messages for remote descriptors.
* py-flow-info: Better daemon status message printing.
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: lns <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
* nDPIsrvd.h: Provide nDPId thread storage.
* nDPIsrvd.py: Fixed instance cleanup bug.
* nDPIsrvd.h: Support for instance/thread user data and cleanup callback.
* nDPIsrvd.h: Most recent flow time stored in thread ht instead of instance ht.
* nDPId: Moved flow logger out the memory profilier into SIGUSR1 signal handling.
* nDPId: Added signal fd to be usable within epoll's event handling (live-capture only!)
* nDPId: Added information about ZLib compressions to daemon status/shutdown events.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* nDPIsrvd: Collector/Distributor logging improved
* nDPIsrvd: Command line option for max remote descriptors
* nDPId: Stop spamming nDPIsrvd Collector with the same events over and over again
* nDPId: Refactored some variable names and events
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
* Improved distributor timeout handling (per-thread).
* flow-info.py / flow-dash.py: Distinguish between flow risk severities.
* nDPId: Skip tag switch datalink packet dissection / processing.
* nDPId: Fixed incorrect value for current active flows.
* Improved JSON schema's.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* bump libnDPI to 8b062295cc76a60e3905c054ce37bd17669464d1
* removed ndpi_id_struct's
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
timestamps.
* py-flow-dashboard: Added color mapping for PieCharts/Graph that make more sense
* nDPId: Renamed `flow_type' to a more precisely `flow_state'
* nDPId: Changed the default setting to process only as much packets as libnDPI does
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will only affect flows with the state `FT_FINISHED' (detection done).
* nDPIsrvd.py: force use of JSON schema Draft 7 validator
* flow-dash.py: gather/use total processed layer4 payload size
* flow-info.py: added additional event filter
* flow-info.py: prettified flow events printing whose detection is in progress
* py-semantic-validation.py: added validation checks for FT_FINISHED
* updated flow event JSON schema
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* fixed GitLab pipeline
* nDPId: added static assert (just for a test)
* nDPId: memory profiling for total bytes compressed
* nDPId-test: enable zLib compression if configured with ENABLE_ZLIB
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
| |
* bump libnDPI to c53c82d4823b5a8f856d1375155ac5112b68e8af
* run_tests.sh: improved execution from non-git directories e.g. via `make dist`
* updated JSON schema to be more restrictive
* nDPId: splitted generic get_ip_from_sockaddr into IPv4/IPv6 to prevent compiler warnings on some platforms
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* Fixed invalid array subscript typ0 (caused some trouble..)
* bump libnDPI to 2cd0479204301c50c6149706fcd4df3058b2a8cc
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* TCP timeout after FIN/RST: switched back to the value from a35fc1d5ea8570609cc0c8cf6edadc81f8f5bb76
* py-flow-info: reset 'guessed' flag after detection/detection-update received
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- nDPId: fixed invalid IP4/IP6 tuple compare
- nDPIsrvd: fixed caching issue (finally)
- added tiny c example (can be used to check flow manager sanity)
- c-captured: use flow_last_seen timestamp from `struct nDPIsrvd_flow`
- README.md update: added example JSON sequence
- nDPId: added new flow event `update` necessary for correct
timeout handling (and other future use-cases)
- nDPIsrvd.h and nDPIsrvd.py: switched to an instance
(consists of an alias/source tuple) based flow manager
- every flow related event **must** now serialize `alias`, `source`,
`flow_id`, `flow_last_seen` and `flow_idle_time` to make the timeout
handling and verification process work correctly
- nDPIsrvd.h: ability to profile any dynamic memory (de-)allocation
- nDPIsrvd.py: removed PcapPacket class (unused)
- py-flow-dashboard and py-flow-multiprocess: fixed race condition
- py-flow-info: print statusbar with probably useful information
- nDPId/nDPIsrvd.h: switched from packet-flow only timestamps (`pkt_*sec`)
to a generic flow event timestamp `ts_msec`
- nDPId-test: added additional checks
- nDPId: increased ICMP flow timeout
- nDPId: using event based i/o if capturing packets from a device
- nDPIsrvd: fixed memory leak on shutdown if remote descriptors
were still connected
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
|
|
| |
* After the first packet was processed, "flow_last_seen" was still 0.
This behaviour is invalid as the first packet may contain l4 payload data e.g. for UDP
and it also breaks nDPId json consistency "flow_first_seen" > 0, but "flow_last_seen" == 0.
* JSON schema: set minimum timestamp value for Epoch timestamps to 24710 for flow_*_seen and
1 for pcap packet ts. Those values are dependant on some manipulated pcap's in libnDPI/tests/pcap.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* nDPId: Renamed some of the misleading terms, still TODO for nDPIsrvd
* CMake improvments
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
nDPId-test: Collect information about JSON string length's.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
|
| |
* fixed linking issue by using CMake to check if explicit link against libm required
* make nDPIsrvd collectd exit if parent pid changed, meaning that collectd died somehow
* nDPId-test restores SIGPIPE to the default handler (termination), so abnormal connection drop's do now have consequences
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
| |
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* w/o zLib
* gcrypt requires to be enabled
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
| |
* print a summary
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
|
|
|
|
| |
* fixed run_tests.sh file check bug, CI compat
* updated results due to libnDPI submodule update
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|