diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | README.md | 53 | ||||
| -rw-r--r-- | examples/README.md | 43 | ||||
| -rw-r--r-- | schema/README.md | 5 |
4 files changed, 92 insertions, 10 deletions
diff --git a/.gitignore b/.gitignore index 56e0c00ec..6be61315e 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ # c executables /nDPId /nDPIsrvd +/nDPId-test /examples/c-collectd/c-collectd /examples/c-captured/c-captured /examples/c-json-stdout/c-json-stdout @@ -1,7 +1,7 @@ # abstract nDPId is a set of daemons and tools to capture, process and classify network flows. -It's only dependencies (besides a half-way modern c library and POSIX threads) are libnDPI (>= 3.4.0, current github dev branch) and libpcap. +It's only dependencies (besides a half-way modern c library and POSIX threads) are libnDPI (>= 3.6.0 or current github dev branch) and libpcap. The core daemon nDPId uses pthread but does use mutexes for performance reasons. Instead synchronization is achieved by a packet distribution mechanism. @@ -12,7 +12,6 @@ nDPId uses libnDPI's JSON serialization to produce meaningful JSON output which High level applications can connect to nDPIsrvd to get the latest flow/packet events from nDPId. Unfortunately nDPIsrvd does currently not support any encryption/authentication for TCP connections. -TODO: Provide some sort of AEAD for connecting distributor clients via TCP (somehow very critical). # architecture @@ -48,36 +47,71 @@ All JSON strings sent need to be in the following format: ## Example: ```text -0015{"key":"value"} +00015{"key":"value"} ``` -where `0015` describes the length of a **complete** JSON string. +where `00015` describes the length of a **complete** JSON string. TODO: Describe data format via JSON schema. -# build +# build (CMake) + +```shell +mkdir build +cd build +cmake .. +``` + +or + +```shell +mkdir build +cd build +ccmake .. +``` + +# build (old style GNU Make) To get an overview over all build options, run: ```shell -make help +make -f Makefile.old help ``` To build nDPId and nDPIsrvd, run: ```shell -make all +make -f Makefile.old all ``` To build nDPId and nDPIsrvd with sanitizer, debug mode enabled and a custom/not-your-distro libnDPI, run: ```shell -make ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all +make -f Makefile.old ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all +``` + +If you get any linker errors, try one of the +```shell +make -f Makefile.old | grep '^NDPI_WITH_' +``` +e.g. +```shell +make -f Makefile.old NDPI_WITH_GCRYPT=yes ENABLE_DEBUG=yes ENABLE_SANITIZER=yes CUSTOM_LIBNDPI=[path-to-libndpi].[a|so] all +``` + +or let pkg-config do the job for you: +```shell +PKG_CONFIG_PATH="[path-to-optional-nDPI-pkg-config-dir]" make -f Makefile.old PKG_CONFIG_BIN=pkg-config ENABLE_DEBUG=yes ENABLE_SANITIZER=yes all ``` To build nDPId and nDPIsrvd and examples, run: ```shell -make all examples +make -f Makefile.old all examples ``` # run +Generate a nDPId compatible JSON dump: +```shell +./nDPId-test [path-to-a-PCAP-file] +``` + Daemons: ```shell ./nDPIsrvd -d @@ -91,7 +125,6 @@ or for a usage printout: ``` And why not a flow-info example? -Run ```shell ./examples/py-flow-info/flow-info.py ``` diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 000000000..676e9a9d2 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,43 @@ +# examples + +Some ready-2-use/ready-2-extend examples/utils. +All examples are prefixed with their used LANG. + +## c-captured + +A capture daemon suitable for low-resource devices. +It saves flows that were guessed/undetected/risky/midstream to a PCAP file for manual analysis. +Basicially a combination of `py-flow-undetected-to-pcap` and `py-risky-flow-to-pcap`. + +## c-collectd + +A collecd-exec compatible middleware that gathers statistic values from nDPId. + +## c-json-stdout + +Tiny nDPId json dumper. Does not provide any useful funcationality besides dumping parsed JSON objects. + +## go-dashboard + +A discontinued tty/ncurses nDPId dashboard. I've figured out that Go + NCurses is a bad idea. + +## py-flow-info + +Prints prettyfied information about flow events. + +## py-flow-undetected-to-pcap + +Captures and saves undetected flows to a PCAP file. + +## py-json-stdout + +Dump received and parsed JSON strings. + +## py-risky-flow-to-pcap + +Captures and saves risky flows to a PCAP file. + +## py-schema-validation + +Validate nDPId JSON strings against pre-defined JSON schema's. +See `schema/`. diff --git a/schema/README.md b/schema/README.md new file mode 100644 index 000000000..186dc4ddf --- /dev/null +++ b/schema/README.md @@ -0,0 +1,5 @@ +# schema + +All schema's placed in here are nDPId exclusive, meaning that they are not necessarily representing a "real-world" JSON string received by e.g. `./example/py-json-stdout`. +This is due to the fact that libnDPI itself add's some JSON information to the serializer of which we have no control over. +IMHO it makes no sense to include stuff here that is part of libnDPI. |