summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* nDPId-test: added JSON distribution + JSON parsing (Multithreaded design ↵Toni Uhlig2021-04-09
| | | | | | | | | | | | | | re-using most of nDPId/nDPIsrvd core) * improved Makefile.old install targets * splitted nDPIsrvd_parse into nDPIsrvd_parse_line and nDPIsrvd_parse_all for the sake of readability * minor Python script improvments (check for nDPIsrvd.py on multiple locations, may be superseeded by setuptools in the future) * some paths needs to be absolute (chdir() during daemonize) and therefor additional checks introduced * test run script checks and fails if certain files are are missing (PCAP file <=> result output file) * removed not very useful "internal format error" JSON serialization if a BUG for same exists * fixed invalid l4 type statistics counters for nDPIsrvd-collectd Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reverted internal ndpi structs to dynamic memory allocation hoping that it ↵Toni Uhlig2021-03-26
| | | | | | | | | | will help to reduce the average memory consumption. How? After the detection finished, internal ndpi structs can be free'd as they are not needed anymore. * Set the amount of max. packets to process via subopt. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* run_tests.sh checks/print-stderr-on-failToni Uhlig2021-03-24
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated README.mdToni Uhlig2021-03-24
| | | | | | * added examples/README.md and schema/README.md Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Updated test outputs and pinned travis-ci's nDPI git repo sync to a specific ↵Toni Uhlig2021-03-24
| | | | | | | | commit hash. * fixed Makefile.old COpts hell Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added diff'able nDPId-test JSON dumps and travis-ci integration.Toni Uhlig2021-03-24
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added nDPId-test as all-in-one JSON dumper.Toni Uhlig2021-03-24
| | | | | | | | | * fixed invalid flow event schema type * added run_tests.sh to generate/diff JSON dumps * renamed lot's of vars/fns in nDPId.c/nDPIsrvd.c, so nDPId-test.c can include "*.c" * improved CMake dependency checks Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reduced code duplication. Preps for nDPId-test.Toni Uhlig2021-03-23
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Switched to CMake build system.Toni Uhlig2021-03-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd collectd-exec overhaul.Toni Uhlig2021-03-15
| | | | | | | * Install targets updated. * Removed nDPIsrvd.h token validation function (done automatically by token_get). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed collectd-exec issues.Toni Uhlig2021-03-12
| | | | | | * Added collectd example config and types.db Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added basic collectd-exec example.Toni Uhlig2021-03-10
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added missing enum strings.Toni Uhlig2021-03-09
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd refactoringToni Uhlig2021-03-05
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed possible NULL pointer deref in dependencies/nDPIsrvd.h and ↵Toni Uhlig2021-02-28
| | | | | | | | | | examples/c-captured/c-captured.c * `make install` C examples * examples/py-flow-info/flow-info.py supports filtering * added breed to examples/py-flow-info/flow-info.py Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JA3 / SSL SHA1 fingerprint blacklists.Toni Uhlig2021-02-27
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Finalized examples/c-captured to dump packet bytes to PCAP for further analysis.Toni Uhlig2021-02-24
| | | | | | * Fixed memory holes in nDPId structs. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Extended nDPIsrvd.h with address parsing.Toni Uhlig2021-02-23
| | | | | | | | | * nDPId supports looading of custom nDPI protocol/category files * extended JSON schemas according to nDPI / nDPId JSON serializing * removed memory holes in nDPId * extended examples/c-captured Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* C-Api uses similiar flow key hash calculation as Python-Api.Toni Uhlig2021-02-19
| | | | | | * Make use of flow id / alias / source which is required for future use cases where multiple nDPId instances (same / different machines) feed one nDPIsrvd. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Reverted f8692c6be11c664e4345ceaaec5c6b35ba9b3febToni Uhlig2021-02-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added JSON schema files and a Python schema validator.Toni Uhlig2021-02-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Aligned nDPIsrvd.py to nDPIsrvd.h for consistency.Toni Uhlig2021-02-16
| | | | | | | | | * Simplified Python interface as well. * c-captured and flow-undetected-to-pcap.py produce similiar results * Removed overloaded nDPIsrvd.py event structures. * flow-info.py prints (with a color-hash) additional information e.g. alias/source and midstream Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Removed superfluous nDPI (header) dependency for examples/c-captured. Use ↵Toni Uhlig2021-02-06
| | | | | | header definitions from libc instead. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd C API overhaul and massive simplification.Toni Uhlig2021-02-06
| | | | | | | | * nDPIsrvd.h does flow mgmt out of the box * dissect received JSON strings via callback * added new JSON key/values for packet-flows (usecTimestamp/L3/L4 info) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed braindead failure - Increase of NETWORK_BUFFER_MAX_SIZE > 9999 means ↵Toni Uhlig2021-01-27
| | | | | | | | | to also increase NETWORK_BUFFER_LENGTH_DIGITS to 5. * Fixed ARM32 xcompile warnings; Other GCC versions, other uint64_t's.. * Replaced ridiculous nDPIsrvd_JSON_BYTES with NETWORK_BUFFER_LENGTH_DIGITS. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased JSON buffer size to 12288 (libnDPI serializes more and more ↵Toni Uhlig2021-01-27
| | | | | | | | | | | information). * Making Compare&Fetch mandatory. * Added some more Compare&Fetch to prevent TSAN complaining about data races. Fixed possible but more ore less harmless data races during shutdown process. * Shrink SIGNAL handler to a minimum. SYSV Signal handling and MT-safety is awkward. Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* All colors are beautiful for py-flow-info.Toni Uhlig2021-01-26
| | | | | | * instance alias is now mandatory (will be required for future use-cases) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Finished (mostly) C guessed/undetected flow dumper.Toni Uhlig2021-01-26
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Switched to a slightly modified base64 encoder basicially from wikibooks.Toni Uhlig2021-01-20
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added simple python json out example used for comparing JSON output.Toni Uhlig2021-01-20
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added internal/external packet processing modes. #2Toni Uhlig2021-01-10
| | | | | | * fixed incorrect handling of skipped flows Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased network buffer size for python scripts (and raise exception if ↵Toni Uhlig2021-01-08
| | | | | | capacity reached). Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Added internal/external packet processing modes.Toni Uhlig2021-01-06
| | | | | | | | * only IPv4 supported for now * refactored nDPId's internal IP address storage * use fresh ndpi_free_flow_data() to free nDPI's dynamic allocated data Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Declared nDPI id and flow structs in nDPId flow info struct.Toni Uhlig2020-12-30
| | | | | | | | Two reasons: * reduce heap memory allocations * nDPId flow info struct may be inflated in the future (more bytes to compress) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Set an optional bpf filter string for pcap based packet capture.Toni Uhlig2020-12-29
| | | | | | * serialize and send datalink layer type Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPIsrvd: Distributor clients which are too slow can cause buffer bloat.Toni Uhlig2020-12-19
| | | | | | | | | Switching back to blocking mode works as a quick fix but is not sufficient. See comments. * nDPId prints more accurate error messages if command line argument validation failed Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Improved testing capabilities via Makefile.Toni Uhlig2020-12-15
| | | | | | * increased nDPId_MAX_READER_THREADS to 32 Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* nDPId option validation is now less restrictiveToni Uhlig2020-12-14
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Generic nDPIsrvd.h event handling integration and flow tracking with uthash.Toni Uhlig2020-12-06
| | | | | | * aims to be re-usable for many different apps and use-cases Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Merge commit 'c8bf38e5fb717d40635a2a89b22ed71b0de4266b' as 'dependencies/uthash'Toni Uhlig2020-12-01
|\
| * Squashed 'dependencies/uthash/' content from commit 8e67cedToni Uhlig2020-12-01
| | | | | git-subtree-dir: dependencies/uthash git-subtree-split: 8e67ced1d1c5bd8141c542a22630e6de78aa6b90
* Removed all LUA leftover as preparation for UTHASH.Toni Uhlig2020-12-01
| | | | | | * Added UNIX socket support for nDPIsrvd.h (and c-captured) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Set rpath when compiling with use of pkg-config.Toni Uhlig2020-12-01
| | | | | | * New Makefile target: mocksrvd, fake nDPIsrvd instamce Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Let travis use pcap files from nDPI to produce some JSON output. (disabled, ↵Toni Uhlig2020-12-01
| | | | | | | | | | | needs further testing..) * Added pcap diff script * Added \n to JSON string end (useful for debugging and readability) * Use first host/server name character for hash calculation as well * Removed error'ing EPOLLHUP handling in nDPIsrvd (connection closing will be detected via read()) Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Moved datalink layer processing into a separate function.Toni Uhlig2020-11-18
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Increased serialization buffer size as required for recent nDPI builds.Toni Uhlig2020-11-18
| | | | | | | * added .gitignore to prevent SCM check-ins of unwanted files * fixed invalid include path for python examples Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Refactored Makefile pkg-config part to be able to work with multiple ↵Toni Uhlig2020-11-18
| | | | | | | | | pkg-config based projects. * added lua as new dependency for examples/c-captured * improved `make help' print Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Provide functionality do deal with tokens from JSMN.Toni Uhlig2020-11-17
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Fixed inconsistent callback function call if a token parsed.Toni Uhlig2020-11-13
| | | | Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
* Add 'dependencies/jsmn/' from commit '053d3cd29200edb1bfd181d917d140c16c1f8834'Toni Uhlig2020-11-13
|\ | | | | | | | | | | git-subtree-dir: dependencies/jsmn git-subtree-mainline: f79991d70c66040426cb2c59876d0c48f80fdfe6 git-subtree-split: 053d3cd29200edb1bfd181d917d140c16c1f8834
Powered by cgit, Themed by Ted Yin.