| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reformatted shell scripts according to [ShellCheck](https://github.com/koalaman/shellcheck/).
I. Most common changes:
1. https://github.com/koalaman/shellcheck/wiki/SC2086
`$var` → `"$var"`
Note: this isn't always necessary and I've been careful not to substitute where it wasn't necessary in meaning.
2. https://github.com/koalaman/shellcheck/wiki/SC2006
`` `command` `` → `$(command)`
3. https://github.com/koalaman/shellcheck/wiki/SC2004
`$(( $a + $b ))` → `$(( a + b ))`
4. https://github.com/koalaman/shellcheck/wiki/SC2164
`cd "$dir"` → `cd "$dir" || exit 1`
5. https://github.com/koalaman/shellcheck/wiki/SC2166
`[ check1 -o check2 ]` → `[ check1 ] || [ check2 ]`
6. https://github.com/koalaman/shellcheck/wiki/SC2002
`cat "${file}" | wc -c` → `< "${file}" wc -c`
Note: this looks a bit uglier but works faster.
II. Some special changes:
1. In file `utils/common.sh`:
https://github.com/koalaman/shellcheck/wiki/SC2112
This script is interpreted by `sh`, not by `bash`, but uses the keyword `function`.
So I replaced `#!/usr/bin/env sh` to `#!/usr/bin/env bash`.
2. After that I thought of replacing all shebangs to `#!/usr/bin/env bash` for consistency and cross-platform compatibility, especially since most of the files already use bash.
3. But in cases when it was `#!/bin/sh -e` or `#!/bin/bash -eu` another problem appears:
https://github.com/koalaman/shellcheck/wiki/SC2096
So I decided to make all shebangs look uniform:
```
#!/usr/bin/env bash
set -e (or set -eu) (if needed)
```
4. In file `tests/ossfuzz.sh`:
https://github.com/koalaman/shellcheck/wiki/SC2162
`read i` → `read -r i`
Note: I think that there is no need in special treatment for backslashes, but I could be wrong.
5. In file `tests/do.sh.in`:
https://github.com/koalaman/shellcheck/wiki/SC2035
`ls *.*cap*` → `ls -- *.*cap*`
6. In file `utils/verify_dist_tarball.sh`:
https://github.com/koalaman/shellcheck/wiki/SC2268
`[ "x${TARBALL}" = x ]` → `[ -z "${TARBALL}" ]`
7. In file `utils/check_symbols.sh`:
https://github.com/koalaman/shellcheck/wiki/SC2221
`'[ndpi_utils.o]'|'[ndpi_memory.o]'|'[roaring.o]')` → `'[ndpi_utils.o]'|'[ndpi_memory.o]')`
8. In file `autogen.sh`:
https://github.com/koalaman/shellcheck/wiki/SC2145
`echo "./configure $@"` → `echo "./configure $*"`
https://github.com/koalaman/shellcheck/wiki/SC2068
`./configure $@` → `./configure "$@"`
III. `LIST6_MERGED` and `LIST_MERGED6`
There were typos with this variables in files `utils/aws_ip_addresses_download.sh`, `utils/aws_ip_addresses_download.sh` and `utils/microsoft_ip_addresses_download.sh` where variable `LIST6_MERGED` was defined, but `LIST_MERGED6` was removed by `rm`.
I changed all `LIST_MERGED6` to `LIST6_MERGED`.
Not all changes are absolutely necessary, but some may save you from future bugs.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Use two separate lists:
* one for the ingress nodes, which triggers a ProtonVPN classification
* one for the egress nodes, which triggers the
`NDPI_ANONYMOUS_SUBSCRIBER` risk
Add a command line option (to `ndpiReader`) to easily test IP/port
matching.
Add another example of custom rule.
|
| |
|
|
|
| |
* added additional (more restrictive) checks
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
| |
* Improved ASN update script
* Ran `utils/update_every_lists.sh'
* `tests/do.sh.in' prints the amount of failed pcap(s)
* `utils/asn_update.sh' prints the amount of failed download(s)
Signed-off-by: lns <matzeton@googlemail.com>
|
| |
|
|
|
|
|
|
| |
addresses/ranges. (#1524)
* Replaced return statements in `utils/*.sh' with exit's (such scripts should never source'd)
* Ran `utils/update_every_lists.sh'
Signed-off-by: lns <matzeton@googlemail.com>
|
| | |
|
| |
|
|
|
| |
* CI will print a warning if ASN/IP addresses changed.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
|
|
|
The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.
The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.
That usually happens for the vast majority of the traffic.
However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.
Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF
This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.
A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.
TODO: IPv6
|