zero config proxies

In most cases, we can export http_proxy=http://localhost:8081. But at times, programs do not recognize the environment variable http_proxy. We may need to refer to their manual to learn how to configure its proxy settings. It is tedious. For example, the way to specify proxy of git over HTTPS different from that of git over SSH. We need some non-intrusive (transparent) way to specify proxies. Applications are required of nothing. No configuration entry for proxies, no environment variable.

VPN

May be you’re wondering what’s wrong with the good old VPN. This is the way to go. It ultimately boils down to the flexibility. We may want leave some traffic as it is, while transparently manipulate some other traffic to make it use the proxy. Here are a few methods to change the proxying policies for VPN.

Routing tables

We need TCP connections to foreign countries to be proxied, while those to domestic servers should not be proxied. VPN solutions normally can only decide to proxy traffic based on simple routing tables.

Network namespaces

To let only specific programs make use of the VPN, we can use Linux’s network namespace. Below is an simple script to run openvpn in network namespace (adopted from Feed all traffic through OpenVPN for a specific network namespace only)

#!/usr/bin/env bash

# Adopted from https://unix.stackexchange.com/a/196116
# sudo $0 --config config.ovpn --auth-user-pass user-pass.txt
# sudo ip netns exec openvpn sudo -u $(whoami) command

set -xeuo pipefail
export PATH=$PATH:/run/current-system/sw/bin

: "${script_type:=}"
: "${netns_name:=openvpn}"
: "${nameserver:=8.8.8.8}"

teardown() {
  if ! ip netns delete "$netns_name"; then
    :
  fi
}

runHook() {
  case "$script_type" in
  up)
    dev="$1"
    mtu="$2"
    ip="$4"
    netmask="${ifconfig_netmask:-30}"
    ip netns add "$netns_name" || true
    ip netns exec "$netns_name" ip link set dev lo up
    mkdir -p "/etc/netns/$netns_name"
    echo "nameserver $nameserver" > "/etc/netns/$netns_name/resolv.conf"
    ip link set dev "$dev" up netns "$netns_name" mtu "$mtu"
    ip netns exec "$netns_name" ip addr add dev "$dev" "$ip/$netmask" ${ifconfig_broadcast:+broadcast "$ifconfig_broadcast"}
    if [[ -n "${ifconfig_ipv6_local:-}" ]]; then
      ip netns exec "$netns_name" ip addr add dev "$1" "$ifconfig_ipv6_local"/112
    fi
    ;;
  route-up)
    ip netns exec "$netns_name" ip route add default via "$route_vpn_gateway"
    if [[ -n "${ifconfig_ipv6_remote:-}" ]]; then
      ip netns exec "$netns_name" ip route add default via "$ifconfig_ipv6_remote"
    fi
    ;;
  down)
    teardown
    ;;
  esac
  exit 0
}

runOpenvpn() {
  script="$(realpath "$0")"
  exec openvpn --script-security 2 --ifconfig-noexec --route-noexec --up "$script" --route-up "$script" --down "$script" "$@"
}

if [[ -z "$script_type" ]]; then
  runOpenvpn "$@"
else
  runHook "$@"
fi

See Routing & Network Namespaces for example usage of wireguard.

VPN in userspace

Another choice for proxying some program’s traffic is to use userspace VPN. This solution does not rely on kernel’s network stack, thus is very flexiable. The downside is that, without kernel’s support, it may be hard for the programs whose source is not under control to use our VPN. See Our User-Mode WireGuard Year for an illustration. See octeep/wireproxy and Tailscale Userspace networking mode (for containers) for real world usage.

More complex proxying rules

Simple forwarding all traffic into the tun device will not work in complicated environment. Sometimes we want our proxying decisions based on the domain name of the request. Traditional VPNs are futile in those cases.

Hooks injection

A few packages can force some programs to use proxy. They work great in their specific use case and if you know how to invoke the program by command line.

proxychains

Dynamically linked programs normally initiate network requests by calling the system libc. Proxychains uses the LD_PRELOAD trick (see ld.so(8) for details) to wrap these connection requests, i.e. they are not really connecting to the target server, instead, they are being sent to the proxy services.

graftcp

The above method does not work with statically linked binaries. For that, we need to hook into the syscalls. graftcp does this. It uses ptrace(2) under the hood.

proxifier

One windows alternative for the above tools is proxifier. In my experience, proxifier does not work in some situation. I think it is because, proxifier hijack win32 API calling like proxychains, which does not always work as programs theoretically can just use system calls.

Linux netfilter

Netfitler is a beast. If only I know how to domesticate it. There are a few ways to proxying all traffic with netfilter. Before we start we should note that it is really easy to cause infinite loop. The traffic from the proxy service itself should not be proxied or there will be an infinite loop.

traffic redirection methods

This section gives an overview on how to redirect the traffic to a backing proxy service. Note that the backing proxy service must support tproxy/redirect proxy mode actively. For a working script to manipulate iptables, see clash-redir.

tproxy

TPROXY stands for transparent proxy. This documentation is more clear than the kernel documentation. The traffic is transparently redirected to the proxy server. The tproxy server captures the traffic, and pretends to be the target server. It is quite easy for the tproxy server to get the original destination, as tproxy server’s receiving socket is set to be of original destination.

iptables -t mangle -N CLASH_EXTERNAL
iptables -t mangle -A CLASH_EXTERNAL -d 0.0.0.0/8 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 127.0.0.0/8 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 224.0.0.0/4 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 172.16.0.0/12 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 169.254.0.0/16 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 240.0.0.0/4 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 192.168.0.0/16 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 10.0.0.0/8 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 100.64.0.0/10 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -d 255.255.255.255 -j RETURN
iptables -t mangle -A CLASH_EXTERNAL -p tcp -j TPROXY --on-port "$CLASH_TPROXY_PORT" --on-ip 127.0.0.1 --tproxy-mark "$CLASH_MARK"
iptables -t mangle -I PREROUTING -p tcp -j CLASH_EXTERNAL
if [[ -z "$(ip rule list fwmark "$CLASH_MARK" table "$CLASH_TABLE")" ]]; then
    ip rule add fwmark "$CLASH_MARK" table "$CLASH_TABLE"
fi
ip route replace local 0.0.0.0/0 dev lo table "$CLASH_TABLE"

A few notes:

The above script makes all TCP traffic to be tproxied to 127.0.0.1:$CLASH_TPROXY_PORT.
I intentionally create a new chain called CLASH_EXTERNAL to make it easier to restore order. Running iptables-save -c | grep -v CLASH_ | iptables-restore -c is enough.
The ip rule and ip route commands make sure all traffic is forwarded, including local traffic and forwarding traffic.
if check is used for idem-potency.

redirect

Iptables redirect just redirects the traffic to the target server. The original server could be a normal server (say a normal HTTP server), or a proxy (say a socks5 proxy which would then forward the traffic to socks5 proxy server).

iptables -t nat -N CLASH_LOCAL
iptables -t nat -A CLASH_LOCAL -m owner --uid-owner "$CLASH_USER" -j RETURN
iptables -t nat -A CLASH_LOCAL -m owner --gid-owner "$NOPROXY_GROUP" --suppl-groups -j RETURN || true
iptables -t nat -A CLASH_LOCAL -d 0.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 127.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 224.0.0.0/4 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 172.16.0.0/12 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 169.254.0.0/16 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 240.0.0.0/4 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 192.168.0.0/16 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 10.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 100.64.0.0/10 -j RETURN
iptables -t nat -A CLASH_LOCAL -d 255.255.255.255 -j RETURN
iptables -t nat -A CLASH_LOCAL -p tcp -j REDIRECT --to-ports "$CLASH_REDIRECT_PORT"
iptables -t nat -I OUTPUT -p tcp -j CLASH_LOCAL

iptables -t nat -N CLASH_EXTERNAL
iptables -t nat -A CLASH_EXTERNAL -d 0.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 127.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 224.0.0.0/4 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 172.16.0.0/12 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 169.254.0.0/16 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 240.0.0.0/4 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 192.168.0.0/16 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 10.0.0.0/8 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 100.64.0.0/10 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -d 255.255.255.255 -j RETURN
iptables -t nat -A CLASH_EXTERNAL -p tcp -j REDIRECT --to-ports "$CLASH_REDIRECT_PORT"
iptables -t nat -I PREROUTING -p tcp -j CLASH_EXTERNAL

A few notes:

Need to add rule for both PREROUTING and OUTPUT for local traffic and forwarding traffic.
NAT under the hood.
It is a little tricky for the redirect proxy to obtain the original destination address and port.
Works perfectly if you don’t need the original destination (like DNS request).

DNAT

Just like redirect.

traffic matching methods

Below are some frequently used traffic matching methods. See iptables-extensions(8) for more methods.

ipset

netfilter itself is able to match a few IPs effectively. When an entire country’s IP addresses need matching, it would be better to use ipset.

cgroup

See cgproxy.

owner, supplementary groups

See gotchas below.

Gotchas

Although it is straightforward to set up transparent proxy on Linux, There are a few delicated situations.

Infinite loop while using the proxy

I use iptables owner module to avoid infinite loop for the proxy itself, supplementary groups to skip proxy for some programs.

iptables -t mangle -I CLASH_LOCAL -m owner --uid-owner "$CLASH_USER" -j RETURN
iptables -t mangle -I CLASH_LOCAL -m owner --gid-owner "$NOPROXY_GROUP" --suppl-groups -j RETURN || true

To let iptables skip traffic from clash, I run clash with

useradd --system --no-create-home "$CLASH_USER" >/dev/null 2>&1 || true
capsh --user="$CLASH_USER" --caps='cap_net_admin+eip cap_setpcap,cap_setuid,cap_setgid+ep' --addamb=cap_net_admin -- -c "clash -f $CLASH_CONFIG"

Change the clash command the $CLASH_USER for your proxy.

I also use the following snippet to skip proxy for a systemd service.

[Service]
SupplementaryGroups=noproxy

To skip proxy temporarily, run sudo systemd-run -p SupplementaryGroups="noproxy" --uid $USER --pty --same-dir --wait --collect --service-type=exec curl https://cloudflare-quic.com/b/ip. You may also try start a cgroup with systemd-run --unit=noproxy --user --shell, and then run sudo iptables -t mangle -I CLASH_LOCAL -m cgroup --path user.slice/user-$UID.slice/user@$UID.service/app.slice/noproxy.service -j RETURN to make traffic within this cgroup bypass proxy.

Proxy not working with docker container in bridge network mode

This is a first world problem. Docker/Kubernetes wants sysctl net.bridge.bridge-nf-call-iptables=1, while libvirt wants sysctl net.bridge.bridge-nf-call-iptables=0. More explanations can be found here, here and here. The following scenery illustrates why docker/Kubernetes insists on enabling bridge-netfilter.

docker run -it --rm -p 8081:8081 nicolaka/netshoot socat -v -v -d -d tcp-listen:8081,fork exec:cat

HOST_IP="$(ip -4 -json addr | jq -r '.[] | .addr_info[] | select(.scope == "global") | .local' | head -n 1)"
docker run -it --rm -p 8082:8082 nicolaka/netshoot bash -c "echo test | socat - tcp:$HOST_IP:8081"
docker run -it --rm -p 8082:8082 nicolaka/netshoot bash -c "echo test | socat - tcp:$HOST_IP:8081,bind=\$(ip -4 -json addr show dev eth0 | jq -r '.[].addr_info[].local'):8082"
docker run -it --rm -p 8082:8082 nicolaka/netshoot bash -c "echo test | socat - tcp:$HOST_IP:8081,bind=127.1.0.1:8082"

When bridge-netfilter is disabled, the last command would time out, but the other two commands will not. This kind of hairpinning support is seldom needed on my machine.

sysctl net.bridge.bridge-nf-call-iptables=0 net.bridge.bridge-nf-call-ip6tables=0 net.bridge.bridge-nf-call-arptables=0

So I disable bridge-netfilter. A further complication is that k3s and docker is so smart as to enable bridge-netfilter on startup. I added a ExecStartPost to disable it.

Proxy not working with docker container when on-ip is missing

To be more precise, sometimes it does not work. I don’t know why. I just banged my head for a few hundreds times and find out --on-ip is a must.

iptables -t mangle -A CLASH_EXTERNAL -p tcp -j TPROXY --on-port 7893 --on-ip 127.0.0.1 --tproxy-mark 0x4242/0xffffffff

DNS resolution

It is of no use for the proxy server to send its requests to a fake server. There are mainly two methods to avoid DNS poisoning.

Bogon IP

Client initiate a DNS request to resolve google.com
The proxy service immediately returns the IP address 192.18.0.22, insert the mapping from 192.18.0.22 to google.com into its internal state
The client initiate a TCP connection to 192.18.0.22
Upon receive the IP packet to 192.18.0.22, the proxy service finds out the request is to google.com. It decides to send the traffic through the proxy server

Redirect DNS requests

Client initiate a DNS request to resolve google.com
The proxy service hijack the request and redirect the traffic to its internal DNS server. The un-posioned address 142.250.66.46 is returned
The client initiate a TCP connection to 142.250.66.46
The proxy service checks the IP database, and decides to redirect the traffic to the proxy server

L4/L7 proxies to L3/L2 VPNs

If you ever used macOS, iOS and android, you will find how easy it is on these platform to set up an VPN service. These VPN services, unlike traditional ones, are much more flexible. They are like PAC proxies, but for all programs. Below is a typical proxy traffic flow in those platforms.

Life of a packet

apps <-> OS <-(L2/L3)-> virtual tunnel <-(L2/L3)-> proxy frontend <-(L4)-> socks5 client <-> proxy client <-> internet <-> proxy server <-> internet

The proxy first creates a virtual tunnel using OS-specific APIs (e.g. TUN/TAP on Linux, VPNService on Android).
Upon receiving app request, the OS constructs L2/L3 packets (ethernet/ip packets), and send those packets to the proxy over the virtual tunnel.
The proxy unwraps those L2/L3 packets and then sends TCP/UDP packets to the socks5 client.
The proxy backend client sends the proxy requests to the proxy backend server over the Internet.
Upon receiving the response, the proxy server sends it back to the proxy client.

A few details

There are a few things requiring special attention.

How does the proxy frontend get L4 packets from L2/L3 packets in the chain OS <-(L2/L3)-> virtual tunnel <-(L2/L3)-> proxy frontend <-(L4)-> socks5 client?
How does the proxy client avoid infinite loop?

The first question is solved by redsocks and tun2socks. There are quite a few solutions on the market. Moreover, apple, by the OS itself, provides such L4 to L2/L3 convertor. It is called NEAppProxyProvider. This is why there are some many proxies on macOS have enhanced mode (effectively an L3 proxy backed by an L3 proxy). Moreover, networkextension also provides useful APIs to change DNS Proxy and filter traffic. Windows users also have a few generic solutions like leaf and maple. Besides, clash premium supports tun, and it does those socks5 proxy to l3 tunnel conversions automatically.

The second question is platform-dependent. Wireguard’s Routing & Network Namespaces gives a good overview for this problem on Linux. A simple generic solution is to add a new routing table entry for the server’s IP so that connection to the server is excluded from redirecting to the tun device. See also openvpn’s implementation, how shadowsocks Android solves this problem.

router in the middle

You can also set up a router in the middle to transparently proxy your traffic.

iptables/tun on openwrt

All you need to enable IP forwarding and following the above instructions.

announce another host as gateway or customize routing table manually

If your router is powerful enough, just set up proxy in the router. Otherwise, announce the gateway to be a proxy server in DHCP. You may also change the default routing table manually.

ARP spoofing

The downsides for above method is that, you either need control to the router or you need to change a few things manually on each device. fqrouter had a slick trick. It fools the hosts in the LAN to believe that this host is the gateway by ARP spoofing. See here for details.

wiki