Bridging Wired and Wireless Networks, Gentoo-style

I want my wired and wireless networks to share a single 192.168.1.x address space (instead of separate 192.168.0.x and 192.168.1.x addresses).

In order to do that, we need to set up a bridge to merge disparate networks into a single space.

Part 1: The Basic Configuration

ADMtek NC100 (uses tulip driver)
Ralink RT61 PCI (uses rt61pci driver)
hostapd
linux 4.1.15-gentoo-r1
net-misc/bridge-utils 1.5
net-wireless/iw 3.17

Part 2: Making It Work

I started out creating a basic bridge, using the Gentoo Wiki as a guide:

cd /etc/init.d
ln -s net.lo net.br0

/etc/init.d/net.br0 start

There’s no need to change how hostapd starts; it still talks to wlan0 (not br0).

# /etc/conf.d/net

modules_wlan0="!iwconfig !wpa_supplicant"
config_wlan0="null"
config_eth0="null"
config_br0="192.168.1.1/24"
brctl_br0="setfd 0
sethello 10
stp off"
bridge_br0="eth0 wlan0"

The Problem

The above config is naive and doesn’t work right.  I got this error:

Can't add wlan0 to bridge br0: Operation not supported

Huh.  There’s nothing indicative in dmesg about the error, the last entry shows the bridge being created on the wired card and then being taken down.  Just to be sure, I created a bridge with just eth0 and it worked:

$ brctl show
bridge name   bridge id           STP enabled   interfaces
br0           8000.00045a42a698   no            eth0

After casting about a bit, I found a serverfault.com page that pointed to this fix:

$ iw dev wlan0 set 4addr on
$ brctl addif br0 wlan0

That works, but that won’t do me much good as a long-term solution.  I would need to pay a visit to the basement after every planned reboot and unplanned power outage, or else nobody can get onto the network.

( More about the 4addr option here. )

You can’t just add the option to modules_wlan0, it doesn’t work that way.  A quick visit back to the wiki suggested the solution, though, which is to define a preup function where we can execute arbitrary commands.

The Working Config

These statements are in addition to the WAN interface config:

# /etc/conf.d/net
modules_wlan0="!iwconfig !wpa_supplicant"
config_wlan0="null"
config_eth0="null"
config_br0="192.168.1.1/24"
brctl_br0="setfd 0
sethello 10
stp off"
bridge_br0="eth0 wlan0"

preup() {
    # br0 uses wlan0, and wlan0 needs to set the
    # 4addr option before being used on a bridge
    if echo "${IFACE}" | grep -q 'br0' ; then
        /usr/sbin/iw dev wlan0 set 4addr on
    fi

    return 0
}

Then do all the accounting to clean up:

rc-update add net.br0 default
rc-update del net.eth0 default
rc-update del net.wlan0 default

I also had to update my iptables config to refer to br0 instead of eth0 and wlan0.

Finally, a reboot to test that everything starts properly.

Setting up a Gentoo-Based Dual-Stack Router

Our network has been based around a home-built router for quite some time, ever since I got fed up with the crappy ActionTec router that Verizon bundled with our FiOS service. (If you’re going to offer high-speed internet, you should probably bundle equipment that can actually keep up.) I had originally followed a slightly older version of these instructions to get a nice basic router going. But I finally wanted better. I wanted the bright, shiny, new thing. I wanted IPv6.

So, here’s my instructions for going from an existing IPv4 router to dual-stack IPv4/6.

Note: I am using dnsmasq for DNS and DHCP, hostapd for wireless management, and an iptables firewall. Since Verizon still doesn’t widely support consumer IPv6, I’m using a tunnel broker to get my /6 address. If you’re using a different setup your mileage may vary. If you find anything that I appear to have forgotten, please let me know!

Step 1: Recompile the Kernel

This should be obvious: if you want to run ipv6 you need ipv6 support in your kernel. In order to trim as much off my kernel as possible I did not have it built in, and had to recompile.

You should also add netfilter support for ipv6 so that your firewall will work.

Networking support  --->
    Networking options  --->
        <*>   The IPv6 protocol  --->
            <*>   IPv6: IPv6-in-IPv4 tunnel (SIT driver)
        [*] Network packet filtering framework (Netfilter)  --->
            IPv6: Netfilter Configuration  --->
                <M> IPv6 NAT
                <M> IP6 tables support (required for filtering)
                <M>   Packet filtering
                <M>   ip6tables NAT support
                <M>     MASQUERADE target support
                ... other filtering options as you may need for your situation

Step 2: Update Your IPv6 Support

Again, it was never compiled in, in order to trim off unused bits of code. Add ‘ipv6’ to your USE variable and emerge --newuse world

Step 3: Install network tools (if they aren’t already)

emerge --noreplace sys-apps/iproute2 net-firewall/iptables

Step 4 (optional): Set up your tunnel

If your ISP doesn’t provide ipv6, and many don’t, you need to request an address range from a tunnel broker. I’m using Hurricane Electric, which is free, but there are others — see this list or just google it.

If you have multiple machines on your network (which is assumed, since this is a router guide), you may prefer a /48, so that autoconfig works nicely, instead of the default /64. This guide assumes a /48.

Going forward, replace 2001:470:891a: with your own /48 range.

Now activate your tunnel:

ip tunnel add he-ipv6 mode sit remote 1.2.3.4 local 5.6.7.8 ttl 255
ip link set he-ipv6 up
ip addr add 2001:470:1f06:2a3::2/64 dev he-ipv6
ip route add ::/0 dev he-ipv6
ip -f inet6 addr

Step 5: Update Your Net Config

I have two wired and one wireless card in my router. Here’s what my /etc/conf.d/net looks like:

# enp2s0 is my exterior wired nic (aka public facing)
# enp3s5 is my interior wired nic
# wlp3s6 is my interior wireless nic

dhcp_enp2s0="nodns" # we choose our own DNS, tyvm

config_enp3s5="192.168.0.1/24 2001:470:891a:0::/64"

modules_wlp3s6="!iwconfig !wpa_supplicant"
config_wlp3s6="192.168.1.1/24 2001:470:891a:1::/48"
dns_servers_wlp3s6="127.0.0.1"

After making appropriate changes, restart your NICs. If you’re working remotely, you may want to be connected via two paths instead of just one (so when you inevitably get bounced and can’t reconnect, you still have a way back in).

A properly-configured set of addresses looks like this:

# ip addr
1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp2s0: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:e0:4d:bf:03:f5 brd ff:ff:ff:ff:ff:ff
    inet 108.20.118.17/24 brd 108.20.118.255 scope global enp2s0
       valid_lft forever preferred_lft forever
    inet6 fe80::cbdf:25c0:c948:f4bb/64 scope link
       valid_lft forever preferred_lft forever
3: enp3s5: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 00:04:5a:42:a6:98 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/24 brd 192.168.0.255 scope global enp3s5
       valid_lft forever preferred_lft forever
    inet6 2001:470:891a::/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::204:5aff:fe42:a698/64 scope link
       valid_lft forever preferred_lft forever
4: wlp3s6: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:1a:ef:07:4d:a7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global wlp3s6
       valid_lft forever preferred_lft forever
    inet6 2001:470:891a:1::/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::21a:efff:fe07:4da7/64 scope link
       valid_lft forever preferred_lft forever
5: sit0@NONE:  mtu 1480 qdisc noop state DOWN group default
    link/sit 0.0.0.0 brd 0.0.0.0
6: he-ipv6@NONE: &lt;POINTOPOINT,NOARP,UP,LOWER_UP&gt; mtu 1480 qdisc noqueue state UNKNOWN group default
    link/sit 108.20.118.17 peer 209.51.161.14
    inet6 2001:470:1f06:2a3::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::6c14:7611/64 scope link
       valid_lft forever preferred_lft forever

Test it with a ping:

ping6 www.kame.net

Step 6: Reconfigure dnsmasq

You’ll need to add router advertisments and your new addresses. Rather than hard-coding an address, dnsmasq offers a ‘constructor’ label which figures it out automatically. Here’s the relevant section from my /etc/dnsmasq.conf:

domain-needed
bogus-priv
domain=jonesling.us
dhcp-authoritative
enable-ra
dhcp-range=192.168.0.20,192.168.0.100,12h
dhcp-range=192.168.1.20,192.168.1.100,12h
dhcp-range=192.168.2.20,192.168.2.100,12h
dhcp-range=::,constructor:enp3s5,ra-names,slaac,12h
dhcp-range=::,constructor:wlp3s6,ra-names,slaac,12h
resolv-file=/etc/resolv.dnsmasq
selfmx
enable-ra

And restart it: /etc/init.d/dnsmasq restart

Step 7: Configure your firewall

Pretty much every iptables reference in your firewall config will be mirrored with an ip6tables command.

Here’s my script to set up iptables (if you see an error or something stupid, I would appreciate your criticism – paired with your reasoning on why it should be changed so I can know better for next time).

#!/bin/bash
# based on http://www.gentoo.org/doc/en/home-router-howto.xml

# set to '0' to lock the kids out
OPEN_INTERNET=1

# these systems can get shut out when OPEN_INTERNET isn't true
declare -a NO_SURFING=( 'wii-u'
                        'kids-computer'
                      )

# these systems never get shut out
declare -a OK_SURFING=( 'parents-computer'
                        'parents-phone'
                      )

# these ports take precedence over CLOSED_PORTS
declare -a OPEN_TCP_PORTS=( 'ssh'
                            'http'
                            'mail'
                            'submission'
                          )

declare -a OPEN_UDP_PORTS=( 'submission' )

# if the port is meant to be closed, we close tcp *AND* udp
declare -a CLOSED_PORTS=( '0:1055'
                          'svn'
                          'distcc'
                          'x11'
                          'nfs'
                          'icpv2'
                          'mysql'
                          'rtsp'
                          '3128' # squid
                          '3130' # squid ICP
                          '3551' # nisport
                        )

declare -a LAN_SERVICES=( "svn" )

# blacklisted IPs and ranges
# http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xhtml
declare -a IP_BLACKLIST=( # APINIC
                          # AFRINIC
                          # LACNIC
                          ...
                        )

LAN=enp3s5
WLAN=wlp3s6
WAN=enp2s0
SIT=he-ipv6

INSIDE=( $LAN $WLAN )

LOCAL_RANGE_IPV4='192.168.0.0/16'
LOCAL_RANGE_IPV6='2001:470:891a::'

# First we flush our current rules
iptables -F
iptables -t nat -F
ip6tables -F
ip6tables -t nat -F

# Setup default policies to handle unmatched traffic
iptables  -P INPUT ACCEPT
iptables  -P OUTPUT ACCEPT
iptables  -P FORWARD DROP
ip6tables -P INPUT ACCEPT
ip6tables -P OUTPUT ACCEPT
ip6tables -P FORWARD DROP

# Then we lock our services so they only work from the LAN
iptables  -I INPUT 1 -i ${LAN}  -j ACCEPT
iptables  -I INPUT 1 -i ${WLAN} -j ACCEPT
iptables  -I INPUT 1 -i lo      -j ACCEPT
ip6tables -I INPUT 1 -i ${LAN}  -j ACCEPT
ip6tables -I INPUT 1 -i ${WLAN} -j ACCEPT
ip6tables -I INPUT 1 -i lo      -j ACCEPT

# block members of IP_BLACKLIST, plus any addresses passed in on the
# command line
for IP in ${IP_BLACKLIST[@]} ; do
    iptables -I INPUT -s ${IP} -p TCP --dport ssh -j DROP
done

for IP in $@; do
    iptables -I INPUT -s ${IP} -d 0/0 -j REJECT
done

iptables  -A INPUT -p UDP --dport bootps -i ${WAN} -j REJECT
ip6tables -A INPUT -p UDP --dport bootps -i ${SIT} -j REJECT
iptables  -A INPUT -p UDP --dport domain -i ${WAN} -j REJECT
ip6tables -A INPUT -p UDP --dport domain -i ${SIT} -j REJECT

# Explicitly allow access to services on the WAN
for SERVICE in ${LAN_SERVICES[@]} ; do
    for IFACE in ${INSIDE[@]} ; do
        iptables  -A INPUT -p TCP --dport svn -i ${IFACE} -j ACCEPT
        iptables  -A INPUT -p UDP --dport svn -i ${IFACE} -j ACCEPT
        ip6tables -A INPUT -p TCP --dport svn -i ${IFACE} -j ACCEPT
        ip6tables -A INPUT -p UDP --dport svn -i ${IFACE} -j ACCEPT
    done
done

# Allow access to our server from the WAN
for PORT in ${OPEN_TCP_PORTS[@]} ; do
    iptables  -A INPUT -p TCP --dport $PORT -i ${WAN} -j ACCEPT
    ip6tables -A INPUT -p TCP --dport $PORT -i ${SIT} -j ACCEPT
done

for PORT in ${OPEN_UPD_PORTS[@]} ; do
    iptables  -A INPUT -p UDP --dport PORT -i ${WAN} -j ACCEPT
    ip6tables -A INPUT -p UDP --dport PORT -i ${SIT} -j ACCEPT
done

# Drop TCP / UDP packets to privileged ports
for PORT in ${CLOSED_PORTS[@]} ; do
    iptables  -A INPUT -p TCP -i ${WAN} -d 0/0 --dport ${PORT} -j DROP
    ip6tables -A INPUT -p TCP -i ${SIT} -d 0/0 --dport ${PORT} -j DROP

    iptables  -A INPUT -p UDP -i ${WAN} -d 0/0 --dport ${PORT} -j DROP
    ip6tables -A INPUT -p UDP -i ${SIT} -d 0/0 --dport ${PORT} -j DROP
done

iptables  -I FORWARD -i ${LAN} -d $LOCAL_RANGE_IPV4 -j ACCEPT
iptables  -A FORWARD -i ${LAN} -s $LOCAL_RANGE_IPV4 -j ACCEPT
ip6tables -I FORWARD -i ${LAN} -d $LOCAL_RANGE_IPV6 -j ACCEPT
ip6tables -A FORWARD -i ${LAN} -s $LOCAL_RANGE_IPV6 -j ACCEPT

if (( OPEN_INTERNET )); then
    echo 'yay, everybody gets internet'
    iptables  -I FORWARD -i ${WLAN} -d $LOCAL_RANGE_IPV4 -j ACCEPT
    iptables  -A FORWARD -i ${WLAN} -s $LOCAL_RANGE_IPV4 -j ACCEPT
    ip6tables -I FORWARD -i ${WLAN} -d $LOCAL_RANGE_IPV6 -j ACCEPT
    ip6tables -A FORWARD -i ${WLAN} -s $LOCAL_RANGE_IPV6 -j ACCEPT
else
    echo "boo, only ${OK_SURFING[@]} get internet"
    for IP in ${OK_SURFING[@]}; do
        iptables -I FORWARD -i ${WLAN} -d $IP/255.255.255.255 -j ACCEPT
        iptables -A FORWARD -i ${WLAN} -s $IP/255.255.255.255 -j ACCEPT
    done
fi

iptables  -A FORWARD -i ${WAN} -d $LOCAL_RANGE_IPV4 -j ACCEPT
ip6tables -A FORWARD -i ${WAN} -d $LOCAL_RANGE_IPV6 -j ACCEPT

iptables  -t nat -A POSTROUTING -o ${WAN} -j MASQUERADE
ip6tables -t nat -A POSTROUTING -o ${SIT} -j MASQUERADE

# This is so when we boot we don't have to run the rules by hand
/etc/init.d/iptables save
/etc/init.d/ip6tables save

# fail2ban should be reloaded after flushing iptables
/etc/init.d/fail2ban reload

Step 8: Update your DNS

Add a AAAA record to your domain’s DNS record.  You may have to keep this one up-to-date yourself.

Interesting to note: you might be thinking “crap, what’s the ipv6 equivalent of these CNAME records?”  Stop worrying, there isn’t.  The CNAME is read like normal, but ipv6 clients will then look up the AAAA (instead of the A) record of the destination host.  It just works.

What?  You built your own router but you don’t have your own domain?  WTF is wrong with you?

Step 9: Reboot your clients

While I was working, I made a bunch of mistakes and my clients had multiple ipv6 addresses – making networking from them unstable as they didn’t necessarily know which address to use. Rebooting will clear them – and make sure your config is proper.

At this point your clients should be in ipv6 and you’re gonna be all excited to see if work.  Browsers take ipv6 addresses a little differently: http://[2001:470:1f06:2a3::2]/

Transferring Large Files

Linux has an impressive tool set, if you know how to use it.  The  philosophy of using simple tools that do one job (but do it well) with the ability to chain commands together using pipes creates a powerful system.

Everyone has to transfer large files across the network on occasion.  scp is an easy choice most of the time, but if you’re working with small or old machines the CPU will be a bottleneck due to encryption.

There are several alternatives to scp, if you don’t need encryption.  These aren’t safe on the open internet but should be acceptable on private networks.  TFTP and rsync come to mind, but they have their limitations.

  • tftp is generally limited to 4 gig files
  • rsync either requires setting up an rsync service, or piping through ssh

My new personal favorite is netcat-as-a-server.  It’s a little more complicated to set up than scp or ftp but wins for overall simplicity and speed of transfer.

netcat doesn’t provide much output, so we’ll put it together with pv (pipeviewer) to tattle on bytes read and written.

First, on the sending machine (the machine with the file), we’ll set up netcat to listen on port 4200, and pv will give us progress updates:
pv -pet really.big.file | nc -q 1 -l -p 4200

  • pv -p prints a progress bar, -e displays ETA, -t enables the elapsed time
  • nc -q 1 quits 1 second after EOF, -l 4200 listens on port 4200

Without the -q switch, the sender will have to be killed with control-c or similar.

On the receiver (the machine that wants the file) netcat will read all bytes until the sender disconnects:
nc file.server.net 4200 | pv -b > really.big.file

  • nc will stream all bytes from file.server.net, port 4200
  • -b turns on the byte counter

Once the file is done transferring, both sides will shut down.