Bringing IPv6 into the LAN Blues

What the ISP offers

I’m using NTT flets as physical provider (fiber), and Biglobe for the packets ontop. Regarding IPv6, Biglobe allows on a webfrontent to request v6, which I did. Then they offer 2 options:

IPv6オプション (“option”)
IPv6オプションライト(“option light”)

I was first on the former, then switched to the latter. Their site on this. With both of these options, I get IPv4 via PPPoE tunnel, and IPv6 plainly onto my ethernet interface facing towards the ONU/modem.

Basic network setup of PPPoE, for IPv4 connectivity

With that, we can swap both NVMe and boot from the new system. No network configured right now, so access via serial console, details here. The previous distro on my Star64 router was Ubuntu based and I used NetworkManager (NM), which seemed preferred by the distro, and I’m also somewhat familiar with NM. It made setting up a Wifi AccessPoint very simple. As a downside: I never got the PPPoE session properly integrated with NM, so I always controlled the PPPoE interface in executing “pon dsl-provider” to activate and “poff” to bring down the interface.

The Debian way for Bookworm and later seems to be systemd-networkd right now, so I’m going with that. My ethernet devices appear as “end0” and “end1”, and wifi as “wlx14f5f9517bcc”.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
### Remove the old interfaces file out of the way
mv /etc/network/interfaces /etc/network/interfaces.save
systemctl enable systemd-networkd
cd /etc/systemd/network
vi end1.network
cat end1.network
[Match]
Name=end1

[Network]
Description=Link to the LAN for my clients

[Address]
Address=fc00::1/64

[Address]
Address=192.168.1.1/24

networkctl reload

This the LAN facing interface, just setting a static v4 and v6 address. After rebooting, this becomes active, “networkctl” and “ip addr s” confirm.

PPPoE and NAT setup

The pppoeconf tool does the setup for us - it’s working outside of systemd-networkd and setting up pieces in /etc/network/interfaces, so pppoe is started after reboots.

1
2
3
pppoeconf
# At this point we have internet access, and can set our clock:
ntpdate ntp1.ptb.de

Setting up IPv4 NAT for the LAN clients, and Wifi clients on 192.168.5.0/24 network:

1
2
3
4
5
6
7
echo 'net.ipv4.ip_forward = 1' >/etc/sysctl.d/20-cust.conf
echo 'net.ipv6.conf.all.forwarding = 1' >>/etc/sysctl.d/20-cust.conf
sysctl -p /etc/sysctl.d/20-cust.conf
iptables -t nat -o ppp0 -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE
iptables -t nat -o ppp0 -A POSTROUTING -s 192.168.5.0/24 -j MASQUERADE
# Adding the last 2 commands to /etc/rc.local to survive reboots
vi /etc/rc.local

Manual triggering of the PPPoE session with “pon dsl-provider” and “poff” works then. The user/password for the pppoeconf tool are the ones used for the data ISP, here Biglobe. pppoe is then getting listed by networkctl, but is not fully managed by systemd-networkd:

1
2
3
4
5
6
7
8
9
$ networkctl
IDX LINK            TYPE     OPERATIONAL SETUP
  1 lo              loopback carrier     unmanaged
  2 end0            ether    routable    configured
  3 end1            ether    routable    configured
  4 wlx14f5f9517bcc wlan     routable    configured
  5 ppp0            ppp      routable    unmanaged

5 links listed.

IPv6 for router and LAN

Now it’s getting ugly.. I struggle with this forever. All ISP’s seem to do this differently. One needs to find out what the ISP is offering, how to get v6 for the router, and then how to best make it available for the LAN.

In my case: first step is requesting v6 with the ISP, via a web frontend from the ISP. Once the ISP has enabled IPv6 from their side, the kernel recognizes announcements and configures interface end0 - even if the PPPoE session is down:

1
2
3
4
5
6
7
8
[root@star64 60c ~]$ ip addr s
[..]
2: end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 6c:cf:39:00:75:5f brd ff:ff:ff:ff:ff:ff
    inet6 2404:7a80:3820:3700:6ecf:39ff:fe00:755f/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2591978sec preferred_lft 604778sec
    inet6 fe80::6ecf:39ff:fe00:755f/64 scope link
       valid_lft forever preferred_lft forever

From that point on, I can do IPv6 traffic from the router/star64.

I then had a journey through various tools, trying to understand what the ISP is offering, and how to best make IPv6 available to the systems on the LAN. The inconvenient truth is that NAT with IPv6 is the only thing I got working. These are the options:

Option 1: Subnet delegation, via systemd-networkd

Best option would be the ISP to provide a delegation of a /56 or bigger, which could then be used from the LAN. The router would then plainly route through. I sniffed traffic on the end0 interface right after activating the interface, seems like I get. One would use “radvd” to advertise the network to the LAN clients, activate forwarding on the router. I had no luck with requesting subnet delegation.

With

1
2
3
4
5
[Network]
DHCP=yes

[DHCPv6]
PrefixDelegationHint=::/56

in my end0.network interface config file. The ISP facing interfact ends then up like this:

1
2
3
4
5
6
2: end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 6c:cf:39:00:75:5f brd ff:ff:ff:ff:ff:ff
    inet6 2404:7a80:3820:3700:6ecf:39ff:fe00:755f/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2591944sec preferred_lft 604744sec
    inet6 fe80::6ecf:39ff:fe00:755f/64 scope link
       valid_lft forever preferred_lft forever

..which looks like I got a /64 network. But then, adding

1
2
3
[Network]
IPv6SendRA=yes
DHCPv6PrefixDelegation=yes

to the interface facing to the LAN, I do not get networks announced on the systems connected on the LAN side.

Option 2: Subnet delegation, via wide-dhcpv6-client

This is another option to request a subnet from the ISP, instead of having systemd-networkd request the subnet. Package wide-dhcpv6-client is available in Debian repos. In removing file /etc/systemd/network/end0.network, I can take interface end0 out of systemd-networkd’s management, and try wide-dhcpv6-client:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
[root@star64 58c ~]$ cat /etc/wide-dhcpv6/dhcp6c.conf
interface end0 {
 send ia-pd 1;
 send ia-na 1;
};

id-assoc na 1 {
};

id-assoc pd 1 {
 # You should also try /56 and /64 here, but for me no
 # difference, unfortunatelly.
 prefix ::/48 infinity;
 prefix-interface end0 {
  sla-id 1;
  sla-len 8;
 };
 prefix-interface end1 {
  sla-id 2;
  sla-len 8;
 };
};

Yet, after starting /etc/init.d/wide-dhcpv6-client, my interface just gets a link-local address. Not even the SLAAC configured 2404[..] address. I can run wide-dhcpv6 in debug mode, and it seems like no PD is offered by the ISP:

1
2
3
4
5
6
/usr/sbin/dhcp6c -Pdefault -c /etc/wide-dhcpv6/dhcp6c.conf -D -f end0
[..]
Oct/06/2024 15:21:25: dhcp6_get_options: get DHCP option status code, len 16
Oct/06/2024 15:21:25:   status code: no prefixes
Oct/06/2024 15:21:25: client6_recvadvert: server ID: 00:03:00:01:00:[..], pref=-1
Oct/06/2024 15:21:25: client6_recvadvert: advertise contains NoPrefixAvail status

Dumping the traffic and investigating with tshark, I see my request for delegation, and the denial from the other side:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
$ tshark -r 2_dump_wide -V|
[..]
    Identity Association for Prefix Delegation
        Option: Identity Association for Prefix Delegation (25)
        Length: 41
        IAID: 00000001
        T1: 0
        T2: 0
        IA Prefix
            Option: IA Prefix (26)
            Length: 25
            Preferred lifetime: infinity
            Valid lifetime: infinity
            Prefix length: 48
            Prefix address: ::
[..]
    Status code
        Option: Status code (13)
        Length: 16
        Status Code: NoPrefixAvail (6)
        Status Message: NOPREFIX-AVAIL

Also re questing /56 or /64 subnets makes no difference.

Option 3: Subnet delegation, via network scripts

After installing package isc-dhcp-client, I can also use

1
2
3
iface end0 inet6 auto
        dhcp 1
        request_prefix 1

in /etc/network/interfaces to get a prefix. After a “ifup end0”, I get the 2404[..] address, but no network.

Option 4: DHCPv6 relay via wide-dhcpv6-relay

The idea is here to have

LAN clients issue DHCPv6 requests
have the router relay these to the ISP facing network
and have the reply also relayed to the LAN client, so it can configure it’s interface
then also ensure routing/proxying for normal v6 traffic from the LAN client

After installing package wide-dhcpv6-relay on the router, it seems like relaying is done, just that the IPv6 interface on the LAN client is not getting an address:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# on router
[root@star64 59c network]$ dhcp6relay -D -f -r end0 end1
Oct/08/2024 07:48:15: main: dhcp6relay started
Oct/08/2024 07:49:35: relay6_recv: from fe80::9ad:54d1:c1ef:d93%end1, size 85
Oct/08/2024 07:49:35: relay6_recv: received solicit from fe80::9ad:54d1:c1ef:d93%end1
Oct/08/2024 07:49:35: copy_option: set relay message (len 85)
Oct/08/2024 07:49:35: copy_option: set interface ID (len 4)
Oct/08/2024 07:49:35: relay_to_server: relay a message to a server ff05::1:3
Oct/08/2024 07:49:36: relay6_recv: from fe80::9ad:54d1:c1ef:d93%end1, size 85
Oct/08/2024 07:49:36: relay6_recv: received solicit from fe80::9ad:54d1:c1ef:d93%end1
Oct/08/2024 07:49:36: copy_option: set relay message (len 85)
Oct/08/2024 07:49:36: copy_option: set interface ID (len 4)
Oct/08/2024 07:49:36: relay_to_server: relay a message to a server ff05::1:3

# on LAN client
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 98:fa:9b:91:06:37 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 brd 192.168.1.255 scope global noprefixroute enp0s31f6
       valid_lft forever preferred_lft forever
    inet6 fe80::9ad:54d1:c1ef:d93/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

Option 5: DHCPv6 relay via dibbler-relay

Also with this, the interface on the LAN system is not getting an address:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# tail -f  /var/log/dibbler/dibbler-relay.log
14:41 Relay Notice    Received SOLICIT on end1/3,trans-id=0xea650, 6 opts: 14 3 39 6 1 8
14:41 Relay Debug     Interface-id option added before relayed message.
14:41 Relay Notice    Relaying encapsulated SOLICIT message on the end0/2 interface to multicast (ff05::1:3) address, port 547.
14:41 Relay Debug     Accepting messages.
14:42 Relay Debug     Received 85 bytes on the end1/3 interface (socket=0, addr=fe80::9ad:54d1:c1ef:d93, port=547).
14:42 Relay Notice    Received SOLICIT on end1/3,trans-id=0xea650, 6 opts: 14 3 39 6 1 8
14:42 Relay Debug     Interface-id option added before relayed message.
14:42 Relay Notice    Relaying encapsulated SOLICIT message on the end0/2 interface to multicast (ff05::1:3) address, port 547.
14:42 Relay Debug     Accepting messages.

Option 6: NDP proxy

Next best option is the router acting as IPv6 NDP proxy: this would proxy IPv6 config requests from the LAN to the ISP, and also answers. I installed ndppd and played around, without success.

Option 7: IPv6 NAT/masquerading

My last resort: a static IPv6 IP on the end1 interface which is facing the LAN, and static IPv6 IP on the LAN system. The router is then getting configured for NAT:

1
# ip6tables -t nat -A POSTROUTING -s fc00::2 -j MASQUERADE

This state is not satisfying.. we should setup an exhaustive list of debug steps for IPv6, to find out what an ISP is offering and then investigate the options.