What the ISP offers
I’m using NTT flets as physical provider (fiber), and Biglobe for the packets ontop. Regarding IPv6, Biglobe allows on a webfrontent to request v6, which I did. Then they offer 2 options:
- IPv6オプション (“option”)
- IPv6オプションライト(“option light”)
I was first on the former, then switched to the latter. Their site on this. With both of these options, I get IPv4 via PPPoE tunnel, and IPv6 plainly onto my ethernet interface facing towards the ONU/modem.
Basic network setup of PPPoE, for IPv4 connectivity
With that, we can swap both NVMe and boot from the new system. No network configured right now, so access via serial console, details here. The previous distro on my Star64 router was Ubuntu based and I used NetworkManager (NM), which seemed preferred by the distro, and I’m also somewhat familiar with NM. It made setting up a Wifi AccessPoint very simple. As a downside: I never got the PPPoE session properly integrated with NM, so I always controlled the PPPoE interface in executing “pon dsl-provider” to activate and “poff” to bring down the interface.
The Debian way for Bookworm and later seems to be systemd-networkd right now, so I’m going with that. My ethernet devices appear as “end0” and “end1”, and wifi as “wlx14f5f9517bcc”.
### Remove the old interfaces file out of the way
mv /etc/network/interfaces /etc/network/interfaces.save
systemctl enable systemd-networkd
cd /etc/systemd/network
vi end1.network
cat end1.network
[Match]
Name=end1
[Network]
Description=Link to the LAN for my clients
[Address]
Address=fc00::1/64
[Address]
Address=192.168.1.1/24
networkctl reload
This the LAN facing interface, just setting a static v4 and v6 address. After rebooting, this becomes active, “networkctl” and “ip addr s” confirm.
PPPoE and NAT setup
The pppoeconf tool does the setup for us - it’s working outside of systemd-networkd and setting up pieces in /etc/network/interfaces, so pppoe is started after reboots.
pppoeconf
# At this point we have internet access, and can set our clock:
ntpdate ntp1.ptb.de
Setting up IPv4 NAT for the LAN clients, and Wifi clients on 192.168.5.0/24 network:
echo 'net.ipv4.ip_forward = 1' >/etc/sysctl.d/20-cust.conf
echo 'net.ipv6.conf.all.forwarding = 1' >>/etc/sysctl.d/20-cust.conf
sysctl -p /etc/sysctl.d/20-cust.conf
iptables -t nat -o ppp0 -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE
iptables -t nat -o ppp0 -A POSTROUTING -s 192.168.5.0/24 -j MASQUERADE
# Adding the last 2 commands to /etc/rc.local to survive reboots
vi /etc/rc.local
Manual triggering of the PPPoE session with “pon dsl-provider” and “poff” works then. The user/password for the pppoeconf tool are the ones used for the data ISP, here Biglobe. pppoe is then getting listed by networkctl, but is not fully managed by systemd-networkd:
$ networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 end0 ether routable configured
3 end1 ether routable configured
4 wlx14f5f9517bcc wlan routable configured
5 ppp0 ppp routable unmanaged
5 links listed.
IPv6 for router and LAN
Now it’s getting ugly.. I struggle with this forever. All ISP’s seem to do this differently. One needs to find out what the ISP is offering, how to get v6 for the router, and then how to best make it available for the LAN.
In my case: first step is requesting v6 with the ISP, via a web frontend from the ISP. Once the ISP has enabled IPv6 from their side, the kernel recognizes announcements and configures interface end0 - even if the PPPoE session is down:
[root@star64 60c ~]$ ip addr s
[..]
2: end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 6c:cf:39:00:75:5f brd ff:ff:ff:ff:ff:ff
inet6 2404:7a80:3820:3700:6ecf:39ff:fe00:755f/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 2591978sec preferred_lft 604778sec
inet6 fe80::6ecf:39ff:fe00:755f/64 scope link
valid_lft forever preferred_lft forever
From that point on, I can do IPv6 traffic from the router/star64.
I then had a journey through various tools, trying to understand what the ISP is offering, and how to best make IPv6 available to the systems on the LAN. The inconvenient truth is that NAT with IPv6 is the only thing I got working. These are the options:
Option 1: Subnet delegation, via systemd-networkd
Best option would be the ISP to provide a delegation of a /56 or bigger, which could then be used from the LAN. The router would then plainly route through. I sniffed traffic on the end0 interface right after activating the interface, seems like I get. One would use “radvd” to advertise the network to the LAN clients, activate forwarding on the router. I had no luck with requesting subnet delegation.
With
[Network]
DHCP=yes
[DHCPv6]
PrefixDelegationHint=::/56
in my end0.network interface config file. The ISP facing interfact ends then up like this:
2: end0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 6c:cf:39:00:75:5f brd ff:ff:ff:ff:ff:ff
inet6 2404:7a80:3820:3700:6ecf:39ff:fe00:755f/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 2591944sec preferred_lft 604744sec
inet6 fe80::6ecf:39ff:fe00:755f/64 scope link
valid_lft forever preferred_lft forever
..which looks like I got a /64 network. But then, adding
[Network]
IPv6SendRA=yes
DHCPv6PrefixDelegation=yes
to the interface facing to the LAN, I do not get networks announced on the systems connected on the LAN side.
Option 2: Subnet delegation, via wide-dhcpv6-client
This is another option to request a subnet from the ISP, instead of having systemd-networkd request the subnet. Package wide-dhcpv6-client is available in Debian repos. In removing file /etc/systemd/network/end0.network, I can take interface end0 out of systemd-networkd’s management, and try wide-dhcpv6-client:
[root@star64 58c ~]$ cat /etc/wide-dhcpv6/dhcp6c.conf
interface end0 {
send ia-pd 1;
send ia-na 1;
};
id-assoc na 1 {
};
id-assoc pd 1 {
# You should also try /56 and /64 here, but for me no
# difference, unfortunatelly.
prefix ::/48 infinity;
prefix-interface end0 {
sla-id 1;
sla-len 8;
};
prefix-interface end1 {
sla-id 2;
sla-len 8;
};
};
Yet, after starting /etc/init.d/wide-dhcpv6-client, my interface just gets a link-local address. Not even the SLAAC configured 2404[..] address. I can run wide-dhcpv6 in debug mode, and it seems like no PD is offered by the ISP:
/usr/sbin/dhcp6c -Pdefault -c /etc/wide-dhcpv6/dhcp6c.conf -D -f end0
[..]
Oct/06/2024 15:21:25: dhcp6_get_options: get DHCP option status code, len 16
Oct/06/2024 15:21:25: status code: no prefixes
Oct/06/2024 15:21:25: client6_recvadvert: server ID: 00:03:00:01:00:[..], pref=-1
Oct/06/2024 15:21:25: client6_recvadvert: advertise contains NoPrefixAvail status
Dumping the traffic and investigating with tshark, I see my request for delegation, and the denial from the other side:
$ tshark -r 2_dump_wide -V|
[..]
Identity Association for Prefix Delegation
Option: Identity Association for Prefix Delegation (25)
Length: 41
IAID: 00000001
T1: 0
T2: 0
IA Prefix
Option: IA Prefix (26)
Length: 25
Preferred lifetime: infinity
Valid lifetime: infinity
Prefix length: 48
Prefix address: ::
[..]
Status code
Option: Status code (13)
Length: 16
Status Code: NoPrefixAvail (6)
Status Message: NOPREFIX-AVAIL
Also re questing /56 or /64 subnets makes no difference.
Option 3: Subnet delegation, via network scripts
After installing package isc-dhcp-client, I can also use
iface end0 inet6 auto
dhcp 1
request_prefix 1
in /etc/network/interfaces to get a prefix. After a “ifup end0”, I get the 2404[..] address, but no network.
Option 4: DHCPv6 relay via wide-dhcpv6-relay
The idea is here to have
- LAN clients issue DHCPv6 requests
- have the router relay these to the ISP facing network
- and have the reply also relayed to the LAN client, so it can configure it’s interface
- then also ensure routing/proxying for normal v6 traffic from the LAN client
After installing package wide-dhcpv6-relay on the router, it seems like relaying is done, just that the IPv6 interface on the LAN client is not getting an address:
# on router
[root@star64 59c network]$ dhcp6relay -D -f -r end0 end1
Oct/08/2024 07:48:15: main: dhcp6relay started
Oct/08/2024 07:49:35: relay6_recv: from fe80::9ad:54d1:c1ef:d93%end1, size 85
Oct/08/2024 07:49:35: relay6_recv: received solicit from fe80::9ad:54d1:c1ef:d93%end1
Oct/08/2024 07:49:35: copy_option: set relay message (len 85)
Oct/08/2024 07:49:35: copy_option: set interface ID (len 4)
Oct/08/2024 07:49:35: relay_to_server: relay a message to a server ff05::1:3
Oct/08/2024 07:49:36: relay6_recv: from fe80::9ad:54d1:c1ef:d93%end1, size 85
Oct/08/2024 07:49:36: relay6_recv: received solicit from fe80::9ad:54d1:c1ef:d93%end1
Oct/08/2024 07:49:36: copy_option: set relay message (len 85)
Oct/08/2024 07:49:36: copy_option: set interface ID (len 4)
Oct/08/2024 07:49:36: relay_to_server: relay a message to a server ff05::1:3
# on LAN client
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 98:fa:9b:91:06:37 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global noprefixroute enp0s31f6
valid_lft forever preferred_lft forever
inet6 fe80::9ad:54d1:c1ef:d93/64 scope link noprefixroute
valid_lft forever preferred_lft forever
Option 5: DHCPv6 relay via dibbler-relay
Also with this, the interface on the LAN system is not getting an address:
# tail -f /var/log/dibbler/dibbler-relay.log
14:41 Relay Notice Received SOLICIT on end1/3,trans-id=0xea650, 6 opts: 14 3 39 6 1 8
14:41 Relay Debug Interface-id option added before relayed message.
14:41 Relay Notice Relaying encapsulated SOLICIT message on the end0/2 interface to multicast (ff05::1:3) address, port 547.
14:41 Relay Debug Accepting messages.
14:42 Relay Debug Received 85 bytes on the end1/3 interface (socket=0, addr=fe80::9ad:54d1:c1ef:d93, port=547).
14:42 Relay Notice Received SOLICIT on end1/3,trans-id=0xea650, 6 opts: 14 3 39 6 1 8
14:42 Relay Debug Interface-id option added before relayed message.
14:42 Relay Notice Relaying encapsulated SOLICIT message on the end0/2 interface to multicast (ff05::1:3) address, port 547.
14:42 Relay Debug Accepting messages.
Option 6: NDP proxy
Next best option is the router acting as IPv6 NDP proxy: this would proxy IPv6 config requests from the LAN to the ISP, and also answers. I installed ndppd and played around, without success.
Option 7: IPv6 NAT/masquerading
My last resort: a static IPv6 IP on the end1 interface which is facing the LAN, and static IPv6 IP on the LAN system. The router is then getting configured for NAT:
# ip6tables -t nat -A POSTROUTING -s fc00::2 -j MASQUERADE
This state is not satisfying.. we should setup an exhaustive list of debug steps for IPv6, to find out what an ISP is offering and then investigate the options.
Last modified on 2024-10-06