Problem creating VLAN interface?

[Dunuin]

Hi,

Already used this hook 2 years ago on Proxmox VE server to unlock a LUKS encrypted LVM via tagged vlan and that is still working fine.
Now I'm trying to set up another Proxmox VE 7.2 ( which is based on Debian 11) server that also needs tagged VLAN but this time to unlock the encrypted ZFS root filesystem dataset.

But with this server this just won't work. After manually unlocking the root filesystem by connecting a keyboard+display and Proxmox VE has booted my network is working fine. This is my /etc/network/intefaces:

auto lo
iface lo inet loopback

iface enp3s0 inet manual

auto enp3s0.43
iface enp3s0.43 inet manual

auto vmbr43
iface vmbr43 inet static
        address 192.168.43.40/24
        gateway 192.168.43.1
        bridge-ports enp3s0.43
        bridge-stp off
        bridge-fd 0

When I check the used driver in Proxmox VE it is reporting to use r8169 and the chip is a Realtek RTL8111GR.

My /etc/initramfs-tools/initramfs.conf looks like this:

VLAN="enp3s0:43"
IP=192.168.43.40::192.168.43.1:255.255.255.0:J3710Unlock:enp3s0.43:off:192.168.43.1

The old server, using a Mellanox Connectx-3, and this config works fine:

VLAN="ens5:43"
IP=192.168.43.50::192.168.43.1:255.255.255.0::ens5.43:off

So the new server uses basically the same, just another NIC, a hostname and an additional DNS server IP.

But when I reboot it looks like initramfs is failing to create the VLAN interface:
Second 4 to 5.5:

...
Running /scripts/init-premount ... done.
Mounting root filesystem ... Begin: Waiting up to 180 secs for enp3s0.43 to become available ... Begin: Running /scripts/nfs-top ... done.
Running /scripts/nfs-premount ... done.
Running /scripts/local-top ... Begin: Bringing up enp3s0.43 ... ip: RTNETLINK answers: Operation not supported
Generic FE-GE Realtec PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC)
r8169 0000:03:00.0 enp3s0: Link Is Down
ip: SIOCGIFFLAGS: No such device
done.
done
...
Begin: Importing ZFS root pool 'rpool' ... Begin: Importing pool 'rpool' using defaults ... done.
done.
Enter passphrase for 'rpool/ROOT':
r8169 0000:03:00.0 enp3s0: Link is Up - 1Gbps/Full - flow control off
IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0: link becomes ready

Then a long time nothing and then 10x:

ipconfig: enp3s0.43: SIOCGIFINDEX: No such device
ipconfig: no devices to configure

And a single:
/scripts/init-premount/dropbear: .: line 329: can't open '/run/net-enp3s0.43.conf' : No such file or directory

When I skip tagged VLAN and use this for the /etc/initramfs-tools/initramfs.conf and set the switch to untagged VLAN 43 everything works fine:

IP=192.168.43.40::192.168.43.1:255.255.255.0:J3710Unlock:enp3s0:off:192.168.43.1

What could be the problem here? Looks like initramfs and Proxmox VE are using the same r8169 driver but somehow initramfs still can't create a vlan interface?

I'm also not sure how to add a driver to initramfs using modprobe. Do I just need to "modprobe r8169" to the /etc/initramfs-tools/initramfs.conf?

[Dunuin]

I installed a new server. Same problem as last time but this time it gets stuck with a endless loop of this after unlocking the root filesystem through the physical console:

Before unlocking the root filesystem it looks like this:

I've added this to the initramfs.conf:

VLAN="ens3:43"
IP=192.168.43.50::192.168.43.1:255.255.255.0:EnterpriseUnlock:ens3.43:off:192.168.43.1

Where ens3 is my ConnectX-3 NIC. Used the same NIC in the old server, where it was called ens5 with this config, and there everything was working fine:

VLAN="ens5:43"
IP=192.168.43.50::192.168.43.1:255.255.255.0::ens5.43:off

What is the best way to recover from this so I can boot my OS and change/disable the config?

[Dunuin]

In case someone also screws up the initramfs and needs to rebuild it to get the linux boot again. You need to boot another linux, chroot into the non-bootlable linux and then rebuild the initramfs. I documented it there: https://forum.proxmox.com/threads/fix-stuck-initramfs.120192/post-522569

Still the question why VLANs are not working and it fails with "RTNETLINK answers: Operation not supported" on both of my servers.

[stcz]

Sorry, but because I'm not using the hook anymore and a lack of time, I'm not able to maintain this repo anymore.

Please fork and start developing :)

[williamdes]

@Dunuin I reported the sleep and cat issue to https://bugs.debian.org/1033802
But until now we did not find an easy way to reproduce the bug.

[malakudi]

The sleep/cat issue is when initramfs has gone (boot continued) and dropbear is stuck in loop inside wait_for_dropbear()

The problem you had is probably with the sourcing of conf.d files, which breaks the script execution if there is no conf file there. I fixed this in #18

[williamdes]

Thank you for making a fix !

The problem you had is probably with the sourcing of conf.d files

But for that, I am not sure. There was only an interface name change involved

[williamdes]

You might find out what dot wrong if you read my message https://bugs.debian.org/1033802#73

Update December 2023, Using "debian-12.2.0-amd64-netinst.iso":

The system was installed by un checking all boxes and only checking "ssh server". Sleep and cat are found on the booted system.

I created a new VM for a client on Proxmox and managed to fall into this sleep cat not found bug.
> I had put an interface name: ens9.123 thinking it would take the VLAN tag.
But it triggered the crash. Removing the ".123" fixes it.
That said, sleep and cat are still not into the initramfs image.

For me all this is related to the network interface name in the IP= setting. But there is much more comments on the thread

[malakudi]

initramfs networking does not support vlans. Adding the files from this repo enables that. But without a .conf file in conf.d, the vlan script would early teminate because the line . conf/conf.d/*.conf returns with an exit code of 1 if no files found and script exits at that point so vlan is never configured. The naming of the interface in IP= as ens9.123 was correct, if the vlan script would load correctly

In your older test you probably had older versions of those files which didn't have this problem

[malakudi]

@williamdes if you want check again with my pull request applied (or download from my repo - fix-conf.d-parsing branch)

[williamdes]

The thing is that I never used this project, only stock Debian. Or is it already included in Debian?

[malakudi]

It is not included in Debian and without this project, you cannot use vlan on initramfs