Additional virtual work party/training 'eliminate rob/owen being a SPOF'

[davidelang]

Rob is becoming a SPOF, we need to train more people on how to do everything.

just saying 'it's in git' isn't enough.

[davidelang]

Owen is also a spof on all things switch releated, he started training Scott this year, but we need to expand the knowledge

This can start with documentation/videos, but at some point we need to do a live (could be through zoom) session to have people actually try to do the things.

[owendelong]

Everyone who attended my talk this evening got a good start on this. Owen

…

On Mar 15, 2024, at 10:30, David Lang ***@***.***> wrote: Owen is also a spof on all things switch releated, he started training Scott this year, but we need to expand the knowledge This can start with documentation/videos, but at some point we need to do a live (could be through zoom) session to have people actually try to do the things. — Reply to this email directly, view it on GitHub <#723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAK6GTXTQE3OGJRKFLIOSWTYYMV33AVCNFSM6AAAAABEYMLHE2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBQGEZDMNBRGM>. You are receiving this because you are subscribed to this thread.

[sarcasticadmin]

@kylerisse and @ruebenramirez got onboarded this year for the server infrastructure.

@MrHamel has been a big help on the AP front and Im hoping we can work together more there through 2024

[MrHamel]

Owen isn't the only Juniper netadmin. My paid job is network engineering (and automation). Do I fully understand the convention center patching? No, however, I got better at handling the catwalk.

Anyway, below are a few ideas off the top of my head:

Add jumpboxes to the VM list to access various things. That way, we do not have to be on specific VLANs or need RPis setup in the NOC. Everything is an SSH session away (including various port forwards).

Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop.

[sarcasticadmin]

Owen isn't the only Juniper netadmin. My paid job is network engineering (and automation). Do I fully understand the convention center patching? No, however, I got better at handling the catwalk.

👍

Anyway, below are a few ideas off the top of my head:

Add jumpboxes to the VM list to access various things. That way, we do not have to be on specific VLANs or need RPis setup in the NOC. Everything is an SSH session away (including various port forwards).

There are no vlan requirements in the NOC switch for infra vlan, its untagged. Everything is an SSH session away.

Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop.

The massflash is a livecd, its can run on any x86 machine. The reason its this way is so we dont have to have the hypervisors for a work party. Ive considered the idea of included similar massflash configuration in the core machine but there needs to be more considerations before its implemented.

[davidelang]

On Thu, 21 Mar 2024, Robert James Hernandez wrote: > Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop. The massflash is a livecd, its can run on any x86 machine. The reason its this way is so we dont have to have the hypervisors for a work party. Ive considered the idea of included similar massflash configuration in the `core` machine but there needs to be more considerations before its implemented.

I would love to have massflash as a pi image as well. given our experience this year of various things not working through dhcp relays, etc. I would really like us to be able to bring up a virual replica of the show at the work parties, including the hypervisors, routers, and layers of switches. Being able to bring this up outside of work parties would be even better, but at least at our final work party, we should be able to try everything we are depending on for the show. David Lang

[owendelong]

On Mar 21, 2024, at 14:03, Ryan Hamel ***@***.***> wrote: Owen isn't the only Juniper netadmin. My paid job is network engineering (and automation). Do I fully understand the convention center patching? No, however, I got better at handling the catwalk. Anyway, below are a few ideas off the top of my head: Add jumpboxes to the VM list to access various things. That way, we do not have to be on specific VLANs or need RPis setup in the NOC. Everything is an SSH session away (including various port forwards).

Since we don’t really block SSH anywhere, I’m not sure what jump boxes would buy us. The only VLANs that are restricted in any meaningful way are the vendor VLANs and we should never be on one of those except for testing the vendor network experience. (which means we shouldn’t have access to stuff, including a jump box).

Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop.

There are some problems with this. The VLAN it requires has to be the same VLID as the Infrastructure VLAN for one of the buildings, so if we uplinked it to the NOC switch, chaos would ensue, cats and dogs getting along, that sort of thing. Could we do it with enough effort, yes, probably. Is it worth that effort and the inherent risks when it goes horribly wrong? Not in my estimation. Eventually, we can likely build a Pi image to replace the laptop, which should greatly simplify the process (burn SD card, stand up Pi, done). Owen

[owendelong]

Once Rob and I get the Dev Server on line, we might be able to do some form of GRE based tunneling of SCaLE VLANs to work parties from my colo. This is probably worth further investigation. I’ll try to get with Rob and possibly Ryan and discuss possible solutions. Owen

…

On Mar 21, 2024, at 15:02, David Lang ***@***.***> wrote: On Thu, 21 Mar 2024, Robert James Hernandez wrote: >> Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop. > > The massflash is a livecd, its can run on any x86 machine. The reason its this way is so we dont have to have the hypervisors for a work party. Ive considered the idea of included similar massflash configuration in the `core` machine but there needs to be more considerations before its implemented. I would love to have massflash as a pi image as well. given our experience this year of various things not working through dhcp relays, etc. I would really like us to be able to bring up a virual replica of the show at the work parties, including the hypervisors, routers, and layers of switches. Being able to bring this up outside of work parties would be even better, but at least at our final work party, we should be able to try everything we are depending on for the show. David Lang — Reply to this email directly, view it on GitHub <#723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAK6GTW4AWH5NCMIHKF6PZDYZNKGFAVCNFSM6AAAAABEYMLHE2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJTHEYDCNJVHA>. You are receiving this because you commented.

[owendelong]

Could we also virtualize massflash to remove the laptop as a dependency? It requires a VLAN allocation, and the massflash switch would uplink to the NOC switch or whatever, instead of the laptop.

We try to avoid VLANs spanning buildings, but we might be able to put it on the CF side server. I would rather not put it on the Expo side server as we never do mass flashing over there and I'm really trying to eliminate as many of the building-spanning VLANs as possible. IIRC, the only one remaining is the point to point between the CF and BR routers, which amounts to a point to point link, so not so much a concern from a fault isolation perspective.

IMHO, a Pi image would be better... We should also add a fiber module to the mass flash switch and plug the host(s) doing the mass flash into copper SFPs in that module. I"ve got a spare 4x1g SFP module for ex4200 we can use and plenty of 1g copper SFPs laying around.

[owendelong]

I think with the combination of the new routers and additional documentation in the Repo for the switch configuration automation, this issue is much less of an issue and can be closed.