Comments (6)
The extremely hacky approach I've applied right now is: https://gist.github.com/abagshaw/a7dba307e5513f35d9314563bcb41ba0
But I'm hoping there is a more reliable way to identify if the instance we are on is of RDMA type or some more reliable heuristic to trigger the renaming by.
from azhpc-images.
@abagshaw The eth1 MAC address is in the range 00155d33ff0b - 00155d34fe07 on current IB clusters. You can rely on this for current clusters, but please note that this may not be always true in future SKUs.
One other way is to check the ib0 MAC address (or deduce from ib0 ipv6 address) and identify/rename the corresponding IB eth1 interface.
eg:
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::519b:214f:8106:3711 prefixlen 64 scopeid 0x20
ether 00:15:5d:33:ff:35 txqueuelen 1000 (Ethernet)
ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 2044
inet 172.16.1.44 netmask 255.255.0.0 broadcast 172.16.255.255
inet6 fe80::215:5dff:fd33:ff35 prefixlen 64 scopeid 0x20
from azhpc-images.
@jithinjosepkl Hmm I see - unfortunately I can't run the risk that the MAC range changes and the rule breaks in the future, I need something more future proof.
I thought about doing it through ib0 but I found that ib0
isn't up when the udev rules run for bringing eth1
up. Also, ib0
only shows up on SR-IOV enabled devices right?
Is there any way to reliably figure out if I am on an RDMA enabled instance type before the network is up? That would be enough for me.
from azhpc-images.
How about lspci and grep for IB device?
Another thought is to check metadata information and do your rules based on SKU type. (https://docs.microsoft.com/en-us/azure/virtual-machines/linux/instance-metadata-service). "r" in vmSize field indicates RDMA enabled VM.
from azhpc-images.
I tried lspci but it doesn't seem to have the IB device populated at the time of the udev
rule running for eth1
.
IMDS doesn't work because the udev
rule runs before the network is up so I can't access IMDS.
from azhpc-images.
@abagshaw , you may want to use two different images here - one for IB-enabled SKUs and one for non-IB. I dont see any other option especially since you are checking this in the udev rules processing time. Also, checking the MAC range is not a bad idea tbh, the range is ~65K. It may take sometime to go over that limit in a single cluster.
I am closing this issue as this is not related to the images.
from azhpc-images.
Related Issues (20)
- Release matching container images?
- topology xml file for an azure cluster with 4 nodes each with 4 K80 GPU
- nfs read-ahead setting causes Lustre slowness HOT 2
- amlfs-lustre-client-2.15.1-29-gbae0abe not found HOT 1
- copy_kvp_client.sh Connection refused HOT 1
- MOFED on secureboot enabled vms HOT 3
- add pmix v4 HOT 6
- Docker containers left over from the test need to be removed
- GPG checks are failing on almalinux 8.7 image HOT 2
- AlmaLinux HPC image hostname change broken HOT 1
- install `nv_peer_mem` kernel for GPUDirect RDMA HOT 5
- NCCL graph and topology incompatible with A100 HOT 3
- Feature request to add support for NCv5 HOT 2
- Support for SMB shares
- Rocky Linux 9 or AlmaLinux 9 support HOT 1
- doc: how to build the image HOT 1
- Why the SUSE contributions got deleted? HOT 1
- CUDA driver version mismatched with CUDA runtime version HOT 1
- Enroot breaks in almalinux:almalinux-hpc:8_7-hpc-gen2:8.7.2024042601 HOT 5
- leftover hello-world container in almalinux:almalinux-hpc:8_7-hpc-gen2:8.7.2024042601 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azhpc-images.