NVIDIA Tesla M60 - Server Install

-

 Installing NVIDIA Tesla M60's on my servers

In this post, I’ll go through how I upgraded my HP Gen8 servers to use NVIDIA Tesla M60 GPUs, including replacing the PCIe riser cards, verifying detection in BIOS, and preparing for driver installation on openSUSE Leap 15.6.

My servers

🖥️ Node 1 – HP DL385p Gen8

Component

Spec

CPU

2 × AMD Opteron 6380

Threads

64

RAM

128 GB DDR3 ECC

GPU

NVIDIA Tesla M60 (16 GB GDDR5 – dual GPU card, 2 × 8 GB)

Storage

11.7 TB total

  ðŸ–¥️ Node 2 – HP DL380p Gen8

Component

Spec

CPU

2 × Intel Xeon E5‑2665

Threads

32

RAM

256 GB DDR3 ECC

GPU

NVIDIA Tesla M60 (16 GB GDDR5 – dual GPU card, 2 × 8 GB)

Storage

512 GB SSD

 ðŸ–¥️ Node 3 – HP DL380 Gen8

Component

Spec

CPU

2 × Intel Xeon E5‑2260

Threads

24

RAM

48 GB DDR3 ECC

GPU

NVIDIA Tesla M60 (16 GB GDDR5 – dual GPU card, 2 × 8 GB)

Storage

2 × 512 GB SSDs + 6 × 1 TB HDDs = ~6.5 TB total

I wanted to install NVIDIA Tesla M60 cards into each server to build a small Beowulf cluster capable of running AI workloads like Ollama.

My server already had the correct hardware and the graphics card just slotted right in, however the servers required the PCI riser card be changed.

Upgrading the Risers and adding the graphics card.

The issue I had was that the PCI risers contained only one x16 PCI lane and the graphics card is double height, so it would not fit.


I bought the correct part HP 634582-001 662525-001 DL380p G8 2Slot 2x16 PCI-E Riser Card

First I removed the screws holding the old card in place then I slide the card out.

The new card slid into place of the old and and the screws were tightened up.

I then put the graphics card in place and attached the power cable
I then put the riser back into the server

Put the lid back on and connected up the peripherals and powered it up. During boot i went into the bios to confirm that the server detected the new hardware

Which it did and was shown on slot 5 which is correct.
With the hardware installed and recognized in BIOS, the next step is to get the NVIDIA drivers set up correctly under openSUSE Leap 15.6

Installing NVIDIA Tesla M60 Drivers on openSUSE Leap 15.6 

I wanted to ensure that I had a clean, minimal install guide that just works as I had three of these to do.
The following is what I done to get the drivers working: -

First I opened up the terminal.

Step 1 — Update the system

Start by making sure everything is up-to-date.

sudo zypper refresh
sudo zypper update
sudo reboot

Reboot so the latest kernel and packages are active.

Step 2 — Disable the open-source Nouveau driver

The open-source nouveau driver conflicts with NVIDIA’s official one.
Create a small blacklist file:

sudo sh -c "echo 'blacklist nouveau' > /etc/modprobe.d/50-blacklist-nouveau.conf"
sudo sh -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/50-blacklist-nouveau.conf"
sudo dracut --force
sudo reboot

After reboot, confirm Nouveau isn’t loaded:

lsmod | grep nouveau

No output means it’s disabled, which is exactly what we want.

Step 3 — Add the official NVIDIA repository

openSUSE makes this easy. For Leap 15.6:

sudo zypper addrepo --refresh https://download.nvidia.com/opensuse/leap/15.6 NVIDIA
sudo zypper refresh

This repo contains all current proprietary driver builds. If your doing this yourself, ensure you have the correct repository for your version of Linux.

Step 4 — Install the Tesla M60 driver

Tesla cards use the G06 driver branch, designed for data-centre and compute GPUs.

sudo zypper install nvidia-compute-G06 nvidia-driver-G06-kmp-default nvidia-gl-G06 nvidia-video-G06

If you see an error about libglvnd, fix it with:

sudo zypper install libglvnd

…and rerun the install command above.

Step 5 — Reboot and verify

sudo reboot
nvidia-smi

If everything worked, you’ll see a table like this:

That means the driver is active and the GPU is ready for CUDA work.

Step 6 - Install the Nvidia Config

Using YaST2 software package installer, we are now going to install the NVidia settings and xconfig so we can see this the graphics card settings within the GUI

I did a search for NVidia and selected the relevant packages to install
The packages I choose were nvidia-settings and nvidia-xconfig, the others wanted to downgrade the driver so I unselected.
With these packages installed, I can now see the settings for the graphics card.

I repeated this process for my other two servers and all worked great.
Now I can finally look at the next part of setting up my Beowulf cluster which I hope to document and show.

Comments

Post a Comment

Popular posts from this blog

Math Behind Logic Gates

-
Image
Understanding the Real Electronics Behind Logic Gates (The Practical Math Behind Digital Circuits) When we build circuits with logic chips like the 74LS series, it’s easy to think only in 1s and 0s. But under the hood, every logic gate follows the same rules as any other circuit, Ohm’s Law, current limits, and voltage thresholds. So yes, there is real-world math behind those blinking LEDs and logic tables.  1. Logic Levels — The Digital View At the logic level, everything is simple: 0 V → LOW (logic 0) 5 V → HIGH (logic 1) We don’t care about exact voltages here just whether a signal crosses a defined threshold. For TTL (Transistor-Transistor Logic) chips like the 74LS86 XOR gate , these thresholds are: This means a logic gate “decides” based on whether the voltage is above or below that boundary, which is how binary logic becomes real voltage levels. 2. Inside the Circuit — Real Electrical Math Even though logic circuits process bits, they still obey Ohm’s Law ...
Read more

6502 - Part 2 Reset and Clock Circuit

-
Image
6502 Retro Console Part 2 Add a Reset and Clock Circuit In Part 1 we powered up the 6502 and watched it running a NOP test with just the CPU and LEDs on the address bus. Now we’ll give it a clock and a clean start every time you power on, rather than relying on a function generator. In this post we will cover a simple 555 Timer Clock circuit and reset circuit. 555 Timer The 6502 needs a steady pulse on its Φ0 (pin 37) to step through instructions. A NE555 timer is perfect for low-speed testing reliable, simple, and easy to wire. Parts List NE555 timer 1 kΩ resistor 10 kΩ resistor 100 nF capacitor (timing) 0.1 µF capacitor (decoupling) Breadboard + jumper wires Circuit Diagram How to Wire It Add a 0.1 µF decoupling cap across the +5V and GND rails stability. Typical speed: ≈ 7–10 kHz perfect for a visible NOP test. Use larger resistors or capacitors to slow it down. Tip: Swap the timing capacitor for 1 µF and the 10 kΩ for 100 kΩ to step slowly enough ...
Read more

Building a 6502 NOP Test

-
Image
 The Simplest Working 8-bit CPU Circuit Before we start building a full retro console, let’s do something beautifully simple get a 6502 CPU to run,  no RAM, no ROM, no glue logic, just the chip, a few LEDs, and a clock. This is called a NOP test (No Operation). It’s the digital equivalent of “does it breathe?” What You’ll Need Part Purpose MOS 6502 CPU The heart of our system Bench power supply (5 V) Power for the CPU Function generator Clock source 8 × LEDs To watch address lines change 8 × 470 Ω resistors Current-limiters for the LEDs 1 × 0.1 µF capacitor Decoupling between VCC and GND Breadboard and jumper wires For connections MOS 6502 Cpu Pin Out Check your IC and its datasheet to ensure that you have the correct pin out map as the steps below may vary slightly. Step 1 — Power and Decoupling Give the 6502 a clean 5 V supply: VCC (pin 8) → +5 V VSS (pins 1 and 21) → Ground Place the 0.1 µF capacitor between VCC and VSS close to the chip. That tiny capacitor...
Read more