Thanks to Verda, formerly Datacrunch.io, for sponsoring today’s episode. Check them out at https://verda.com, and use Coupon Code CRAFT-COMPUTING-V100 for $25 off at checkout!
Grab yourself a Pint Glass or Coffee Tumbler at https://craftcomputing.store
Implementing any form of AI workflow into your business is prohibitively expensive. From just the cost of hardware to the power and cooling infrastructure of modern AI servers, it’s enough to turn anyone off. But you know me… I love digging up old servers out of eWaste piles and giving them new life. But how well does Nvidia’s Tesla V100s stack up to modern cards in AI? Today, we’re testing Eight Tesla V100s in 70B and 120B LLMs to see if there’s life still in these eight year old GPUs.
But first… What am I drinking???
Deschutes Brewing (Bend, OR) Fresh Squeezed IPA NA (0.5%)
HUGE THANKS to UnixSurplus for sending over the Inspur DGX V100 system for me to take a look at. Check them out at https://UnixSurplus.com
Or their eBay store: https://ebay.us/6BPOyd
*Links to items below may be affiliate links for which I may be compensated*
Inspur DGX V100 Server from UnixSurplus: https://ebay.us/1U6I0H
Dual Intel Xeon 8260 24-Core / 48-Thread
256GB DDR4-3200 REG-ECC
2x MZ-7LH1T90 1.92TB SSD Drives
8x Nvidia V100 32GB SXM2 GPU
1x 4x 10GB SFP+ Mezzanine Card
OB NIC: 100GB Nvidia CX6 network card
Follow me on Bluesky @CraftComputing.bsky.social
Support me on Patreon and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
https://www.patreon.com/CraftComputing
Timestamps
0:00 – Intro
2:11 – Sponsor – Verda.com
3:29 – Speeds and Feeds
6:10 – Llama 3.1 8B (MLPerf)
8:16 – Phi 4 Reasoning 14B (MLPerf)
9:06 – LM Studio (Llama 3.3 70B + GPT-OSS 120B)
14:56 – As Always, I’m Mikey
20:15 – Wrapping Up
22:57 – Deschutes Fresh Squeezed NA
