Writing a 100-word email using ChatGPT (GPT-4, latest model) consumes 1 x 500ml bottle of water It uses 140Wh of energy, enough for 7 full charges of an iPhone Pro Max
Writing a 100-word email using ChatGPT (GPT-4, latest model) consumes 1 x 500ml bottle of water It uses 140Wh of energy, enough for 7 full charges of an iPhone Pro Max
Does that account for cooling? Storage? Networking? Non-H100 compute and memory?
Nope. Just GPU board power draw. 60 seconds is also pretty long with how fast these enterprise cards are but I’m assuming they’re using a giant 450B or 1270B model.