For decades, communication networks have been made up of servers, each known as a "node" in the overall network. Along comes AI with its neural network architecture comprising a matrix of mathematical nodes known as "neurons."
AI nodes are not physical hardware or software. They are abstract computations that are mapped onto the GPU hardware, which also uses data from memory.
The Recipe/Cooking Analogy
An intriguing analogy for a neural network node is that a node is a "step in a recipe" that uses "ingredients" stored in memory while the GPU does the "cooking." See
AI network,
neural network and
server node.