Is stochastic gradient descent pseudo-stochastic?
4
$begingroup$
I know that stochastic gradient descent randomly chooses 1 sample to update the weights. An epoch is defined as using all $N$ samples. So with SGD, for each epoch, we update the weights $N$ times. My confusion is doesn't this make it so you have to go through all $N$ samples before you can see the same sample twice? Doesn't that effectively make it pseudo-random/stochastic? If it was entirely random, then there would be a possibility of seeing the same sample more than once before going through all $N$ samples.
machine-learning neural-networks gradient-descent sgd
share | cite | improve this question
edited yesterday
...