The Stochastic Gradient Descent and Mini-batch gradient descent

[frameproject]

Hello Aurélien

The program loop is longer than in ordinary gradient descent:

for epoch in range (n_iterations):
     for i in range (m):
......

I don't see a speed advantage here compared to the ordinary gradient descent with a loop size of 1100:

eta = 0.01
n_iterations = 1100
m = 5
theta = np.random.randn(2,1)
theta[0]=0.1
theta[1]=0.1
print("theta=", theta);

for iteration in range(n_iterations):
gradients = 2/m * X_b.T.dot(X_b.dot(theta) - y)
theta = theta - eta * gradients

print ("theta", theta)

Many thanks
Frederic