Gradient Descent

Implement something from scratch is the best way to learn how it works. This python script implements gradient descent algorithm to train a linear regression model based on a very simple dataset

Results

At the end of the run, a plot is shown to illustrates how cost decreases through iterations until it converges to zero. Tweaking with parameters (learning rate, initial conditions) yields new shapes of the curve and help to undersand what impact performance

Next Steps

• Optimize algorithm execution to achieve acceptable execution time
• Improve code quality in order to swap algs and visualizations easily and make further experimentations
• Implement new cost functions and new estimation algorithms (SGD , MLE)
• Extend for N-dimensional feature space
• train with test/crossvalidation datasets