Tensorflow Model Accuracy Not Increasing

I am currently doing the Deep Learning course on Udacity and am presently trying to complete the 4th assignment, where you are supposed to create your own model and see what the be

Solution 1:

I agree with @cyniikal, your network seems too complex for this dataset. With a single layer model, I was able to achieve 93.75% accuracy on the training data and 86.7% accuracy on the test data.

In my model, I used GradientDescentOptimizer that minimized cross_entropy just as you did. I also used a size 16 batch-size.

The main difference I see between your approach and mine is that I:

1. OneHot Encoded the labels
2. Used a single-layer network rather than VGG-16

See this notebook with my single layer model code sample.

If you would like to add layers to your neural network (the network will converge with more difficulties), I highly recommend reading this article on neural nets. Specifically, since you added sigmoid as your last activation function, I believe you are suffering from a vanishing gradient problem. See this page to address the vanishing gradient.

Solution 2:

Playing around with the learning_rate might yield better results, but it could be that your network is just too complex (computing a super non-convex function) for simple Gradient Descent to work well here. Other than that I don't spot any immediate issues, but debugging a neural network implementation can be pretty tricky sometimes.

Try out a quick switch to AdamOptimizer or another advanced optimizer or toying around with the learning_rate. If you're still getting super low test accuracy, then I'll try to go through it with a fine-toothed comb later.