Adding Regularizer To Skflow

I recently switched form tensorflow to skflow. In tensorflow we would add our lambda*tf.nn.l2_loss(weights) to our loss. Now I have the following code in skflow: def deep_psi(X, y)

Solution 1:

You can still add additional components to loss, you just need to retrieve weights from dnn / logistic_regression and add them to the loss:

defregularize_loss(loss, weights, lambda):
    for weight in weights:
        loss = loss + lambda * tf.nn.l2_loss(weight)
    return loss    


defdeep_psi(X, y):
    layers = skflow.ops.dnn(X, [5, 10, 20, 10, 5], keep_prob=0.5)
    preds, loss = skflow.models.logistic_regression(layers, y)

    weights = []
    for layer inrange(5): # n layers you passed to dnn
        weights.append(tf.get_variable("dnn/layer%d/linear/Matrix" % layer))
        # biases are also available at dnn/layer%d/linear/Bias
    weights.append(tf.get_variable('logistic_regression/weights'))

    return preds, regularize_loss(loss, weights, lambda)

```

Note, the path to variables can be found here.

Also, we want to add regularizer support to all layers with variables (like dnn, conv2d or fully_connected) so may be next week's night build of Tensorflow should have something like this dnn(.., regularize=tf.contrib.layers.l2_regularizer(lambda)). I'll update this answer when this happens.