How can I visualize the weights in cnn in Tensorflow ?

How can I visualize the weights in cnn in Tensorflow ?

Asked on December 19, 2018 in Tensorflow.
Add Comment


  • 3 Answer(s)

          In Tensorflow to visualize the weights, use a tf.image_summary() op to transform a slice of a filter into a summary proto, Using a tf.train.SummaryWriter write it to a log and then visualize it using TensorBoard.

    Here is an simplified program:

    filter = tf.Variable(tf.truncated_normal([8, 8, 3]))
    images = tf.placeholder(tf.float32, shape=[None, 28, 28])
     
    conv = tf.nn.conv2d(images, filter, strides=[1, 1, 1, 1], padding="SAME")
     
    # More ops...
    loss = ...
    optimizer = tf.GradientDescentOptimizer(0.01)
    train_op = optimizer.minimize(loss)
     
    filter_summary = tf.image_summary(filter)
     
    sess = tf.Session()
    summary_writer = tf.train.SummaryWriter('/tmp/logs', sess.graph_def)
    for i in range(10000):
      sess.run(train_op)
      if i % 10 == 0:
        # Log a summary every 10 steps.
        summary_writer.add_summary(filter_summary, i)
    

          And then start TensorBoard to visualize the logs in /tmp/logs, which show the visualization of the filter.

    Answered on December 19, 2018.
    Add Comment

          Use tf.image_summary. For cifar10_train.py, Apply the following code under def train(). Note to access a var_scope ( ‘conv1’ )

    # Visualize conv1 features
    with tf.variable_scope('conv1') as scope_conv:
      weights = tf.get_variable('weights')
     
      # scale weights to [0 255] and convert to uint8 (maybe change scaling?)
      x_min = tf.reduce_min(weights)
      x_max = tf.reduce_max(weights)
      weights_0_to_1 = (weights - x_min) / (x_max - x_min)
      weights_0_to_255_uint8 = tf.image.convert_image_dtype (weights_0_to_1, dtype=tf.uint8)
     
      # to tf.image_summary format [batch_size, height, width, channels]
      weights_transposed = tf.transpose (weights_0_to_255_uint8, [3, 0, 1, 2])
     
      # this will display random 3 filters from the 64 in conv1
      tf.image_summary('conv1/filters', weights_transposed, max_images=3)
    
    Answered on December 19, 2018.
    Add Comment

    Extract the values as numpy arrays by using the below code:

    with tf.variable_scope('conv1', reuse=True) as scope_conv:
        W_conv1 = tf.get_variable('weights', shape=[5, 5, 1, 32])
        weights = W_conv1.eval()
        with open("conv1.weights.npz", "w") as outfile:
            np.save(outfile, weights)
    

    Then adjust the scope and variable name.

    Example to visualize numpy arrays:

    #!/usr/bin/env python
     
    """Visualize numpy arrays."""
     
    import numpy as np
    import scipy.misc
     
    arr = np.load('conv1.weights.npb')
     
    # Get each 5x5 filter from the 5x5x1x32 array
    for filter_ in range(arr.shape[3]):
        # Get the 5x5x1 filter:
        extracted_filter = arr[:, :, :, filter_]
     
        # Get rid of the last dimension (hence get 5x5):
        extracted_filter = np.squeeze(extracted_filter)
     
        # display the filter (might be very small - you can resize the window)
        scipy.misc.imshow(extracted_filter)
    
    Answered on December 19, 2018.
    Add Comment


  • Your Answer

    By posting your answer, you agree to the privacy policy and terms of service.