{"id":1095,"date":"2021-02-10T08:47:26","date_gmt":"2021-02-10T08:47:26","guid":{"rendered":"https:\/\/www.gravityjack.com\/?p=1095"},"modified":"2023-11-28T13:33:37","modified_gmt":"2023-11-28T13:33:37","slug":"understanding-neural-network-part-3","status":"publish","type":"post","link":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/","title":{"rendered":"UNDERSTANDING YOUR NEURAL NETWORK \u2013 PART 3: MACHINE LEARNING EXPLAINED"},"content":{"rendered":"\n
How Useful is Feature Visualization?<\/strong><\/h5>\n\n\n\n

Welcome back! This post marks Part 3 in our ongoing series on neural understanding, where we work through a practical example with some representative techniques in order to understand what and how a neural network has learned. <\/p>\n\n\n\n

While Part 2<\/a> walked through a sample problem, this post contains a collection of results to analyze! Quite a bit of results, in fact. So, maybe take some time out to stretch and do some calisthenics while following along.<\/p>\n\n\n\n

As always, the Jupyter notebooks may be found here<\/a>. <\/p>\n\n\n\n

What is Feature Visualization?<\/strong><\/h5>\n\n\n\n

In the old days of Machine Learning (ML) a feature was an attribute of an example. For example, when trying to classify a sample data set concerning cancer biopsies<\/a>, features might be the average width of the cells in the sample, the average perimeter of the cells, and the opacity of the cells. A feature describes an object that the ML algorithm is trying to reason about. In numerical datasets, a feature represents a column of data. In visual datasets, a feature is a collection of pixels, perhaps the edge of an object or the texture of part of the object.<\/p>\n\n\n\n

In pre-deep learning times, some sort of feature engineering took place before feeding a dataset to an ML algorithm. In visual domains, this included things like detecting all the edges in an image<\/a> or finding all the \u201cblobs\u201d<\/a>. In the era of Deep Learning, part of the job of the neural network is to find these features themselves (or at least that is the theory). Unfortunately, due to the black box nature of neural networks, it is not obviously clear that this actually happens. <\/p>\n\n\n\n

Feature visualization<\/strong> is the umbrella term for creating an image to suggest what a given neuron or group of neurons have learned. Google has done a great deal of work <\/a>on feature visualization recently, with DeepDream<\/a> being perhaps the most famous.<\/p>\n\n\n

\n
\"\"
Image 1: An adorable floppy ear detector.<\/figcaption><\/figure><\/div>\n\n\n

This example comes from Google\u2019s inception network that was trained to identify roughly 20,000 different objects, many of them different breeds of dogs. This feature visualization is from one of the neurons in a deep layer of the network. Notice that it looks remarkably like a dog\u2019s ear, furthermore, a specific type of dog ear. What this is saying is that when attempting to identify a species of dog in an image, the network is using the shape of the ear to make a determination. This is clear evidence of how the network operates! <\/p>\n\n\n

\n
\"\"
Image 2: Neurons from inception. The leftmost 6 are nearer to the surface than the middle 6 with the rightmost 6 being the deepest.<\/figcaption><\/figure><\/div>\n\n\n

Upon examining the visualizations of their lower-level layers, viewers can see that they start with simple line detectors and then move up to more complicated textures. This is exactly in line with the theory of deep learning.<\/p>\n\n\n\n

How Does It Work?<\/strong><\/h5>\n\n\n\n

The basic version of feature visualization asks, for a given neuron, what input image results in the highest activation? This is known as HappyMap. Basically, what sort of input to the network makes this neuron the most active or \u201chappy\u201d? Google has also experimented with the inverse, SadMaps. Here\u2019s the basic gist: <\/p>\n\n\n\n

    \n
  1. Select a neuron.<\/li>\n\n\n\n
  2. Create a random image. In this type of image, each pixel takes on a random value independent of nearby pixels. The result looks very much like TV static. <\/li>\n\n\n\n
  3. Feed the image to the network and observe the output of the selected neuron.<\/li>\n\n\n\n
  4. Adjust the image so that the output of the neuron is greater than the observed amount.<\/li>\n\n\n\n
  5. Repeat steps 3 to 5 until the output of the neuron cannot be increased.<\/li>\n\n\n\n
  6. Output the adjusted image as the HappyMap. <\/li>\n<\/ol>\n\n\n\n

    In practice, this is accomplished by performing a gradient descent search on the input space, keeping the weights constant. Unfortunately, this process alone doesn\u2019t usually result in interpretable HappyMaps.<\/a> As each pixel is adjusted independently of its neighbors, the result can still be somewhat noisy and hard to interpret. So, usually, a normalizing function is added to the optimization. The normalizing function penalizes an image for not being \u201cnatural enough\u201d. This can be required that the pixels conform to some standard or that neighborhoods of pixels do not look too different. <\/p>\n\n\n\n

    What We Did<\/strong><\/h5>\n\n\n\n

    Google makes their experiments on inception freely available in collab notebooks that ship with Lucid<\/a>, the software behind the experiments. They also provide scattered instructions on how to replicate these experiments by using a tensorflow trained network. However, we use Keras here, and though Keras does run on top of tensorflow, there was no way to save a Keras model such that the Google collab notebooks would work. Even following the instructions didn\u2019t work! <\/p>\n\n\n\n

    In the end, we did find a wonderful tool, keras-vis<\/a> \u2014 that allows the same types of visualizations that Lucid allows, but works out of the box on Keras.<\/p>\n\n\n\n

    The notebooks built can be found in experiments\/experiment1<\/strong> and various subdirectories. <\/p>\n\n\n\n

    A few notes are in order. First of all, keras-vis uses two regularizers or normalizers. The first is basically an old fashioned p norm. The idea here is to minimize the values that the input pixels can take. This keeps the values from going crazy, beyond the range of allowable RGB values. The second norm is the total variation norm. Total variation norm tries to minimize the variation in a group of pixels. This one turned out to be crucially important for getting good visuals. Keras-vis allows the user to weight the main activation maximization function with the two normalizing functions. In practice, we experimented with various levels of total variation, from 1e-5 to 1e3. It was observed that good visuals tended to occur for filters at 1e-3 or 1e-2 and 1e1 for dense neurons. Our experiments did not find any significant effect of changing the norm weight. <\/p>\n\n\n\n

    Secondly, the input to this process is a random image. Any optimization process that starts from a random input is not guaranteed to end up with the same outputs. Put simply, each time you generate a HappyMap from a given neuron, it is likely to be different than any previous attempts. The hope, and our experience, is that the maps do tend to look similar (you will need to watch out for random noise). In several cases, a HappyMap showed an interesting pattern that was not seen to generate again across multiple attempts \u2013so be careful before drawing conclusions. <\/p>\n\n\n\n

    Initial Results<\/strong><\/h5>\n\n\n\n

    Take a look at experiments\/experiment1\/visualization_first_pass.ipynb<\/strong>. At the top of the notebook you can find some code for loading the model and functions for wrapping the keras-vis visualize_activation<\/strong> function. After seeing the results that Google achieved on their visualization results and the results displayed on the keras-vis documentation, we were excited to see what our output neurons would show. Multiple weight levels were tested and displayed the different results in the notebook.<\/p>\n\n\n

    \n
    \"\"
    Image 3: The left HappyMap is for \u201cnot left\u201d, the right is for \u201cfacing left\u201d<\/figcaption><\/figure><\/div>\n\n\n

    It\u2019s clear that our results are not nearly as straightforward or, well, exciting. Neuron 0 (labeled as filter 0) is the one that codes for \u201cnot facing left\u201d. Neuron 1 (labeled as filter 1) codes for \u201cfacing left\u201d. The HappyMap for neuron 1, does sort of look like a face pointing to camera left (if you squint). One interesting thing to note is that the HappyMap for not facing left appears to be the complement of the facing left neuron. In a softmax layer, the sum of all the activations must be one. Since in the two class problem there are only two neurons, whatever maximizes the activation of one neuron naturally minimizes the other neuron. <\/p>\n\n\n\n

    This begs the question: Would this get better results without using a softmax layer? Or using a multi class problem? <\/strong><\/p>\n\n\n\n

    Continuing on in the notebook, the first three convolution layers were visualized. Some of the layers were visualized many times with different weights for the total visualization norm. Looking at conv2d_1<\/strong>, the first convolution layer, there are some interesting results.<\/p>\n\n\n

    \n
    \"\"
    Image 4: Staticky maps<\/figcaption><\/figure><\/div>\n\n\n

    The first is that some neurons, when viewed with different weights always seem to produce a pattern that appears to be random noise. These appear to be neurons that don\u2019t have a pattern, nothing makes them particularly happy. So they are likely not used deeper into the network. This signals that there may be more convolution filters than necessary.<\/strong> <\/p>\n\n\n

    \n
    \"\"
    Image 5: Always off or always on?<\/figcaption><\/figure><\/div>\n\n\n

    There are also some neurons that under various weights appear to be most happy when the input is bright. Are these ambient light detectors? Or are they neurons that simply like anything? Is this more evidence for having excess convolution filters?<\/p>\n\n\n

    \n
    \"\"
    Image 6: Beautiful edge detectors!<\/figcaption><\/figure><\/div>\n\n\n

    The rest of the neurons in that layer exhibit the same characteristics that were seen in the Google examples. There are wavy lines that appear to be the edge detectors that Google talked about.<\/p>\n\n\n

    \n
    \"\"
    Image 7: What are these? We don\u2019t know. They change each time we run the visualizer.<\/figcaption><\/figure><\/div>\n\n\n

    There are also quite a few neurons, about half, for which no discernable pattern forms. At some weights, these neurons present a blue background with graceful curves. It\u2019s tempting to think that these might be some sort of curve operator? But running the results multiple times didn\u2019t yield any consistent patterns. So the question arises again: Are there more filters than necessary?<\/p>\n\n\n

    \n
    \"\"
    Image 8: Deeper convolutional layers form more complex textures.<\/figcaption><\/figure><\/div>\n\n
    \n
    \"\"
    Image 9: Neurons from a deeper level convolutional layer.<\/figcaption><\/figure><\/div>\n\n\n

    conv2_2<\/strong> and conv2_3<\/strong> show some pretty interesting results. The neurons seem to be activated by more complicated patterns than just lines. Often, there is a two dimensional structure, as if these layers are looking for corners and angles. In fact, almost every neuron in this layer yields a recognizable pattern. Additionally, many of the neurons have HappyMaps that seem to code for edges of various thickness as well. It seems as if something about the texture of an image is being learned.<\/p>\n\n\n

    \n
    \"\"
    Image 10: Dense layer neurons. Some of them look repetitive.<\/figcaption><\/figure><\/div>\n\n\n

    Now turn to experiments\/experiment1\/visualization_dense_layers.ipynb<\/strong>. This notebook shows the results of running on the dense layers. Each of the neurons in these layers responds well to visualization in that a pattern is evident. What seems to be happening is that there are neurons that respond to events in certain locations. Could these be the object or part of object detectors that we were promised? If so, they must encode for something entirely different because they\u2019re much unlike Google\u2019s neurons. Are they looking for ears? Noses? Something in the environment? There certainly isn\u2019t a dog or floppy ear detector like Google\u2019s. Hmmm\u2026. <\/p>\n\n\n\n

    WHAT\u2019S NEXT?<\/strong><\/h5>\n\n\n\n

    Why are our results so different? Do we not have enough layers? Do we not have enough wide enough layers? Or, do we have too many dense neurons? <\/strong>Answering these questions will need some additional finesse. To address each individual point, we\u2019ll have to run some quick follow-ups and  see what happens. But that\u2019s a debacle for another day. Join us next time as we run some test experiments! <\/p>\n\n\n\n

    <\/p>\n\n\n\n

    <\/p>\n\n\n\n

    <\/p>\n","protected":false},"excerpt":{"rendered":"

    How Useful is Feature Visualization? Welcome back! This post marks Part 3 in our ongoing series on neural understanding, where we work through a practical example with some representative techniques in order to understand what and how a neural network has learned.  While Part 2 walked through a sample problem, this post contains a collection of results … UNDERSTANDING YOUR NEURAL NETWORK \u2013 PART 3: MACHINE LEARNING EXPLAINED<\/span><\/a><\/p>\n","protected":false},"author":9,"featured_media":1106,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[9,3],"tags":[],"class_list":["post-1095","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ar-news","category-news"],"acf":[],"yoast_head":"\nUnderstanding Your Neural Network - Part 3: Machine Learning Explained<\/title>\n<meta name=\"description\" content=\"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Understanding Your Neural Network - Part 3: Machine Learning Explained\" \/>\n<meta property=\"og:description\" content=\"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/\" \/>\n<meta property=\"og:site_name\" content=\"Gravity Jack\" \/>\n<meta property=\"article:published_time\" content=\"2021-02-10T08:47:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-28T13:33:37+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"344\" \/>\n\t<meta property=\"og:image:height\" content=\"194\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Gravity Admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gravity Admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/\",\"url\":\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/\",\"name\":\"Understanding Your Neural Network - Part 3: Machine Learning Explained\",\"isPartOf\":{\"@id\":\"https:\/\/www.gravityjack.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg\",\"datePublished\":\"2021-02-10T08:47:26+00:00\",\"dateModified\":\"2023-11-28T13:33:37+00:00\",\"author\":{\"@id\":\"https:\/\/www.gravityjack.com\/#\/schema\/person\/63c79a7e30c630a69e6e003ef7476fcc\"},\"description\":\"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage\",\"url\":\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg\",\"contentUrl\":\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg\",\"width\":344,\"height\":194},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.gravityjack.com\/#website\",\"url\":\"https:\/\/www.gravityjack.com\/\",\"name\":\"Gravity Jack\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.gravityjack.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.gravityjack.com\/#\/schema\/person\/63c79a7e30c630a69e6e003ef7476fcc\",\"name\":\"Gravity Admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.gravityjack.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/aa3d42e00068a5cdefa39bab3155196c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/aa3d42e00068a5cdefa39bab3155196c?s=96&d=mm&r=g\",\"caption\":\"Gravity Admin\"},\"url\":\"https:\/\/www.gravityjack.com\/author\/gjmagravityjack-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Understanding Your Neural Network - Part 3: Machine Learning Explained","description":"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/","og_locale":"en_US","og_type":"article","og_title":"Understanding Your Neural Network - Part 3: Machine Learning Explained","og_description":"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!","og_url":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/","og_site_name":"Gravity Jack","article_published_time":"2021-02-10T08:47:26+00:00","article_modified_time":"2023-11-28T13:33:37+00:00","og_image":[{"width":344,"height":194,"url":"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg","type":"image\/jpeg"}],"author":"Gravity Admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Gravity Admin","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/","url":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/","name":"Understanding Your Neural Network - Part 3: Machine Learning Explained","isPartOf":{"@id":"https:\/\/www.gravityjack.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage"},"image":{"@id":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage"},"thumbnailUrl":"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg","datePublished":"2021-02-10T08:47:26+00:00","dateModified":"2023-11-28T13:33:37+00:00","author":{"@id":"https:\/\/www.gravityjack.com\/#\/schema\/person\/63c79a7e30c630a69e6e003ef7476fcc"},"description":"We built a neural network to classify images using artificial intelligence. Part 3 of our machine learning series analyzes some experimental results!","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.gravityjack.com\/news\/understanding-neural-network-part-3\/#primaryimage","url":"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg","contentUrl":"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Neural-networking-pt3-1.jpg","width":344,"height":194},{"@type":"WebSite","@id":"https:\/\/www.gravityjack.com\/#website","url":"https:\/\/www.gravityjack.com\/","name":"Gravity Jack","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.gravityjack.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.gravityjack.com\/#\/schema\/person\/63c79a7e30c630a69e6e003ef7476fcc","name":"Gravity Admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.gravityjack.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/aa3d42e00068a5cdefa39bab3155196c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/aa3d42e00068a5cdefa39bab3155196c?s=96&d=mm&r=g","caption":"Gravity Admin"},"url":"https:\/\/www.gravityjack.com\/author\/gjmagravityjack-com\/"}]}},"_links":{"self":[{"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/posts\/1095","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/comments?post=1095"}],"version-history":[{"count":0,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/posts\/1095\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/media\/1106"}],"wp:attachment":[{"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/media?parent=1095"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/categories?post=1095"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gravityjack.com\/wp-json\/wp\/v2\/tags?post=1095"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}