Usually, the team gathers around and shouts a million and one suggestions (\u201cit must be the blue in the background\u201d, \u201cmercury is in retrograde\u201d, or an office favorite, \u201cmaybe remove the labels!\u201d). While some of these suggestions are testable, the effort needed to verify these features isn\u2019t necessarily feasible. Even if there isn\u2019t anyone shouting suggestions, we still find ourselves scrambling around using instinct and experience to diagnose the problem. The end result could be tons of wasted cycles spent guessing at solutions and then testing them again. <\/p>\n\n\n\n
Or perhaps a wonderful network is built that passes all the tests created for it. It\u2019s proudly presented it to a client only to have them ask, \u201chow can we trust it?\u201d. One way is pointing to all the tests the neural network completed and present various metrics, but unless the clients are already well versed in Machine Learning<\/a>, they won\u2019t feel very satisfied. <\/p>\n\n\n\n Instead, in a second scenario, we again have constructed a fully functional network and it again passes all tests, solving the client\u2019s problem. But then someone asks \u201cwhat have we learned?\u201d Beyond predicting some value, how can we use this network to help plot future strategy?<\/p>\n\n\n\n In the first case, we hope to see light at the end of a seemingly endless cycle of guess and check that usually accompanies network development. We want to understand why a network made its decision and use that to guide our experiments.<\/p>\n\n\n\n In the second case, we seek to provide another measure of the network\u2019s success and seek to bolster \u201cconfidence\u201d in its decisions. One of the reasons people love algorithms like SVMs<\/a>, Nearest Neighbor<\/a>, or Linear\/Logistic <\/a>Regression<\/a> is that the training algorithms output some data structure that can be easily examined. There are a few coefficients or exemplars that can be pointed to or visualized. One can easily say how a model makes its decisions and show that they are explainable. With Neural Networks, we don\u2019t get that benefit right away.<\/p>\n\n\n One of the reasons why linear models and the like are so popular in the Social Sciences is that experimenters want to both predict their dataset accurately and know the reason in regard to the underlying data structure. Which variable is likely to predict student success? Does the variable have the same effect for two sub-populations? If we want to boost sales, which strategy is more likely to succeed? These are questions that are easy to answer if there are any coefficients. Not so with a neural network.<\/p>\n\n\n\n These are the two biggest subfields of neural network understanding. In traditional machine learning, we typically refer to features as the direct input to our learning algorithm \u2014 which can be viewed as a pipeline. Start with raw data, transform that data into features, and then run the machine learning algorithm. For example, in face detection, it was the standard to first run edge detection on an image to produce an image of edges, then feed that image to the machine learning algorithm. Generally, the selection of features was a carefully considered part of the process and was often called feature engineering.<\/p>\n\n\n\n In today\u2019s deep networks, the process of feature engineering is one of the tasks that the network must accomplish. The choice of features is something that the network is supposed to learn through training. The theory is that the lower-level layers in the model learn features like edge detection while the upper levels learn more abstract concepts. How true is that? That is the question that feature visualization is supposed to answer. This is usually done by taking a group of neurons within a network and deriving a picture of inputs that lead to group activation. Is there a picture of what typically makes this group of neurons happy? The hope is that these pictures will reveal which features were learned. For example, when examining select neurons in GoogleNet<\/a>, researchers generated an image for a group of neurons that looks suspiciously like a dog\u2019s ear. They reasoned that these neurons were a dog ear detector.<\/p>\n\n\n\n Attribution asks the question, \u201cwhich portions of the input are responsible for the network\u2019s response?\u201d When the inputs are graphical images, the question changes to what portion of the input image are most influencing the eventual outcome of the network. The hope here is to produce a graphic that highlights which image portions the network \u201cpays attention to\u201d and which parts drive network decisions.<\/p>\n\n\n\n Thankfully, there has been a lot of work done in these areas. In one hour of searching, we managed to find some 50 academic papers on the subject and several blog posts. While these papers are fascinating and fun to read, they don\u2019t answer the practical question of \u201chow useful are these ideas and tools?\u201d Can these concepts be used in a production environment? Do they meet the business needs outlined above?<\/p>\n\n\n\n We find current theoretical and mathematical research quite fascinating. However, we have practical work that involves getting Deep Neural Networks to perform specific tasks. We would like to see if any of these insights and tools are going to help us with that. Can these tools actually tell us what is wrong or right with our network? Will this knowledge actually help us to build a better network?<\/p>\n\n\n\n To that end, we propose a set of ongoing experiments. First, we will find a small enough object classification dataset. We will then construct a small network that attempts to classify the dataset. This network will have some flaws. We will then try out various neural understanding methods and see if it helps us do our job and fix our network. Along the way, we intend to release as much code as we can (no proprietary technology) as well as HOW TOs and tutorials on these techniques. We will also try to explain the techniques in a simple way. Our hope is to demonstrate the relative usefulness of these techniques and give you the tools to use them in your own workflows. To that point, don\u2019t forget to follow us on social media as we release our findings!<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":"![\"\"](\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/regression-equation-cartoon-1.png\")
Feature Visualizations And Attribution<\/h5>\n\n\n
![\"\"](\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/edgedetection-1.png\")
The Questions We Want To Answer<\/h5>\n\n\n
![\"\"](\"https:\/\/www.gravityjack.com\/wp-content\/uploads\/2023\/11\/Gravity-Jack-Neural-Netowrks-1-300x300-1.jpg\")
<\/figure><\/div>\n\n\n