Optimizing Assumptions

One of the reasons we assume anything is to optimize our thinking. Rather than rederiving an empirical truth each time we need to do anything, we assume that our prior experience holds true for the current situation. This prior experience is generally a result of living long enough to build up a sufficient number of priors to hold assumptions that are generally valid. So having experienced a system in the past, that the behavior of the system in the current situation should behave the same way. Other times the prior experience is passed from person to person via written or spoken word. Our education system is based on the premise we can instill correct assumptions in the masses of people, and when those assumptions are not based in fact, correct opinion is instilled. After all this, simplifying assumptions optmize our thinking in the common case, and only catches up with us in edge conditions. This is why Bayesian analysis is so successful as a machine learning technique, as it provides a simple model for accumulating experience and generating assumptions that generally hold true. This analysis is not based on cause and effect, it is merely a correspondence. You don't need to be right, just right enough most of the time.

Now challenging assumptions can be tricky. Often when you take the balance of your experience, you don't want to challenge your assumptions because that requires additional work. Should you discover that your assumption still holds true, you have wasted considerable effort relearning something you already knew. Challenging assumptions in fields with highly specialized individuals, and especially among those who derive inordinate status and power from being preceived as being knowledgeable, can be dangerous. When one challenges an accepted tenent of any faith, be it economics, computer science, physics, or contemporary art, they are undermining the lifestyle of those who base their careers on that assumed authority. Practicitioners of each faith have a strongly vested intrest in the preservation of the belief system, not born out of pride but mere survival. These societies create rites of passage in which new would be hierophants perform a ritual debunking of some fringe of the doctrine (typically known in the industry as getting your PhD), but at the same time reaffirming their faith in the central tenents on which the modern incarnation of the faith is established. The ritual proves that the supplicant is both capable of defending the faith and at the same time obedient to it. Any who are foolhardy enough to pursue truth over blind devotion to the faith are carefully weeded out and denied funding from "legitimate" sources.

We can find this same religious behavior in nearly every aspect of everyday life. We trade worthless pieces of paper or numbers in a bank's database in exchange for actual goods and services. The optimizing assumption that this money will be good tomorrow works in the general case, but obviously fails when you have state sanctioned hyperinflation, global financial collapse, or simple theft. Each day, when we go to work we assume that the company paying our paycheck will pay us. Once again this is generally the case, unless the company goes bankrupt between now and then, or that the CEO doesn't just flee the country with a suitcase of our earnings. These assumptions optmize our daily lives because the alternative is to be paid as you go in something immediately useful to you, shoes, cows, milk, land, a new computer, but that would make the evening commute a nightmare beyond imagining.

In the field of computers, we have several competeing priesthoods, each with their own rituals, jargon, and religious tenents. Not only do some of these religions espouse technical truths, but they also claim dominion of business practices, operations, and design. If one wanted to look for some of the worst excuses for magical thinking to ever infect an industry, they need only look at the long litinay of programming fads. Even issues as simple as the nature of which hammer to use becomes a holy war, between vi and emacs, between tabs and spaces, between C and Go, between good and evil. Popularity becomes a substitute for sound techinical judgement. It is easier to hire young impressionable programmers keyed into the hip new language or platform, than it is to wade through the veterans of crusades past. No boss wants someone to whip out experience to justify not doing something stupid but popular. Those old optimizing assumptions are simply out of place in the new culture, because they aren't optimizing the problem the high priest needs to solve to receive his tithes, converting fresh recruits into true believers.

With this in mind, there is a cure for religious thinking, but the way is hard an fraught with mathematics. Sufficient data can be used to invalidate both spoken and unspoken assumptions. Using the hammer of mathematics, you can discern if your assumptions are inline with the data. The goal of this analysis is not to establish the truth, but rather to remove the most eggregiously wrong assumptions. If your beliefs about the behavior of the system are out of line with observed behavior, this illustrates a gap between your mental model and reality. The data does not, however, validate your assumptions. The abscence of evidence of a gap does not mean that your model is a true represenation. It merely means that for the observable behavior, your model is indistinguishable from the true one. Your model merely is accurate within the limits of your measurement. Data allows you to say if your assumptions are good enough, not correct.