Interesting Title You Have There

It would be a shame if it made no sense to most readers, wouldn't it?

Fair point. My last (and first) post was on some philosophical sensible nonsense about why certain things are critically important to moving forward in life. Another principle that I firmly believe in is Toyota's Kaizen principle. If you haven't heard of it, check it out. The idea is often summarized as doing 1% better every day on your life's highest priority item(s). The mathematician in me is screaming that you cannot possibly improve by 1% every day, and then I realize the results can never be consistent nor that large a number. In the words of those annoying university textbooks, I'll leave that as a thought exercise to the reader as to why that's my opinion. (Feel free to comment below if you agree, or don't).

Kaizen in Action: Language Learning Imagine you're trying to learn a new language. Instead of trying to master it overnight, you decide to improve by 1% every day. This might mean learning a few new vocabulary words or having a short conversation with a native speaker. Over time, these small daily efforts will compound, and you'll find yourself becoming more fluent and confident in the language. I have a few friends who are learning new languages, and an App like Duolingo helps them quite a bit here.

Today's post is on the second part of the blog title:

Optimization, one educated guess at a time

How in the world does that work? What does an educated guess even mean? Interestingly, many of the current hot topics in the tech world today use this principle under the hood. If I had the time to do the research, I'd wager it's not only many but most of them do! The people who spend their time in the belly of this beast will already know where this is going. If you don't, don't worry - hopefully, this post will make it clear!

Let's start with linear things

Wikipedia describes linear programming (in extraordinary amounts of detail as Wikipedia does) as linear optimization. The first thing to note here is the word linear - what does that mean? It's just a straight line, right?

Well, yes and no. There are different classifications of linear systems, however, when dealing with linear programming (I'll just start referring to it as LPs at this point, deal?) it quite literally refers to straight lines. More precisely, the straight lines here refer to constraints on some objective function. 

Understanding Linear Programming Linear programming (LP) involves optimizing an objective function subject to linear constraints. Picture it as plotting a straight-line graph where each line represents a constraint. The goal is to find the best possible solution within these constraints.

Linear Programming Example - Easy to spot the Maximum!

In other words, what characteristics must the answer adhere to while maximizing some value described by an equation (or set of equations!). Now, it should be noted when dealing with systems (of equations) that are described this well linearly, the optimization process is less of a guessing game and more of a methodical strategy to obtain the best possible answer. It's very similar to chess - there is a very well-defined set of rules and clear objectives. Now, those familiar with the mathematics behind possible combinations in chess will be weary of the claim that a methodical strategy can obtain the best possible answer - but that claim holds true. Game theory simply requires perfect information for a turn-based game to be solvable. 

The issue?

The issue is a numbers game. We can describe the best possible outcome in some way, shape, or form. Many people have attempted this, but "solving" the game of chess is a hard problem that is not only computationally (time) complex but also spatially complex. Solving it for six pieces requires around 150GB of space, and for seven, 1.8TB. The solutions for eight pieces become jaw-dropping and are estimated to require 2 PB of space.

My next post will be on non-linear optimization, which I suspect is more closely aligned with what the rest of life looks like. LP problems are a simplification of real-world problems (that does not mean that it isn't useful!).

So what?

Well, many things don't require analytically perfect solutions. Importantly, most practical optimization strategies don't need exact answers. Ignoring the risk of being stuck on a local maximum (for the purposes of this post at least), you only need to keep moving in the right direction with every iteration you make. If you do 1% better than the previous guess, on every guess, then at some point you will reach a threshold where improving by 1% is infeasible (let's say there's less than 1% between the current approximation and perfection for example). The trick? Knowing which direction is the right way to move. In mathematics, many gradient-descent or other approaches can assist with that, while in life, I find having just the right book (or blog!) at the right time will help with that guidance.

Why Perfection Isn't Necessary Think of optimization like learning to play a musical instrument. You don't need to be perfect from the start. Each practice session, you aim to improve a little bit more. Over time, these small improvements add up to significant progress.

I suspect most people aren't blessed with unending foresight, although some seem to be playing life by a different set of rules (Look at any of the somewhat incomprehensible accomplishments of Bezos, Musk, Mr Buffet, Bill Gates, Steve Jobs, and so many more).

Great stuff

Perfection isn't always necessary; doing a bit better each time can lead to significant progress over time, even if you don't have perfect foresight like some of the world’s most successful people.

"Success is the sum of small efforts, repeated day in and day out." – Robert Collier