The XY Problem

Problem-solving is an increasingly necessary skill for knowledge employees these days.

For instance, software engineers have to deal with problems of various complexity on a daily basis. If you have the proper experience as a developer, you can quickly solve the problem by implementing the right solution. However, sometimes you deal with a problem that is novel to you, and you don’t have the relevant experience to solve it.

The good news is that the problems are rarely unique and never encountered before by anyone else. In 99.99% of cases, someone somewhere dealt with such an issue previously. So you need to search and adapt their knowledge and findings to your situation.

The Main Problem When Solving Problems

So we have a problem “X” we need to solve but which we haven’t encountered before. The first thing we do, even before googling stuff, is trying to reason about it based on our previous experience. Quite often, that is the first stumbling block toward a painless and effortless solution.

Depending on the novelty of the problem for us, we might know how to solve a similar issue or just hypothesize about a solution that could eventually work. Therefore, we focus on the “Y” solution that we think will solve the “X” problem.

But, as the work on the Y solution progresses, you encounter some roadblocks and turn to the wisdom of the Internet. You ask “how to solve Y.” After some time chasing for the answer that will allow you to advance, you find out that the Y solution is inappropriate for X, and you find out a better approach to your X problem.

Therefore, it is crucial to understand and correctly address the right problem, as sometimes we are spending the effort in a direction misaligned with our goals.

The XY Problem

It turns out this is such a common issue that it has its own name: the XY Problem:¹

• User wants to do X.
• User doesn’t know how to do X but thinks they can fumble their way to a solution if they can just manage to do Y.
• User doesn’t know how to do Y either.
• User asks for help with Y.
• Others try to help the user with Y but are confused because Y seems like a strange problem to want to solve.
• After much interaction and wasted time, it finally becomes clear that the user really wants help with X and that Y wasn’t even a suitable solution for X.

Causes Of The XY Problem

There are two primary causes why the XY problem occurs:

1. The person does not fully understand the domain of the X problem.

For instance, the person wants to get the extension of files on their computer programmatically (the X problem). But because they do not fully understand how the filesystem works, they ask how to get the last 3 characters in a file name (the Y problem). Therefore, the Y solution is not suitable because it is not guaranteed that an extension is always three characters long.

2. The person does not dive into the root cause of their problem.

Not fully understanding the problem itself is prone to adopting the wrong solutions. For instance, the car would not start (the X problem). The initial finding is that the battery is dead (the Y problem). After replacing the battery, the car works normally, but the battery dies again after two weeks. After an investigation, it turns out that the alternator is not functioning and does not charge the battery. Therefore, the Y solution is a short-term one that does not fix the X problem long-term.

These are the most prominent causes of the XY problem, but there are many more.

Potential Solutions

So how to overcome these situations? As with everything in life, there are no clear, one-fit-all solutions. Here are some actions that can alleviate these cases, though:

1. Start with Why? There is a technique named Five whys, developed initially at Toyota Motor Corporation. This technique’s primary goal is to determine the root cause of a defect or problem by repeating the question “Why?” The number 5 in the name derives from an anecdotal observation on the number of iterations needed to ask the question and is not a hard rule.

For instance, the card does not start:

1. (Why?) The battery is dead.
2. (Why?) The alternator is not functioning.
3. (Why?) The alternator belt has broken.
4. (Why?) The alternator belt was well beyond its useful service life and not replaced.
5. (Why?) The vehicle was not maintained according to the recommended service schedule.

2. Ask clarifying questions. Asking questions allows a better understanding of the problem that is tried to be solved. This is a straightforward approach that yields great results.²

<n00b> How can I echo the last three characters in a filename?
<feline> If they're in a variable: echo ${foo: -3}
<feline> Why 3 characters? What do you REALLY want?
<feline> Do you want the extension?
<n00b> Yes.
<feline> There's no guarantee that every filename will have a three-letter extension,
<feline> so blindly grabbing three characters does not solve the problem.
<feline> echo ${foo##*.}

Conclusion

Solving problems is an indispensable part of our workflow today. And because the amount of information is increasing and the nature of problems continuously evolve, it is impossible to know the solution to all of the issues we encounter.

Therefore, a critical aspect of one’s success is the ability to identify and communicate what we are trying to obtain.

If you liked this article, consider subscribing below and following me on twitter (@iuliangulea).

---

1. The XY problem article on Wikipedia focuses more on support and customer service as areas where this problem is most often encountered. ↩︎

2. This example is taken from xyproblem.info ↩︎

---