It is a truism that "understanding your customer is important in business". And yet it seems remarkably easy to identify examples where insufficient attention is paid to this -- particularly in the software and hardware industries.
It probably doesn't help to speculate the causes. Executive ego, inwardly facing engineering cultures, inexperienced product managers, shallow "marketing" perspectives, along with many other factors may be to blame. A quick perusal of Dilbert cartoons offers many candidates for consideration. The real issue, however, is what to do about it.
For such a challenging question, the answer is remarkably pedestrian: decide to do something about it (and then do it). Get out into the field and observe your customers. Don't limit yourself to the vocal few who seek you out; go and find a representative sample. If circumstances permit it (i.e. you have a large enough customer base and they are sufficiently cooperative) make use of well-crafted surveys, which are quick and easy to implement now thanks to web-hosted survey services such as SurveyMonkey and PollDaddy.
Sometimes surveys will directly answer your strategic questions. But more often than not, they instead shine a light on where to look more closely in more one-on-one forms of customer research. Also they often provide a convenient validation to whether your one-on-one interactions are representative of your customers as a whole.
Having focus to your customer research will help drive to a positive result. Some fundamentally simple questions can lead to illuminating answers, including:
While it is tempting to applaud any investment of time and money into "knowing your customer", experience has taught me to "look behind the curtain" before putting too much faith in the results. The process of data collection and interpretation matters. Some of the "corruptions" of the process that I have observed include: projecting "tradeshow buzz" or "PR buzz" as genuine customer feedback; putting too much faith in "experts" (who just happen to have relationships with the manager that brought them in); skewing survey questions to elicit a predetermined response; surveying an unrepresentative sample of customers.
There is a lot we can tell from out customers by looking at their purchasing patterns, also. For this to be effective, you need good sales data, and some skill in processing and interpreting the results. Looking at sales data tells us both things about the customers, and also things about our own company (what I would term here as "knowing ourselves"). In the latter respect, I've often found that internal perceptions about the business (which products are "hot"; which products "pay the bills", etc.) don't exactly match the objective numbers. If you don't have someone keeping an eye on product line trends for revenues, units sold, etc. then it is easy for perceptions in a company to be influenced by other factors (e.g. executive desires, PR buzz, and other internal agendas).
"Knowing ourselves" also extends to having a realistic picture of the capabilities of the organization to respond to opportunities and product requirements. Having a list of product ideas that the organization cannot deliver doesn't really help. Good product management involves balancing the needs and opportunities in the market with the ability of the engineering team to deliver (unless, of course, there is executive sponsorship for deep partnerships or M&A activity).
But beyond the technical aspects of delivering a product, I always encourage product managers to think about the "Go To Market" infrastructure that would be necessary to pursue an opportunity. For example, a software product designed for business professionals might be relatively easy to adapt for a consumer audience. But a company with only an existing Go To Market infrastructure for selling high-priced professional software in a B2B scenario would likely have to make a big investment in a consumer-oriented marketing and channel infrastructure to be successful. Understanding your existing Go To Market infrastructure, and having a good insight into how your product is sold by your channel, will help guide strategic decisions going forward.
There are, of course, many other factors to consider in strategic planning and marketing. These include competition, market trends, technical advances, alignment to corporate vision/mission, appeal for the board or shareholders, and so on. But a good understanding of the customer and characteristics of one's own business make a foundation on which good decisions can be built.