Flash memory is rapidly moving into the data center market as was recently reported in BusinessWeek. Now that it is possible to collect more data, more analyses can be performed. In a continuously changing world, it means the data is continuously changing as well. And we can’t be looking at the same data forever, because it is sure to become stale and lose its value. So companies are anxious to process that data faster. And so we are seeing quite a rapid migration of flash memory into data centers, in order to serve up that data faster.
The Hadoop platform has brought distributed computing power for faster number crunching with all of the parallel cores being put to work. And CDNs have enabled providers to minimize the number of network hops that data streams have to travel through in order to serve up web pages with minimal latency. But now we have bottlenecks that may be arising just in pulling large data sets off those spinning disks. So there is a move to Flash memory, despite the premium price.
This really underscores how markets are working today. We need data to support operational decisions. But that data is changing. So we need to make our calculations on what happened and measure what changes we are seeing. It is always about trying to understand, “Why is this happening” and “How should we respond”. If the context of what is happening can be isolated, our understanding will improve. So whether you need to make decisions about: inventory management, production orders, sales support staff, product mix, sales promotions, geographic hot pots, or correlate results to outside events, you need to analyze the data to understand.
But, are you a scientist or engineer? This is about the mindset you bring to how you go about identifying, gathering and acting on data. Following the science path, you may indeed end up with an insatiable desire for data. And you clearly don’t want to end up in a state of paralysis-by-analysis. You really need to follow an engineering approach, where you need to build a process that is good enough, and can give you actionable information. Approximations and assumptions are what make this possible.
One thing that history has taught us is that it is all about adapting to change. Those who adapt successfully will be those who are in a position to prosper. The absolute key here is that you acknowledge the need to analyze the results and changes that are happening in your business. It is then vital to determine the measures that matter to your business, and act on them.
But, no data set is perfect. And, no model is perfect. You can continue to arm yourself with more data and more data analytics tools. But you need to assess to what extent you are improving. Just as the high frequency trading industry is coming to understand, front running will only get you so far. At some point, you have to revisit how you are creating value in your business, and whether you are still doing the right things. Just digesting data faster is not a panacea.