Small Data sometimes leads to the real life answers that Big Data Doesn’t.

Think Small: Time For Marketers To Move Beyond The Big Data Hype

Everyone seems to be talking about big data. From feature stories in popular media, such as Time magazine, to breathless predictions from Fortune that big data could generate millions of new jobs, plenty of steam is coming out of the big data hype machine.

Want to uncover hidden patterns about customer behavior, predict the next fashion trend, or see where to focus ad spend? There’s an app for that.

It shouldn’t have been a surprise that last year Deloitte Consulting predictedmore than 90 percent of the Fortune 500 would have some big data initiatives under way by the end of 2012. Yet despite significant investment in big data technologies and services–spending is expected to reach $10 billion in 2013, according to IDC–many organizations continue to struggle with turning their big data vision into practical insights and tools for everyday workers.

Moreover, some experts are starting to wonder if and when the economic payoff for big data will come. As one health-care services executive put it, “People are getting infatuated with the shiny object of big data, yet it needs to have a purpose.”

That purpose can be found in small data.

I have been exploring the state of big and small data for the past year, and recently concluded a study of how organizations are looking to bring the power of big data to the masses. As part of this research (supported by Adobe, along with Actuate, HubSpot, and Visible), we interviewed more than a dozen planners, practitioners, and strategists to get their perspectives on the state of big data, what tools and approaches they are using, and how they see the demand for consumer-style apps within their organizations and client bases. We also examined the influential literature and related research in this space, and aimed to put today’s developments in a historical perspective to frame out use cases, foundations, and benefits of “thinking small.”

This article shares some of what we learned.

The Benefits Of Thinking Small
More than any other group, marketing organizations have been on the frontlines of Web and social innovation, and they have seen the potential of database marketing, social listening, and even data-driven mobile applications. Plus, among the many marketers we have talked to, the majority are clamoring for new approaches to harnessing the power of data and turning it into practical tools, apps, and campaigns for customers and those who serve them.

While big data (especially predictive analytics) has great potential, it must be actionable and accessible beyond the small number of “experts” who have access (and aptitude) to high-end tools in order to deliver value. Plus, to serve the broadest set of business objectives and users, the goal isn’t just to accumulate more data assets. Rather, it’s about collecting data that is already available, discovering its meaning in the context of the task at hand, and delivering the right data in the right format to the broadest set of users.

This is the essence of the small data philosophy–where apps and tools are simple enough for marketers who aren’t data scientists to get just the information they need, precise enough to deliver insights and answers where users need them, and easy enough for users to add new insights and  share them with peers.

While the term “small data” can also apply to the size of data sets or the amount of data that can be conveniently stored by an average user, our definition focuses on both the type and use of data assets to create value for nontechnical users: Small data connects people with timely, meaningful insights (derived from big data and/or “local” sources), organized and packaged–often visually–to be accessible, understandable, and actionable for everyday tasks.

This definition applies to end-user apps as well as the analyst workbenches and tools for turning big data sets into actionable small data–a key priority for business and IT executives, the majority of whom struggle to convert their volumes of data into actionable intelligence. As one marketing analyst for a tech firm stated, “We have to make data digestible by everybody!”

So what is the right data to look for upstream, from apps and tools? At the risk of oversimplifying the world of data management, Digital Clarity Group separates customer-related data into three main groups:

• Transactional data: The “classic sources,” typically the domain of data warehousing, customer relationship management (CRM) reporting, and large-scale analytics. It consists of mostly “inside” data. The ability to create rich applications, dashboards, and reports to bring this type of data to life–and make it more consumable and actionable by more users–is a core element for delivering value in the last mile of big data.

• Online data: The “digital sources,” delivered as Web reports, user profiles, or predictive models. This group consists of mostly “outside” data. Web adoption has created massive volumes of customer preference, behavior, and user-generated product data that complement transaction data.

• Social and mobile data: The “new sources,” gathered from monitoring and listening tools, and processed via text and sentiment analysis. Social data can be both “inside” and “outside” data, but it tends to be uniquely conversational in nature, as opposed to behavioral or transactional data. Social channels are rich with “local” small data that is ready to be collected to inform marketing and buyer decisions. It can also add context to transactional records as well as amplify digital campaigns via word of mouth and social sharing.

Against this backdrop, it should be easy to see why it’s time to re-envision the “last mile” of big data. Not only are many new data sources, formats, and delivery channels in play, but there are also increasingly savvy customers and support teams who need tools tailored to their needs.

This is where small data comes in. Taking a user-centric view means shifting focus beyond the three Vs (volume, velocity, and variety) of big data, and adding a new V: the elements that create value for the end user.

Even more so, if big data has been largely about machines and processing power, then small data is about people, context, and individual requirements. This means empowering users with visual elements and engaging experiences. It also means employing intuitive listening tools and campaign platforms. The result: better market insight, targeting, offers–and, ultimately, more deals.

Learn More: This article is based on a new DCG report, “Thinking Small: Bringing the Power of Big Data to the Masses,” which is available for download from Adobe (free, registration required).