Big Data Brokers and Open Data Sources

The current state of the world is that we almost every action and fact we take are tracked and recorded; ranging from mobile phone usage, driving behaviors, social networks, and our shopping transactions.

Due to the nature of data and the fact that it exists in an invisible layer that most of us don’t comprehend, the result is that many of us don’t realize the ‘big business’ of our data.

Without our knowledge and comprehension, there are hundreds of big data brokers who sell and barter our information for a wild variety of uses. These organizations not only make massive amounts of profit by selling information we created, but they also make tremendous impacts on the bottom line revenue of businesses that understand how to use big data.

As I go through this article  (which has turned into a book) I’m going to give you a thought as to the scope of this business and why you should pay great attention to it both personally and professionally, along with giving you pointers to some of the available sources of data (ranging from open, governmental, local, industry, etc.)

We’ll examine some critical areas around big data

  • Why to use big data and open data sources regardless of how big or small your business is
  • Common areas to browse through and familiarize yourself with
  • Understanding what trends are affecting your future

Getting Started with the big question…

What is my data worth?

Part of answering the bigger questions around big data brokers and open data sources is the need to understand what your data is worth and that for every person, group, and organization there is an enormous scale of data being created.

Consider this, for every person: They interact with 100+ other individuals, 100+ local businesses, and 1000+ global businesses.
Each interaction creates a data point. Each data point has value. 

This federal action helps illustrate the scope of value: Oobama-executive-order-open-datan May 9th (2013) President Obama issued an Executive Order directing historic steps to make government data more accessible for economic growth and innovation. The order declares that the information is a valuable resource that needs to be used as a strategic asset for the nation. President Obama called for an Open Data Policy, requiring machine-readable formats of data for government records that can drive long-term economic growth.

The origination of the order comes from growing concerns including personal privacy, regulation, industry development, and competitive positioning of ‘who owns your data’

If the President of the United States is issuing an Executive Order regarding government data and considers it a national asset, chances are you should really be paying attention to the types of data you create and interact with.

By the People, for the People.

In an on-going series of efforts going back a decade, our government has a series of sites providing what I would refer to as an elementary view into some of the data we create. On a rating of 1 to 10 and of usability to the common person or business this data is a 1, it has questionable freshness, unidentified quality, and lack luster categorization.

The idea is there, but what the government is providing is really ‘only data’ –  It doesn’t come in fancy spreadsheets or colorful visualization that make you understand the significance of the data. This prevents the common professional and business owner from making decisions regarding ongoing trends ranging from local, regional, and global perspectives.

Examples of basic government data

  • Data.gov
  • Recovery.gov – how government spending is tracked
  • CIO.gov – resources from the Chief Information Officer
  • BLS.gov – Bureau of Labor Statistics (employment data)
  • Census.gov – the U.S. census data
  • Census.gov Maps – the census data by map
  • USAspending.gov – basic visualization tool/feeds of U.S. spending
  • FedStats.gov – over 400 different federal statistics from A-Z
  • MapStats – 400 different statistics by state, city, and judicial districts

We also have Open Data

Open data is publicly accessible information that has been established by public interest groups, government, corporate entities, and expert practitioners.

The general idea behind Open Data is that even big corporations need to release some ideas about the information they can access so that innovative entrepreneurs and community-minded leaders can build interesting things with that data.

Example sources of Open Data

Big Data Brokers

Take all of your data, along with governmental and open data, then buy some more!

Big Data Brokers collect all sorts of data ranging from transaction and purchase behavior, consumer interest information, and usage patterns. The basic concept of a big data broker is that they collect merchant details at the point of sale, social data from your relationship network, and demographic identifiers from millions of different databases that have been collected under a few hundred organizations that fall into the category of being a ‘data broker.’

They sell basic pieces of data such as names, contact information, and additional demographics, like age, race, occupation, and education level. There are lesser-known data areas that include everything from how fast you drive your car (insurance reports, traffic records) to your entire employment history.

You also have big data brokers that sell information about products (such as Amazon store data), places (Google Maps), events, vendors, manufacturers, employee names, and stocks (Nasdaq, Dow Jones), etc.

The list is truly endless.

On the downside of big data brokers is the fact that there is someone willing to track everything and anything related to you. I’ve had the pleasure of speaking about the topic of privacy and invasive trends that affect us on a multitude of levels, but in most cases the ‘big data’ being collected about you is years ahead of legislation and regulation.

EXAMPLE:

Think about everything you type into Google, Facebook, or your Mobile Phone.
Take every e-mail, text, conversation, or transaction and think about where all of the data goes.
Imagine the past five years of your  data collected on a computer and offered up for sale for $1

While the privacy issues are enormous, the beneficial areas are enormous as well.

Overlapping Points to Consider

We can think of the overlapping open, governmental, and big data segments in several different viewpoints that range across three primary categories: the individual, the business, and the community.

Each of these categories has signals of data that reach far beyond the boundaries of the specific element. The visual below helps demonstrate how waves of data interact at dozens of points from the three categories.

big-data-individual-business-community

  • For the individual big data has the risk of invasion of privacy, with the benefit of streamlined performance and increasing quality of life. The same data about your home electric consumption can be used against you to market goods and services, or used for you to help reduce usage and be environmentally aware.
  • For the business side of data, we run the risk of being the instigators of breaching personal privacy, while we have the benefits of improving income and employee/consumer lifestyle. We need to be aware of local and global differences in ethics, morality, and legality as we engage our businesses as consumers and providers of competitive marketplaces.
  • For the community portion of big data, we run a balancing act of considering the privacy of the individual, the trends of our community, and the financial interactions of our community businesses. We also need to consider the asset of data as a community, as the risk and benefit for both individuals and businesses are almost entirely controlled by representation as a community (in the form of a real-world community, association, employer, etc.)

What is all of this data used for?

That is a pretty big question. The most typical use is one of two things:

  • Option 1: attempting to collect enough data to reach a critical mass that is easy to sell. Most social networks (Facebook, etc) and search engines (Google, etc) need to collect billions of data points before there is enough quantity to sell the data over and over again in some form of profitable model.
  • Option 2: they are attempting to find areas of overlapping data that have a point of interaction that identifies a recurring situation or life-event trigger such as buying a new smartphone, getting married, or ordering a movie on demand.

How do I use this in business?

If we imagine both option 1 and option 2 happening at the same time, we could look at an example of several business entities sharing a common community. This could be a group of industry businesses, a few stores working together at the local mall, or perhaps international trade partners shipping into the same city.

big-data-brokers

When a new individual interacts with the community or any one of the three businesses, a data exchange is possible.

In the real world, this could be thought of as a friendly referral (it can have darker tones as well.)

When the new individual is introduced, the data points they bring with them have dozens of interaction points with the three vendors.

big-data-brokers-for sale

The are important things to realize about the above chart.

  • When an individual first engages it is typically with a community layer.
  • The individual has data that reaches into the core of one business (perhaps a transaction) and has only secondary data touching the other two.
  • All three businesses collect different data points on the same individual.
  • The individual may be aware of one, two, three or none of the businesses.

How should I use Big Data?

(Disclaimer: this is entirely my opinion.)

Most big data is used to sell you stuff, you can use e-commerce scraping and use that data to compare your business with your competition. There exists a huge data about a tremendous number of things and you can use it to sell all sorts of ‘stuff.’

Making money and driving a strong economy can be a great reason to use big data.

As a serial entrepreneur and innovation consultant, I prefer to give people a big idea to go with big data. 

You can sell the data…You can buy the data… Or…

Change the World with Big Data.

You can build things with data.

You can inspire people with data.

You can change lives with data.

If you want to so do something big-  think about using big data to help your community, people who do not have access to the data, or your family. Look at your network of friends, family, and co-workers and consider the insights that are hidden in your data.

  • Take a look at local charities and think about how your data aligns with something that makes you feel good.
  • Look at local employment rates and use the data to make jobs for people who need them.
  • Use data to reduce city spending, improve the education system, or help solve homelessness.
  • Spend some time looking through the open data and government data above.
  • Think about ways you can visualize data to help people understand.

I hope this has given you some insight into
the world of Big Data.
Go and do something useful with it!