Competitive Intelligence and Online Privacy, NSA Leaks from Edward Snowden

In preparation for attending the Privacy Identify Innovation conference here in Seattle with peers, my recent conversations have been around the  kind of data that Edward Snowden and related NSA leaks stirred up.


These conversations live in another dimension almost as spooky as the Twilight Zone.

While the NSA is in the center of this issue, we should think about the thousands of companies that have access to our personal information.

Through these companies there are an untold number of individuals who have access to our digital homes without our knowledge. Some of them knowingly allow access (while others do it accidentally.)

We have dropped the ball: not only as companies and organizations in the U.S., but as a global society we have absolutely failed to understand how the Internet of everything relates to metro, regional, ethical, and moral audiences around the world.

If this was my house it would  be leaving my front door unlocked and leaving it unattended. When we have a digital version of our home we need to give a little bit of thought about the benefits and risks associated to it.

  • We have to keep in mind that data isn’t limited to live where it is created.
  • It isn’t limited to a time or place.
  • It is a universe of information that covers both historical and future possibilities.
  • Our digital neighborhood isn’t local- it is global.

We need to ask questions about who may visit our home.

What do they know about me?

This is a big question. 

You need to have a grasp of the questions below before you can really begin to identify what they know about you.

If someone is interested in you they have decided you are worth a certain amount of effort.

The reality of the situation is that there are a lot of rocks to kick over and look under.

When competitive intelligence professionals like myself get involved there is a level of strategic and investigative research that takes place using all sorts of systems that help look under rocks quickly.

Examining what is under each rock cost a little bit of money and effort. If I have intent and reason to turn over thousands of rocks I will find out more about you than being lazy and turning over just one.

The speed at which this investigation can take place depends on the tools and the likelihood that the research effort will be rewarded with knowledge.

Thinking about who really has access to the data

When I turn on the typical desktop computer there are dozens of companies involved with the security of the data.

A basic example of typical computer privacy points:

  • The Brand/Creator of the PC
  • CPU Manufacturer
  • RAM Memory Manufacturer
  • BIOS programmer
  • Video Card
  • Wireless Processor
  • High Speed Modem
  • Network Router


  • Adobe Flash
  • Java
  • Microsoft Office
  • Firefox, Chrome, Internet Explorer (with several dozen plugins)
  • Internet is supplied by Comcast (Comcast router boxes, relay junctions, data stores, etc)

Web Services

  • Banking
  • Utility Payments
  • Mobile Services
  • Entertainment (Netflix, etc)
  • Social Networks (Facebook, Linkedin, etc)
  • Communication tools (email, digital phone, etc)

Securing the basic elements above almost requires a degree in rocket science.

Yet as a society of web enabled users we’ve thrown caution to the wind and have opted-in to all sorts of things like social networks and freemium web services.

Multiply the basic example above for laptops, tablets, mobile phones, etc.

Facebook is an amazing example of our blunder.

Over a billion people have joined the site and have given access to our profiles, social networks, and personal communications.

In addition to what we share on, Facebook scripts and widgets run on tens of millions of sites.

These scripts cover a range of login, recommendation, analytics, and sharing functions.

If you were to figure out the combined data collection of Facebook across all of these sites you’d have trillions of interactions.

But we also have Google who has access to sites with

  • Google Analytics
  • Google Webmaster
  • Google Adsense
  • Google Content Networks
  • Google Gmail
  • Google Docs

When you overlap just the properties and data collection points of Google and Facebook you end up with information on almost the entire web using population. 

The NSA may have access to that data… but who else?

While the NSA may have access, I wouldn’t typically focus on whether the NSA has it.

You should be focusing on people and organizations that seek you harm (if the NSA has reason to cause you harm, then worry about the NSA.)

Google and Facebook don’t necessarily want to harm us, but they do want to make a few dollars in profit.

The core item to think about is that there are thousands of businesses in the industry of collecting and monetizing our data or using it for harmful or monetary purposes. These include data brokers, financial organizations, major employers, and big retail brands. They also include criminal and military organizations.

The information they are collecting is specifically important to the intent of why they are collecting it.

You can help figure out where you sit in the big picture as both an individual and as a business by running through a series of questions.

What are the problems?

Data will always be created and collected by some process.

The core problem comes from the question of Good vs Evil?

Good uses:

  • Trying to use the data to improve the education system of a local town.
  • Supporting entrepreneurs to create green, sustainable business.
  • Helping third would countries raise the standard living.

Bad Uses:

  • Identifying an individual’s commute time to work so robbers know when the house is empty.
  • Discriminating against employees based on what was perceived as private.
  • Disabling a city utility by crashing the utility grid.

Why are they collecting data?

Most organizations use online data to define and segment millions of users into a size they can interact with.

They want to strategically locate communities and individuals who matter to them.

This usually revolves around simple items such as:

  • How many interactions?
  • How many relationships?
  • How many transactions?
  • How many habits?

 Who is using it?

The answer to who is using it creates a number of tangents to think about:

  • Where are they?
  • How do they store it?
  • Do they sell it?
  • Do they abuse it?
  • Do they learn from it?
  • Where do they have interests?

What laws am I dealing with?

As you answer the above questions about you begin to identify the legal structures of where your data lives.

In the U.S. we have some very specific ideas about privacy and freedom of speech. These same ideas may not apply around the globe.

  • Where does all that data live?
  • Who owns the lines it moves across?
  • What jurisdictions apply to the servers?
  • What companies have access?
  • What employees have access?
  • What criminals have access?

What ethics am I dealing with?

With some of the legal concepts detailed we can begin to think about ethical and moral uses of the data.

Some cultures and countries have wildly different ethical and moral concepts.

  • Do they want to hurt/help me?
  • Do they want to hurt/help my family/friends?
  • Do they want to hurt/help my company?
  • Do they want to hurt/help my country?

What can I do about it?

The key to protection is understanding.

#1 – write down a list of things that are most important to you.

#2 – write down a list of people who want to hurt you.

#3- ask an expert to detail ways #1 and #2 interact.

#4- Create a plan for protecting the things most important to you can be used by people wanting to hurt you.

#5- Apply a scenario to two or three organizations you don’t like and ask yourself what you can do to them.

These basic steps will wildly vary in results depending on if  individual and group perspective.

By understanding value vs risk you can allocate where your effort will produce the most protection.


Big Data Brokers and Open Data Sources

The current state of the world is that we almost every action and fact we take are tracked and recorded; ranging from mobile phone usage, driving behaviors, social networks, and our shopping transactions.

Due to the nature of data and the fact that it exist in an invisible layer that most of us don’t comprehend, the results is that many of us don’t realize the ‘big business’ of our data.

Without our knowledge and comprehension there are hundreds of big data brokers who sell and barter our information for a wild variety of uses. These organization not only make massive amounts of profit by selling information we created, but they also make tremendous impacts to the bottom line revenue of businesses who understand how to use big data.

As I go through this article  (which has turned into a book) I’m going to give you a thought as to the scope of this business and why you should pay great attention to it both personally and professionally, along with giving you pointers to some of the available sources of data (ranging from open, governmental, local, industry, etc.)

We’ll examine some critical areas around big data

  • Why to use big data and open data sources regardless of how big or small your business is
  • Common areas to browse through and familiarize yourself with
  • Understanding what trends are affecting your future

Getting Started with the big question…

What is my data worth?

Part of answering the bigger questions around big data brokers and open data sources is the need to understand what your data is worth and that for every person, group, and organization there is an enormous scale of data being created.

Consider this, for every person: They interact with 100+ other individuals, 100+ local businesses, and 1000+ global businesses.
Each interaction creates a data point. Each data point has value. 

This federal action helps illustrate the scope of value: Oobama-executive-order-open-datan May 9th President Obama issued an Executive Order directing historic steps to make government data more accessible for economic growth and innovation. The order declares that the information is a valuable resource that needs to be used as a strategic asset for the nation. President Obama called for an Open Data Policy, requiring machine readable formats of data for government records that can drive long-term economic growth.

The origination of the order comes from growing concern including personal privacy, regulation, industry development, and competitive positioning of ‘who owns your data’

If the President of the United States is issuing an Executive Order regarding government data and considers it a national asset, chances are you should really be paying attention to the types of data you create and interact with.

By the People, for the People.

In an on-going series of efforts going back a decade, our government has a series of sites providing what I would refer to as an elementary view into some of the data we create. On a rating of 1 to 10 and of usability to the common person or business this data is a 1, it has questionable freshness, unidentified quality, and lack luster categorization.

The idea is there, but what the government is providing is really ‘only data’ –  It doesn’t come in fancy spreadsheet or colorful visualization that make you understand the significance of the data. This prevents the common professional and business owner from making decisions regarding on-going trends ranging from local, regional, and global perspectives.

Examples of basic government data

We also have Open Data

Open data is publicly accessible information that has been established by public interest groups, government, corporate entities, and expert practitioners.

The general idea behind Open Data is that even big corporations need to release some ideas about the information they have access to so that innovative entrepreneurs and community minded leaders can build interesting things with that data.

Example sources of Open Data


Big Data Brokers

Take all of your data, along with governmental and open data, then buy some more!

Big Data Brokers collect all sorts of data ranging from transaction and purchase behavior, consumer interest information, and usage patterns. The basic concept of a big data broker is that they collect merchant details at the point of sale, social data from your relationship network, and demographic identifiers from millions of different databases that have been collected under a few hundred organizations that fall into the category of being a ‘data broker.’

They sell basic pieces of data such as names, contact information, and additional demographics, like age, race, occupation and education level. There are lesser known data areas that include everything from how fast you drive your car (insurance reports, traffic records) to what your entire employment history looks like.

You also have big data brokers that sell information about products (such as Amazon store data), places (Google Maps), events, vendors, manufacturers, employee names, and stocks (Nasdaq, Dow Jones), etc.

The list is truly endless.

On the downside of big data brokers is the fact that there is someone willing to track everything and anything related to you. I’ve had the pleasure of speaking about the topic of privacy and invasive trends that affect us on a multitude of levels, but in most cases the ‘big data’ being collected about you is years ahead of legislation and regulation.


Think about everything you type into Google, Facebook, or your Mobile Phone.
Take every e-mail, text, conversation, or transaction and think about where all of the data goes.
Imagine the past five years of your  data collected on a computer and offered up for sale for $1

While the privacy issues are enormous, the beneficial areas are enormous as well.

Overlapping Points to Consider

We can think of the overlapping open, governmental, and big data segments in several different viewpoints that range across three primary categories: the individual, the business, the community.

Each of these categories have signals of data that reach far beyond the boundaries of the specific element. The visual below helps demonstrate how waves of data interact at dozens of points from the three categories.


  • For the individual big data has the risk of invasion of privacy, with the benefit of streamlined performance and increasing the quality of life. The same data about your home electric consumption can be used against you to market goods and services, or used for you to help reduce usage and be environmentally aware.
  • For the business side of data we run the risk of being the instigators of breaching personal privacy, while we have the benefits of improving income and employee/consumer lifestyle. We need to be aware of local and global differences in ethics, morality, and legality as we engage our businesses as both consumers and providers of competitive marketplaces.
  • For the community portion of big data we run a balancing act of considering privacy of the individual, the trends of our community, and the financial interactions of our community businesses. We also need to consider the asset of data as a community, as the risk and benefit for both individuals and businesses are almost entirely controlled by representation as a community (in the form of a real world community, association, employer, etc.)

What is all of this data used for?

That is a pretty big question. The most typical use is one of two things:

  • Option 1: attempting to collect enough data to reach a critical mass that is easy to sell. Most social networks (Facebook, etc) and search engines (Google, etc) need to collect billions of data points before there is enough quantity to sell the data over and over again in some form of profitable model.
  • Option 2: they are attempting to find areas of overlapping data that have a point of interaction that identifies a recurring situation or life-event trigger such as buying a new smartphone, getting married, or ordering a movie on demand.

How do I use this in business?

If we imagine both option 1 and option 2 happening at the same time, we could look at an example of several business entities sharing a common community. This could be a group of industry businesses, a few stores working together at the local mall, or perhaps international trade partners shipping into the same city.


When a new individual interacts with the community or any one of the three businesses, a data exchange is possible.

In the real world this could be thought of as a friendly referral (it can have darker tones as well.)

When the new individual is introduced, the data points they bring with them have dozens of interaction points with the three vendors.

big-data-brokers-for sale

The are important things to realize about the above chart.

  • When an individual first engages it is typically with a community layer.
  • The individual has data that reaches into the core of one business (perhaps a transaction) and has only secondary data touching the other two.
  • All three businesses collect different data points on the same individual.
  • The individual may be aware of one, two, three or none of the businesses.

How should I use Big Data?

(Disclaimer: this is entirely my opinion.)

Most big data is used to sell you stuff. This data exists about a tremendous number of things and you can use it to sell all sorts of ‘stuff.’
Making money and driving a strong economy can be a great reason to use big data.

As a serial entrepreneur and innovation consultant, I prefer to give people a big idea to go with big data. 

You can sell the data…You can buy the data… Or…

Change the World with Big Data.

You can build things with data.

You can inspire people with data.

You can change lives with data.

If you want to so do something big-  think about using big data to help your community, people who do not have access to the data, or your family. Look at your network of friends, family, and co-workers and consider the insights that are hidden in your data.

  • Take a look a local charities and think about how your data aligns with something that makes you feel good.
  • Look at local employment rates and use the data to make jobs for people who need them.
  • Use data to reduce city spending, improve the education system, or help solve homelessness.
  • Spend some time looking through the open data and government data above.
  • Think about ways you can visualize data to help people understand.

I hope this has given you some insight to
the world of Big Data.
Go and do something useful with it!


Some interesting context to specific Big Data issues

Health Data

Forbes: What the Government’s Big Data Dump Tells Us About the Hospital Market –  I believe in the power of markets to create efficiency and innovation, so I will go there. To unleash the power of entrepreneurs to create transparency and choice in the hospital services market. Fortunately this starting to happen. Companies like CastLightHealthInReach, and Pokitdok are good examples.

Data Value

AdAge: FTC Sting Operation Results in Warnings to 10 Data Brokers – The FTC “test-shopped” for data from the companies and determined that they may be in violation of the FCRA, according to the missives, dated last week. The firms in the crosshairs are ConsumerBase, Brokers Data, US Data Corporation,, 4Nannies, U.S. Information Search, People Search Now, Case Breakers and USA People Search. The tenth company was not named by the FTC.

 Privacy Data

IT World: Web Trackers are Out of Control – The amount of data compiled about me was jarring. BlueKai and its tracking partners (DataLogix, Lotame, and others) had aggregated some 471 separate pieces of data. On the other hand, BlueKai correctly determined that I have two teenage kids, I’ve lived in the same place for more than 7 years, carry a mortgage, and like sci-fi movies, travel, and politics.