Competitive Intelligence and Online Privacy, NSA Leaks from Edward Snowden

In preparation for attending the Privacy Identify Innovation conference here in Seattle with peers, my recent conversations have been around the  kind of data that Edward Snowden and related NSA leaks stirred up.


These conversations live in another dimension almost as spooky as the Twilight Zone.

While the NSA is in the center of this issue, we should think about the thousands of companies that have access to our personal information.

Through these companies there are an untold number of individuals who have access to our digital homes without our knowledge. Some of them knowingly allow access (while others do it accidentally.)

We have dropped the ball: not only as companies and organizations in the U.S., but as a global society we have absolutely failed to understand how the Internet of everything relates to metro, regional, ethical, and moral audiences around the world.

If this was my house it would  be leaving my front door unlocked and leaving it unattended. When we have a digital version of our home we need to give a little bit of thought about the benefits and risks associated to it.

  • We have to keep in mind that data isn’t limited to live where it is created.
  • It isn’t limited to a time or place.
  • It is a universe of information that covers both historical and future possibilities.
  • Our digital neighborhood isn’t local- it is global.

We need to ask questions about who may visit our home.

What do they know about me?

This is a big question. 

You need to have a grasp of the questions below before you can really begin to identify what they know about you.

If someone is interested in you they have decided you are worth a certain amount of effort.

The reality of the situation is that there are a lot of rocks to kick over and look under.

When competitive intelligence professionals like myself get involved there is a level of strategic and investigative research that takes place using all sorts of systems that help look under rocks quickly.

Examining what is under each rock cost a little bit of money and effort. If I have intent and reason to turn over thousands of rocks I will find out more about you than being lazy and turning over just one.

The speed at which this investigation can take place depends on the tools and the likelihood that the research effort will be rewarded with knowledge.

Thinking about who really has access to the data

When I turn on the typical desktop computer there are dozens of companies involved with the security of the data.

A basic example of typical computer privacy points:

  • The Brand/Creator of the PC
  • CPU Manufacturer
  • RAM Memory Manufacturer
  • BIOS programmer
  • Video Card
  • Wireless Processor
  • High Speed Modem
  • Network Router


  • Adobe Flash
  • Java
  • Microsoft Office
  • Firefox, Chrome, Internet Explorer (with several dozen plugins)
  • Internet is supplied by Comcast (Comcast router boxes, relay junctions, data stores, etc)

Web Services

  • Banking
  • Utility Payments
  • Mobile Services
  • Entertainment (Netflix, etc)
  • Social Networks (Facebook, Linkedin, etc)
  • Communication tools (email, digital phone, etc)

Securing the basic elements above almost requires a degree in rocket science.

Yet as a society of web enabled users we’ve thrown caution to the wind and have opted-in to all sorts of things like social networks and freemium web services.

Multiply the basic example above for laptops, tablets, mobile phones, etc.

Facebook is an amazing example of our blunder.

Over a billion people have joined the site and have given access to our profiles, social networks, and personal communications.

In addition to what we share on, Facebook scripts and widgets run on tens of millions of sites.

These scripts cover a range of login, recommendation, analytics, and sharing functions.

If you were to figure out the combined data collection of Facebook across all of these sites you’d have trillions of interactions.

But we also have Google who has access to sites with

  • Google Analytics
  • Google Webmaster
  • Google Adsense
  • Google Content Networks
  • Google Gmail
  • Google Docs

When you overlap just the properties and data collection points of Google and Facebook you end up with information on almost the entire web using population. 

The NSA may have access to that data… but who else?

While the NSA may have access, I wouldn’t typically focus on whether the NSA has it.

You should be focusing on people and organizations that seek you harm (if the NSA has reason to cause you harm, then worry about the NSA.)

Google and Facebook don’t necessarily want to harm us, but they do want to make a few dollars in profit.

The core item to think about is that there are thousands of businesses in the industry of collecting and monetizing our data or using it for harmful or monetary purposes. These include data brokers, financial organizations, major employers, and big retail brands. They also include criminal and military organizations.

The information they are collecting is specifically important to the intent of why they are collecting it.

You can help figure out where you sit in the big picture as both an individual and as a business by running through a series of questions.

What are the problems?

Data will always be created and collected by some process.

The core problem comes from the question of Good vs Evil?

Good uses:

  • Trying to use the data to improve the education system of a local town.
  • Supporting entrepreneurs to create green, sustainable business.
  • Helping third would countries raise the standard living.

Bad Uses:

  • Identifying an individual’s commute time to work so robbers know when the house is empty.
  • Discriminating against employees based on what was perceived as private.
  • Disabling a city utility by crashing the utility grid.

Why are they collecting data?

Most organizations use online data to define and segment millions of users into a size they can interact with.

They want to strategically locate communities and individuals who matter to them.

This usually revolves around simple items such as:

  • How many interactions?
  • How many relationships?
  • How many transactions?
  • How many habits?

 Who is using it?

The answer to who is using it creates a number of tangents to think about:

  • Where are they?
  • How do they store it?
  • Do they sell it?
  • Do they abuse it?
  • Do they learn from it?
  • Where do they have interests?

What laws am I dealing with?

As you answer the above questions about you begin to identify the legal structures of where your data lives.

In the U.S. we have some very specific ideas about privacy and freedom of speech. These same ideas may not apply around the globe.

  • Where does all that data live?
  • Who owns the lines it moves across?
  • What jurisdictions apply to the servers?
  • What companies have access?
  • What employees have access?
  • What criminals have access?

What ethics am I dealing with?

With some of the legal concepts detailed we can begin to think about ethical and moral uses of the data.

Some cultures and countries have wildly different ethical and moral concepts.

  • Do they want to hurt/help me?
  • Do they want to hurt/help my family/friends?
  • Do they want to hurt/help my company?
  • Do they want to hurt/help my country?

What can I do about it?

The key to protection is understanding.

#1 – write down a list of things that are most important to you.

#2 – write down a list of people who want to hurt you.

#3- ask an expert to detail ways #1 and #2 interact.

#4- Create a plan for protecting the things most important to you can be used by people wanting to hurt you.

#5- Apply a scenario to two or three organizations you don’t like and ask yourself what you can do to them.

These basic steps will wildly vary in results depending on if  individual and group perspective.

By understanding value vs risk you can allocate where your effort will produce the most protection.


Big Data Analytics – 5 ideas to consider

As someone who drives complex ideas through and around dozens of roadblocks, I am often reminded that one of the best ways to help people is simply to highlight the right resources that will eventually transform into knowledge and wisdom.

In one of the most complex areas of my endeavors is the current ‘seizure’ mentality that is stopping the right technology and trends at the appropriate levels. This is especially true of the Big Data conversation that is challenging all levels of business.

Ironically the small and mid-sized business teams are having an easy time gaining traction in understanding the issues that affect their business. The small and medium businesses bring another problem: not having enough Big Data to play with and the budget to bring the expertise to make the initial change.

So the key to think about:

Enterprise Player = focus on tactical and economy of scale.

Small and Mid-sized Business = focus on wisdom of larger data providers.

If you are in an organization that is currently considering a Big Data initiative, please  take a moment to reach this article by John Weathington over at TechRepublic = To get Big Data buy-in, IT should let go of proof of concept …

Once you have had a chance to read it, come back and explore some of these other Big Data articles with the mindset of initiating the process with the right team and executive support in place. The simple reality is that Big Data isn’t an IT issue; it is a game changing opportunity for executive leadership and owners who choose to be nimble.


Dynamic data analysis the future of big data: Brainmates – ZDNet

Big data isn’t all about beefing up marketing strategies for organisations, according to founder of the product management …

“As a predictive tool, big data can deliver better products, but to effectively achieve this, product managers should use big data to test the questions that have previously been impossible to answer until after the product has gone to market,” he said. “Too often, we are led by what we think we can do and are trapped on those rails, whereas if we imagined the impossible, pretty soon we will find it has already been done elsewhere.”

Can Big Data make government cheaper? – ZDNet (blog)

The movie “Moneyball” celebrated a “Big Data” approach to maximizing returns from investments. Can Big Data do the same for government?…

“Today’s young people – hammered by the Great Recession, the loss of middle class jobs and gridlock in Washington – may take a more pragmatic approach. Stress – like the Great Depression and WWII – seems to focus Americans on solutions rather than ideology.”

Putting a Dollar Value on Big Data Insights – Wired

The Big Data phenomenon produces some mind-boggling statistics, such as the fact that the volume of global data produced doubles every two years. At the University of California at Berkeley, researchers …

” To cite just one example, retail giant Walmart was able to use Big Data analysis to drive a 10%-15% increase in completed online sales for $1 billion in incremental revenue, which is a well-planned ROI using data.”

5 Big Data Projects That Could Impact Your Life – Mashable

GCN argues that the first “big data” software was IBM’s DB2, a database management system it released back in 1983. What’s new is the ability to compile and process everything at such large scale …

“It’s a swanky term for a not-so-sexy concept: the idea that mass amounts of information can be analyzed to find hidden patterns, buried beneath terabytes of numbers, in Facebook posts,Google searches and Amazon purchases. These patterns can predict social trends and, in some cases, reengineer the way we live.”


Enhanced by Zemanta

Big Data Brokers and Open Data Sources

The current state of the world is that we almost every action and fact we take are tracked and recorded; ranging from mobile phone usage, driving behaviors, social networks, and our shopping transactions.

Due to the nature of data and the fact that it exist in an invisible layer that most of us don’t comprehend, the results is that many of us don’t realize the ‘big business’ of our data.

Without our knowledge and comprehension there are hundreds of big data brokers who sell and barter our information for a wild variety of uses. These organization not only make massive amounts of profit by selling information we created, but they also make tremendous impacts to the bottom line revenue of businesses who understand how to use big data.

As I go through this article  (which has turned into a book) I’m going to give you a thought as to the scope of this business and why you should pay great attention to it both personally and professionally, along with giving you pointers to some of the available sources of data (ranging from open, governmental, local, industry, etc.)

We’ll examine some critical areas around big data

  • Why to use big data and open data sources regardless of how big or small your business is
  • Common areas to browse through and familiarize yourself with
  • Understanding what trends are affecting your future

Getting Started with the big question…

What is my data worth?

Part of answering the bigger questions around big data brokers and open data sources is the need to understand what your data is worth and that for every person, group, and organization there is an enormous scale of data being created.

Consider this, for every person: They interact with 100+ other individuals, 100+ local businesses, and 1000+ global businesses.
Each interaction creates a data point. Each data point has value. 

This federal action helps illustrate the scope of value: Oobama-executive-order-open-datan May 9th President Obama issued an Executive Order directing historic steps to make government data more accessible for economic growth and innovation. The order declares that the information is a valuable resource that needs to be used as a strategic asset for the nation. President Obama called for an Open Data Policy, requiring machine readable formats of data for government records that can drive long-term economic growth.

The origination of the order comes from growing concern including personal privacy, regulation, industry development, and competitive positioning of ‘who owns your data’

If the President of the United States is issuing an Executive Order regarding government data and considers it a national asset, chances are you should really be paying attention to the types of data you create and interact with.

By the People, for the People.

In an on-going series of efforts going back a decade, our government has a series of sites providing what I would refer to as an elementary view into some of the data we create. On a rating of 1 to 10 and of usability to the common person or business this data is a 1, it has questionable freshness, unidentified quality, and lack luster categorization.

The idea is there, but what the government is providing is really ‘only data’ –  It doesn’t come in fancy spreadsheet or colorful visualization that make you understand the significance of the data. This prevents the common professional and business owner from making decisions regarding on-going trends ranging from local, regional, and global perspectives.

Examples of basic government data

We also have Open Data

Open data is publicly accessible information that has been established by public interest groups, government, corporate entities, and expert practitioners.

The general idea behind Open Data is that even big corporations need to release some ideas about the information they have access to so that innovative entrepreneurs and community minded leaders can build interesting things with that data.

Example sources of Open Data


Big Data Brokers

Take all of your data, along with governmental and open data, then buy some more!

Big Data Brokers collect all sorts of data ranging from transaction and purchase behavior, consumer interest information, and usage patterns. The basic concept of a big data broker is that they collect merchant details at the point of sale, social data from your relationship network, and demographic identifiers from millions of different databases that have been collected under a few hundred organizations that fall into the category of being a ‘data broker.’

They sell basic pieces of data such as names, contact information, and additional demographics, like age, race, occupation and education level. There are lesser known data areas that include everything from how fast you drive your car (insurance reports, traffic records) to what your entire employment history looks like.

You also have big data brokers that sell information about products (such as Amazon store data), places (Google Maps), events, vendors, manufacturers, employee names, and stocks (Nasdaq, Dow Jones), etc.

The list is truly endless.

On the downside of big data brokers is the fact that there is someone willing to track everything and anything related to you. I’ve had the pleasure of speaking about the topic of privacy and invasive trends that affect us on a multitude of levels, but in most cases the ‘big data’ being collected about you is years ahead of legislation and regulation.


Think about everything you type into Google, Facebook, or your Mobile Phone.
Take every e-mail, text, conversation, or transaction and think about where all of the data goes.
Imagine the past five years of your  data collected on a computer and offered up for sale for $1

While the privacy issues are enormous, the beneficial areas are enormous as well.

Overlapping Points to Consider

We can think of the overlapping open, governmental, and big data segments in several different viewpoints that range across three primary categories: the individual, the business, the community.

Each of these categories have signals of data that reach far beyond the boundaries of the specific element. The visual below helps demonstrate how waves of data interact at dozens of points from the three categories.


  • For the individual big data has the risk of invasion of privacy, with the benefit of streamlined performance and increasing the quality of life. The same data about your home electric consumption can be used against you to market goods and services, or used for you to help reduce usage and be environmentally aware.
  • For the business side of data we run the risk of being the instigators of breaching personal privacy, while we have the benefits of improving income and employee/consumer lifestyle. We need to be aware of local and global differences in ethics, morality, and legality as we engage our businesses as both consumers and providers of competitive marketplaces.
  • For the community portion of big data we run a balancing act of considering privacy of the individual, the trends of our community, and the financial interactions of our community businesses. We also need to consider the asset of data as a community, as the risk and benefit for both individuals and businesses are almost entirely controlled by representation as a community (in the form of a real world community, association, employer, etc.)

What is all of this data used for?

That is a pretty big question. The most typical use is one of two things:

  • Option 1: attempting to collect enough data to reach a critical mass that is easy to sell. Most social networks (Facebook, etc) and search engines (Google, etc) need to collect billions of data points before there is enough quantity to sell the data over and over again in some form of profitable model.
  • Option 2: they are attempting to find areas of overlapping data that have a point of interaction that identifies a recurring situation or life-event trigger such as buying a new smartphone, getting married, or ordering a movie on demand.

How do I use this in business?

If we imagine both option 1 and option 2 happening at the same time, we could look at an example of several business entities sharing a common community. This could be a group of industry businesses, a few stores working together at the local mall, or perhaps international trade partners shipping into the same city.


When a new individual interacts with the community or any one of the three businesses, a data exchange is possible.

In the real world this could be thought of as a friendly referral (it can have darker tones as well.)

When the new individual is introduced, the data points they bring with them have dozens of interaction points with the three vendors.

big-data-brokers-for sale

The are important things to realize about the above chart.

  • When an individual first engages it is typically with a community layer.
  • The individual has data that reaches into the core of one business (perhaps a transaction) and has only secondary data touching the other two.
  • All three businesses collect different data points on the same individual.
  • The individual may be aware of one, two, three or none of the businesses.

How should I use Big Data?

(Disclaimer: this is entirely my opinion.)

Most big data is used to sell you stuff. This data exists about a tremendous number of things and you can use it to sell all sorts of ‘stuff.’
Making money and driving a strong economy can be a great reason to use big data.

As a serial entrepreneur and innovation consultant, I prefer to give people a big idea to go with big data. 

You can sell the data…You can buy the data… Or…

Change the World with Big Data.

You can build things with data.

You can inspire people with data.

You can change lives with data.

If you want to so do something big-  think about using big data to help your community, people who do not have access to the data, or your family. Look at your network of friends, family, and co-workers and consider the insights that are hidden in your data.

  • Take a look a local charities and think about how your data aligns with something that makes you feel good.
  • Look at local employment rates and use the data to make jobs for people who need them.
  • Use data to reduce city spending, improve the education system, or help solve homelessness.
  • Spend some time looking through the open data and government data above.
  • Think about ways you can visualize data to help people understand.

I hope this has given you some insight to
the world of Big Data.
Go and do something useful with it!


Some interesting context to specific Big Data issues

Health Data

Forbes: What the Government’s Big Data Dump Tells Us About the Hospital Market –  I believe in the power of markets to create efficiency and innovation, so I will go there. To unleash the power of entrepreneurs to create transparency and choice in the hospital services market. Fortunately this starting to happen. Companies like CastLightHealthInReach, and Pokitdok are good examples.

Data Value

AdAge: FTC Sting Operation Results in Warnings to 10 Data Brokers – The FTC “test-shopped” for data from the companies and determined that they may be in violation of the FCRA, according to the missives, dated last week. The firms in the crosshairs are ConsumerBase, Brokers Data, US Data Corporation,, 4Nannies, U.S. Information Search, People Search Now, Case Breakers and USA People Search. The tenth company was not named by the FTC.

 Privacy Data

IT World: Web Trackers are Out of Control – The amount of data compiled about me was jarring. BlueKai and its tracking partners (DataLogix, Lotame, and others) had aggregated some 471 separate pieces of data. On the other hand, BlueKai correctly determined that I have two teenage kids, I’ve lived in the same place for more than 7 years, carry a mortgage, and like sci-fi movies, travel, and politics.

WordPress Security & Backup

This is one of those stories about WordPress you should read, because it affects you even if you don’t use (or even know about) WordPress.


On April 15th, the United States Department of Homeland Security, Computer Emergency Readiness Team issued a warning of an “ongoing campaign targeting the content management software WordPress…”

On the Department of Homeland Security site… 

CloudFlare, a web performance and security startup, has to block 60 million requests against its WordPress customers within one hour elapse time. The online requests reprise the WordPress scenario targeting administrative accounts from a botnet supported by more than 90,000 separate IP addresses. A CloudFlare spokesman asserted that if hackers successfully control WordPress servers, potential damage and service disruption could exceed common distributed denial of service (DDoS) attack defenses.

Why should we care?

According to Wikipedia,

WordPress is used by over 14.7% of Alexa Internet’s “top 1 million” websites and as of August 2011 manages 22% of all new websites. WordPress is currently the most popular blogging system in use on the Web, powering over 60 million websites worldwide.

According to, WordPress 3.5 has been downloaded 19,370,337 times (as of this article)

So what?

Imagine for a minute that 20,000,000 websites were running code for personal blogs, corporate websites, and local stores.

Imagine if four million of those were out of date and had security holes.

You don’t have to imagine.

WordPress 3.5 is currently the most current release of WordPress.

Roughly 65% of them are out of date.
*pie chart is directly from



Two-thirds of the WordPress sites are out of date…

Creating a  ‘feeding frenzy‘ of criminal hackers looking for easy victims.

We can see by searching Google Trends that search volumes for words such as WordPress Security and WordPress Backup are trending up with no sign of slowing down. If we look at small segments of time related to keywords in the marketplace we can see that backup and security solutions for WordPress as follow-up remedies after the situation has escalated.

In other words: WordPress users wait until a problem happens, then they complain they left the door unlocked.



How does this affect me?

Imagine the value of your personal data.

Think about the previous line item “WordPress is used by over 14.7% of Alexa Internet’s “top 1 million” websites”

  • How many times a month do you visit a WordPress site?
    (hint… you are on a WordPress site right now.)
  • How many of those sites have you joined up as a member?
  • Did you release your e-mail or privately talk with another member?
  • Does your e-mail, username, and password match one of your utility or bank logins?
  • Did you click on a message link from a friend only to realize it was spam?
  • Did you download a nasty virus on your computer off the web?
  • Did you wait a few extra seconds waiting for a site to load?

Simply said: there are hundreds of ways that a WordPress virus or hack can affect you, your business, or the people you care about. Some of them are simply nuisance items and others are a complete panic.

If you own a site that has been attacked, the resulting damage can cause irreparable damage to your business and your community members. Repairing a site can often cost thousands of dollars in lost income and technical fees (and the trust of your users.)


If you use WordPress: UPDATE IT along with ALL of the plugins you use.

Share this article with anyone who uses WordPress.

Encourage them to update the software they use and make frequent back-ups to protect themselves and their users.

We all have a responsibility to help each other have a positive and safe experience on the web.


Big Names (outside of the Department of Homeland Security) are talking about it.

Techcrunch, ArsTechnica, Hostgator…


Global WordPress Brute Force Flood – Gator Crossing – HostGator
Apr 11, 2013 Global WordPress Brute Force Flood The main force of this attack began last week, then slightly died off, before picking back up again

Fix For Recent WordPress Brute Force Attack Is Easier Than You …
Apr 14, 2013 Over the past couple of days it has been widely reported that WordPress based sites are being targeted by a massive brute force attack, one

Brute Force Attacks Build WordPress Botnet — Krebs on Security
Apr 12, 2013 Brute Force Attacks Build WordPress Botnet. Security experts are warning that an escalating series of online attacks designed to break into

Hackers Point Large Botnet At WordPress Sites To Steal Admin …
Apr 12, 2013 For the most part, this is a bruteforce dictionary-based attack that aim to that CloudFlare saw attacks on virtually every WordPress site on its

Huge attack on WordPress sites could spawn never-before-seen …
Apr 13, 2013 According to CloudFlare’s Prince, the distributed attacks are attempting to brute force the administrative portals of WordPress servers,

Worldwide WordPress Security Admin Attacks: Secure Your Blogs Now
Apr 16, 2013 WordPress Security is always an important part of running your blog, but right now it is even more important than ever. There is a severe

Update WP Super Cache and W3TC Immediately – Remote Code …
Apr 23, 2013 Shame on us for not catching this a month ago when it was first reported, but it seems that two of the biggest caching plugins in WordPress


Competitive Research Tools & Tips

Competitive research isn’t just for major corporations

There are hundreds of competitive research tools out there that provide *FREE* insight to what you should be doing as a business.

This article is going to go through a few of my favorite free tools. If you have any questions about how to apply them, please make sure to zip me off an e-mail or leave a comment below.

As an example for this article I am going to use the Samsung Galaxy S4 product launch.

*************** Read more