from arstechnica.com: Recent leaks about the NSA’s Internet spy programs have sparked
renewed interest in government surveillance, though the leaks touch
largely on a single form of such surveillance—the covert one. But
so-called “open source intelligence” (OSINT) is also big business— and
not just at the national/international level. New tools now mine
everything from “the deep Web” to Facebook posts to tweets so that cops
and corporations can see what locals are saying.  

Due to the sheer scale
of social media posts, many tools don’t even aim at providing a complete
picture. Others do.

For instance, consider BlueJay,
the “Law Enforcement Twitter Crime Scanner,” which provides real-time,
geo-fenced access to every single public tweet so that local police can
keep tabs on #gunfire, #meth, and #protest (yes, those are real
examples) in their communities. BlueJay is the product of BrightPlanet,
whose tagline is “Deep Web Intelligence” and whose board is populated with people like Admiral John Poindexter of Total Information Awareness infamy.

BlueJay allows users to enter a set of Twitter accounts, keywords,
and locations to scan for within 25-mile geofences (BlueJay users can
create up to five such fences), then it returns all matching tweets in
real-time. If the tweets come with GPS locations, they are plotted on a
map. The product can also export databases of up to 100,000 matching
tweets at a time.

A look at the BlueJay interface shows it to be a fairly basic tool,
but one that gets its power from full access to the Twitter “firehose”
of all tweets. Users who want to search the Twitterverse have three
basic options: Twitter’s search API, Twitter’s streaming API, or full
firehouse access to Twitter from third party providers like GNIP and
DataSift. The first two are free but limited; BrightPlanet notes
that even the broader streaming API returns somewhere between one and
40 percent of the relevant tweets depending on Twitter’s load at the
moment. The firehose requires some serious infrastructure and a paid
contract with Twitter, but it provides all relevant tweets.

BlueJay partners with firehose providers and touts this fact in its marketing copy:
“BlueJay captures tweets from the entire Twitter stream, unlike all
other products on the market that only get a fraction of the tweets that
are being posted.” BlueJay is meant only for local monitoring of
tweets, but this also keeps the price in reach of local police
departments—$150 a month.

Of course, once you have this basic data, you may want to do more
with it. Say a suspect is tweeting from a GPS-enabled phone client and
appears to be dealing drugs. Forget bothering with the paperwork needed
to track the phone through a cell phone provider. BrightPlanet also
offers GeoTime, a separate data visualization tool that can take
exported BlueJay data and mine it to show where and when the target
travels, what he tweets about at various locations, and where his phone
resides at night. (BrightPlanet describes this as using “pattern
recognition to automatically detect and annotate time-space behaviors,
such as meetings, gaps, connections, clusters, and motion.)

“The bad guys are out there, and they’re talking to each other online,” BrightPlanet says in its sales pitch
to law enforcement. “We’re intimately familiar with OSINT needs and
deliverables, and we will deliver the Deep Web Intelligence that finds
the bad guys and lets you get them behind bars.”

As with most general purpose surveillance tools, though, the tech can
be turned to any purpose. In a sales flyer, BrightPlanet suggests using
BlueJay to “monitor large public events, social unrest, gang
communications, and criminally predicated individuals” to “identify
potential witness and indicators for evidence” and “track department
mentions.” The company has also expanded its tools to provide OSINT to
the pharma and financial services industries.

Do people really tweet about things like #meth? They certainly do,
though not all tweets provide any sort of actionable intelligence (and
many are about Breaking Bad):

Criminals do just come right out and tweet about their crimes,
but BlueJay appears to be more useful as a way to “listen in” on people
who would not ordinarily be talking to police. Used well, such tools
should make police departments more aware of both local problems and
complaints about their own work. Used less than well, it can be a bit
creepy, sort of on par with having a kid’s uncle listen outside her
bedroom during a slumber party. And used badly, it can make a nice tool
for keeping an eye on critics/dissenters.

BlueJay is a reminder that tweets are public. It’s an obvious point,
but one that’s easy to forget. No matter how many followers you have,
your tweet stream is really being broadcast to the world. And the world
is watching.