New Tricks: Data Mining With Google Spreadsheets

March 22, 2010

Happily, I stumbled across the following link:

Now You Can Mine Data With Google Queries Too

The interesting bit is below the comic where they actually reveal a method I hadn’t thought of: 

Using a query embedded in Google Spreadsheets to mine and graph data in Google’s engine.

While the actual instructions are terse, I was able to get things up and running by visiting the actual example, and then copying and pasting the individual cells for examination.

Here is the blow by blow:

First, decide what you want to mine.  One of the examples is for income, we will use this one.

Open up Google spreadsheets and in cell A2 put (complete as printed here):

=”"”"&”I make $”&B2&” per year”"”

[NOTE:  WordPress jacks up the quotes, so you are going to have to replace all of the quotes in the above with double quotes, or it won't work!]

Initially it is gonna look like this “I make $ per year”.

Now in B2 put a dollar amount: 45,000.

You should see your number populate in B1 now.

Finally, the magic that actually gets the query info.

Put the following in C2:

=importXML(“http://www.google.com/search?num=100&q=”&A2,”//p[@id='resultStats']/b[3]“)

[NOTE: Same problem here - WordPress tries to mess with the multiple quotes.  Replace all double AND single quotes manually and you will be fine, otherwise you will get an Error.]

After a brief load time you should see a number returned.  This is the number of returns that included your statement in cell A2.

Now copy and paste A2 and C2 down the line and change your values accordingly as you move down.

To create the graph, simply open “Insert->Chart” and choose your graph type.

To populate the graph with your data, make sure to clear the box right under “What Data?” and then click and drag down column C on your spreadsheet.  Make sure to remove Column C as labels.  You should see your data represented in the preview.

That’s it!  The world is now your oyster!  I can’t wait to apply this in some cases I am working on, I am still mulling over where this can be most useful, but the possibilities boggle the mind.

3 Comments | Tools and Tips | Tagged: , , , | Permalink
Posted by inforensics

Open Source and the Digital Forensics Lab

March 18, 2009

A while back I wrote an article for Evidence Technology magazine entitled “Seven Uses of Open-Source Software for the Digital Forensic Lab.” The article was primarily targeted towards law enforcement agencies that were having trouble getting funding for their labs.  In addition to building the case regarding cost savings, I discussed other advantages to running open sourced tools.

At recent conferences I have been increasingly approached by law enforcement as well as corporate investigation teams for advice on dealing with budgetary constraints, so it seems time to resurrect the topic.

Here is a summary of the “Seven”, the original article is here:

  1. Case Management: Although designed for CRM functions, SugarCRM actually makes a great inexpensive case management system.  It has the added advantage of allowing you to maintain a local copy instead of “the cloud”.
  2. Acquisition: The flexibility of “dd” for everything from imaging to memory and file carving makes it the number one contender in this category.  If you must have a MS based solution then you can also try FTK’s Imager lite (not mentioned in the original article).
  3. Analysis: Brian Carrier’s work on The Sleuth Kit with the optional graphical front-end of Autopsy is very worthy of support (tip of the hat to Dan Farmer and Wietse Venema for their original work on “The Coroner’s Toolkit”).  TSK has the added benefit of being scriptable (I use shell or PERL to get the job done).  You can check out TSK here.
  4. Miscellaneous: Stegdetect for dealing with steganography, Ophcrack for system passwords, Foremost or Scalpel for scriptable file carving.
  5. OS support: Linux.  You have access to libraries for NTFS, HFS++, etc. as well as everything you need for MS documents via OpenOffice 3.0. I have had great success with Ubuntu and variations (Mint).
  6. Virtual Platforms: At the time I wrote the article VMWare was offering their player and pre-made virtual systems for download.  If you are running off of a Mac you can use Parallels (not free, but very inexpensive) to run various pre-builds of Linux.  Even more compelling is Live View, which allows you to virtually mount and run a dd image without modifying the underlying image.  You can find Live View here.
  7. Mobile Acquisition and Analysis: Helix is no longer free, but those guys at e-fense  have given so much value to the rest of the world for so long via Helix that I say “Good on them!”.   You can also check out Backtrack 3 – just be aware that you run the risk of altering data if you boot up incorrectly with Backtrack.

What are some other “Can’t miss tools”?  Drop a comment in and tell the rest of us.

Leave a Comment » | Tools and Tips | Tagged: , , , , , , , | Permalink
Posted by inforensics

Firewire Target Mode and Other Apple Goodness

March 5, 2009

When performing information forensics on Apple platforms we have a few options for acquisition:

  • Firewire Target mode
  • BackTrack or Helix 3 (tested on intel platforms – works great, some caveats, though)
  • Pull the drive and do your thing!

Here is an article that describes yet another use for Firewire target mode.  It is good to be reminded of the flexibility available through some of these features:

Macworld: “Firewire target disk mode to the rescue“

While I am at it, here is some more wonderful Mac goodness:

TUAW: “Keyboard Shortcuts During Mac OSX Startup“

Somewhat related to the Firewire target mode discussion above.

Download YouTube (in HD as well) using Safari or FireFox

(Also useful for other streams).  Make sure to use the “HD” format so you can get .mp4 format in iTunes – otherwise you will need an FLV player.

Teleport: Control Multiple Macs With One Keyboard Mouse (Mac-centric Synergy-like program)

I have long used Synergy, but if you watch your logs you quickly realize that Synergy on a Mac is very “chatty”.  This is a good stand in for Mac only control.  If you need multiple OS support, then Synergy is for you.  Here is a Synergy version that is friendlier to Macs.

Are there any “Can’t live without them” features I have left out?

Leave a Comment » | Tools and Tips | Permalink
Posted by inforensics

Have Cellphone, Will PDF

January 13, 2009

From Twitter: “http://twitter.com/vidocrazor/statuses/1109866285“

This is another Gina Trapani “Upgrade Your LIfe” tip I found that has been HUGE for me.

I am a major whiteboard fanatic – if it were up to me all surfaces of every wall in my life would be covered with whiteboard.

The folks at Qipit.com will take a photo of a page , whiteboard, etc. and convert to PDF.  This service works straight from your mobile phone (assuming that you have a camera and email – you DO have that, right?), requires no software install, and is absolutely free – a killer combination.

As an example:  I have registered my number with Qipit and placed the “copy@qipit.com” email address in my contacts as “Qipit”.  Now when I need notes from a whiteboard I simply snap a shot with my iPhone 3g and email it to user “Qipit”.  Within seconds I have a PDF version of my whiteboard.

Occasionally the transmogrification will fail, simply realign the shot and repeat – it takes seconds.

The website is here: http://www.qipit.com/

Leave a Comment » | Tools and Tips | Permalink
Posted by inforensics

Lifehacker Goodness

January 13, 2009

From Twitter: “http://twitter.com/vidocrazor/statuses/1108280366“

I discovered Lifehacker not too long ago – there are some incredible gems mixed in amongst some of the cruft.

Gina Trapani’s book “Upgrade Your Life” is essentially a Best of… for the site and well worth the price of admission.

Some of my favorites from the book:

Quicksilver: Has sped up the use of my Macbook considerably.

Email and File Management: Includes some advanced uses for Spotlight.

Firefox Add-ons: Some of the add-ons recommended by the book have completely altered my management of investigative information gleaned from the web.

Time Management: Several tips from the book have increased my daily output.

Leave a Comment » | Tools and Tips | Permalink
Posted by inforensics