The Era of Tera : Technology : MobileMag

I think this has a lot of good points. Gelsinger, in particular, has a lot of good points. Now, his points are certainly to help drive sales of ever faster CPUs because, really now, who needs even current CPUs for most basic word processing? Certainly, it takes a good CPU to do the advanced word processing we all do now, with real time spelling and grammar checking as well as real time formatting and what not. Especially since we need our MP3s playing in the background. ;) But I digress... (shocking, huh?)

The real problem for us, at least, is how to manage our images (video and sound, too, but that's just an extension). Back on our France trip, we averaged 1 Gigabyte a day of images. We also had to go through the pain of dumping images multiple times a day onto my Archos using a card reader for the Canon DSLR, the Sony DSC-U20, and the Casio EX-4U. Our largest card was a 512Megabyte card in the Canon (thanks Ray!) with a backup 256MB card, followed by a 256MB for the Casio and a 128MB for the Sony. What this essentially means is that we often had to wait for a good dump point and couldn't just shoot all out with the Canon.

Enter now... for Laurie's company, we've just ordered her a 2.2GB card and we'll be getting her a 4GB card once quantity returns (another topic entirely). Given the largest card we own is 256MB, this will give 9 times the storage. I anticipate that this will mean significantly more images will be taken. We're going on a road trip soon to the Vegas area. That will provide a good test.

But anyway, I digress again. The whole point is, within a couple of months when we go on trips we could be easily rolling 5Gigabytes of images out a day. How do we sort, archive, modify, keep track of modifications, find, search, etc. this massive (by today's standards -- wait 'til we start shooting in RAW!) amount of data??? And this is just images which are, by comparison to audio and video, easy to flip through and view.

We have some ideas of an in-house solution that might work. (Laurie is an excellent db schema designer -- she's had daily "practice" on large eterprise-wide solutions over the last couple of years. My goal with this is that it'll be a combination hardware and software solution. I could see a machine hosting a service like this easily carrying a terabyte or so of storage just for today's needs. Well, let's think. To store a 10 day trip of all the original data, we'd need 50 Gigabytes of storage. 20 trips and we need 1 Terabyte. Now, that's not taking into account the removal of rejects, except that the act of removing rejects means sorting through images. The Canon is a 6Megapixel camera. The images are noticeably slow to load on my laptop. And this is all for now. Tomorrow we'll need so much more.

The Era of Tera??

How about the Pita of Peta computing... ;)

Posted by Shane on February 22, 2004 10:09 AM | Permalink