In the year 2000 storage cost about $12/GB [1]. This article claims 10s of TB, so let's say 20 TB, which is the most conservative possible estimate. 20 TB = 20000 GB. At $12/GB, that comes out to $240,000. That doesn't include retransmission cost (for busted drives) or shipping. I'm not sure the analysis is correct here.

Presumably their minimum commit was a lot lower than $240k, because otherwise they wouldn't have shrunk from just using the connection. They probably saved a lot of money by doing it their way.

[1] http://www.mkomo.com/cost-per-gigabyte

* I wasn't around then so I don't actually know.

* * It also depends what part of the year they needed to do the transfer. By [1], costs had dropped to $8/GB by the end of the year.

"storage cost about $12/GB "

Hard drives cost $12/GB. Tapes/optical media maybe $1/GB. Maybe I'm missing something but it seems like they would be fine for this.

Enterprise tape drives ain't cheap iirc. But anyway, I was responding specifically to the HDD idea.

Right - but they had to buy those hard drives anyways, so it didn't cost them anything more than the transportation cost of the drives to buy them at the source, load them up, and then ship them.

It would be hell to get the data off before imaging the drives. What do you transfer it to? The shuffling would require a lot of technical work, which also costs money. Also, we don't know if they were on homogenous hardware by then -- they may have still been using budget pcs and refurb'd parts. So it's hard to know if your plan could work. The investment of inventing a process for mass data transfer versus using the normal process but only during a certain time period is also significantly higher.

I have to think someone ran the numbers. It's not like the folks in 2000-google were dummies.

Admittedly, it would have been a lot of drives. Back in circa 2000, according to http://www.mkomo.com/cost-per-gigabyte, 20 Gigabyte Drives were in vogue @$200/pop. So, a Terabyte would have been 50 drives, 10 Terabytes @500 Drives.

10s of terabytes at a minimum 1000 drives. I'm beginning to question how "simple" an operation this might have been after all....

> It would be hell to get the data off before imaging the drives. What do you transfer it to?

I'd guess you'd transfer it to the same thing that was accepting the data when it was coming over the 1Gbps link?

The parameters of that scenario were that you'd ship the data on the disks that were eventually to store it (presumably without duplicating the OS on each during the transfer process). Therefore, there'd be nothing to move the data to.

You'd have to buy the drives anyway.

Even if you didn't consider cross-country-shipped drives to be data-center reliable (e.g.: the transfer media weren't re-used in the datacenter), you could simply cycle through drives. If your goal was 1 month to transfer 20 GB, you utilized overnight shipping, and required a day to load/unload data from drives at either end, that's 200 drives in flight, 50 of which are outbound, 50 returning, 50 writing, and 50 reading, on any given day.

You even get to take weekends off.

It's $48,000 in media. Plus shipping (not inconsiderable). You'd save on the cost of dedicated data transfer equipment (datacenter, router, links), though you'd need infrastructure for reading/writing data, and of course, storage at the remote side if you're not simply shipping disks for installation (which, frankly, is probably your best bet).

Resync should be fairly reasonably accomplished over the wire.

You can vary the math by taking longer (fewer drives, less cost) or shorter (more drives, higher cost), or even by utilizing ground shipping (7 days) at far lower cost.

What about the physical size of 1 GB in year 2000? How many HD do they need? and how much does it cost them to transport those drivers?

Your 16.2TB estimate is for one month. The article says this continued for multiple months.

You'd trust your hard drives with USPS?

Edit: I'm not implying that USPS is necessarily unreliable, I'm curious to hear if this is common practice.

Presumably I'm not sending the originals. Also, powered down, hard drives are pretty robust - their read/write heads are locked up. Remember - those drives have been bouncing around a LOT to get to you through retail/wholesale channels.

[edit] Also - they obviously had to buy hard drives for the destination, right? Presumably, early stage google was all about the open-chassis on plywood slam drives in approach, and wouldn't have been averse to loading up 30 or 40 drives, mailing them out, and simply diffing and resyncing as needed if 1 or 2 drives had issues, which would have been unlikely.

Also, presuming you needed to have one of your people go to the remote data center, there is always the old, move the datacenter with hard drives in my luggage trick. Even in 2000 you could put a lot of data into carry-on.

You've always got to expect a certain amount of packet loss over any connection.

RAID = Redundant Array of Independent Deliveries

You could make up any small number of botched drives with over-the-wire transfers. If it were a sufficient number of drives, you could simply re-image and re-transmit the data.

When they're adequately backed up, as in this circumstance, why not? Send the data in triplicate if you want, just to avoid the hassle of re-sending any that get damaged. Regardless, the cost would be trivial, even for young-Google.

I can assure you that there were drives larger than 20G in 2000. I had a server with 4 23G Seagate Elites at that time, and the Elite was an _old_ 5 1/4 FH SCSI drive. 18G Seagate Barracudas were certainly alive and kicking since 1997 at minimum (the year I started building real production servers).