Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

Backup of a virgin 14hour TiVo took 5 CD"s

I just did a backup of a 14 hour TiVo that had never been powered up. Below is some information for those who care on how well it compressed. This is using gzip with the "best" compression setting.

I imaged the drive into 645MB chunks and then gzipp'ed each one.

I ended up with 21 gz files for a total of 2,891,456,088 bytes. That is about 2.7GB of space needed for the files. So it took 5 CD's to hold it all.

The first two 645MB chunks compressed down to 83MB and 11MB respectively.

The next 12 645MB chunks compressed down to 642KB each! This is because each of them was completely full of zeroes.

The next 7 645MB chunks are where the mpeg data kicks in. You can see how the data tapers off near the middle and then starts back up again.

510MB
638MB
638MB
147MB
18MB
579MB
126MB

Those numbers are rounded up to the nearest megabyte in case you are checking against the total up at the top!

So if anybody actually compressed an entire virgin 14 hour unit on to 1 CD I am amazed. There may be better compression algorithims than gzip uses, but not that good.

One thing could possibly be that this 14 hour unit is newer and possibly contains more mpeg data (demo mode ?) and that causes it to be bigger than older units which might backup to a single CD.



------------------
Belboz
belboz@cinci.rr.com

POST #1 | Report this post to a moderator | IP: Logged

Registered: Jan 2000
Location: Charlotte, NC USA
Posts: 1439

Dr. Belboz,

What sentence did you use in Linux? Can't you compress the chunks while copying? Did you then try to restore the backup from CD? What sentence did you use for that?

Forgive my ignorance and thanks for your help.

Xaa

POST #2 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

Wow, I am suprised. I am currently backing up a virgin 30GB. I will post the single compressed file size and how long it took as soon as it's finished. How long did your backup take, Mr. BelBoz?

Justin

POST #3 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

Thanks for the info. Any thought's of piping the dd into gzip first, then split? I realize their technially quite different. The method you used could allow you to restore directly from cd, where dd into gzip would force you to assemble the complete image before dd'ing to the target. Thoughts? Thanks.

POST #4 | Report this post to a moderator | IP: Logged

Registered: Jun 2000
Location: None Of Your Business
Posts: 7

quote:

---

Originally posted by Justin_Thyme:
Wow, I am suprised. I am currently backing up a virgin 30GB. I will post the single compressed file size and how long it took as soon as it's finished. How long did your backup take, Mr. BelBoz?

Justin

---

POST #5 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

Xaa,

Yes I could have compressed it one step, but I wanted to create the big image chunks first and examine them.

I did the images with the following command.

dd if=/dev/hdc bs=512 | split -b 645m - /mnt/windows/virgin

That created files named virginXX in this casre virginaa to virginau

I gzipped them with the --best parameter.

I don't have a Quantum drive to fully test the backup (its on its way) but I did restore the image to a Maxtor drive and it "looked fine". Will test the image for real when I get the Quantum drive.

-----------------

Justin,

It took about 3 1/2 hours to do image and compress.

-----------------

Tom,

I could have done the one big gzip and then split that, but I did it my way for a couple reasons.

1) I wanted to examine each chunk first.

2) I was going to put it on CD anyway so I figured why split one big archive? By keeping the images seperate and gzipping each one individually I think it is more fault tolerant. By that I mean if one gz file has a bad spot on the CD the other files are fine. If its one big gz file you lose it all. I was also able to "double up" some of the backup archives that were smaller and place them on some of the other CD's in the backup set for redundancy.

------------------

Sorphin,

Seems like we had very similar results. I take it yours was a 14 hour unit also?

------------------


All,

I can't really believe anyone got a backup on one CD. If they did I would love to hear from them and see some information on their image file posted. I think the one CD backup is an urban myth!




------------------
Belboz
belboz@cinci.rr.com

POST #6 | Report this post to a moderator | IP: Logged

Registered: Jan 2000
Location: Charlotte, NC USA
Posts: 1439

Dr. Belboz,

Thank you. You are indeed a credit. You efforts and comunication are appreciated.

Xaa

POST #7 | Report this post to a moderator | IP: Logged

Registered: Mar 2000
Location:
Posts: 1031

quote:

---

Originally posted by sorphin:
Tried doing that myself, partitions 1-10 compressed to a total size of ~150 megs or less
and partition 11, ended up at 2 Gigs within the 1st 4 files (that were 650M each).. and this is a new one wiht the demo stuff on it too, and a very virgin image.

---

Ahh the demo stuff. I hadn't thought of that. My Tivo never had the demo stuff on it..

Yah, it wouldn't compress well with that on there.

Doc Belboz: The Demo stuff has got to be the reason for the 1 cd backup. They didn't have any of that until recently. My machine was 1.2.1 when I got it, with only a little intro movie on it. That would probably have compressed to a single cd when I got it. Not anymore of course, but still.


------------------
Moderator - AVS Forum - Tivo Underground
"If once you start down the dark path, forever will it dominate your destiny. Consume you it will!" -- Yoda

[This message has been edited by Otto (edited July 20, 2000).]

POST #8 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by Otto:
Ahh the demo stuff. I hadn't thought of that. My Tivo never had the demo stuff on it..

Yah, it wouldn't compress well with that on there.

Doc Belboz: The Demo stuff has got to be the reason for the 1 cd backup. They didn't have any of that until recently. My machine was 1.2.1 when I got it, with only a little intro movie on it. That would probably have compressed to a single cd when I got it. Not anymore of course, but still.

---

POST #9 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Atlanta, GA, USA
Posts: 83

Has anyone considered the following:

1) Delete all shows from TIVO
2) Write a program that fills the file system with one big file that is all zeros and then deletes that file.
3) Do your drive backup

That would replace all that uncompressible deleted MPEG info with very
compressible deleted material.

Just curious.....

POST #10 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by David Scavo:
Has anyone considered the following:

1) Delete all shows from TIVO
2) Write a program that fills the file system with one big file that is all zeros and then deletes that file.
3) Do your drive backup

That would replace all that uncompressible deleted MPEG info with very
compressible deleted material.

Just curious.....

---

POST #11 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

quote:

---

Originally posted by Justin_Thyme:
I am currently backing up a virgin 30GB. I will post the single compressed file size and how long it took as soon as it's finished.
Justin

---

POST #12 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by Justin_Thyme:
The backup is finished. It took under 14 hours and perhaps much less than that, I last checked it's status at the 8 hour mark. The whole image file is only 584,428 bytes. I used the following commands:
mount -t vfat /del/hda1 /mnt
cd /mnt
dd if=/dev/hdb bs=512 | gzip -9c > rawimage.gz

It's worth noting that this virgin 30GB drive has ver 1.21 on it.

Justin

---

POST #13 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

quote:

---

Originally posted by Justin_Thyme:
The whole image file is only 584,428 bytes.
Justin[/B]

---

POST #14 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by Justin_Thyme:
[B]
Mr. BelBoz, does the difference in image size from your test to mine seem logical? Seems too different to me . . .

Justin

---

POST #15 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

quote:

---

Originally posted by Mr. Belboz:
Justin,

List the contents of your gzip archive. Look at the size of that file.

That should be the total size in bytes of your drive. If you have the original drive connected to Linux still you can do the following.

cat /proc/ide/hdX/capacity

---

POST #16 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by Justin_Thyme:
I still have it connected and the answer is 5863344, roughly what the figure mentioned earlier. I too am awaiting a pair of LCT10 drives (hopefully) tommorrow that I can expand this image back onto a drive to test . . .

Justin

---

POST #17 | Report this post to a moderator | IP: Logged

Registered: Jun 2000
Location: None Of Your Business
Posts: 7

quote:

---

Originally posted by Mr. Belboz:
Justin,
.....
My original thought about the demo mode must be it. I did some reading through the old forum and it turns out some 1.2.1 and 1.3 units can have it. Once you run guided setup you lose the demo.

When I get my spare Quantum drive I will check this further.

---

POST #18 | Report this post to a moderator | IP: Logged

Registered: Not Yet
Location:
Posts: N/A

quote:

---

Originally posted by Mr. Belboz:
I mean the size of the file uncompressed, not the archive itself. Sorry if the previous message was not specific enough. You can get that by listing the gz files contents.

If you do a gzip -l filename.gz it will show you the files actual size and compressed size (along with the ratio of compression). If you take the actual uncompressed size that number should be the total capacity of the drive in bytes.

I look forward to hearing how your restore does.

---

POST #19 | Report this post to a moderator | IP: Logged

Registered: Dec 1999
Location: Cincinnati, OH
Posts: 185

quote:

---

Originally posted by Justin_Thyme:
I issued the following commands:
mount -t vfat /dev/hda1 /mnt
cd mnt
ls

I got a directory listing including the rawimage.gz file

gzip -| rawimage.gz

Linux responded with
rawimage.gz: not found

Figuring that I needed to specify the location, I typed:

gzip -| /mnt/rawimage.gz

It accepted the command but has not responded with anything. It's been a long time and no drive lights are flickering. Is that syntax wrong?

Justin

---

That command is a gzip -l with the parameter being a lowercase "L" character. It looks like AVS forum uses a different font for display versus what is shown when you type your message in the edit box.

If you did type a lower case L and it is taking that long something is wrong. When I do a list on even one of my large 600+MB gz files it is instaneous with the results.

Hopefully that redirection pipe doesnt kill your original gz file



------------------
Belboz
belboz@cinci.rr.com

POST #20 | Report this post to a moderator | IP: Logged