TiVo Community Forums Archives - is it quicker if I delete bad files before dd copy?

TiVoCommunity.com
(c)opyright 1995-2005 All rights reserved
indexcheckTC
This area is a static history of posts in the TiVo Community Forum Archive.
This archive history was made for the simple indexing of search sites like Google.

Pages:1

is it quicker if I delete bad files before dd copy?
(Click here to view the original thread with full colors/images)

Posted by: newsposter

I have a bunch of bad programs that caused reboots and have SUID them to prevent any new programs from being recorded on that part of the disk. I'm coming up on the 30 day deadline to return the bad drive to western digital so have to copy to the replacement drive soon.

I was wondering

1 will it be better to delete the bad programs before I copy so that there is less to copy?

2 assuming the answer is yes, should I run the western digital diagnositics to try to fix the bad sectors before I dd or just dd with the bad programs deleted off the bad drive first? I'm guessing if I delete the SUID, then there won't be a chance of bad stuff copying anyway, right?

should I use bs=512, bs=512k or bs=32k I've heard 512 is the longest but that 32k would be very accurate as well and take less time. since I'm only looking to get my 137 out of 160gig, I didn't understand why on my first dd attempt I got an error I was out of space. I used bs=1024k.

Thank you for your help.

Posted by: stevel

No - for dd, it makes no difference at all whether you delete old files or not.

Posted by: Robert S

When you use 'sync', dd pads any partial blocks out to the full block size. You need this to cope with bad blocks so that the blocks after the bad ones end up in the right place. However, if you use a large block size, the last block will be partial (because the data on the source ran out). 'sync' will pad it out to a full block, thus causing an overflow.

This is an expected error and does not relate to your problem with the copied drive.

If you want to try dd again, I'd recommend bs=32k. If you want to use bs=512, I suggest you use dd_rescue instead, which copies in large blocks when things are reading correctly, but switches to small blocks when it hits errors to salvage as much as possible.

Posted by: newsposter

quote:
Originally posted by stevel
No - for dd, it makes no difference at all whether you delete old files or not.

Interesting, glad I asked because I just assumed the video would be deleted and not need to be copied, thus saving me time. But now that I think about it after your comment, I guess it's just like PC file where when you delete, it really doesn't delete the file.

can I assume running extended diagnostics wouldn't help either and would be possibly destructive to my existing programs?

I'm gonna shoot for bs=32k and prepare for a week run

Posted by: Robert S

dd just copies the entire drive - it has no idea whether what it's copying represents a recording, a deleted recording, or just zeroes, it copies it just the same.

Diagnostics are a good idea. If you're lucky, it might repair some of the bad blocks. Although the data in those blocks is already lost, repairing them may smooth the way for the copying of the drive.

Posted by: stevel

dd just does a disk sector copy - it doesn't understand the file system.

Some of the extended diagnostics are data-destructive. They'll typically warn you before you commit to the operation.

Posted by: newsposter

quote:
Originally posted by stevel
dd just does a disk sector copy - it doesn't understand the file system.

Some of the extended diagnostics are data-destructive. They'll typically warn you before you commit to the operation.

yes the WD program warns you to do a backup, which is impossible in my case. However on my 1st dd try i didn't get an error until 59000 in, which someone else said means the tivo system files were OK. So maybe I'd be ok but still lose my programs.

quote:
dd just copies the entire drive - it has no idea whether what it's copying represents a recording, a deleted recording, or just zeroes, it copies it just the same.

I believe your statement is true except when you are dealing with my hard drive :) I am cursed. The copying corrupted even a lot of my good programs that worked perfectly on the error ridden drive. go figure.

Posted by: BlankMan

quote:
Originally posted by newsposter
I believe your statement is true except when you are dealing with my hard drive :) I am cursed. The copying corrupted even a lot of my good programs that worked perfectly on the error ridden drive. go figure.

That's because of the blocking factor you're using in the bs= options and is exactly what Robert mentioned you would have to deal with. When dd gets an error it zeros the rest of the block when it encountered the problem and that could spill over into the next recording.

If you use dd_rescue as Robert suggested you can possibly avoid that. With the parameters set as I recommend when a bad sector is encountered you only lose that sector and not possibly the remaining 31.5K of that block (assuming you were using 32K blocks). Also, because of dd_rescue's dual blocking factor it is much faster then dd and it is more informative as to its progress.

Posted by: newsposter

quote:
Originally posted by BlankMan
That's because of the blocking factor you're using in the bs= options and is exactly what Robert mentioned you would have to deal with. When dd gets an error it zeros the rest of the block when it encountered the problem and that could spill over into the next recording.

If you use dd_rescue as Robert suggested you can possibly avoid that. With the parameters set as I recommend when a bad sector is encountered you only lose that sector and not possibly the remaining 31.5K of that block (assuming you were using 32K blocks). Also, because of dd_rescue's dual blocking factor it is much faster then dd and it is more informative as to its progress.

Someone else may have said it but the 'spill over' you mentioned in your posting just made the whole thing click in my head finally.

I looked into ddrescue (and your posting) on my last crash in Feb. and thought it was over my head. But I do plan on bs=32k for this dd copy so I guess I will have much better luck than 1024k now that I understand the possible spillover thing.

Thanks to everyone who has helped me understand my problem. I will definitely update my thread with my results so maybe it will help someone else understand in the future. And maybe someday I'll d/l or order the dd rescue program and tackle it.

Posted by: BlankMan

Glad I could help. If you use and understand dd, dd_rescue is not over your head, it's an offshoot of dd written by someone that wanted the same functionality I wanted when I went looking for something better then dd to do this save a TiVo disk operation. Think of it as dd on steroids.