ddrescue dd_rescue gddrescue gnuddrescue myrescue ... which to choose?

I hadn't done a recovery in a while that needed special attention, and ended up wasting a bunch of time fiddling the the ddrescue variants before I found the one I had used in the past.

I fumbled around with dd_rescue for a while thinking that it was the one that I had wanted, since I thought I had remembered using the one that had a dash or underscore in the name previously. Unfortunately after having left it running over night, I found that it was not the one I was expecting, since I cancelled (CTRL-C) the command in the morning, and then tried to continue by simply entering in the same command again, only to find it repeated copying the same areas it had already done, so it wasn't being "smart" ... even though I had told it to use a logfile, and a bad block bbfile. This sent me on a search trying to see if I was running it wrong, or was missing something. Then I ran across a comment on the digg article talking about dd_rescue (2006), that happened to mention

"Dave Burton Says:
February 3rd, 2007 at 9:56 am

Rakesh,

Here’s a better link:
http://freshmeat.net/search/?q=ddrescue
(That link finds both ddrescue and dd_rescue.)

The program described in this article is Garloff’s original dd_rescue.

The program which I very much prefer is Diaz’s newer GNU ddrescue."

This twigged my memory, as well as reading the rest of the comment, it pointed out that gddrescue (diaz's) provided for "remembering" what blocks it had recovered so it doesn't try to go over the same area twice.

I promptly installed gddrescue using "apt-get install gddrescue" and then tried to view the man pages, of course using "man gddrescue" ... however, this is where some of the ddrescue confusion comes in, "gddrescue" simply uses ddrescue for its binary and for its man pages... (confusing), since "dd_rescue" actually uses a package name of "ddrescue" (apt-get install ddrescue), while using dd_rescue for its binary and man pages (once again, confusing).

gddrescue (package name=gddrescue, binary=ddrescue, man=ddrescue) is the one I wanted all along (hence why I'm now writing a blog post so I don't forget in the future).

I had also tryed "myrescue", it has a really good concept, however, its execution leaves a little to be desired... I tried using it on a 200MB partition (that had no errors), and then used dd, dd_rescue, gddrescue on the same partition, all to different output files. After "rescuing" the partition 4 times, I compared all the output files to each other, the first obvious difference is the file size of the myrescue image was 1024 bytes smaller than the other 3 files. The "other 3" files are the same as each other according to diff, while the 4th (myrescue) file is tagged as being different from the others (obviously, if its size isn't even the same). I also noticed that if I used a different block size (larger generally), that when myrescue reached the "end" of the partition, it never "completed" while still using a bunch of cpu processing time, and just never letting go, or saying it was "done". I'm guessing there's a bug in there somewhere regarding end position or something like that.

Thats the end of the post for now.

edit:
Some useful links:
- Ubuntu DataRecovery Howto, do it yourself
- Comment regarding which ddrescue to use, reminding me that it is GNU ddrescue (the gddrescue package, but 'ddrescue' binary) that I found most useful in the past
- A nifty looking ubuntu image (iso or memory stick) already setup for data recovery (removing the need for internet to download most tools you would need for recovery work), and providing a lower over-head environment (command line only, no GUI), but also offers a repository that you can use to add the same recovery tools to an already installed or live copy of ubuntu, which you might have a GUI on already.

notes:
- for first run of gddrescue, it may be wise to add the --no-split (or -n) option, as described in the texinfo manual, it says:

--no-split: Do not try to split error areas. Allows a fast recovery of the non-damaged part of the file.

- after the first run, or if the first run is having issues, you may want to try the following:

hdparm -a0 -A0 /dev/bad-device

and then re-running ddrescue using the logfile you hopefully told it to use on the first (and any successive) runs of ddrescue, this command will disable any read-ahead that the hard-drive tries to do on your behalf, but also brings your copying rate to a dismally slow speed (100k-200k/sec in my case of recovery on an 80gb laptop drive attached directly by ide), so you may want to try your luck without disabling it at first

- check the texinfo manual for some data recovery execution examples as well

VMWare Server 2.0 2.0.1 2.0.x DNS via NAT broken

scenario:
- created a Linux machine (Ubuntu 8.04.3) hosting VMware server 2.0.1
- two network cards, one to WAN (DSL), one with NO ip address used to bridge a physical network (wireless access points) to the captive portal "fenced" network
- created a virtual guest using downloaded VM image of pfSense 1.2.2 (and tried 1.2.3 beforehand) which is running Freebsd 7.0
- created a second virtual guest using Ubuntu Server 9.10 beta as a client for testing the pfSense captive portal
- scenario worked during testing off-site and after the installation on-site at clients location

- intermittent issues began cropping up with either slow, or no internet access for the kiosks using the captive portal, or the wireless guests
- a reboot of the Linux host machine seemed to "fix" the issues
- issues came back a few days later, or maybe sooner, but were not as apparent

- I start troubleshooting the issues using the Ubuntu server guest VM, and find that I can ping hosts directly by IP on the Internet, but not by their hostnames... DNS requests fail almost all the time
- I login to the pfSense via SSH and try some ping tests... once again, direct IP addresses work, while domain names do not resolve and fail
- I try changing to different DNS servers, doesn't help
- I try using nslookup on the Ubuntu server guest VM, issuing the "server ip.ip.ip.ip" command to switch to a known working DNS server, and issue a lookup for a domain such as www.google.ca or www.yahoo.ca which promptly (or not so promptly) fails after a few seconds
- I try the same nslookup command on the Linux Host machine, which works perfectly fine.
- I start googling for VMWare issues with NAT networks (which is what the pfSense WAN network was using with VMware to get to the internet
- I find some issues in the forums at vmware, but the ones I found were mostly issues with Windows XP Host machines, which had a work-around editing the NAT config file for vmware and setting the DNS lookups to ordered instead of burst
- I look into this file, on the Linux host, and I find the dns section, but am told by a comment in the file that this section is ignored by Linux Hosts, and only used with Windows... what luck (not) :/
- I continue googling trying to narrow it down to vmware server issues since many of the issues were related to VMware Workstation I believe.. but really get nowhere
- I start thinking of new ways to avoid using the NAT networking in VMware
- idea1: install a 3rd physical network card into the Linux Host machine, provide NAT services via Linux on this interface, and bridge the pfSense WAN network to this interface
- problem with this idea.. server is a 3hr round-trip drive away from me
- idea2: switch to KVM for virtual hosting instead of VMware
- problem with this idea.. server processor does not support AMD-V or Intel VT.. I don't mind swapping the CPU out, but once again, 3hr round-trip not desirable
- idea3: switch to XEN for virtual hosting instead of VMware
- problem with this idea.. time, and working on a server remotely is slightly more dangerous if I break something mid-work, once again, 3hr trip to fix if I do break something
- idea4: try putting a virtual network interface or "alias" interface (nothing to do with VMware) on the primary NIC that attaches to the internet on the Linux Host machine, give it a private subnet address different than any other physical or virtual network currently in use (192.168.50.1), and enable forwarding on this virtual interface on the Linux Host and bond the pfSense WAN nic in VMware to this physical interface, set the pfSense WAN to static IP (192.168.50.129) and gateway of the Linux Host's alias IP of 192.168.50.1
- problem with this idea.. don't know if it'll work, somewhat risky playing with networking on remote server, seems complicated
- I went ahead with this idea after weighing the other ideas.

- after choosing idea4, I went ahead and implemented the changes, setting up the virtual alias of eth0:50 with an IP of 192.168.50.1, enabled IP forwarding manually for testing, then later in the sysctl config plus 2 other settings related to forwarding, created a small script to run the 4 iptables rules that implement the masking/masqing/masquerading/NAT redirection functions of linux
- I modified the pfSense WAN IP to be static 192.168.50.129 netmask 192.168.50.0/24 (255.255.255.0)
- I adjusted the pfSense virtual guest's first virtual network card to be bridged to the Linux Host's primary network card (default network name in VMware is "Bridged") which is used as the WAN interface in pfSense
- maybe some reboots involved since things weren't working initially
- also made a correction to my pfSense netmask, since I mistakenly chose /32 initially instead of /24 netmask
- can now ping from pfSense interface to linux host's eth0:50 ip of 192.168.50.1, and vice-versa
- can ping outbound from pfSense to the internet
- nslookup requests from pfSense work
- switch to the testing guest Ubuntu Server VM, and reboot it to get a new lease from the pfSense captive portal, probably not required, but reboots are the order of the day it seems
- nslookup requests from the testing guest using the captive portal work
- call client to have them test using their Kiosks
- wait to hear from client, and continue to test with testing guest to see that the intermittent issue side of things doesn't crop up again

- waiting to know if it all worked out

VMware Server 2.0 (free) gotchas

Just a couple gotchas I figure I would document for my own use later and for anyone else that finds it using google.

Gotcha1: USB Devices in VMware guests

Using VMware Server 2.0 (free), I ran into my first requirement of needing a USB Software Dongle Key to work inside a virtual machine running Windows XP. All the documentation kept mentioning the steps of a) add a USB Controller to the virtual guest inside the VMware Server control web interface (this was the easy part) and then b) Enable the USB Device in the web interface on the VM.

I for the life of me couldn't find where it meant there was somewhere to enable the specific plugged in USB device that you had plugged into the Host machine, allowing it to work on a specific guest... after much googling and not coming up with something, I finally noticed a new button on the VMware guest page in the web interface, to the right of the regular 4 buttons (Stop, Pause, Play, Restart), a new button appears with the USB Icon, and you can enable/connect the USB Device to the VMware guest by clicking on it, and checking off the appropriate USB device... as simple as that. (gee whiz)

Gotcha2: Adding Datastores to VMware 2.0 (free)

I was trying to determine how to add additional datastores to VMware Server 2.0 to no avail... everything kept mentioning or even showing pictures of the screen with the boxes to fill in, but nothing mentioned how to get this screen up in the first place, or at least not enough for me to clue in and I was unable to find it... after googling for a while, I went back to the web interface, and eventually lucked out (and apparently I am blind... or was very tired), and determined that if you just click on the Server in the list on the far left of the web interface, on the far right is your standard options such as "Create Virtual Machine", "Add Virtual Machine to Inventory", and... you guessed it, "Add Datastore" ... not sure how I missed that one... but I actually didn't "SEE" it until I clicked on the one original/default datastore that was added during configuration, from the list/display window in the middle of the web interface with the server selected... which showed some additional options for that datastore (rename, refresh, remove datastore).. and I finally saw the "Add Datastore above all those new options.

So, I guess the 2nd one wasn't really a gotcha... just more of a blind mistake or I was more tired than I originally thought. Anyways, a heads-up for anything that may be blind like myself.

Gotcha3: VMware Server 2.0 (free) 64-bit Host, but unable to successfully use 64-bit Guests

I am using some second-hand HP DL585 G1 servers with 4 x Opteron 848 2.2Ghz CPU's in them... I totally forgot about having earlier checked if they had VT/AMD-V capabilities months earlier (they don't with this cpu revision of CG or C0 (I forget which C* they are, but anything C* generation or older do not work ie. stepping CG does not, but E1, E4 or E6 does support it))

So, off to HP's site to see if there are firmwares that work with the E* revision cpu's, and then off to Ebay to try and poach some.. since they seem to be quite reasonable prices.. assuming they work with my server(s). Just for anyone else that might be trying to run older gen Opteron's without AMD-V support, and trying to setup 64-bit virtual guests unsuccessfully.

fi.

Windows Update fails in Vista using Shaw Internet using "Automatically Detect Settings"

If you've got Vista installed on a PC or Laptop computer, but your Windows Update is failing (I can't remember with what code off-hand, as I'm not typing this the same day/week I ran into this problem, though I will update the post if I run into it again, or feel like re-creating the problem), simply try the following:

In Internet Explorer 7 or 8, open the Tools Menu (press the ALT key once if the menu bar is hidden), choose Internet Options down at the bottom of the Tools Menu, Choose the Connections Tab from the list of tabs across the top, Choose LAN Settings down near the bottom right of the window, and make sure to UNCHECK "Automatically Detect Settings".

Problem should be solved, simply try Windows Update again, this has fixed the error for me on several machines... I don't know if its my local shaw proxy server that is being detected, and failing for some unknown reason.. but, thought I would blog about it since its happened several times in the past, and seems to be better to simply disable the proxy anyways.

fi.

edit: apparently i have an older post in my blog that describes this exact issue, including the error codes... and here I was thinking my blog could act as my memory on some things... go figure :p

"Grub hard disk error"

short story:

- installed ubuntu server 8.04.2 LTS on machine using 160gb IDE drive (hda), including a single /boot partition and an LVM partition for root (/) and another LVM partition for swap.

- installed new 500gb Sata drive into machine

- created similar partition schema (partition 1 = 250mb /boot partition (sda1)), (partition 2 = logical of the rest of the space (sda2=extended info, sda5=data partition that was created inside the extended partition

- used dd to copy hda1 (250mb /boot) to sda1 (new 250mb /boot)

- made sda1 bootable using cfdisk

- added sda5 partition to main volume group using pvcreate

- used pvmove to tell LVM to move anything on the /dev/hda5 physical volume

- after ~4 hours (150gb @ 12mb/sec average, since its slow due to using atomic writes, etc), it was done moving all data off of the old 150gb physical volume

- used vgreduce to remove the hda5 physical volume from the main volume group (since nothing is on that partition now (well, technically its probably got all the data if you wanted to try recovery of some kind, I don't believe it writes 0's or anything to it during a pvmove))

- tryed using grub to re-install to new hard-drive (sda), all messages seemed to be fine

- shutdown machine and removed the 160gb IDE drive

- got "Grub hard disk error" ... with no other useful information, or ability to "enter" grub

- attempted to manually fix grub using rescue mode of ubuntu server install CD

- googled for a while

- didn't find anything that really helped me, but some useful information none-the-less

- then realized one of the options in the rescue menu was "reinstall grub" or something to that effect, I chose to re-install to (hd0) (thats hdZERO)

- I then chose to reboot the system

- Magic happened, and grub loaded correctly, and then promptly loaded Ubuntu Server for me, along with my VMWare instances that were set for autostart.

- Problem solved... why my manual grub flailing around didn't fix anything (tried many different things between grub-install, setup-grub, setup (hd0), etc, I have no idea why, but I'm happy that this route worked for me

Windows Vista CBS.Log and SFC Scannow failure

original error after running sfc /scannow:

windows resource protection found corrupt files but was unable to fix
some of them. details are included in the CBS.log
windir\Logs\CBX\CBS.log

C:\windows\logs\CBS\CBS.log

links:

how to take ownership of files in windows vista

well written howto on troubleshooting/repairing windows vista

Examples of taking file ownership/permissions:

C:\>takeown /f "C:\program files\windows sidebar\settings.ini"

SUCCESS: The file (or folder): "C:\program files\windows sidebar\settings.ini" now owned by user "Owner-PC\Owner".

C:\>icacls "C:\program files\windows sidebar\settings.ini" /grant Owner:F
processed file: C:\program files\windows sidebar\settings.ini
Successfully processed 1 files; Failed processing 0 files



http://jsbi.blogspot.com/2006/12/how-to-recover-boot-failure-in-windows.html

http://forums.wincustomize.com/?postid=348387

http://www.vistax64.com/tutorials/86959-access-vista-install-dvd-files.html

http://support.microsoft.com/kb/928228

Set Grub Default to Windows or Last Selected Choice

I was installing a used hard-drive in a friends computer after his drive failed, so had to re-install windows, etc. I took the opportunity to install Ubuntu Linux on the last 6 gigs of the hard-drive as well, in case he is ever having troubles (virus, root-kit, etc) with Windows, he can boot to linux if he still needs to get on the Internet, etc.

I wanted to make it so Windows would be the default on the grub menu. This in itself is actually rather easy, just setting the 'default' option in grub to the line number that Windows shows up in the Grub boot menu. This becomes a problem later in "life" if Ubuntu is updated, and a new kernel is installed in addition to the old kernel, this changes the line that Windows will show up on. One way around this was to move the lines that add Windows to the grub menu up to the top of the menu.lst, therefore it shouldn't change (hopefully)... but I came up with a better suggestion with a little more digging on google. Instead, I simply enabled the usefulness of the savedefault option in Grub, so that any grub menu options (kernel options/operating systems) that have the savedefault in their list of commands would change the default O/S to THAT O/S. If you don't have a "default 0" option, thats fine, just make sure you add the new "default saved" option to your config somewhere in the global area (near the top area).

** The lines with #XX in front of them have been replaced by the respective following line of each of them.

# You can specify 'saved' instead of a number. In this case, the default entry
# is the entry saved with the command 'savedefault'.
# WARNING: If you are using dmraid do not change this entry to 'saved' or your
# array will desync and will not let you boot your system.
#XXdefault 0
default saved

## should update-grub adjust the value of the default booted system
## can be true or false
#XX# updatedefaultentry=false
# updatedefaultentry=true

## should update-grub add savedefault to the default options
## can be true or false
#XX# savedefault=false
# savedefault=true

If my friend ever goes into Ubuntu, it will change the default to Ubuntu, but all he has to do to get it to default back to windows again, is simply choose to boot into windows again from the grub menu, and it will then default to windows again. Problem solved.

One other option could be to remove the savedefault from any menu entries that you don't want to make changes to the default O/S menu choice during boot-up, if you happen to boot that entry once in a while and don't ever want it to be the default.

Using sfdisk to backup and restore partition tables

I needed to backup a partition table and restore it to a new hard-drive, I tried a little bit of googling using parted with my search terms because I had forgotten that it was actually sfdisk I needed/wanted (from previous similar endeavours), so after coming up with only a few posts that poorly inform readers that copying the first 512bytes of a drive will get them their entire partition table in case of disaster, I finally came across something that mentioned sfdisk, which twigged my memory and I thusly changed my google search to include sfdisk as one of my search terms instead of parted. I came upon the article/post that this post's Title links to (also here), which nicely points out a good method of backing up your hard-drive's partition tables properly using sfdisk, which I will quote the relevant snippets here with a couple pieces edited for clarity or additional information:

Using sfdisk with the -d option we can get a dump of the current partition table in a regular file, and if needed we can restore it from that file:

sfdisk -d /dev/sda > sda_table

and to restore the partition table to the same drive (or change device name for another destination drive):

sfdisk /dev/sda < sda_table

Regardless on the method used, having a backup of the partition table might be handy.

One more tip: another usage of sfdisk with -d is to create an identical partition table from another hard disk. In this example we are partitioning sdb with the exact same partitions as sda (ie. copying partition table from SDA to SDB):

sfdisk -d /dev/sda | sfdisk /dev/sdb

of course this can be handy when the hard drives are identical, in raid1 setups, etc. (actually, they don't even have to be identical, but the destination drive needs to be the same size or larger (exactly the same size, not "sales pitch gigabytes size")).

End Quote.

So there you have it (and I have it for my permanent memory now!).

Sidenote: copying the first 512bytes of your drive may copy many partition tables, but only if they do not include extended partitions, since the main/base partition table actually uses a reference that links to another part of the drive where it stores the rest of the extended partition table data. Don't ask how I know this, other then it was a late night, and interesting things were learned using hexedit! :) ask Jaymes if one must know! :P

Microsoft Outlook XP 2002 0x8004060c

Was receiving the following error message in Outlook 2002 when trying to send/receive email:

Sending and Receiving reported error 0x8004060C (0X8004060C) Unknown Error: 0X8004060C

After googling, came to this page (http://support.microsoft.com/kb/304863) which explains the error in more or less words.

Short Answer: your PST file is full, you need to either start using a new PST file and use your old one as an Archive (quickest method I think), or create a new PST File, and move a bunch of your old emails into it (from the old PST file), and then compact your old PST file (google for how if you don't know how).

Problem fixed.

sidenote: a person may want to setup auto-archiving of older emails to another PST file to help avoid this, until the archive pst file becomes full, and then you'll have to adjust the destination to an additional (new) archive pst file after creating another one to use.

fi.

Vista Windows Update Error 0x80072EFE 80072EFE

I encountered the error code of: 0x80072EFE on windows update on a Dell laptop after "upgrading" it (repair install) .

In the end, it ended up being that since the "Automatically detect proxy" was enabled in Internet Options in Internet Explorer, it was trying to detect my ISP's proxies, and in the C:\Windows\windowsupdate.log file, it showed it as failing via a proxy (when I didn't have a specific proxy set).

Anyways, long story short, I simply disabled the Automatically Detect Proxy option in Internet Options in Internet Explorer, windows update now stopped trying to use a proxy that vista was trying to detect (maybe my isp's proxy was down.. or broken or ?). Windows update worked flawlessly after disabling the detect proxy option.

Articles (rather, posts) I found on the problem were often unanswered (successfully) or pointed towards an internet related issue with your system. I had gone through most of a microsoft howto (http://support.microsoft.com/kb/836941/en-us) (found via this post: http://www.vistaheads.com/forums/microsoft-public-windowsupdate/185095-windows-update-fails-error-80072efe.html)on troubleshooting the issue (after manually trying some things myself beforehand), and the howto pointed/reminded me to read the windowsupdate.log file, so I went and checked, and saw the proxy trying to be used (unsuccessfully). In this instance, it was definitely an internet related issue... (a broken proxy or some other issue with it).

fi.