I learned all about Time (Machine) so you don’t have to
I’ve been learning a lot about the Time Machine feature of Apple’s OS X 10.5 (Leopard) and wanted to take the time to share my experience. In a nutshell, Time Machine watches the files that get modified on your system and copies them to a secondary hard drive or network volume so that you have automatic backups available for anything that was deleted or modified.
It is a very nice user experience and it takes a lot of the work out of making backups. Of course, by taking work out Apple is really just making decisions for the majority of users and for most users, that decision is quite acceptable. I’m not most users. Fortunately, Apple directly or indirectly provides methods for changing the behavior of much of the OS and Time Machine is no exception. Here are many of the things that I’ve gathered over the last few months that might benefit you if you want to get more out of your experience.
Overview
I won’t go into too much detail as I’m going to presume you’ve already used Time Machine and are familiar with how it works. Generally, it performs a backup of your system every hour to an external drive. It is efficient in that only the modified files are copied. Hourly copies are kept for for 24 hours, daily copies for a month, and weekly copies until the destination drive fills up. You are free to put other data on that Time Machine disk, but it will eventually consume all available space.
Space is pretty efficiently used in that a file is copied only if it has changed but ineffecient in that a small change to a large file results in the entire file being backed up again. This can be a problem if the file really is big and changes frequently. This is the case with the virtual hard disk used for a virtual machine under VMware Fusion or Parallels Desktop.
Backing-up to a network volume
Any self-respecting geek has more than one computer in his or her home. Often times, you are using a laptop and have a dedicated computer with more storage sitting by your TV or in the den, closet, or garage. Unfortunately, while Time Machine can back up to a network volume, by default it only supports a specially presented volume served via Apple’s network filling protocol (AFP) from Leopard Server. You can back up to other choices (e.g., Windows or Linux) via Microsoft’s filing protocol (CIFS). To do this, you first need to have a native Apple file system to copy the data to.
Creating a sparseimage
OS X has supported disk images since the beginning but has recently added a format called sparseimage that starts small but grows as more data is added to it. To use a network volume, you first need to create the image that your machine will be storing the actual data in. Type the following on a mac creating the disk on a local hard drive first:
hdiutil create -library SPUD -size $SIZESPEC -fs HFS+J -type SPARSEBUNDLE \
-tgtimagekey sparse-band-size=262144 -volname "Backup of $MACHINENAME" \
$MACHINENAME_$MAC.sparsebundle
Here are explanations on the options that you’ll want to change:
$SIZESPEC – the size of the virtual disk when filled. A good choice is twice the size of what you’re backing up.
$MACHINENAME – The name of your machine
$MAC – The network address of the primary network interface without separators (e.g., aabbccddeeff0011). You can get this from System Profiler or Network Utility.
For education purposes, here is some explanation of some of the other items:
-fs HFS+J – make the filesystem HFS+ with Journaling (the default OS X filesystem)
sparse-band-size – the size of the ‘chunks’ that make up the sparseimage (they aren’t just one single file). Normally, the images are made in 8M chunks but those perform quite poorly over the network. The value provided above represents 128M chunks (2 * 128 * 1024 512-Byte-Blocks = 262144 = 128MB) which is a good size for a hard disk backup.
Copying the image to to the network volume is as simple as dragging the sparseimage file via the Finder or do it in the terminal:
cp -r $MACHINENAME_$MAC.sparsebundle /Volumes/path/to/destination
The last part is to tell the system to allow any network volume as a destination and not just AFP:
defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
Using just part of a drive
For some reasons you’ll see below, I wanted to limit the usage of the destination drive local to the system. Normally, Time Machine will use all the space of an external drive chosen as the destination. If you are storing other information on it, that can become a problem. The solution is to convert the physical Time Machine destination volume into a disk image.
The first step is to create a disk image with the same information above. Next, mount the newly created disk image:
hdiutil mount /path/to/created/imagefile
You can also mount it simply by double-clicking on the disk image file. You’ll then copy the contents of the current Time Machine volume to the created disk image. Note, it is best to ensure Time Machine is disabled as you don’t want anything changing as you are copying it:
sudo asr restore --source /Volumes/srcvol --target /Volumes/Backup\ of\ $MACHINENAME
asr is Apple’s System Restore utility and it performs copy between volumes but ensures that everything is copied properly. You need the sudo portion to have the command be able to access every file which may not be the case if run just as yourself.
Of course, this copy can take some time depending on the speed of your system but mostly due to the size of the data being copied.
Once completed, you can unmount the disk image and configure Time Machine to use the disk that the disk image is stored on. Time Machine is smart enough to use the image if it sees it (the file has to be at the top-level of the destination hard drive).
Trigger a Time Machine backup from the command-line
If you ever want to trigger a Time Machine backup on a system from the command-line, perhaps on a remote system via ssh, issue the following command:
/System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd-helper -auto
Exclude an individual file from being backed up
You can exclude files from being backed up (like previously mentioned virtual disks) using the GUI in the Time Machine preferences (under Options), but it can also be done using the extended attributes in the command-line:
xattr -w com.apple.metadata com_apple_backup_excludeItem /path/to/file/to/exclude
Similarly, you can see if a file is being excluded but listing the extended attributes:
xattr -l /path/to/file/in/question
Hide the Time Machine disk in the Finder
If you aren’t getting all fancy and writting to a disk image, you may want to hide the destiation disk. You can do that just by making the volume invisible:
SetFile -a V /path/to/Time/Machine/Volume
Conclusion
I hope this has been helpful to you and that with the information are able to make a useful tool even more useful. To give credit where credit is due, I’ve included below the links to where much of the information was originally found.
Update
20090527 I’ve revised the sparse-band-size value from 1G to 128M which is the largest band currently support in 10.5.7. Files with larger band files cannot be mounted in 10.5.7. Once I determine the correct procedure for converting an old image, I’ll post a new article. In the meantime, you can get more information here.
Reference
10.5: Set up Time Machine on a NAS in three easy steps
10.5: Improve networked Time Machine performance
Hi. Thanks for your post. Do you by chance know how one can tell which files have been backed up by TM after a specific backup? I occasionally have instances where a large BU is performed even though I don’t think much has changed on my disk. I have already excluded the usual culprits from TM.
Thanks,
Ben
Ben-
I just did a follow-up post that describes a tool (tms) that will give the information you are looking for.
–Mike
Do you have to eject the external drive before you shut down your computer? or can you just shut down the computer and it will eject it properly for you?
Michelle-
OS X is graceful when handling external drives during sleep and shutdown. I’m not sure if you were referring to a physically attached drive (e.g., via USB or Firewire) or a network mount, but it will not be a problem. Of course, it is wise to make sure the system is idle before doing so, though that, too, is handled gracefully.
two things about your hdiutil line
-library SPUD should be -layout SPUD.
your quotation marks are fancy. they should be regular ” otherwise hdiutil gets confused.
. . . that last comment is wrong. -library SPUD is correct.
I cannot find any reference to -library in the hdiutil command reference. -layout however is a possible argument and SPUD can be used with it, so I would say that -layout SPUD is correct…
I’ve googled a bunch of Time Machine related things and decided to post to yours for advice. Where I work, one of the employees had a pirated copy of Maya on his G4 mac mini running 10.5.5. We told him to remove Maya and he did so using Maya’s ‘uninstall’. Much to his and our chagrin, the Maya uninstall started deleting all the applications and even the system preferences. The computer is somewhat crippled now. BUT he did have Time Machine running and I am wondering if I need to jump all the way back to TM’s first backup to restore the computer to the way it was prior to the malicious uninstall? From what I’ve read, TM only backups the apps and OS settings on the first backup? My idea is to copy the 40GB of work related files to an external and then “reset” the computer using TM and then copy the work files back over afterwards. Does that sound feasible? Thanks in advance.
Matt-
To be honest, I haven’t had the occasion to do a significant restore other than the occasional file. I do know that you can perform a full OS install from the DVD and use a TimeMachine backup to effectively revert the system back to a prior state. How that works in practice, however, I cannot say.
Hi,
I do have OS X Server and would like to setup TM for backing up our stuff.
Attached to the server is an external 1.5TB drive (over USB) partitioned into 1TB + remaining space.
The 1TB partition has been shared using AFP and checkmarked ….use as TM backup in the sharing tab of the admin tools. However, when I try to connect to the visible TM share from within TM on the client, I get an error: Operation could not be completed. OSStatus error -6602
Thoughts???
I figured it out …. a simple reboot of the server fixed it.
This is not directly related to your post, but you seem to know a good deal about TM, so…
I have a Time Capsule at home, and a firewire disk at work. It appears that if I manually tell TM which disk is available, it will do an appropriate incremental backup to that disk. That’s good. The question is, is there anyway to tell TM to look for both disks and use whichever one it finds? Apparently TM has no Applescript dictionary. Have you seen any suggestions?
Tom-
You have correctly identified the behavior when switching between destinations. Unfortunately, I have not seen any tool to switch the destination by any method other than via the Preference pane. I even looked in the Time Machine prefs file (/Library/Preferences/com.apple.TimeMachine.plist) to see if the defaults command could be used. No luck.
Interestingly, Apple indicated at a WWDC before Leopard shipped that it would support multiple destinations simultaneously (e.g., have more than one set), but that didn’t make the final release.
Thanks for the post – I’d missed that you could set the size of each band within the sparsebundle, so I’m looking forward to the improvements from 128 MB bands.
I have been using TM on my borrowed laptop which I need to give back to my school. I have been backing up to my external hard drive in hopes to easily keep the files and then use them on my emac. The problem is that I can’t even see the volume of the external hard drive when I plug it into the emac. Do you think it is just because the emac is older? It has OS X. The other question I have is, do you know how I can get that volume to show up again on the emac. I can transfer the files I need manually, but I need to see that volume. I have already tried deleting the backup file on the hard drive and it still won’t show up on the emac. Any advice would be helpful. Thanks! I may be buying a new mac within the next year, but it scares me that I wouldn’t be able to access those files until then.
Cat-
Chances are you just aren’t displaying external disks in the Finder. Open the Finder preferences and in General you can check “External disks” to have it be listed. At that point, you can navigate it easily. You can also choose “Computer” from the Go menu to see all the disks attached to the system. If you’ve hidden the disk with the above command, you can reverse it with “
SetFile -a v /path/to/Time/Machine/Volume“. More information be found on macosxhints: http://www.macosxhints.com/article.php?story=20080124184544791Hi, I am new to Mac and bought an external drive (USB). I plugged it in and started TM. That is fine, no problem. My question is can you do a system restore like in Windows? Say I put a program on my Mac and run TM before installing it. I install it and my Mac starts acting funny so I uninstall but still acting funny so I want to return it to its previous state it was in before the install. How do I do that? All I have seen so far is if I lose a document or picture I can go and search for it, but I have not yet seen how to actually turn everything back to a certain date.
Thanks and keep up the good work!
Tony-
It works a bit differently than in Windows. If you include system files with your TM backup (which is default behavior), then you can boot from your OS DVD and restore your system from a TM backup. I would stop short of equating that with Windows roll-back. As you point out, TM is much more reasonable for file-level restores than system-wide changes. On the plus side, OS X doesn’t suffer from something like Windows registry and uninstalling a program is usually as simple as deleting it. You can use something like AppZapper which tries to get the other ‘bits’ associated with a program. Anything more intrusive should provide an uninstaller program to unwind it from the system.
dont work for me, I have Prestigio netdrive NAS, and after all give me a (OSStatus error -6602.)
10.5.8, but stuck here
Sorry you’re having problems. Are you mounting as AFP or SMB/CIFS? Have you created the disk image on (or copied it to) the network share? Time Machine backups can only be written to HFS+ filesystems so if you are looking to use any network attached storage, you’ll need a HFS+ disk image on it.
On a Mac laptop, a Leopard upgrade resulted in the creation of a second, duplicate Applications folder on the desktop. Seeking to get rid of this to reclaim the space, I used Appzapper to trash the duplicate software. The Applications folder in the Hard Drive icon showed all the installed software as still there. But what I think might have happened is that Appzapper also trashed some shared, associated files and now Garageband won’t work properly. Other installed software is also giving problems. Thankfully I use Time Machine. How do I use it to restore the hard drive on the laptop as it was before I started Zapping the duplicated software? For the moment, I just want to turn the clock back on the internal drive as a whole, so that it’s how it was before I started getting rid of that duplicated spftware.
Nigel-
Yeah, the mistake was using AppZaper to delete the duplicate software as it likely removed supporting files which made the original copy incomplete. Time Machine is intended for file restoration and not a full system, though if you didn’t exclude system files, you can boot from the install DVD and restore the system from a Time Machine backup. You can also do an install of just the applications from the install DVD to restore GarageBand or other Apple-supplied apps. In the future, simply deleteing (move to trash) of the duplicate Applications folder would keep things functional.
To begin, please note that I am a very non-techical user with ZERO Apple experience. I just purchased the new iMac (21.5) after so many years using a PC. The reason for the switch was the fact that the PC crashed. It could not be fixed without wiping out everything on the hard drive. I took it to a reputable repair shop in town and they said they could get all my files, photo’s etc. but not the programs. Okay that’s a nightmare, but I’ll live. Along with the iMac, I purchased the Time Machine. My question is this…the Store rep said it would back up everything on my computer, including the programs. Can you confirm that his statement is true? Does Apple refer to programs as Applications?
Thanks!!
Bob
Bob-
Welcome to the fold. I can’t promise that you’ll not have any problems with your iMac (computers are computers), but I can confirm that OS X and Macs are generally more problem-free than Windows on PCs. By “Time Machine” I believe you are referring to Apple’s Time Capsule (http://www.apple.com/timecapsule/) which is basically a hard drive you put on your network to use as a destination with OS X’s Time Machine. Regardless of destination (Time Capsule or directly attached USB hard drive), Time Machine, when enabled, will back up your primary boot drive and all the files on it.
As an IT professional I have to tell you that data isn’t really safe unless there are two copies with at least one of them not at your location. Time Machine can configured to switch destinations so a decent option would to use a USB drive every few weeks or month that you then keep at work or a friend’s house and then use your Time Capsule for your “regular” backups.
Have fun!
What do i put for the $SIZESPEC if i want an 80GB file?
You can simply use 80g. Of course you could also use 8192m.
I have an external USB drive I used for Time Machine on one Mac (Mac A) which I connected to another Mac (Mac B) which messed up access to the backups for Mac A so they don’t appear anymore in Time Machine but they’re still in a folder. I messed up the Mac OS on Mac A and now can’t restore it with Time Machine when I use the OS X 10.6 installation disk, I just get a spinning beach ball forever. Is there a way to create a disk image from the first Time Machine backup folder for Mac A and restore it with Disk Utility on the OS X 10.6 installation disk?
Brian, I haven’t had the occasion to do a Time Machine restore from the installation disk but I would believe it would properly recognize a Time Machine disk image. Another option is to do a basic OS install and then use the Migration Assistant (in the Utilities folder) which can also restore users and/or applications from a Time Machine backup.
I have a 1TB Time machine that I wish to use as a standard disk for periodical backups. How do I reconfigure the disk ? Do I just reformat it with a standard file system ? Is it possible to change the characteristics of the disk by using the Airport client ?
Thanks,
Ken
Ken, there’s nothing special you need to do to use a disk with Time Machine other than to be sure it is formatted as HFS+. I don’t know if that can be done when connected to an Airport base station, but you can easily do it when directly connected to the system. You should know that if you use a disk directly with Time Machine and not with a disk image created within it, Time Machine will keep using space util the drive is full. If you want to use the drive with other data as well, that may be a problem. I’d suggest an image to help limit the portion used. You can alternatively partition the drive to achieve similar results.
Hi Mike,
Glad I found this blog – hopefully you can help me out. I have a 1TB external hard drive (seagate) connected to my Airport Extreme base station – we have 2 macbooks (mine and my wife’s) – so far on my laptop TM is running nicely and backing up wirelessly to the seagate hard drive – the problem is that my wife’s laptop cannot seem to find the drive to “Choose Backup Disc…” – do you have any ideas how to get TM running on a 2nd macbook to the same backup drive? Thanks!!
I’ve had a problem like that before. The fix that worked for me was to mount the destination volume first (Finder->Go->Connect to Server…->afp://servername) and then select the volume in the Time Machine preference pane.
I recently installed a new hard drive.. and when i try to restore the system from backup (time machine) absolutely nothing happens. It just won’t load. What should I do?
David-
If your TM backup included system files, you can restore the system by booting the OS DVD and choosing to restore. You can also do a new install and then use the Migration Assistant in the Utilities folder to restore applications and users from a TM backup.
I have 2 1TB external drives hooked up to my mac. I use one for my working files and I want to use the other as a backup. I want to use time machine to backup one to the other. It looks as if time machine defaults to backing up the hard drive. Is there a way to designate an external drive as the info I want to backup?
Thanks for any help!!
David- Time Machine is only intended (and only supports) backing up the system drive. If you want to back up the one external to the other, I would suggest Carbon Copy Cloner which can do that quite well. It won’t have the revisions that Time Machine offers, but it is a copy.
Putting on my IT professional hat, for a moment, I will say that backups don’t count if they don’t follow the 3-2-1 rule: 3 copies in at-least 2 locations with 1 being off-site.
Any chance that this has changed? I would like to use time machine to back up both locally and to a network drive, changing the destination periodically. I was thinking that I could schedule a job to change the destination.
Tom-
As far as I know that hasn’t changed. It’s a shame, really, as you have a reasonable use case. Interestingly, before Time Machine was release with 10.5 it was going to support multiple drives (e.g., office and home) without requiring any change. For whatever reason, that feature didn’t make to release.
My dad has stumped me with a Time Machine question, as I don’t have it to walk through with him. (Windows at home, Mac at work with other backup). He says his LaCie backup drive shows the Time Machine files all with a date of April 24. . . the info’s all there, but the date’s goofy. Any thoughts that could help him? Thanks very much, I appreciate your blog!
So, the System Preferences shows the last backup as April 24? What happens when he selects backup now? Does it attempt? Does it succeed? Worst case, he can unconfigure Time Machine, wipe the drive, and set up Time Machine again. I’ve encountered some problems with my machines (not like this) and will try to write up new posts covering the problems and solutions (if I resolve them). Sorry I can’t be of more help.
Tom, Blogs 39&40 refer. I’m the inept dad who writes to refine his Time Machine problem. Time Machine records everything properly except for it’s hangup on the day/date. I can go to History, count down any number of lines,and read what was backed up on that actual day. Only trouble is that the software thinks that every day/date is Sat 24 April. FYI my oldest backup is 30 Dec 2009 ( date installed) and the latest is today 20 min ago. Hope this helps. Ever grateful.
Interesting, if I’m understanding correctly, the timeline shown on the right when you enter Time Machine all show the same 4/24 date? That’s strange, indeed. If it works, I guess you don’t have worry unless your OCD is getting the better of you. If that is the case, you can research on Apple’s support boards to see if others have encountered (and fixed) the problem.
Yes,every date from the beginning (Dec 09) is Sat 24 April except for Today and Yesterday. I’ll try the Apple support boards as you suggest. If I come up with a solution I’ll pass it along. Many thanks.