Pants Full of Unix

Recently, I was asked to provide the justification for moving an existing NetBackup environment from performing duplication jobs within Vault to performing them with Storage Lifecycle Policies (newly available with NetBackup 6.5, which is already rather dated). These are also the reasons that you should use SLPs for duplication in any new environment. Please, never, ever configure duplication in Vault again. If you hire me (directly or through my employer) to come fix your NetBackup environment, this will be in my top 5 recommendations, right up there with “write to disk first”, “use MPX if you must write to tape”, “use multistreaming always”, and “increase media server bandwidth, reduce client backup network bandwidth”.

Before Storage Lifecycle Policies (SLPs), the only way to automate duplication of backup images in NetBackup was the duplication step of a Vault profile. While that does function, it was only ever clumsy at best. In the old (pre 6.5) NBU environment, there were two logical constructs, controlling the who/what/where/when/how of backup images through their lifetime. These were:

Backup Policy

• who - which clients to back up
• what - which files to back up
• where - storage location for FIRST* copy of the backup
• when - backup start windows but, more importantly here, how long to keep that first image copy
• how - details about client communication/snapshots/so forth

Vault Profile duplication

• who & what - which backup images to work on (including for duplication)
  • limitation: vault must be constrained to only look for backups whose primary copy was written between the last N days ago and M hours ago
• where - storage location for SECOND copy of the backup
• when - how long to keep the duplicated copies

* … unless inline tape copy is in use. But nobody actually uses ITC.

And then Vault also does things like catalog backups and ejecting tapes to go off-site, but it does those things well, so I’m not considering them here.

SLPs are inserted as a configuration tier between the above two tiers, allowing one to specify in a single place how a given set of backup images are treated throughout their entire life time, including all copies of the image, regardless of logical or geographical location. In a post-6.5 world, with SLPs in use, the sections look like this:

Backup Policy

• who - which clients to back up
• what - which files to back up
• where - explicit reference to an SLP
• when - backup start windows only (retention managed by SLP)
• how - details about client communication/snapshots/so forth

Storage Lifecycle Policy

• where - list of all copies that will be created for a given backup image whose policy/schedule targets this SLP
  • permits hierarchical definitions (duplication Y should come from backup image X, duplication Z should come from duped image Y is a common use)
• when - retention period for each hierarchical tier

Vault Profile duplication

• responsible only for catalog backups and tape ejects

So, what SLPs buy us is a single location (or just a couple of locations; most environments end up needing more than two, fewer than five) to manage where all copies of a backup image are kept and how long they’re kept. This fixes two major flaws in the old method:

1. Should business policy change and retention periods change, in the old model, every single backup policy would have to be changed. In the new model, only the SLPs need to be modified.
2. In the old model, if backup images aren’t successfully duplicated before they age out beyond Vault’s look-back window, they will never be duplicated unless the vault profile is modified and a one-off job run, which is precisely the tangle that Guy’s in now. In the new model, the SLP engine will begin attempting to duplicate backup images as soon as the first copy is successfully written and will not give up on a given image unless explicitly told to do so; Vault’s only job is ejecting the tapes that the SLP duplications produce.

So, there you have it. Go forth, shun Vault duplications, and sin no more.