Understanding OnePass Archiving Retention

Commvault's OnePass Archiving solution consolidates traditional backup and archiving processes into a single operation. In a OnePass operation, data gets backed up as part of the backup operation and source files or messages that meet archiving rules can be replaced by a stub that provides transparent recall capability. The benefits of OnePass archiving are efficient use of processing resources, reduction in the size of backups, and more disk space made available on the source. OnePass archiving is supported for File Systems and Microsoft Exchange Mailbox agents.

But backing up data and archiving data usually have different data management requirements with archived data traditionally being retained longer.  And storage policy retention is job-based, so how does Commvault differentiate retention if both archived and backed up data are in the same job?

At the storage policy copy level, OnePass archived data is managed by the same Basic Backup Retention Rule for All Backups criteria as backed up data. Basic backup retention cannot differentiate between backed up data or OnePass archived data.  That ability to do that comes from two features - Subclient retention settings and Synthetic Full backups. The Basic Retention Rules for Data/Compliance Archiver Data criteria you see on a storage policy copy’s properties dialog box applies to Classic Data Archiver and Compliance Archiver Agent data only.

You can switch a OnePass subclient to use archive retention rules (Days only) vice backup retention rules (Days and Cycles). Essentially you’re turning the multi-purpose OnePass subclient into a single-purpose Archiver subclient. Only incremental backup job types will be honored. The archive retention rules will apply to all data backed up by the subclient – not just the data that qualifies for archiving. In this case, we recommend you select the subclient properties content tab option to “Only backup files that qualify for archiving”.

You enable OnePass subclient’s use of archiver retention rules by adding an additional setting to the CommServe properties (bShowHonorArchiverRetention) and another to the client computer properties (nSkipIndexLookup). Once these settings have been added, an option to “Honor Archiver Retention” will appear in the File System Agent’s general properties tab.  Once this option has been enabled, you cannot disable it later.

Subclient retention determines when storage policy copy retention is applied. Under normal (non-OnePass) backup operations, job-based storage policy retention is immediately applied.  Subclient retention allows the administrator the option to delay the start of storage policy copy retention.

For File System subclients, the delay is based on the start time of the first backup job that does not contain the file or its stub. With a Mailbox subclient, the delay is based on a message’s received time. The fact a message is deleted or stubbed does not matter.

File system stubs that are detected in a non-full backup scan will persist an object on media as long as the stub is present on the source. It is only when the stub is no longer found on the source will subclient retention be applied.  File system subclient retention does allow the administrator to set retention differently for files deleted by the user and files deleted by an archiving operation.

Setting the subclient retention setting is only half of the solution. The other key component for OnePass retention is the Synthetic Full backup. A synthetic full backup creates a full backup on media using objects already on media. To do this, a synthetic full backup uses the most recent non-full backup scan results as the collection list for objects already on media.  With subclient retention, deleted objects can be appended to the collection list causing those objects to be included in the job as if they still existed on the source. Eventually, an object will exceed the subclient retention criteria and its presence in subsequent synthetic full backups will cease. At that time, the job-based retention of the storage policy takes over.

For example, let's say subclient retention was set for 90 days and storage policy copy retention was set for 2 cycles and 15 days. Weekly synthetic full backups will carry the deleted object forward for 90 days. Storage policy retention will keep the last synthetic full job containing the object for another 15 days at least.

For another example, let’s say a Mailbox subclient is associated with a deduplicated storage policy copy that has retention criteria set to 0 cycles and 0 days.  Subclient retention is set for messages older than 1825 days based on received time.  Synthetic full backups are run daily. The ability to prune individual objects from deduplicated storage combined with the exclusion of storage policy retention makes this a purely object-based retention solution using a message’s received time.

Some important caveats to remember –

First, enabling OnePass operations disables the subclient’s Full backup job option. Unlike a synthetic full backup, a Full backup will not carry forward deleted objects and will break OnePass retention. This means when OnePass is NOT enabled using subclient retention runs the risk of an inadvertent full backup disrupting expected retention.

Second, stubs that have been renamed or moved outside the scope of the originating subclient are treated as deleted for retention purposes.  There is a backup job option to Check for deleted stubs which will cause a synthetic full backup to run a scan of local drives on the source to look for deleted stubs outside the subclient’s defined contents.  Selecting this option consumes resources on the source client and extends the time it takes to complete the synthetic full backup, but for found stubs it will persist the ability to restore the associated data from media.

Lastly, OnePass archived objects should be provided with the same - if not more protection than backed up file – purely for the fact that there is no longer a copy of the object on the source.  The purpose of backups is to provide an alternative copy should the source copy be lost.  Archived objects should also have another copy.  Best practice says to always maintain at least two versions of an archived object.