Symptoms
The Avamar 7.0 paging file cache that is used for file system backups to the Avamar and Data Domain integrated solution consumes considerably more disk capacity than the monolithic file cache.
If the Avamar /var directory is on a client file system or volume with a limited amount of disk capacity, the larger "on-disk" file size for the paging file cache may cause disk capacity management issues on the Avamar file system client.
Cause
In the
EMC Avamar 7.0 Operational Best Practices document, we documented:
"In comparison to the original file cache method, backups that implement the demand-paging file cache require up to 20 times more disk space."
There are two reasons why the paging file cache file is approximately 20 times larger than the monolithic file cache:
- Extra 20 bytes per file for CDSF offset
The monolithic file cache uses 44 bytes per file: 4-byte header, 20-byte file attribute hash, and 20-byte file content hash. The paging file cache uses 64 bytes per file. The additional 20 bytes are used to store information about the offset within the Common Data Streaming Format (CDSF) backup container where the file is located. If both the paging file cache and monolithic file cache had the same format, this causes the paging file cache to be approximately 1.5x larger.
- No sharing of entries across backups
Both file caches store the hashes for up to 16 backups. With the monolithic file cache, after the first backup is completed, approximately 2% of the files change daily. After the first backup, most of the entries are shared across backups. However, with the paging file cache, each page of elements is unique to a specific backup, and so there is no sharing of entries across backups. This causes the paging file cache to store approximately 10x more entries than the monolithic file cache.
These two contributors cause an approximate 15 to 20 times increase in the size of the paging file cache relative to the monolithic file cache when backing up the same dataset.
If you know how many files are being backed up in the dataset definition, then you can estimate the eventual size of the paging file cache from the following formula:
<paging file cache size in MBs> = <file count in millions> * 1700
Resolution
There are three ways to mitigate the potential impact of the larger paging file cache:
A) Change the location of the paging file cache by using "cachedir" in avtar.cmd
This is the preferred option and has no drawbacks providing that the client has a large enough volume to store the paging cache.
If the Avamar /var directory, which stores the client cache files, is on a volume with limited capacity, relocate the paging cache to a more spacious volume as described below.
- Create a folder where you want to store the cache files.
- Copy the existing cache files from /usr/local/avamar/var/ or C:\program files\avs\var\ to the new folder created in Step 1.
- Create a file in the client /var directory called "avtar.cmd." If the file exists, edit it.
- Specify the new "cachedir" location in the "avtar.cmd" flag file. For example, if you created D:\avamarcache for the paging file cache, you should have an entry like this in C:\program files\avs\var\avtar.cmd:
--cachedir=D:\avamarcache
- Run a backup.
- Confirm that the new cache directory was used correctly.
- Remove the copy of the client caches from the original Avamar var directory.
B) By applying flags which enable paging cache size limitation
In Avamar 7.2 and later, flags exist to limit paging cache size to a percentage proportion of the size of the volume where the cache resides. For more information about this option, see KB article 19517: How to limit the size of the Avamar demand paging cache (f_cache2.dat).
The trade-off of preventing the cache file from growing to file size is reduced backup performance due to increases cache misses.
C) Limit the number of full backups stored within the paging file cache.
By creating some backups with a small dataset and setting those backups to never expire, we can limit to only eight or less backups of the full dataset that is stored in the paging file cache, thus reducing the size.
This is the least desirable option and requires advanced tuning. It also has caveats. Contact Dell EMC technical support for more information.
Additional Information
Avamar 7.0 file system backups to the Avamar and Data Domain-integrated solution.
For more details about the avtar.cmd file, see KB article 81546:
Avamar: How to gather log files for troubleshooting Avamar client backup and restore issues.