Article Number: 000019245
Avamar: Tuning backups to complete quickly
Summary: This article discusses how to improve the efficiency of Avamar backups to ensure that they complete as quickly as possible.
Article Content
Instructions
This article aims to help improve the efficiency of your Avamar backups and minimize the time needed for them to complete.
Before starting:
This article is complementary to Avamar: Troubleshooting slow backup performance. Read that also.
Define the scope of the dataset as narrowly as possible:
Ensure that only the necessary file systems, or ideally, only the required folders are in the dataset.
This minimizes the total number of files to be scanned during the backup.
The screenshot below shows a dataset that includes only two folders with essential data. Avoid backing up unnecessary data.
Use exclude statements, but try to avoid using 'include' statements:
Exclude statements help avoid backing up file types which:
Although exclusion statements can help performance, there is a trade-off.
There is an overhead cost associated with checking whether each file matches any of the defined exclude patterns.
This overhead cost is low on a file-by-file basis, but it increases if there are many (thousands) of exclude statements or many (millions) of files.
Keep the number of exclusions statements to a sensible number (tens, not hundreds, or thousands) to prevent the overhead from becoming noticeable.
If 'inclusion' statements are used, they are applied against the entire dataset.
Avtar checks excluded folders for the presence of patterns that match the inclusion list. From a backup runtime or performance perspective, avoid using include statements unless necessary.
For specific advice on setting up include or exclude statements, see:
Avamar: How to configure the inclusions and exclusions in a dataset to define the scope of a backup.
Schedule the backup to run when the client is not busy doing its daily jobs:
While a file server is busy serving files, it is not a good time to back up that data. The disks storing the data and the network interface are busy with I/O requests and the CPU is also under a high load.
Similarly, if a server performs batch job processing every day at midnight, overlapping that activity with an Avamar backup of the same data would be unwise.
Understand where the client's backup bottleneck lies:
As discussed in Avamar: Troubleshooting slow backup performance, bottlenecks can be:
Before starting:
This article is complementary to Avamar: Troubleshooting slow backup performance. Read that also.
Define the scope of the dataset as narrowly as possible:
Ensure that only the necessary file systems, or ideally, only the required folders are in the dataset.
This minimizes the total number of files to be scanned during the backup.
The screenshot below shows a dataset that includes only two folders with essential data. Avoid backing up unnecessary data.
Use exclude statements, but try to avoid using 'include' statements:
Exclude statements help avoid backing up file types which:
- Do not require backing up.
- Are typically large or change frequently.
- Are known to de-duplicate poorly. This affects Avamar server capacity, rather than backup performance.
- Avamar Administrator Guide
- Avamar Operational Best Practices Guide
Although exclusion statements can help performance, there is a trade-off.
There is an overhead cost associated with checking whether each file matches any of the defined exclude patterns.
This overhead cost is low on a file-by-file basis, but it increases if there are many (thousands) of exclude statements or many (millions) of files.
Keep the number of exclusions statements to a sensible number (tens, not hundreds, or thousands) to prevent the overhead from becoming noticeable.
If 'inclusion' statements are used, they are applied against the entire dataset.
Avtar checks excluded folders for the presence of patterns that match the inclusion list. From a backup runtime or performance perspective, avoid using include statements unless necessary.
For specific advice on setting up include or exclude statements, see:
Avamar: How to configure the inclusions and exclusions in a dataset to define the scope of a backup.
Schedule the backup to run when the client is not busy doing its daily jobs:
While a file server is busy serving files, it is not a good time to back up that data. The disks storing the data and the network interface are busy with I/O requests and the CPU is also under a high load.
Similarly, if a server performs batch job processing every day at midnight, overlapping that activity with an Avamar backup of the same data would be unwise.
Understand where the client's backup bottleneck lies:
As discussed in Avamar: Troubleshooting slow backup performance, bottlenecks can be:
- File scan time (due to many files in the dataset or slow file scan performance)
- File processing time (a function of the total size of all files which change in the dataset)
- Network transmission time (due to low bandwidth or high changing backups)
- An Avamar server side issue (typically rare but may affect all concurrent backups).
Additional Information
File types to consider for exclusion:
Media files - these are already highly compressed and are often large.
These are saved as .pst files.
To understand why, see Avamar: Performance considerations when using Avamar to back up Outlook archive .pst files
Log files of any type:
These tend to deduplicate well but change regularly and can be large. The avtar process takes a long time processing these files. Do not back them up unless required.
Compressed files - .tgz, tar.gz, .zip, .rar, .ace, bz2.
These contain highly unique data and can sometimes be large. If possible, back up the data uncompressed.
Database files
If an official Avamar plug-in exists for the database, use that. Avoid backing up databases as flat files using a file system type backup.
More generally
Avoid backing up large or changeable files.
Desktop or Laptop clients
For Desktop or Laptop clients, specify the maximum size of files to be backed up.
--maxfile=MB (If not zero, skip files which are larger than this size in MB).
Further reading
Dell KB articles:
Media files - these are already highly compressed and are often large.
- Video files: mp4, .mkv, .mov, .avi, .flv, .wmv, .wma
- Audio files: mp3, .ogg, .flac, aac, .m4a, .wav
These are saved as .pst files.
To understand why, see Avamar: Performance considerations when using Avamar to back up Outlook archive .pst files
Log files of any type:
These tend to deduplicate well but change regularly and can be large. The avtar process takes a long time processing these files. Do not back them up unless required.
Compressed files - .tgz, tar.gz, .zip, .rar, .ace, bz2.
These contain highly unique data and can sometimes be large. If possible, back up the data uncompressed.
Database files
If an official Avamar plug-in exists for the database, use that. Avoid backing up databases as flat files using a file system type backup.
More generally
Avoid backing up large or changeable files.
Desktop or Laptop clients
For Desktop or Laptop clients, specify the maximum size of files to be backed up.
--maxfile=MB (If not zero, skip files which are larger than this size in MB).
Further reading
Dell KB articles:
Article Properties
Affected Product
Avamar
Product
Avamar, Avamar Client
Last Published Date
05 Feb 2024
Version
8
Article Type
How To