Files and Folders

The "Files and Folders" option lets you back up files and folders on the PC.

Use the Plus button to browse files and folders for backup. You can also drag-and-drop files and folders into the Comet Backup client to select them.

The "Choose files" dialog displays a live estimate of the selected filesize (before compression and deduplication).

Filesystem snapshots

On Windows, the "Take filesystem snapshot" option creates a VSS snapshot.

This enables Comet to back up files that are currently in-use. It also provides disk-wide "crash-consistency", as all files for backup will originate from the precisely same timestamp.

Non-Windows platforms do not currently require the use of this feature to back up in-use files, however, they also do not benefit from the disk-wide consistency.

Current versions of Comet cannot take filesystem snapshots of a Protected Item located on a network share; be sure to disable the "Take filesystem snapshot" option on the Protected Item.

Windows supports taking VSS snapshots of NTFS and ReFS filesystems. Comet can snapshot these filesystems for backup.

Windows does not support taking VSS snapshots of FAT or exFAT filesystems. Protected Items backing up these filesystems must not use the "Take filesystem snapshot" option.

This feature is enabled by default for new Protected Items.

Network shares

Comet supports backing up Windows network shares (SMB / CIFS). You should prefer to install Comet Backup directly on the network device; this will offer increased backup performance.

On Windows,

• The "Choose Items" dialog lets you browse through mapped network drive letters. You can also use the "Options > Browse UNC Path" option to directly browse a UNC path.
• Comet supports entering custom Windows Network Authentication credentials via the "Log in to network share" option. If you select a network share for backup, you may need to enter custom credentials in order for the service user account to access the network share.
• Because mapped network drives are private to a user session, Comet Backup automatically converts mapped network drive letters into their UNC path equivalent, so that it can still be accessed by the service user account.

Current versions of Comet cannot take (VSS) filesystem snapshots of a Protected Item located on a network share; be sure to disable the "Take filesystem snapshot" option on the Protected Item.

On non-Windows platforms,

• You should mount the network share locally before backing it up.

Encrypted files (Windows EFS)

EFS is a Windows feature that allows you to encrypt individual files on disk. Comet supports backing up EFS-encrypted files on Windows. The files will be silently decrypted if possible (e.g. if Comet Backup is running as the encryption user, or if Comet Backup is running as the EFS Recovery Agent user).

If it is not possible to automatically decrypt the file for backup, Comet will back up the file in its encrypted form, and will only be able to restore it in its encrypted form. EFS-encrypted files are displayed with green text in the Restore browser dialog in Comet Backup.

If you have a PC failure, the EFS encryption keys may be lost. In this situation, the EFS-encrypted files may be unusable, even after restoring from backup. Comet warns you about this situation by adding a warning message in the backup job log.

In order to safely prepare for this scenario, you should export the PC's EFS encryption keys, so that the files can be accessed after a PC failure. On Windows, you can do this via certmgr.msc; or on Windows Server, taking a System State backup may be sufficient.

Once you have safely backed up the PC's EFS encryption keys, you can suppress the warning in Comet Backup by enabling the "I confirm EFS keys are exported" option in the Protected Item settings.

If you have only a partial PC failure (e.g. files lost, but OS installation and user accounts remain intact) the EFS-encrypted files will be restorable without any further attention to the EFS keys.

Finding files using EFS

You can use the cipher /u /n command to list all files on the local PC that are EFS-encrypted.

Alternatively, you can use this Powershell snippet, to list all EFS-encrypted files on C:\ into a new EFS-FileList.txt file:

get-childitem C:\ -Recurse -Force -ErrorAction SilentlyContinue | Where-Object {$_.Attributes -ge "Encrypted"} | format-list fullname, attributes | Out-File -FilePath .\EFS-FileList.txt -NoClobber

Finding the certificate used to encrypt a file

You can use the cipher /C C:\path\to\file.txt command to display the user accounts and certificates that are able to decrypt a file. This may indicate which user originally encrypted the file and/or which EFS certificates are necessary for backup.

Windows Server Deduplication

Windows Server 2012 and later have a data deduplication feature that is separate- and unrelated- to Comet's own deduplication, that can be used to increase free disk space on NTFS volumes. A scanning process runs in the background to find and merge duplicate file content. By default, the scanning process runs overnight.

Deduplicated files look and behave like normal files; however, they are stored on disk in a special format, that can only be read by Windows Server (and Linux). Non-Server versions of Windows are entirely unable to read these files from disk.

When backing up deduplicated files with Comet, it backs up the full (rehydrated) file content, and then applies its own deduplication to it. This means that Windows Server deduplicated files can be safely restored to non-Server versions of Windows.

When restoring deduplicated files from Comet, the files are restored in their full (rehydrated) format, and are not re-deduplicated until Windows runs its next background scanning pass. This means that you may not have enough free disk space to completely restore a Comet backup to the same source drive.

Advanced filters

You can include or exclude a range of files or directories with Comet.

Include filters

An inclusion filter will scan out the matched files or directories under the specified top directory from backup, using either a glob pattern or a regex (regular expression).

There is no limit to the number of inclusion filters you can add to a single Protected Item.

The speed of file-matching relies upon the top directory location and the complexity of the glob pattern or regex.

Glob pattern match

Comet can include files based on a glob pattern. Any files matching the glob pattern under the specified top directory will be included from the backup job.

The expression is tested against the top directory path to the file.

Some special characters and wildcards are allowed:

• A star (*) skips zero or more characters within a filename
• A double-star (/**/) skips any number of directory components
• A question-mark (?) skips a single character within a filename
• Square brackets denote a character class (e.g. [0-9]) within a directory or file name

Example: Top directory is C:\Users\, pattern is *\Documents

• This will include all Documents directories under the first subdirectories of C:\Users\ .

An invalid glob pattern will prevent a backup job from running.

Regular expression match

Comet can include files based on a regex (regular expression). Any files matching the regex under the specified top directory will be included from the backup job. The specific syntax flavor is that of the Go regexp library.

The regex is tested against the top directory path to the file.

By default, the regex is:

• case-sensitive. You can perform a case-insensitive match by adding an (?i) expression
• non-anchored. You can restrict your regex to the start- or end- of the file path by using the ^, $, \A and/or \z expressions.

Forwardslash (/) is not a special character and does not require escaping with \/.

An invalid regex will prevent a backup job from running.

Popular examples

On Windows,

Top Directory	Path	Match Option
C:\Users\	*\Documents	pattern
C:\Users\User	**\*.docx	pattern
C:\Users\	[^\\]+\\Documents	regex

On Linux,

Top Directory	Path	Match Option
/etc/	**/*.conf	pattern

On MacOS,

Top Directory	Path	Match Option
/Users/	*/Documents	pattern

Exclude filters

You can exclude a range of files from the backup job. An exclusion filter checks whether to exclude each file from backup, using either a glob pattern or a regex (regular expression).

There is no limit to the number of exclusion filters you can add to a single Protected Item.

Glob pattern match

Comet can exclude files based on a glob pattern. Any files matching the glob pattern will be excluded from the backup job.

The expression is tested against the full disk path to the file. Your glob expression can be a partial match (e.g. *.txt) or a fully anchored match (e.g. C:\path\to\file.docx).

Some special characters and wildcards are allowed:

• A star (*) skips zero or more characters within a filename
• A double-star (/**/) skips any number of directory components
• A question-mark (?) skips a single character within a filename
• Square brackets denote a character class (e.g. [0-9]) within a directory or file name

Example: C:\Users\**\AppData\Local\Microsoft\WindowsApps

• This will exclude the automatically re-downloaded non-critical and normally non-accessible windows-app files.

An invalid glob pattern will prevent a backup job from running.

Regular expression match

Comet can exclude files based on a regex (regular expression). Any files matching the regex will be excluded from the backup job. The specific syntax flavor is that of the Go regexp library.

The regex is tested against the full disk path to the file. This enables filtering by path component, or (on Windows) drive letter.

By default, the regex is:

• case-sensitive. You can perform a case-insensitive match by adding an (?i) expression
• non-anchored. You can restrict your regex to the start- or end- of the file path by using the ^, $, \A and/or \z expressions.

Forwardslash (/) is not a special character and does not require escaping with \/.

Note: An invalid regex will prevent a backup job from running.

System exclusions

The Comet Backup system automatically excludes certain folders.

On Windows, Comet skips

• The $RECYCLE.BIN and System Volume Information directories in any drive root
• The C:\Recovery directory
• The C:\pagefile.sys, C:\hiberfil.sys, and C:\swapfile.sys files
• Reparse points of the IO_REPARSE_TAG_APPEXECLINK type, such as MicrosoftEdge.exe or the WindowsApps/ subdirectory
  • This is intended to work around an issue with The file cannot be accessed by the system messages affecting Windows 1803 and later.
• Invalid .DFSFolderLink files

On Linux, Comet skips

• The /proc directory
• The /sys directory
• The /selinux directory
• The /dev directory

On macOS, Comet skips

• The /dev directory
• The /Users/.../Library/VoiceTrigger/SAT directory
  • This is a protected AF_DATAVAULT directory that is inaccessible with SIP enabled (the default).

Rescan unchanged files

In a regular "Files and Folders" backup, Comet will skip over files that have the same file size and modification time as the last backup job. If these properties are the same, Comet will refer to previous chunks and not re-chunk the file. This dramatically improves performance.

If you are working with certain types of files that change content without updating their modification time attribute on the filesystem - for instance, applications that use direct disk I/O instead of filesystem functions; some database data files; or VeraCrypt container files - then the above is obviously unsatisfactory for ensuring backup integrity. In this case, you can enable the "Rescan unchanged files" feature to cause Comet to chunk every encountered file. This has some performance penalty but does ensure backup integrity in the presence of such files.