"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over July 2023.
There were five Comet software releases during July - one in our quarterly 23.5.x Thebe release series, plus four releases in the 23.6.x Voyager release series.
Single sign-on with Microsoft, Google, and OIDC
Comet Server is adding support for administrators to single sign-on (SSO) to the Comet Server web interface, using a supported OpenID Connect (OIDC) identity provider:
OIDC is a framework for authentication and authorization, based on the OAuth 2.0 standard. It's widely used by many providers for "Log in with..." buttons. This new feature extends our existing single sign-on support using the LDAP protocol.
The additional identity providers (IdPs) now supported are:
- Microsoft Entra ID (formerly Azure AD)
- Google (Google Cloud, Google Workspace, or personal)
- Any other OIDC-compatible provider that uses a discovery document (usually at the
.well-known/openid-configurationURL path).
You can configure a new OIDC provider from the Comet Server web interface > Settings screen > "Admin Accounts" tab > "External Authentication Sources" button:
To use this feature, you should first visit your IdP's settings page, register a new application credential within the IdP, and copy the credentials to this settings page. You will then need to copy Comet's generated "Redirect URI" field back into your IdP's settings page.
When the administrator user uses the new "Log in with..." button and performs a successful login operation via the IdP, a new Comet Server administrator account will be dynamically created for them on-demand. As with LDAP, you can specify which Comet Server permissions are granted to the newly generated account. This new account is marked as "Externally managed" within the "Admin Accounts" table, ensuring that valid IdP login is required to access this administrator account.
If your IdP enforces two-factor authentication (2FA), you can configure Comet Server to skip enforcing its own internal 2FA on the account, so that the user is not bothered twice.
You can request custom scopes, and enforce claim values against either custom scopes or standard OIDC scopes. This allows you to enforce that the only members of certain Microsoft, Google, or OIDC groups within your IdP are allowed to log in to the Comet Server.
This feature is available both for the top-level Comet Server administrator as well as individually for each tenant.
Price change notice for Comet Storage powered by Wasabi
Our Comet Storage service gives you the option to purchase Wasabi Cloud Storage directly from Comet, offering all-in-one billing and providing a more integrated experience. This month, we've passed along the latest price changes from Wasabi, to their new price of $6.99 / TB.
For more information, please see Wasabi's official announcement.
Improved job start performance
When backing up a Files and Folders-type Protected Item, one of the first steps is for Comet to enumerate all the selected files, in order to calculate their total size. The total size is used to enforce the "All Protected Items Quota" feature, as well as to properly determine the progress bar's expected upper bound. If you are running a headless device with no GUI to render the progress bar, and you are not using the "All Protected Items Quota" feature, then there's no remaining purpose for this scan phase, and so Comet will skip it to save time.
We heard mixed feedback about this - a discussion in our feature voting system uncovered some use-cases where the progress bar would still be desirable even on headless devices with no GUI. But also, there was competing feedback that spending time on file size measurement is still slow and undesirable even in some cases where the GUI was present.
In the latest version of Comet, we've come up with a new and better approach to this issue. If the "All Protected Items Quota" feature is used, then we require an accurate measurement up-front regardless. But if this feature is not used, then we can rapidly create an approximate progress bar size based on the previous backup job's size plus some small estimated buffer amount. This should provide a great speed improvement for GUI users, a reasonable progress bar for headless users, and at the same time, provide an accurate measurement for quota users. The reported size measurement will always be completely accurate after the backup job finishes.
Improved low-memory modes
For users using Comet on devices with low RAM, our software has long since offered the "Prefer temporary files instead of RAM (slower)" option for backup jobs, to toggle whether Comet stores the deduplication index either in-memory or on-disk in a temporary database file. Enabling this option can significantly reduce Comet's memory usage, allowing the backup job to complete on low-memory devices, at the expense of a longer backup job duration.
The latest version of Comet extends this option to also use a small in-memory bloom filter. This allows Comet to perform some of the deduplication operations in-memory with minimal overhead. This new combination technique can significantly improve the performance of this option for low-memory devices.
The deduplication index is needed for almost all operations involving the Storage Vault, not just backup operations. This month, we've also added an option to use temporary files instead of RAM during a restore, extending the possible use cases for Comet on low-memory devices.
Performance improvements for new servers
The performance improvements this month are not limited to the Comet app itself. We've also significantly improved the account.cometbackup.com system: downloading the large Self-Hosted Comet Server installer is now implemented via an Amazon CloudFront cache, improving download speeds between 2-6x in our testing.
We have also been able to significantly improve the speed of creating new Comet-Hosted Comet Server instances. The creation time has been reduced from 60-90 seconds down to 10-15 seconds, owing primarily to some changes in the default generated DNS names.
Configuration change notice for PKCS11 codesigning
Comet supports Authenticode codesigning for Windows using either an on-disk file (PKCS #12), or a hardware security module such as a USB device (PKCS #11), or a cloud HSM on Azure Key Vault. With the file-based approach no longer being supported for new Authenticode certificates, we are seeing increased use of the alternative PKCS #11 and Azure Key Vault options, as partner Authenticode certificates come up for renewal.
If you are using a physical USB device for Authenticode codesigning, we have updated the available settings options to improve compatibility with a wider range of devices. The new settings screen should be clearer and easier to use, but you may be required to update your configured settings, as depicted:
If your Comet Server is running in a cloud VM, it's not feasible to plug in a USB hardware device for codesigning. We would recommend Azure Key Vault as an excellent cloud-based solution to this issue, but we've also recently tested the compatibility of the third-party Virtualhere software for remotely forwarding a physical USB device to another PC, and we can confirm this solution works for PKCS#11 codesigning when running Comet Server on a cloud VM.
Quick search
When using the Comet Server web interface, the quick search bar (using the Alt + Q keyboard shortcut) could previously search through usernames, Account Name field values, Protected Item and Storage Vault names, settings pages, and more. In the latest version of Comet, we have extended the search capabilities to also find users by their email address.
You can also now enter the ID of a Protected Item, Storage Vault, or even a backup job, and the quick search bar will try to match it with the corresponding user or job. This is particularly helpful for troubleshooting some situations.
That's all for this month - the blog will return next month with more news about all the latest changes to Comet.