What's New?
"What's New?" is a new blog series covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over July 2022.
It's been a busy month for Comet, and we've been lucky to welcome many new staff members that have joined the development team. This month, the 22.6.x "Voyager" release track has seen five minor updates. We've also released two minor updates for our 22.5.x "Lysithea" quarterly release track. There were many interesting improvements:
IDrive e2 support
We're pleased to announce that IDrive e2 are now available as a built-in option in the Storage Vault selection type. This direct integration uses the IDrive e2 API to automatically determine the target bucket hostname, reducing the number of configuration steps compared to the S3-compatible option.
As well as using IDrive e2 directly, you can also add them as a Storage Template provider (previously known as "Requestable" Storage provider) in the Comet Server settings area. If you enter your top-level credentials in this field, Comet Server can use them to provision private buckets with separate credentials for each user, allowing seamless direct-to-cloud backup without traversing through the Comet Server Storage Role.
Mention Comet Backup when signing up to get 1TB free storage for 30 days.
A quick reminder - when using the Storage Templates feature, it's likely you'll want to configure cleanup for cloud buckets that are no longer being used by any user account. Comet's solution for this problem is called Constellation Role, and it's available from the settings page of your Self-Hosted Comet Server. In Constellation, you can enter the same IDrive e2 top-level credentials for it to search through. When a Constellation report runs, it finds all your buckets; checks all the user accounts; cross-references the user's Storage Vaults with the available buckets; and deletes any discovered buckets that are not in use by any user account.
Constellation is designed to scale up to clusters of multiple Comet Servers, with any mix of server replication, Storage Role, or direct-to-cloud buckets created by the Storage Template feature. You should have precisely one Constellation Role server amongst your entire cluster of Comet Servers.
In Comet Server 22.6.6, Constellation now has an extra safety guard to prevent deleting any data if no Auth Role server is configured. This feature can help prevent accidental misconfiguration. This feature joins the many other safety features in Constellation, such as the ability to turn off deleting buckets until you are satisfied with the discoveries it makes.
Custom S3-compatible regions
Amazon AWS first launched their ground-breaking S3 service ("Simple Storage Service") back in 2006. In the 16 years since then, the S3 protocol has evolved and grown, adding support for V4 signing, path-vs-subdomain bucket addressing, multiple regions, batch processing, streaming processing, and many more features. As "S3-compatible" has become the de-facto standard cloud protocol for object storage, there are many varied implementations of the protocol and many different providers require different quirks for compatibility.
Comet has an extensive set of auto-detection for various S3-compatible cloud storage providers, which all require slightly different configuration. We've recently encountered several S3-compatible providers all needing the same type of custom configuration, related to the region field. For providers that we officially support and partner with, such as Wasabi and Storadera, we were able to specify the necessary fields internally, but it wasn't an available configuration option for any custom S3-compatible provider. This is now available in Comet 22.6.7.
For example, configuring a custom S3-compatible Region field helps Comet connect to Scaleway Object Storage in their Paris, Amsterdam, or Warsaw regions with the latest V4 signing mode instead of requiring legacy V2 signing. We're happy to improve support for Scaleway as a well-established European cloud provider offering 6+3 data redundancy (6 data drives plus three parity drives).
Job Execution Delay
Last month, we launched the Job Execution Delay feature, to pseudo-randomly delay a backup job schedule by a different amount each time. If you apply this random job execution delay across your entire customer base, it can spread out the load on your storage infrastructure, reducing the peak spike of CPU usage that happens when many jobs all start simultaneously.
This month in Comet we've expanded our support for this feature. In Comet Server 22.6.3 you are able to enforce this setting at the Policy level and also at the whole-server level, where it will override any user or policy settings.
Test Connections
In the Comet Server web interface, when looking at a user's Protected Item or Storage Vault, a "Test Connection" button is available that uses Comet's code to reach out to the target and ensure the device is able to reach it successfully. Because Protected Items are private for each device in a user account, it's clear that we should send this live-connected signal to the correct device, but for a Storage Vault, any device in the account could potentially respond with this information for us. Up until now, Comet Server has picked a random device in the account for the connection.
For Storage Vaults, the situation can be a little different. In the case of shared Storage Vaults in the cloud, it's likely that any device can reach them. However, there are various situations where only some devices in an account might be able to access a Storage Vault. For example, a Local Path vault for an external harddrive; or troubleshooting if one specific device is unable to reach the target network. To help in these situations, the "Test Connection" button now shows a dropdown if multiple device connections could be used for the test.
We also fixed a security issue related to the Test Connections button in the web interface in Comet 22.5.2 and 22.6.4. If you are running an older version of Comet Server, please upgrade at your earliest convenience, or avoid using the web interface button.
For the programmers in the audience - the AdminDispatcherListActive API that underpins the live-connection selection here was also extended to support choosing the target username. This change is available now also in the latest updates to our PHP and Ruby SDKs on GitHub - and watch this space for more programming language SDKs coming very soon.
A tale of two compression libraries
This month we've upgraded the main compression library used by Comet Backup - the industry-leading Zstandard - to its latest 1.5.2 release. This brings with it a major improvement in compression performance. The practical impact will be limited - backup jobs are almost always bottlenecked by the speed of reading files from the local disk, or by the network speed of uploading them to the Storage Vault - but even in that case, a more efficient compression system should result in lowered CPU usage during backup jobs.
There is a second compression-related story to discuss this month - I'd like to thank everyone who reported a recent issue with the RPM installer for Comet Server. In last month's "What's New" report, we described the new option for pre-built clients. Bundling these pre-built clients with Comet Server resulted in a large size increase for the Comet Server installer. To compensate for this, we adjusted the compression settings used by the Comet Server installer to use a larger window size, so that the pre-built clients would compress effectively with the piecemeal client components that are used with custom branding. This was highly effective, and the pre-built client feature ultimately had a negligible effect on Comet Server install sizes.
However, this compression change caused issues with some RPM-based Linux distros. After some research and testing, we were not able to resolve the issue while still maintaining the optimized file size of the RPM installers - so apologies to RPM users, the Comet Server installer for 22.6 "Voyager" has jumped from ~150MB to ~250MB. This is still a relatively small file size for the great functionality packed within, so we're sure you'll be happy to have the RPM installer functioning correctly again.