What's New?
"What's New?" is a new blog series covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over May 2022.
May was a busy month for us at Comet - we've continued to expand our development team, and we are making great progress improving Comet. We've also grown our customer success team and support team, so if you see some new faces in the support tickets, please give them a warm welcome!
22.5.0 "Lysithea" release
The headline news of this quarter is that we've rolled up the 22.3.x Voyager series into a new long-term-support release. If you're not comfortable with weekly Voyager software updates, we strongly recommend the quarterly rollup series to build your business upon.
We're hosting a webinar for the release on June 7th - please pre-register your interest here to get a notification before we go live.
The codenames for Comet follow a consistent naming pattern, as we voyage from the inner solar system (Comet 17.2.0 "Mercury"), through the inner planets, Earth, Mars, the asteroid belt, Jupiter, and now looking at the moons of Jupiter. "Lysithea" is the tenth moon of Jupiter; it was discovered by Seth Barnes Nicholson in 1938, and is named after the Lysithea of Greek mythology, one of Zeus' lovers.
You can read the 22.5.0 "Lysithea" release changelog here, or if you've been following the monthly What's New blog posts or the weekly Voyager series, please read on for a bit more in-depth look.
Cloud Storage update
It's been an exciting month for cloud storage providers. Just a quick refresher that Comet performs at its best when you minimize the latency from the end-customer through to the storage location, so when choosing a cloud provider, we recommend selecting a storage region as close as possible to the end customer.
Our friends at Wasabi have continued to launch new regions, with the latest addition being Australia (launched on May 9th). At our Comet offices in New Zealand, we're particularly pleased with the latency and performance of the new Australia region. The new region is automatically detected in Comet's software, and no region-specific settings are required.
Another new storage provider hitting the market is IDrive e2. This is an S3-compatible service following in the proud tradition of naming cloud storage as a single letter and number! IDrive e2 offers storage regions in the US and EU, high performance, and an industry-leading price of $4/TB/mo, with free egress and 90% off for the first year. They also join an exclusive club of providers that implement bucket versioning for storage immutability - the perfect defense against ransomware.
IDrive e2 works very well with Comet, and you can expect to hear more about them in the future. Mention Comet Backup when signing up to get 1TB free storage for 30 days.
Requestable Storage for AWS and IAM-compatible
Comet Backup supports many different cloud storage providers if you bring your own storage credentials. If you have configured the customer's Comet Backup app to back up directly to a cloud storage provider, you should avoid reusing storage credentials across customers. For each customer, you should either create new cloud storage credentials manually; or proxy the data through Comet Server Storage Role; or ideally, have Comet Server provision new cloud storage credentials automatically. We call this the "Request Storage Vault" or "Requestable" system.
Previous versions of Comet Server supported this "Requestable" provisioning for Backblaze B2, Wasabi Cloud Storage, and custom HTTP integrations. We're very happy to announce that Comet Server extends this support to Amazon AWS and to custom IAM-compatible cloud services.
There are a lot of S3-compatible cloud storage providers on the market, but having "Requestable" support to automatically provision private credentials is a more advanced level of integration than just S3-compatible, and there's not yet an industry standard for providers to use. We're working with individual storage partners to expand our "Requestable" integrations over time.
Hourly schedules
When configuring a backup job schedule in Comet, you can now tell Comet to back up hourly, but only at certain times of day:
The main two use cases for this feature are
- to back up only 9am to 5pm, Monday to Friday, when you know the customer's computer is on and working, to avoid any meaningless Missed backup job reports; and
- the exact opposite, to back up from 5pm to 9am when you know the customer isn't interactively using their computer.
It's long since been possible to select an hourly schedule, or a daily schedule for weekdays, or individual backup times - but rolling them all together is a nice improvement that makes this kind of configuration much simpler.
This was a popular suggestion from our Feature Voting page. If you haven't checked out our Feature Voting page, i'd really recommend it. We look at this system often to help guide our priorities, so it's a great way to help shape the future of Comet.
Synology
Our Synology .spk package was first publicly released in February this year, as part of Comet 21.12.7. Since then, we've continued to improve the installer and our compatibility.
The platform compatibility has been extended to include more types of ARM CPU, including Synology NAS models based on the Armada, Comcerto, and Monaco platforms. The Synology package has been improved to better understand NAS timezones, provide clearer warnings on login failure, and to calculate device IDs in a more repeatable way. You should find that the installer just works in many more cases.
In the DSM 7 version of the installer, some additional steps are required to grant file permissions to the Comet Backup app. Previously, these steps were only listed in our official documentation, and if you are white-labeling the Comet Backup software for your end-users, then it is difficult to share a link to this information. The latest versions of the installer now include a new built-in help guide explaining how to configure the necessary permission settings for backup.
All of the changes mentioned above are included in our latest quarterly release 22.5.0 "Lysithea".
Comet Server Web interface
The tables in the Comet Server web interface now properly remember their state when refreshing the page or using the back button. This may sound like a subtle change, but it has had an immediate impact in making the Comet Server web interface much more intuitive and easy to use.
This was a long-standing request from our Feature Voting system. It's taken a while to get to this point because Comet Server's web interface is an SPA (Single Page App). This has been an excellent technical foundation for us, as our development team reuses the official Comet Server API, ensuring that it is stress-tested and has broad feature coverage, allowing any external API developers to get the same experience we do internally. However, this API-first design has meant we're not technically using the web browser's traditional Back button handling, and that has resulted in minor quirks like this. Thank you all for your patience with us as we developed this feature.
As of Comet Server 22.3.6, and available now in Comet 22.5.0 "Lysithea", when using the browser back button, all table settings will be restored exactly how they were, including rows-per-page, sorting, current page number, search term, and custom column selection. Your rows-per-page setting will also be preserved globally, even when you log out and log in again.
Your custom column selections will be preserved for each individual table, even when you log out and log in again, with your current version of Comet Server. However, if the Comet Server version changes, the custom column selections will be reset, because new versions of Comet Server might have introduced a different set of available columns.
As well as this improvement to the table system, there were many other small improvements to the Comet Server web interface this month. For example, "Internal Error" popup messages can now show you the information directly in more cases, such as when an administrator tries to use the "Request Storage Vault" feature.
Self-backup
Comet Server has a feature called Server Self-Backup to create backup copies of Comet Server's own configuration, such as the cometd.cfg and comet-users.db files.
The Self-Backup feature does not use the full chunking and deduplication infrastructure like a regular Comet Backup job, because if the Comet Server itself experiences an outage to the point where you need to restore from backup, then this chunking and deduplication infrastructure is unavailable. Instead, the Server Self-Backup only creates simple passworded zip files in any supported storage location. This self-contained design is a way to break the "chicken and egg" problem compared to using the full Comet infrastructure.
For providers with very large Comet Servers with thousands of user accounts and millions of job logs, the overhead of performing a Self-Backup might be considerable. The latest version of Comet Server includes two key improvements to the Server Self-Backup feature to reduce the CPU load in these cases.
Firstly, you can use the new slider component to adjust the compression level, to make a trade-off between 'Faster' compression with lower CPU impact but a larger file-size, and 'Smaller file-size' with a higher CPU impact.
If your Comet Server is hosted on a cloud provider where you pay for both CPU usage and for storage usage, you might find that the storage cost of a larger Self-Backup is actually lower than the CPU cost of performing more compression.
In addition, a new option has been added to exclude the job database from the Comet Server Self-Backup.
The job database is almost always the largest component of a Self-Backup, responsible for as much as 90% of the archive size and CPU impact. It's still important to back up this job database. However if it was excluded, then perhaps the other parts of the Comet Server configuration could be backed up more frequently. Excluding the job database also makes the generated zip archives smaller and more accessible when you only want to restore the user-profile settings or the server configuration.
For large partners in this situation, we recommend having two Self-Backup schedules: one frequent schedule that excludes the job database, and another less frequent backup that includes all databases.
Restore as Archive
When you restore data in Comet, you can restore files and folders directly; you can restore Office 365 emails back to the cloud; you can stream restore data into another program without spooling (great for MySQL); you can restore to nowhere, just to test that Comet is working; and, you have the option to restore as an archive format.
Multiple archive formats are available, including the ubiquitous zip, and we might as well have stopped there - but there is a time and place for other options. In particular, Linux users are better served by a compressed tar archive that can be used with standard system tools, without needing to install a Zip program.
This month, the Restore as Archive feature was expanded to support tar.zst, the latest type of tar format that uses zstd compression instead of gzip. This results in a smaller filesize while also being faster to compress and to decompress. The ordinary Linux untar command (tar xaf file.tar.zst) works with these files since GNU tar 1.31, which can be found in Debian 11 "Bullseye", Ubuntu 21.10 "Impish Indri", Fedora 30, and RHEL 9.
The .tgz and .tzst file extensions are now understood as common aliases for the .tar.gz and .tar.zst extensions, and the target directory for restore is now created automatically, aligning this behavior with how an everyday File and Folder restore works.
Tenant support
Comet Server can be configured for multi-tenancy, to give each of your partners an isolated zone within your Comet Server for their own individual customers. We've seen this feature used for reseller support and for hosted service offerings. Tenants (previously known as Organizations) have been a standard feature of Comet Server for a long time now, and we're continuing to improve the support.
When logged in to the Comet Server web interface as the top-level administrator, looking at a cross-tenant customer, you will find that the available Policy options now more accurately reflect the tenant's policies, instead of showing unusable options from other tenants. The "Request Storage Vault" dropdown option also now correctly finds the tenant's list of "Requestable" storage providers. These changes should make it much more seamless to manage a tenant's customers from the top-level account.
If you're a software developer making use of the Comet Server API to manage your Tenants, you might be interested to know that these improvements come thanks to some new API parameters in our AdminAddUser, AdminPoliciesList, AdminRequestStorageVaultProviders, and related APIs. New features for the API are generally mentioned in the official changelogs, but you can also stay up-to-date by following the changelogs for our PHP SDK and Ruby SDK on GitHub.
Translations
Internationalization is a high priority for us at Comet. It enables you to serve your customers in local markets much more effectively.
Comet is available in 13 fully translated languages, and the Comet 22.5.0 release includes updated translations for 10 of those - Danish, German, Spanish, French, Croatian, Italian, Portuguese (Brazil), Portuguese (Europe), Russian, and Hebrew. All of Comet, from the end-user desktop app and customer-facing email reports, through to the admin section and the new user guide from last month, can be used in translation.
We've also found- and fixed- many small gaps where the underlying English was showing through, such as the pagination footer for tables in the Comet Server web interface.
Our staff mostly only speak English, so we do hire translation contractors for the bulk of this work - but over the years we have consistently found that the best-quality translations come from our customers, who have a full understanding of both the software and the IT / MSP backup industry. If you see a typo in our translations, or if you want to help translate Comet into a new language, we'd love your contribution and it couldn't be easier to get started - just head to our GitHub page and follow the instructions.
In particular this month, I'd like to thank one of our partners for getting the Chinese (Traditional) localization started! This will be our 14th language.
Upcoming Webinar
If you'd prefer to watch rather than read, I'm hosting a live webinar to discuss the latest 22.5.0 "Lysithea" release and everything in it since our previous quarterly release. Please register for a notification before we go live on June 7th to catch up on all the latest Comet news with me - there will a free live Q&A session after the presentation.
As well as that, we have many more videos available on our YouTube channel, including guides on getting started with Comet, individual features, and webinars for previous quarterly software releases.