"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over November 2023.
There were seven Comet software releases during November - four in our 23.9.x Voyager release series, two point releases for our 23.8.x Adrastea quarterly series, and the official launch of our latest 23.11.0 Saturn quarterly release.
Last month, we launched our newest Protected Item type to add support for backing up VMware virtual machines using Comet. Over the last month of Voyager series releases, we've seen great uptake of the new feature and have been working to improve it based on your feedback, including improved performance, compatibility, and correctness.
The VMware vSphere hypervisor platform encompasses both the ESXi hypervisor as well as the vCenter centralized management application. ESXi is available in multiple licensed editions, including a free license with various limitations such as restrictions on the number of virtual CPUs that can be assigned to a given virtual machine.
In our first release, Comet supported ESXi versions with a paid license ("vSphere Essentials" or higher), but in the latest Comet 23.9.11 we took a closer look at the problem, and have extended our compatibility to include support for the free version of VMware ESXi. Our implementation supports the same versions (6.7, 7.x, 8.x) of ESXi and is built solely using VMware's officially supported vSphere VADP backup APIs, including Changed Block Tracking (CBT) support. No configuration changes are required to your free-license ESXi host server to support this backup mode.
As part of working on virtual machine features this month, we put a particular focus on the granular restore option. The existing granular restore mode in Comet allows you to take a virtual disk backup at the block level - either of VMware, Hyper-V, or of a physical Disk Image - and then restore individual files and folders from the interior NTFS filesystem.
Our granular restore feature gives competitive restore performance and offers flexible interactive browsing, all while supporting multiple disk image and partition formats, using Comet's encrypted deduplicated cloud storage.
This month, we've made granular restore even faster and have reduced its memory and network traffic requirements, by using an intelligent caching system that helps Comet traverse the virtual NTFS file table in a more efficient way. We have streamlined the restore and browsing workflows to reduce edge cases, improving support for virtual machines with multiple disks, disk images in subdirectories, disk images that span multiple files, and other such cases. In addition, Comet is now able to display each partition's name in more cases.
Outside of granular restore, we've also optimized our ordinary file restore further by removing bottlenecks that could stop Comet from effectively multithreading. In the latest versions of Comet, it should be much faster to restore very deep directory trees.
Comet Server has an extensive permission system for administrators, tenant administrators, and end users. Comet recently added support to allow a top-level administrator to restrict a tenant administrator into using certain policies. However, as this restriction was not transitive, the tenant administrator could still allow a tenant end-user to bypass those policy restrictions. The latest version of Comet adds a new ability to enforce settings and preferences across an entire Comet Server.
Without careful consideration, adding features can sometimes result in a cluttered user interface. As Comet Server has grown to offer more and more control over user permission levels, we have split out the administrator permission settings onto a separate tab.
The web interface and API for Comet Server make use of a built-in web server, based on the standard library from the Go programming language. This embedded web server is secure, performant and powerful, and includes many recent features such as TLS 1.3 and HTTP/2. As this is a bundled component, any security updates for this component must be delivered as part of a Comet Server update. Recently, the HTTP/2 Rapid Reset Attack (CVE-2023-44487) made headlines around the world, affecting major corporations including Google and Cloudflare. This month, we were able to take advantage of newly available mitigations to resolve the issue directly within Comet Server.
We've also implemented additional security protections against XSS attacks, as a defense in depth measure. The Comet Server web interface now uses an extremely strict Content Security Policy (CSP) to help prevent any remote attacks on your administrator login session.
We've kept the CometCon tradition going, bringing remote and local Comet team members together for another week-long conference.
During the week, we made it a priority to focus on deep technical brainstorming and planning, with speakers from various departments. But it wasn't all work - the collaborative environment was combined with social events including decorating the office, renting out a cinema, learning to mix cocktails, taking a flight lesson, and the infamous "Mandatory Fun".
We're very pleased to announce our latest Quarterly release series - Comet 23.11 Saturn.
Our product's first codename started many years ago with the planets of the inner solar system, and our Voyager track continues ever more outward into distance parts of space. Recently, all our quarterly releases have been named after moons of Jupiter. But with the recent addition of VMware support this quarter, we thought that adding a big new feature deserved a big new planet!
Saturn needs no introduction as the sixth planet from the Sun. It is the second-largest planet in our solar system after Jupiter, large enough to fit the Earth inside more than 760 times over. Its iconic rings were discovered in 1610, and we are continuing to find additional moons - over 140 have been discovered, most recently in 2020.
Comet 23.11.0 Saturn includes 8 new features and 23 enhancements that were developed over the course of the previous 23.9.x Voyager series, including our VMware Protected Item type, a lobby for silent installed devices, and improvements for Object Lock and Comet Storage.
We are currently working towards releasing our new Hyper-V Backup 2.0 offering next year, which means we’ll be retiring our existing Hyper-V backup at the same time. To prepare for this, we’ll preview new technology in Comet’s Voyager releases and add further capabilities to Hyper-V backup as we work towards the launch of Hyper-V Backup 2.0 in 2024!
Comet Backup Offers All-In-One Backup Solution for Businesses and IT Providers with Integrated Cloud Storage Powered by Wasabi
CHRISTCHURCH / BOSTON – November 1, 2023 –Comet Backup, announced an expansion of their long-standing partnership with Wasabi Technologies today. This partnership fully integrates Wasabi’s cloud storage offering into Comet Backup’s software allowing IT providers and IT teams an easy all-in-one backup solution.
Comet Backup empowers IT professionals to manage backup data for all of their clients through a single interface, complete with built-in monitoring and reporting features. By integrating Wasabi hot cloud storage into Comet Backup’s platform, IT providers now have a convenient all-in-one solution for backup and cloud storage. At 80% less than the cost of Amazon S3 with no fees for data egress or API requests and no vendor lock-in, Wasabi provides a scalable, long-term cloud storage option at an affordable price.
“The IT industry is seeing an increased demand for fast, easy-to-use solutions to protect business-critical data. With organizations facing growing security threats, it is vital for companies to have systems in place to ensure business continuity,” said Josh Flores, General Manager of Comet Backup. “We equip IT teams with the tools they need to deliver simple, affordable data protection. We’re excited to partner with Wasabi to allow our customers to centralize their backup strategy and streamline their workflows. The flexibility and cost-savings Wasabi offer make them the perfect partner for this solution.”
By offering Comet Storage powered by Wasabi, IT teams no longer need to set up a separate storage account, or manage multiple vendors. Comet users will pay a single bill for both backup and storage, at the same low price as buying storage directly from Wasabi. With a unified backup and cloud storage solution, companies can spend less time managing their technology stack, and instead focus on their customers, their core competencies, and their bottom line.
“Comet Storage powered by Wasabi is now making it easier than ever for managed service providers and IT professionals to leverage the cloud through a single billing, monitoring and reporting interface. Wasabi is excited to be integrated into this solution,” said Laurie Mitchell, Wasabi Technologies SVP Global Marketing. “The result is a cohesive backup and storage solution that’s perfect for MSPs and IT teams to keep things simple and cost effective.”
For more information on Comet Backup and Wasabi, please visit our dedicated landing page.
Comet provides fast, secure backup software for IT professionals and businesses worldwide, localizing to 13 languages. It enables organizations to secure their data, deliver business continuity and disaster preparedness. Trusted by customers across 120 countries, Comet has been recognized as a momentum leader by G2 and awarded “Best Software” and “Most Affordable” by Software Suggest. Founded by the same team who built MyClient and Nexus (now a J2 Global company), Comet is a privately held company based in Christchurch, New Zealand.
Wasabi provides simple and affordable hot cloud storage for businesses all over the world. It enables organizations to store and instantly access an unlimited amount of data with no complex tiers or egress or API fees, delivering predictable costs that save money and industry leading security and performance businesses can count on. Trusted by customers worldwide, Wasabi has been recognized as one of technology’s fastest-growing and most visionary companies. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is a privately held company based in Boston. Wasabi is a Proud Partner of the Boston Red Sox, and the Official Cloud Storage Partner of Liverpool Football Club and the Boston Bruins.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over October 2023.
There were four Comet software releases during October, all in the 23.9.x Voyager release series.
Comet 23.9.7 adds support for backing up VMware virtual machines.
This has been a long-standing request on our Feature Voting page. After running a successful beta program over the last few months, we're very happy to be able to deliver this feature to partners in our Voyager track.
Comet's new VMware Protected Item type allows you to easily connect to your ESXi or vCenter server. You can pick individual VMs for backup across all datacenters, or choose "All VMs" to ensure all VMs are backed up with targeted exclusions.
The feature supports Changed Block Tracking. After completing an initial backup job, any future backup jobs will coordinate with the VMware host server to identify which ranges of the disk have changed since the previous backup job. These changed ranges from the VMware server are adapted into content-defined boundaries for Comet's deduplicating chunking engine. This results in an extremely efficient, incremental-forever backup.
As this is a new Protected Item type, it must be configured to run from an installed device that will perform the compression and encryption workload. Installing Comet Backup on a VM within the VMware server itself is recommended for reduced end-to-end latency. In this first released version in Comet 23.9.7, this feature requires the device to be running Windows x86_64.
We are excited to bring this new Protected Item type to our entire Comet Community, so look out for our upcoming quarterly software release at the end of this month. We'd love your feedback and are here to help if you need any assistance getting started, reach out via our support ticket system.
It's official - Comet Storage is a new cloud storage offering from Comet, in partnership with Wasabi. We offer Wasabi's same great S3-compatible service, at no additional cost above their public pricing. The feature is fully integrated and managed from within your account.cometbackup.com account, giving you unified billing and reporting across both cloud storage and your backup software licenses.
Comet Backup will continue to support a wide range of cloud storage providers. However, the new all-in-one Comet Storage offering is both highly convenient and excellent value. If you are interested in migrating to Comet Storage from an existing Wasabi account or from another cloud provider, please contact us for migration assistance.
Comet Storage also supports S3 Object Lock, allowing the backed-up data to be marked as immutable. This is a complete defense against ransomware attacking the backup storage location itself, giving you a fixed number of days to identify and mitigate the issue. In the latest version of Comet, we've also made Object Lock easier to use for all supported Object Lock-compatible providers, by simplifying the configuration options for both Storage Vaults and Storage Templates.
For more information, please see the documentation, or check out our latest YouTube video:
New account.cometbackup.com user interface design
Every element on the page has been given a fresh coat of paint - from buttons to popups, from paying your bill to raising a support ticket. We've also grouped some pages together in a more logical way, so you'll find it simpler to make your way around the site.
Inspired by the similar change to the Comet Server web interface earlier this year, the new design has moved the main navigation bar from the top to the left-hand side. This change brings our branding more closely in line across these two interfaces. On devices with smaller screens, such as laptops and tablets, you can click the small arrow button to collapse the navigation bar and regain horizontal screen real estate.
Comet supports backing up physical Disk Images, Hyper-V virtual machines, and VMware virtual machines. All of these different Protected Item types result in virtual disk files. Comet supports granular restore for all three types, allowing you to browse through partitions and supported filesystems, to restore individual files from within a full disk backup.
In the latest versions of Comet, we've significantly improved the speed of granular restores from Disk Image backups. Some particular use-cases seeing a large improvement are granular restores involving a large number of directories, or a large quantity of small files. We're committed to continuing to improve Comet's performance and this work has identified more opportunities for improvement across all three types, so watch this space!
We've also added a feature to restore Disk Image backup jobs as VMware vSphere-compatible virtual disks.
Both Disk Image and our new VMware Protected Item type generate virtual *.vmdk files inside Comet's deduplicated Storage Vault. However, the subformat of the files does differ slightly. Until now, users who are using Comet to perform a physical-to-virtual (P2V) migration from a physical disk to a VMware virtual machine have been required to perform an extra file format conversion after the restore, requiring extra time and temporary disk space. With the new option in Comet to restore the disk in VMware vSphere-compatible file format, the conversion takes place dynamically as part of the restore job, simplifying the process and helping meet your recovery time objective (RTO).
Earlier this year, we added Audit Logging support to the self-hosted Comet Server product, to help our partners meet their compliance obligations. Since then, we've expanded the list of audit properties, and added a helpful option to configure this feature directly from the Comet Server web interface from the Settings page on the "License & Access" tab.
The new controls should make it much more accessible to configure Audit Logging support for your Comet Server.
We sat down for a chat with Natalie, Comet’s Marketing Manager, who has been with the company since July 2020. This piece has been lightly edited for length and clarity.
I grew up in Silicon Valley, where having a passion for emerging technologies was instilled in me from a young age. I started my career at Apple and then worked for a few different companies, including a large MSP on California’s Central Coast.
I saved up and traveled the world for a year, then moved to New Zealand. I’ve lived in Christchurch for five years and absolutely love the culture here.
What initially attracted me to working here was the team. Everyone gets to be themselves at Comet. I've worked in companies before where you show up and you're professional, but you don't necessarily get to bring your full self to work. At Comet, you get to show up and be your authentic self, which is much more fulfilling.
We've had some really fun companywide homegrown initiatives. For instance, one of our developers, Ben Frengley, was really into growing spicy chili peppers. He gave seedlings to anyone who was interested. A number of us planted them in our gardens. We take pictures and share updates on them occasionally which has been fun.
Other team members have carpooled for ski trips or gone hiking together. We do company barbecues and potlucks from time to time. There’s a very natural cohesion in the way that the team comes together and rallies around things.
You’ve recently returned from parental leave. How was your transition back into the office?
When I was on maternity leave, I missed the team. The fact that we only hire really nice, good people makes the experience of coming to work so enjoyable. The team has been really supportive. There are a couple of us in the office who work part time: one of our team members is passionate about doing volunteer work, and I’ve transitioned back into work at a pace that makes sense for my family right now.
Tell me about Comet’s approach to work life balance.
We have great work life balance, better than at any other company I’ve worked for. That’s something a lot of workplaces say they offer, but at Comet we really do have that flexibility. For example, if I miss an hour in the morning because I need to take my daughter to an appointment, it’s easy for me to make up the time later in the evening after she goes to sleep so I don’t miss those precious family dinnertime moments.
Some people prefer to start work early in the morning, and finish earlier in the day to go surfing or pick up their kids. We also have the option to work from home two days a week. It’s great to have everyone’s lifestyles and schedules supported.
What are your favorite things about working at Comet?
I've bounced back and forth between large corporates and small companies. Ultimately I prefer the outsized impact my contribution makes at a rapidly growing company. You get to have autonomy and take ownership of your work in a meaningful way. There’s no red tape or bureaucracy, if you have a great idea, you get to run with it.
Because we’re constantly experimenting at Comet, there isn’t a culture of blame. A couple years ago, I recommended we go all in on a big podcast sponsorship. Halfway through I realized the campaign was failing and I decided to pull it. When I told the leadership team what had happened, the response was, “What did you learn from that experience? Okay, what’s next? Let’s keep moving.”
What keeps you passionate and inspired working in this field?
Although on the surface, the idea of working in backup software might not be as sexy as working for a gaming company or in an industry that has a lot of cache, at the end of the day, what we do really matters. We’re helping people protect their businesses and their livelihoods. We’re not selling a consumer product that will end up in a landfill. We’re providing an essential service, which makes it a very rewarding industry to contribute to.
Before working at Comet, I specialized in brand and event-based marketing. When I started here, I needed to level-up as a generalist. I had this amazing opportunity to rapidly grow and learn about all these other areas of marketing I had never touched before.
When you’re on a large corporate marketing team, there are entire teams dedicated to each area – campaigns, advertising, sponsorship, events. At Comet, there’s a huge variety to what you get to work on day to day, which keeps me interested and motivated.
Working at Comet is my dream job. I spent the first decade of my career trying to find a company to work at where I would be this happy. Frankly, my early career was a rough and bumpy road at times. It took a while to get here but I’m deeply grateful to have finally arrived at a place where I’m so professionally fulfilled.
Christchurch is the perfect size city. You have all the big city amenities without any of the big city downsides, like traffic. There’s arts, events, festivals, hot pools, great places to eat and drink with friends. The outdoors is also very accessible – it’s only a 20 minute drive to beautiful beaches and you can escape to the mountains within 1-2 hours.
There are such good people here. We’ve made great friends and built a strong community, which has allowed us to settle and make Christchurch our home.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over September 2023.
There were five Comet software releases during September - four releases in the 23.9.x Voyager release series, plus one for our quarterly 23.8.x Adrastea release series.
We've been hard at work developing a new way to register devices you would like to protect, right from the Comet Server admin web interface.
With the new remote registration feature you can remotely register a device and assign it to a user profile, right from the Comet Server admin web interface when the device has been silently installed using the /LOBBY flag.
We've renamed the Connected Devices page to the Devices page and made a number of improvements to this area.
You can now see all devices that have been connected to your Comet Server, not just online devices. This extra information will allow for easier troubleshooting of what state each of the devices connected to your server are in.
We have also added bulk actions to the Devices page. These bulk actions allow you to manage the devices connected to your server much faster than before.
As a final touch we have updated how clicking a device status on the "My Devices" widget works. If you click on "Online," this will take you to the Devices page and apply a filter so that only online devices are shown.
In rare circumstances it is possible for a backup or restore job to be stuck in a running state. For example, this can occur if a device becomes damaged when running a backup job and never starts up again to report to the Comet Server. These jobs stay visible on the Comet Server and impact how easy it is to monitor other jobs that are actively running.
As a first step to provide you with the tools to manage stuck running jobs, we have added the ability to manually mark a stuck running job as abandoned from the admin web interface.
An abandoned job is a job that Comet has automatically determined is no longer running after communication with the client was lost, or one that has been manually marked as abandoned by an admin. We have updated our documentation to add clarity to what each of the job statuses mean.
Marking a job as abandoned has been added to the "cancel a job" workflow. To remove a stuck running job, simply attempt to cancel the job and the Comet Server will help you with the rest.
A job marked as abandoned will revive itself and show as running if the Comet Server ever receives an update about that job again.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over August 2023.
There were six Comet software releases during August - five releases in the 23.6.x Voyager release series, plus one release for the launch of our new quarterly 23.8.x Adrastea release series.
Our goal for the new interface design was to make the product simpler for new users, while still keeping it familiar for existing users. The tabs, icons, buttons, fonts and colors have all been refreshed with a modern rounded style, but you are still greeted with your familiar Protected Items and charts on the home screen.
The most fundamental change was made to navigation within the app: the previous left-side navigation bar has been removed, meaning your Protected Items are always clearly visible, and we have refreshed the breadcrumb bar appearance to help orient you to this landmark interface element. The removed left-side navigation bar has been replaced with a top navigation bar which puts more emphasis on your branded company logo, and offers clearer, more distinctive focus on the calls-to-action for the key backup and restore functionality.
In the top-right corner, you will see a a dropdown overflow menu, using modern and intuitive UX iconography. Inside this menu, you can get quick access to detailed job history; creating recovery media for restoring Disk Image backups; and a new Settings dialog. The new Settings dialog contains Storage Vault management, account and login settings, email reporting settings, your devices, and the ability to import settings from other supported backup products.
The details are subject to change before release, but we expect to be able to support major versions of VMware ESXi (6 / 7 / 8) and vCenter, using either free or paid licenses. Your backup jobs can be accelerated using changed block tracking (CBT) to produce synthetic full disk images that are deduplicated inside your Storage Vault. The new Protected Item type will work seamlessly with Comet's scheduling, deduplication, compression, encryption, job reporting, tenants, and granular restore of single files from supported virtual disk filesystems.
We're in the final stages of the beta program and are currently accepting new partners to help us ensure that this new Protected Item type is a good fit for your production VMware infrastructure. If you are interested in getting early access to this feature, please follow this link to register your interest - we would greatly appreciate any feedback you might be able to give us before the official launch later this year.
Comet has supported Backblaze B2 as a storage platform for over six years, since our 17.6.4 release back in July 2017. Over the years Backblaze B2 has proven to be a reliable, trustworthy, performant, and cost-effective solution. Together with Wasabi, these are the two most popular cloud storage providers amongst Comet users.
This month, Backblaze B2 have announced a price change. The base storage cost is increasing from $5 USD / TB to $6 USD / TB, but egress bandwidth costs are being reduced. For full details, please see their official announcement.
WebDAV is a storage protocol like FTP, SFTP, or the S3-compatible protocol, that can be used to store files in a remote location. It's based on HTTP technology and supports password based authentication, as well as transport layer security over HTTPS. The protocol has been around since 1996 and was standardized by the IETF in RFC 4918.
Accessing a remote WebDAV server is a built-in feature of Windows Explorer, as well as macOS Finder and the KDE and GNOME file managers.
Because the capability for accessing remote storage is built into the operating system, WebDAV is simple to use with a very low barrier to entry, helping it maintain a broad user base amongst enterprises, universities, and commercial service providers including Hetzner Storage Box, DreamHost, Yandex Disk, pCloud, and many others.
Clicking the "Feedback" text will open a short survey asking for any short thoughts you have about your impressions of Comet and how easy our product is to use. Our Customer Success team would really appreciate any answer you give. After submitting feedback, the tab will disappear, but you can submit more feedback at any time by clicking the "Give us Feedback" link in the page footer area.
Earlier this week, we put the finishing touches on our latest quarterly release, Comet 23.8.0 Adrastea. This is the the latest entry in our quarterly rollup series, that branches off from our main rolling Voyager development into a fixed target for you to qualify and build your service offering upon.
As with all our recent quarterly release series, Adrastea is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is the second-closest moon to Jupiter and the smallest of the four inner moons, orbiting at the edge of Jupiter's main ring. It is thought to be the main contributor of material to the rings of Jupiter.
For users coming from the previous 23.5 Thebe quarterly release series, Adrastea adds 7 features and 24 enhancements, including the new Comet Backup desktop app design and WebDAV support mentioned above; single sign-on support with OpenID Connect (OIDC); Protected Items that can stay linked with the user's Policy; additional admin permission options; and many performance improvements.
The 23.8.0 series does remove support for some old versions of macOS. If you have users with old Mac machines that are not able to upgrade the OS, the previous 23.5.x Comet Backup client will remain capable and working when connected to a 23.8.x Comet Server.
The full set of changes can be found in the release notes.
If you'd prefer to watch rather than read, we're hosting a webinar next week to discuss this new quarterly release and all the new changes. Please register for a notification before we go live on September 5th (5pm EDT / 2pm PDT) to catch up on all the latest Comet news with me - and as usual, there will be time for a live question-and-answer session at the end of the presentation.
As well as that, we have many more videos available on our YouTube channel, including guides on getting started with Comet, individual features, demonstrations with our technology partners, and webinars for previous quarterly software releases.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over July 2023.
There were five Comet software releases during July - one in our quarterly 23.5.x Thebe release series, plus four releases in the 23.6.x Voyager release series.
Comet Server is adding support for administrators to single sign-on (SSO) to the Comet Server web interface, using a supported OpenID Connect (OIDC) identity provider:
Google (Google Cloud, Google Workspace, or personal)
Any other OIDC-compatible provider that uses a discovery document (usually at the .well-known/openid-configuration URL path).
You can configure a new OIDC provider from the Comet Server web interface > Settings screen > "Admin Accounts" tab > "External Authentication Sources" button:
To use this feature, you should first visit your IdP's settings page, register a new application credential within the IdP, and copy the credentials to this settings page. You will then need to copy Comet's generated "Redirect URI" field back into your IdP's settings page.
When the administrator user uses the new "Log in with..." button and performs a successful login operation via the IdP, a new Comet Server administrator account will be dynamically created for them on-demand. As with LDAP, you can specify which Comet Server permissions are granted to the newly generated account. This new account is marked as "Externally managed" within the "Admin Accounts" table, ensuring that valid IdP login is required to access this administrator account.
If your IdP enforces two-factor authentication (2FA), you can configure Comet Server to skip enforcing its own internal 2FA on the account, so that the user is not bothered twice.
You can request custom scopes, and enforce claim values against either custom scopes or standard OIDC scopes. This allows you to enforce that the only members of certain Microsoft, Google, or OIDC groups within your IdP are allowed to log in to the Comet Server.
This feature is available both for the top-level Comet Server administrator as well as individually for each tenant.
Price change notice for Comet Storage powered by Wasabi
Our Comet Storage service gives you the option to purchase Wasabi Cloud Storage directly from Comet, offering all-in-one billing and providing a more integrated experience. This month, we've passed along the latest price changes from Wasabi, to their new price of $6.99 / TB.
When backing up a Files and Folders-type Protected Item, one of the first steps is for Comet to enumerate all the selected files, in order to calculate their total size. The total size is used to enforce the "All Protected Items Quota" feature, as well as to properly determine the progress bar's expected upper bound. If you are running a headless device with no GUI to render the progress bar, and you are not using the "All Protected Items Quota" feature, then there's no remaining purpose for this scan phase, and so Comet will skip it to save time.
We heard mixed feedback about this - a discussion in our feature voting system uncovered some use-cases where the progress bar would still be desirable even on headless devices with no GUI. But also, there was competing feedback that spending time on file size measurement is still slow and undesirable even in some cases where the GUI was present.
In the latest version of Comet, we've come up with a new and better approach to this issue. If the "All Protected Items Quota" feature is used, then we require an accurate measurement up-front regardless. But if this feature is not used, then we can rapidly create an approximate progress bar size based on the previous backup job's size plus some small estimated buffer amount. This should provide a great speed improvement for GUI users, a reasonable progress bar for headless users, and at the same time, provide an accurate measurement for quota users. The reported size measurement will always be completely accurate after the backup job finishes.
For users using Comet on devices with low RAM, our software has long since offered the "Prefer temporary files instead of RAM (slower)" option for backup jobs, to toggle whether Comet stores the deduplication index either in-memory or on-disk in a temporary database file. Enabling this option can significantly reduce Comet's memory usage, allowing the backup job to complete on low-memory devices, at the expense of a longer backup job duration.
The latest version of Comet extends this option to also use a small in-memory bloom filter. This allows Comet to perform some of the deduplication operations in-memory with minimal overhead. This new combination technique can significantly improve the performance of this option for low-memory devices.
The deduplication index is needed for almost all operations involving the Storage Vault, not just backup operations. This month, we've also added an option to use temporary files instead of RAM during a restore, extending the possible use cases for Comet on low-memory devices.
The performance improvements this month are not limited to the Comet app itself. We've also significantly improved the account.cometbackup.com system: downloading the large Self-Hosted Comet Server installer is now implemented via an Amazon CloudFront cache, improving download speeds between 2-6x in our testing.
We have also been able to significantly improve the speed of creating new Comet-Hosted Comet Server instances. The creation time has been reduced from 60-90 seconds down to 10-15 seconds, owing primarily to some changes in the default generated DNS names.
Configuration change notice for PKCS11 codesigning
Comet supports Authenticode codesigning for Windows using either an on-disk file (PKCS #12), or a hardware security module such as a USB device (PKCS #11), or a cloud HSM on Azure Key Vault. With the file-based approach no longer being supported for new Authenticode certificates, we are seeing increased use of the alternative PKCS #11 and Azure Key Vault options, as partner Authenticode certificates come up for renewal.
If you are using a physical USB device for Authenticode codesigning, we have updated the available settings options to improve compatibility with a wider range of devices. The new settings screen should be clearer and easier to use, but you may be required to update your configured settings, as depicted:
If your Comet Server is running in a cloud VM, it's not feasible to plug in a USB hardware device for codesigning. We would recommend Azure Key Vault as an excellent cloud-based solution to this issue, but we've also recently tested the compatibility of the third-party Virtualhere software for remotely forwarding a physical USB device to another PC, and we can confirm this solution works for PKCS#11 codesigning when running Comet Server on a cloud VM.
When using the Comet Server web interface, the quick search bar (using the Alt + Q keyboard shortcut) could previously search through usernames, Account Name field values, Protected Item and Storage Vault names, settings pages, and more. In the latest version of Comet, we have extended the search capabilities to also find users by their email address.
You can also now enter the ID of a Protected Item, Storage Vault, or even a backup job, and the quick search bar will try to match it with the corresponding user or job. This is particularly helpful for troubleshooting some situations.
That's all for this month - the blog will return next month with more news about all the latest changes to Comet.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over June 2023.
There were six Comet software releases during June - two in our quarterly 23.5.x Thebe release series, plus four releases in the 23.6.x Voyager release series.
Comet has a comprehensive Policy system that allows you to apply rules to entire sets of customers at a time, such as restricting their ability to open the desktop app or restricting what features and settings can be used.
One of the existing Policy options allowed you to create a Protected Item that would be applied by default to all new devices that register in the account. However, it was limited to when the device was registered for the first time, it only defined the default settings, and after that the Protected Item could drift from what was configured in the policy. Having a default value that can then change is an important use case, but over time, we heard feedback that people would like these default Protected Items to remain tied to the policy version. In Comet 23.6.0, we made that possible.
You can choose whether the Policy Protected Item applies only for new devices, or if it remains linked to the policy after creation. This allows you to configure a Protected Item for a specific application, file-path, or integration target; apply it by default to all users of the Policy; and still be able to change those settings centrally for all users of that policy.
This was a highly popular request on our Feature Voting page. Thank you to all the partners who shared their support for this feature. If you haven't seen this system before, the Feature Voting system allows you to upvote existing requests, raise new requests, discuss features with other Comet partners, and get notified when a change happens in the product. We look at this system often to help guide the priority of new development.
As an administrator, when configuring permissions for other administrator accounts, additional options are now available.
These new options allow you restrict the capabilities of other administrators. This is intended for the use case of tenant administrators, or limited-permission administrators of a shared Comet Server. The use cases are varied but are often from the perspective of restricting the visibility of the Comet branding for white-label purposes.
You can hide the Server History and Server Info widgets from the homepage, including Comet's version numbers and update information. This simplifies the main screen for the limited-permission administrators who would be unable to act upon this information;
You can enforce that the target administrator is unable to create new Storage Vaults via Storage Templates or via custom storage; and
You can filter and restrict which cloud storage providers are available for these limited-permission administrators to use.
This work was also inspired from our Feature Voting system. Thanks to everyone for your patience as we combined several separate requests to refine down the essential requirements for this use case.
From Comet Backup 23.6.3 onwards, your customers must have macOS 10.13 High Sierra (2017) or later to continue running new versions of Comet Backup.
If your customers are still using macOS 10.11 El Capitan (2015) or 10.12 Sierra (2016), the last releases that can be run on these older versions of macOS are the Comet 23.5.x Thebe series, or 23.6.2 Voyager. These versions of macOS are no longer receiving security updates and we would recommend that all end-users upgrade if possible.
There is no change to Comet Server's minimum OS requirements as we do not offer Comet Server on macOS. It is safe to use the bulk deployment system in Comet Server to continue deploying software updates to all macOS customers - the affected users will skip the install and remain on their currently installed version of Comet Backup.
We last upgraded the minimum macOS version back in September 2022. We expect to be able to keep the 10.13 baseline for a longer period of time.
Comet has long supported a "Run when PC starts" option in the job schedule settings. This option is highly useful for laptops that are often offline or sleeping, where a regularly scheduled backup job is unlikely to occur at the odd moments when the device is online. By running a backup job immediately when the PC boots, the device is more likely to complete a successful backup job.
However, recent versions of Windows have expanded how widely "Fast Startup" is applied across different PCs, to the point that a Windows laptop will often not restart, but only simulate a restart via hibernation-related techniques. This has a negative effect on the "Run when PC starts" option as it was only triggered via a full restart, which is not the default behavior in current versions of Windows.
As a result, we have extended the "Run when PC starts" option to count waking up from sleep as "starting the PC". This is more in keeping with the behavior of current-generation laptop computers and it should make the option work as-expected in more cases. Compared to the old behavior, it is likely that additional backup jobs could be run unexpectedly, but our expectation is that this is not a bad thing, and the result will be more reliable overall.
The 23.5 Thebe release last month included a major redesign of the Comet Server web interface. We've continued to build upon this throughout June in the 23.6.x Voyager series with many enhancements for server operators.
One area of focus was on ease-of-use. When configuring a Protected Item for Application Aware Writer, Hyper-V, or Microsoft Exchange, you can now use the top-right button to remotely browse the customer's PC to select items for backup. Compared to finding individual virtual machine IDs, this is a significant improvement to the ease-of-use of this feature.
Additionally, when saving changes in the Settings page, the page will now intelligently wait for settings to be applied, instead of leaving you at a web browser refresh screen. This makes for a much more intuitive and user-friendly experience. The Comet Server web interface will now also warn you more clearly about invalid bucket names for cloud storage.
There was also a particular focus on performance enhancements for the Comet Server web interface. Loading the web interface should be snappier and with fewer loading spinners, as the necessary information is able to be loaded in parallel, as well as reused from previous lookups. The Quick Search area is now faster and has a new design style. When remotely controlling a connected device to start a restore job, the restore job will reuse information known by the web interface instead of recalculating it, resulting in a restore job that starts faster and finishes faster.
A third area of focus throughout the month was on fixing known issues that have been reported. Thank you to all partners who reported issues - the latest 23.5.x and 23.6.x updates should have addressed all the web interface issues raised to this point.
Following on from last year's successful event, we hosted another week-long meetup with Comet team members from all over the country. With a strong focus on technical talks, networking, and a special coffee-tasting event, everyone had a great week! I'd like to extend a special thank you to our guest speakers!
That's all for this month - the blog will return next month with more news about all the latest changes to Comet.
"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over May 2023.
There were four Comet software releases during May, all in the weekly 23.3.x Voyager release series. The work this month included a mix of core application development as well as projects in the surrounding ecosystem:
Earlier in May, Mason gave a presentation of the history and overview of backup technologies, starting with simple file copies, through chain-based approaches, and culminating in the Content-Defined Chunking technology used by Comet.
Docker is a popular system for running applications in "containers". In the same way that freight shipping containers have a consistent size and shape, in order to fit within a larger shipping ecosystem, Docker containers also have some consistent behaviors in order to fit within a larger application runtime ecosystem: a "containerized" application has a standard way to start, stop, and log messages; how their storage and configuration is managed, how ingress and TLS termination works, and how sandboxing and permission restrictions are applied.
By offering Comet Server as a Docker (OCI) container, it should be easy for existing Docker users to get started with the Self-Hosted app and to manage it going forward. It also enables new functionality such as running multiple self-hosted Comet Servers on the same PC or VM. This does not replace the existing Self-Hosted installers, which will continue to be available.
There has been a long-standing open feature request for an official Docker container. Thank you to everyone who voted and commented on this feature, and a special thanks to "Hobadee" for providing an unofficial solution to the community so far.
When you visit the the Comet Server web interface, your web browser loads and runs a "Single Page App" (SPA) locally inside your browser. This webapp speaks to the Comet Server backend solely using our public API. This API broadly covers every action you take within the Comet Server web interface, which means if you can do an action in the web interface, then it can be automated by a developer.
During May, we put the finishing touches on the latest addition to this group - a new public .NET SDK. It's available as CometBackup.CometAPI.SDK on NuGet and is developed in C# for use with any compliant .NET Standard 2.0 runtime, including .NET Framework 4.6, .NET Core 2.0, .NET 5, or any later versions.
Getting started with the SDK should be straightforward:
using (var client = new CometAPI("http://127.0.0.1:8060", "admin-user", "admin-password")) { var users = client.AdminListUsers(); foreach (var user in users) { Console.WriteLine(user); } }
Every API method is also available as an Async() version to allow for concurrency.
Our SDK version numbers generally adhere to Semantic Versioning and any enthusiastic readers may have noticed this SDK uses a "version zero" version number at present. We don't necessarily expect to make breaking changes, but we are actively requesting feedback about this new SDK from existing C# users to ensure we've created something idiomatic. We'd love to hear any feedback about the SDK design or any bug reports before we commit to v1 compatibility, so if you are a C# developer then please reach out by ticket or on GitHub.
To help our partners meet their compliance obligations, we've added advanced audit logging capabilities into Comet Server. Almost every action - including login events for administrators and users, restoring data, changes to user and server settings, and more - can now be logged server-side into a secure file.
This work was based on expanding Comet Server's live event streaming features to cover a much more comprehensive set of data. If you are making use of the existing Webhook feature or the Websocket API feature, when upgrading to Comet Server 23.3.7 or later, you may start receiving a larger amount of data. You can control this by declaring a restriction on the streamable event types you want to receive.
In current versions of Comet, this feature must be configured by setting the AuditFileOptions property inside the cometd.cfg file. We expect to make it available via the Comet Server web interface soon!
As we approached the quarterly release, we took a special focus on quality improvements to give us a solid foundation.
The fresh new design for the Comet Server web interface had its debut in 23.3.3, and since then throughout the 23.3.x Voyager release track, we have spent additional time and effort to ensure quality. In the latest versions of Comet Server you will see fewer cosmetic issues as this work has stabilised nicely.
We've also made significant performance improvements to key areas of the client application. Performing granular restore of single files from inside a Disk Image or Hyper-V backup may now be 3-8x faster while using 40% less memory. Comet also now uses less memory for loading files from S3-compatible Storage destinations like Wasabi, and when loading very large files from a Storage Vault.
The Office 365 Protected Item type in Comet has been a continued development focus this quarter, including a partial rewrite. Comet is now better able to handle incremental changes to mailboxes; the reported error messages are more detailed; and Office 365 backup jobs taken with the latest version of Comet should be more resilient against network errors and rate-limiting from the Microsoft Graph API. We would encourage all users to raise a support ticket if you experience any issues with Office 365 backups as we continue to improve the sync capabilities.
On the 1st of June, we put the finishing touches on our latest quarterly release, Comet 23.5.0 "Thebe".
Thebe is the latest entry in our quarterly rollup series. It branches off from our main rolling Voyager development into a fixed target for you to qualify and build your service offering upon.
Like Comet's previous recent quarterly software releases, "Thebe" is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is a very small moon with only 0.004x Earth's gravity, meaning it is less likely to shape itself into a sphere - in fact a large impact crater covers around 40% of its surface area:
It was discovered by the Voyager 1 and 2 spacecraft in 1979, but owing to its small size and position, almost nothing more could be discovered until the next spacecraft Galileo visited two decades later. Scientists believe it contains frozen water.
As always for a new quarterly release, there are two changelogs for 23.5.0 "Thebe" depending on whether you are coming from the previous quarterly release or the previous Voyager release. You can see the full details in the release notes.
That's all for this month! Thanks for reading - there are some more great features currently under development that we're excited to be able to share with you soon. As always, please follow @CometBackup on Twitter and you can always contact us if you have any questions.
