"What's New?" is a series of blog posts covering recent changes to Comet in more detail. This article covers the latest changes in Comet Voyager over May 2023.
There were four Comet software releases during May, all in the weekly 23.3.x Voyager release series. The work this month included a mix of core application development as well as projects in the surrounding ecosystem:
Learn about Chunking
Earlier in May, Mason gave a presentation of the history and overview of backup technologies, starting with simple file copies, through chain-based approaches, and culminating in the Content-Defined Chunking technology used by Comet.
Comet Server in Docker
The Self-Hosted Comet Server is now available as a Docker container:
Docker is a popular system for running applications in "containers". In the same way that freight shipping containers have a consistent size and shape, in order to fit within a larger shipping ecosystem, Docker containers also have some consistent behaviors in order to fit within a larger application runtime ecosystem: a "containerized" application has a standard way to start, stop, and log messages; how their storage and configuration is managed, how ingress and TLS termination works, and how sandboxing and permission restrictions are applied.
By offering Comet Server as a Docker (OCI) container, it should be easy for existing Docker users to get started with the Self-Hosted app and to manage it going forward. It also enables new functionality such as running multiple self-hosted Comet Servers on the same PC or VM. This does not replace the existing Self-Hosted installers, which will continue to be available.
There has been a long-standing open feature request for an official Docker container. Thank you to everyone who voted and commented on this feature, and a special thanks to "Hobadee" for providing an unofficial solution to the community so far.
C# .NET SDK and expanded API documentation
When you visit the the Comet Server web interface, your web browser loads and runs a "Single Page App" (SPA) locally inside your browser. This webapp speaks to the Comet Server backend solely using our public API. This API broadly covers every action you take within the Comet Server web interface, which means if you can do an action in the web interface, then it can be automated by a developer.
The Comet Server API is comprehensively documented, and during May we added many more comments and explainer notes to the constants, endpoints, and type definitions. Developers using the existing PHP SDK, Javascript / Typescript SDK, Ruby SDK, and Go (Golang) SDK will see these new comments appear in their IDEs during autocomplete.
During May, we put the finishing touches on the latest addition to this group - a new public .NET SDK. It's available as CometBackup.CometAPI.SDK on NuGet and is developed in C# for use with any compliant .NET Standard 2.0 runtime, including .NET Framework 4.6, .NET Core 2.0, .NET 5, or any later versions.
Getting started with the SDK should be straightforward:
using (var client = new CometAPI("http://127.0.0.1:8060", "admin-user", "admin-password")) {
var users = client.AdminListUsers();
foreach (var user in users) {
Console.WriteLine(user);
}
}
Every API method is also available as an Async() version to allow for concurrency.
Our SDK version numbers generally adhere to Semantic Versioning and any enthusiastic readers may have noticed this SDK uses a "version zero" version number at present. We don't necessarily expect to make breaking changes, but we are actively requesting feedback about this new SDK from existing C# users to ensure we've created something idiomatic. We'd love to hear any feedback about the SDK design or any bug reports before we commit to v1 compatibility, so if you are a C# developer then please reach out by ticket or on GitHub.
Audit logging
To help our partners meet their compliance obligations, we've added advanced audit logging capabilities into Comet Server. Almost every action - including login events for administrators and users, restoring data, changes to user and server settings, and more - can now be logged server-side into a secure file.
This work was based on expanding Comet Server's live event streaming features to cover a much more comprehensive set of data. If you are making use of the existing Webhook feature or the Websocket API feature, when upgrading to Comet Server 23.3.7 or later, you may start receiving a larger amount of data. You can control this by declaring a restriction on the streamable event types you want to receive.
In current versions of Comet, this feature must be configured by setting the AuditFileOptions property inside the cometd.cfg file. We expect to make it available via the Comet Server web interface soon!
Quality, performance, and memory enhancements
As we approached the quarterly release, we took a special focus on quality improvements to give us a solid foundation.
The fresh new design for the Comet Server web interface had its debut in 23.3.3, and since then throughout the 23.3.x Voyager release track, we have spent additional time and effort to ensure quality. In the latest versions of Comet Server you will see fewer cosmetic issues as this work has stabilised nicely.
We've also made significant performance improvements to key areas of the client application. Performing granular restore of single files from inside a Disk Image or Hyper-V backup may now be 3-8x faster while using 40% less memory. Comet also now uses less memory for loading files from S3-compatible Storage destinations like Wasabi, and when loading very large files from a Storage Vault.
The Office 365 Protected Item type in Comet has been a continued development focus this quarter, including a partial rewrite. Comet is now better able to handle incremental changes to mailboxes; the reported error messages are more detailed; and Office 365 backup jobs taken with the latest version of Comet should be more resilient against network errors and rate-limiting from the Microsoft Graph API. We would encourage all users to raise a support ticket if you experience any issues with Office 365 backups as we continue to improve the sync capabilities.
Comet 23.5 "Thebe"
On the 1st of June, we put the finishing touches on our latest quarterly release, Comet 23.5.0 "Thebe".
Thebe is the latest entry in our quarterly rollup series. It branches off from our main rolling Voyager development into a fixed target for you to qualify and build your service offering upon.
Like Comet's previous recent quarterly software releases, "Thebe" is named after a moon of Jupiter, which in turn takes its name from an ancient Greek mythological figure. It is a very small moon with only 0.004x Earth's gravity, meaning it is less likely to shape itself into a sphere - in fact a large impact crater covers around 40% of its surface area:
It was discovered by the Voyager 1 and 2 spacecraft in 1979, but owing to its small size and position, almost nothing more could be discovered until the next spacecraft Galileo visited two decades later. Scientists believe it contains frozen water.
As always for a new quarterly release, there are two changelogs for 23.5.0 "Thebe" depending on whether you are coming from the previous quarterly release or the previous Voyager release. You can see the full details in the release notes.
That's all for this month! Thanks for reading - there are some more great features currently under development that we're excited to be able to share with you soon. As always, please follow @CometBackup on Twitter and you can always contact us if you have any questions.