Other APIs

Introduction

The main Comet Server API is described in detail in the API Guide. As well as making API requests to your Comet Server instances as described there, additional APIs are available for different purposes.

Comet Server web interface postMessage API

An API is also available for the Comet Server web interface frontend.

This additional API allows you to control some parts of the web interface by making postMessage calls in Javascript. Messages will be sent in either direction, as Javascript objects, in the general form {msg: "message_type", other_parameters: .... }.

To use this API, create a Comet Server web interface window (e.g. by using window.open() or <iframe>), and add a message event listener to wait for the AppLoaded API message described below.

AppLoaded

Notification sent to the opener window when the Comet Server web interface is ready to receive other postMessage requests.

Message format: Object
Direction: Response (Sent by Comet Server web interface to caller)

Object parameters:

Parameter name	Value
`msg`	`app_loaded`

SessionLogin

Perform a single sign-on (SSO) login to the Comet Server web interface, as an administrator account.

Message format: Object
Direction: Request (Sent by caller to Comet Server web interface)

Object parameters:

Parameter name	Value
`msg`	`session_login`
`username`	Admin username
`sessionkey`	Pre-generated admin session key from the `AdminAccountSessionStart` or `HybridSessionStart` server API

UserSessionLogin

Perform a single sign-on (SSO) login to the Comet Server web interface, as an end-user account.

Message format: Object
Direction: Request (Sent by caller to Comet Server web interface)

Object parameters:

Parameter name	Value
`msg`	`user_session_login`
`username`	Customer username
`sessionkey`	Pre-generated end-user session key from the `UserWebSessionStart` or `AdminAccountSessionStartAsUser` or `HybridSessionStart` server API

Comet Server live event streaming

Additional Comet Server APIs are available to receive live, realtime notifications from the Comet Server of new jobs, completed jobs, and user profile configuration changes.

Because of the reversed direction of information, these APIs do not entirely fit the general documented pattern and are documented separately here.

There are two ways to configure these live event notifications: either as a webhook or as a websocket.

Webhooks

This feature is available in Comet Server 20.6.1 and later.

You can configure a URL to receive events. The Comet Server will submit events by HTTP POST and must be able to reach the configured URL from its network position. The webhook's body is an event structure in JSON as described below in the "Message types" section.

An x-Comet-Tracing-Id header is present in the webhook request. Your webhook endpoint should respond with an HTTP 200 OK code to all POST events. If your webhook endpoint fails to do so, the failure will be logged in the Comet Server log, along with its corresponding x-Comet-Tracing-Id header value; and Comet Server will retry the request a finite number of times.

Custom headers to be included in the webhook POST request can be configured using the CustomHeaders field of WebhookOption. This field can be set using the API endpoints as described below, or directly in the cometd.cfg file:

    ...
    "WebhookOptions": {
        "ExampleOrganization": {
            "URL": "http://example.com/webhooks",
            "WhiteListedEventTypes": [],
            "Level": "full",
            "CustomHeaders": {
                "Authorization": "Bearer f1d2d2f9",
                "X-Custom-Header": "Custom header value"
            }
        }
    },
    ...

Webhook messages are sent in parallel and may be unordered when received. When a webhook experiences an error and retries, the submission is not ordered.

You can use the webhook feature as follows:

Use the AdminMetaWebhookOptionsGet API to retrieve the current webhook configuration
Modify the data structure to add your own additional webhook endpoint in WebhookOption format.
- If the WhiteListedEventTypes array is empty, all event types are sent. Otherwise, only the specified event types are sent.
Use the AdminMetaWebhookOptionsSet API to replace the server's current webhook configuration with the updated configuration.
You should immediately receive the initial SEVT_META_HELLO webhook event to your configured endpoint.
- A SEVT_META_HELLO message will be sent every time the webhook queue is flushed. For instance, it will occur if the Comet Server is restarted.

The webhook configuration will be persisted in the cometd.cfg file. You may also choose to make changes to this file directly.

Websocket

This feature is available in any version of Comet Server.

Instead of using a Webhook, you can make an outbound websocket connection. This method does not require your application to have a public URL. Messages will always be delivered sequentially in-order and will be delivered for as long as the websocket is held open.

If the websocket client is unable to drain messages sufficiently quickly from the TCP stream, event messages will be queued for publication up to an implementation-defined limit. After this limit, the websocket connection would be dropped by the Comet Server; in this case, a client should reconnect and perform a remedial sync using the standard Comet Server APIs to catch up on any missed status changes.

You can use the websocket feature as follows:

Make a GET request to /api/v1/events/stream.
- Set the Origin header to a URL of your choice (e.g. https://cometbackup.com). No filtering is yet done on this value, but that may change at a later time.
- Optionally, limit which events are received by adding the query param allowList with a comma-delimited list of streamable event numbers to filter on (e.g. ?allowList=4100,4101).
The server will perform an HTTP Upgrade to a WebSocket connection.
The client must emit five text message frames in order, containing Username, AuthType, Password, SessionKey, and TOTP parameter values respectively. An empty-string text message frame can be used to skip providing a value.
If the authentication failed, the server will emit a text message frame containing the string 403 Unauth, and then drop the connection. If the authentication succeeded, the server will emit a text message frame containing the string 200 OK.
The server will then send a text message frame for each event, containing an event structure in JSON as described below in the "Message types" section.

Message types

For both webhook and websocket sources, you will receive a JSON-encoded StreamableEvent structure. The Type field indicates the type of the event and will be one of the SEVT_ constant values.

The first message you receive will have the SEVT_META_HELLO type. This indicates that a new live-event-streaming session has been created.

Two-Factor Authentication for the Comet Server API

We would recommend against using two-factor authentication for the Comet API. It is possible to do this, but (A) it's not really meaningful for an automated process to have two factors of authentication; and also (B) it's not currently supported by our PHP SDK on GitHub.

We would recommend creating a new Comet Server admin account just for API purposes with a long random password.

Advanced usage

If it's really seriously needed, it is possible to make any of the API requests using these authentication types: any Comet Server API request needs to be authenticated, and you can authenticate by setting the AuthType parameter to any one of Password, SessionKey, PasswordTOTP, or PasswordU2F; the latter two (TOTP / U2F) being the two currently supported methods of two-factor authentication in Comet Server.

We would suggest only using PasswordTOTP and PasswordU2F only with the AdminAccountSessionStart API, that will give you a single SessionKey token to use with other API calls. Performing the entire TOTP / U2F handshake on every API call is probably an unreasonable amount of overhead. To authenticate using SessionKey mode, the API request should contain a valid SessionKey parameter that you get from the AdminAccountSessionStart API back in the SessionKeyRegeneratedResponse response structure.

To authenticate using Password mode, the API request should contain a valid Password parameter (of the admin account's password), as per the examples in https://cometbackup.com/docs/api#quick-start-examples . Then, if a password-only is insufficient for the account, the API will respond with a HTTP 449 status ("Retry With"), and a X-Comet-TOTP-Requested: 1 header or a X-Comet-U2F-Challenge: ... header. You should use this information to determine if a TOTP or U2F login is suitable.

To authenticate using PasswordTOTP mode, the API request should contain both a valid Password parameter (of the admin account's password) and a valid TOTP parameter. A TOTP code is a 6-digit number that changes every 30 seconds. Both the client and server calculate it from a shared secret. You should find the current secret by decoding the QR code for the user, and then use a library in your programming language to generate the current 6-digit value for the current timestamp.

To authenticate using PasswordU2F mode, the API request should contain a valid Password parameter (of the admin account's password) and a valid U2FSign POST parameter. You can use the information from the previous X-Comet-U2F-Challenge response header, to generate a U2F signature with your U2F hardware device, and fill in the U2FSign POST parameter as a U2FSignResponse struct in JSON encoding.

Custom Remote Bucket

The "Custom Remote Bucket" is a possible Storage Template option in the Comet Server configuration settings. The Storage Template system allows administrators and authorized users to easily provision new, private per-user Storage Vaults inside a user account that are attached to a real storage location created by the storage provider.

As well as the built-in provider types for allocating Comet Server Storage Role, Backblaze B2, or Wasabi buckets, you can use the "IAM-compatible" or "Custom Remote Bucket" provider type to implement a custom storage location.

This is of interest for MSPs that want to bring an advanced direct-to-cloud storage option, or storage providers that want to develop an integration that easily plugs into Comet Servers to provide direct-to-cloud storage. The resulting Storage Vault can use any storage protocol supported by Comet, such as S3-compatible, Local Path, or SFTP.

To use this feature, you should provide a URL endpoint.

Comet Server will make an HTTP POST request to your configured endpoint
Your endpoint should respond with a HTTP 2xx status code.
Your endpoint should respond with a DestinationLocation structure in JSON format
Comet Server will add the resulting DestinationLocation structure into the requesting user's profile as a new Storage Vault.

Sample endpoint implementation (Python)

#!/usr/bin/env python3
#
# This is an example script to show how Comet Server can request custom
# buckets from a web server script
#
# Usage:
#   1. python3 ./sample-custom-remote-bucket.py
#   2. Set up a Custom Remote Bucket in Comet Server using the URL: http://127.0.0.1:8000/request-bucket
# 
# Configuration:

BIND_IP   = "127.0.0.1" # could be any IP; leave blank to bind to all IP addresses
BIND_PORT = 8000

#

import http.server
import socketserver

# CustomRemoteBucketHandler is the HTTP server that is able to respond to
# Comet Server's requests when they happen.
class CustomRemoteBucketHandler(http.server.BaseHTTPRequestHandler):

    def __init__(self, request, client_addr, server):
        """Boilerplate for a custom HTTP service"""
        super().__init__(request, client_addr, server)

    def do_GET(self):
        """Handle GET requests (regular page loads)"""
        if self.path == "/":
            self.do_homepage()
        else:
            self.send_error(404)

    def do_POST(self):
        """Handle POST requests (form submissions)"""
        if self.path == "/request-bucket":
            self.do_requestBucket()
        else:
            self.send_error(404)

    def do_homepage(self):
        """Render the home page for users who visit this server directly"""
        self.send_response(200)
        self.send_header('Content-Type', 'text/html;charset=UTF-8')
        self.end_headers()
        self.wfile.write("""<!DOCTYPE html>
<p style="color:red;font-weight:bold;">
This is a test server for the custom remote bucket feature! Not to be used directly!
</p>
        """.encode())

    def do_requestBucket(self):
        """Accept a Custom Remote Bucket request from Comet Server"""
        self.send_response(200)
        self.send_header('Content-Type', 'application/json')
        self.end_headers()

        # Talk to your storage platform to allocate new access credentials
        # TODO

        # Respond with a Comet DestinationLocation object in JSON format
        # The DestinationType should be a DESTINATIONTYPE_ number from https://docs.cometbackup.com/latest/api/api-constants#destinationtype
        # Use any of the keys from https://docs.cometbackup.com/latest/api/api-data-structures/#api-struct-DestinationLocation
        self.wfile.write("""{
    "DestinationType": 1000,
    "S3Server": "minio.example.com",
    "S3UsesTLS": false,
    "S3AccessKey": "",
    "S3SecretKey": "",
    "S3BucketName": "",
    "S3Subdir": "",
    "S3UsesV2Signing": false
}""".encode())

# Application starts up here if it is run directly
if __name__ == "__main__":
    with socketserver.TCPServer((BIND_IP, BIND_PORT), CustomRemoteBucketHandler) as httpd:
        print("Serving at %s:%s..." % (BIND_IP, BIND_PORT))
        httpd.serve_forever()

Sample endpoint implementation (PHP)

<?php

// This is an example script to show how Comet Server can request custom buckets from a web server script.
// Save this file as request-bucket.php

if ($_SERVER['REQUEST_METHOD'] !== "POST") {
    header("HTTP/1.1 400 Invalid Request");
    die('Expected POST');
}

// Talk to your storage platform to allocate new access credentials
// TODO

$result = [
    DestinationType => 1000,
    S3Server => "minio.example.com",
    S3UsesTLS => false,
    S3AccessKey => "",
    S3SecretKey => "",
    S3BucketName => "",
    S3Subdir => "",
    S3UsesV2Signing => false
];

header('Content-Type: application/json');
echo json_encode($result);

Introduction​

Comet Server web interface postMessage API​

AppLoaded​

SessionLogin​

UserSessionLogin​

Comet Server live event streaming​

Webhooks​

Websocket​

Message types​

Two-Factor Authentication for the Comet Server API​

Advanced usage​

Custom Remote Bucket​

Sample endpoint implementation (Python)​

Sample endpoint implementation (PHP)​

Introduction

Comet Server web interface postMessage API

AppLoaded

SessionLogin

UserSessionLogin

Comet Server live event streaming

Webhooks

Websocket

Message types

Two-Factor Authentication for the Comet Server API

Advanced usage

Custom Remote Bucket

Sample endpoint implementation (Python)

Sample endpoint implementation (PHP)