Python API Event Handler#

Introduction#

SFTPPlus allows developers to write custom event handlers using the Python programming language.

The handlers are executed in separate independent processes / CPU cores, without shared memory. This is why the handle() method of the extension needs to be a @staticmethod and always receives the configuration.

The handler is initialized multiple time. One instance is created in the main process and extra instances are created for each CPU.

A single extension instance can have the onStart(parent) / onStop() method called multiple times during its lifetime. onStart(parent) and onStop() methods are only called for the instance running in the main process.

Most custom extension are created to only handle a few event IDs. You can set the list of event IDs that can be handled by your extension via the TARGET_EVENTS instance member. The TARGET_EVENTS is accessed after onStart(parent).

The code for the event handler needs to be placed in a Python file (module) inside the extension/ folder from the SFTPPlus installation folder.

You can find an extensive example inside the extension/demo_event_handler.py folder of the default SFTPPlus installation.

Below is an example extension code that is also used to document the available API and functionalities.

  1# Place this file in `extension/demo_event_handler.py`
  2# inside SFTPPlus' installation folder.
  3#
  4# For the event handler set
  5# type = extension
  6# entry_point = python:demo_event_handler.DemoEventHandler
  7#
  8import json
  9
 10
 11class DemoEventHandler(object):
 12    """
 13    An event handler which just emits another event with details of the
 14    received events.
 15
 16    Events handler API extensions can emit the events:
 17
 18    * 20174 - Emitted by SFTPPlus for critical errors.
 19              Should not be explicitly emitted by the extension.
 20    * 20000 - For debug messages.
 21    * 20200 - For normal messages.
 22
 23    This is also used as a documentation for the API and is included in
 24    the automated regression testing process.
 25
 26    TARGET_EVENTS defined the events for which this handler will be triggered.
 27    """
 28
 29    TARGET_EVENTS = [
 30        '99',
 31        '100',
 32        '102',  # Custom event ID.
 33        '20156',  # Component started.
 34        '20157',  # Component stopped.
 35    ]
 36
 37    def __init__(self):
 38        """
 39        Called when the event handler is initialized in each worker and
 40        in the main process.
 41        """
 42        self._configuration = None
 43
 44    def onStart(self, parent):
 45        """
 46        Called in the main process when the event handler starts.
 47
 48        `parent.configuration` is the Unicode string defined in the
 49        extension configuration option.
 50
 51        Any exception raised here will stop the event handler from
 52        starting.
 53        """
 54        self._parent = parent
 55        self._configuration = parent.configuration
 56        # Extensions have access to the trusted certificates.
 57        # This returns the PEM content of all concatenated certificates.
 58        self._trusted_certificates = parent.getVaultItemContent(
 59            'LETS-ENCRYPT-CA'
 60        )
 61
 62    def onStop(self):
 63        """
 64        Called in the main process when the event handler stops.
 65        """
 66        self._configuration = None
 67
 68    def getConfiguration(self, event):
 69        """
 70        Called in the main process before dispatching the event to
 71        the worker process.
 72
 73        It can be used as validation for the event,
 74        before handling the event in a separate CPU core.
 75
 76        Return the configuration for the event as Unicode.
 77
 78        Return `None` when the event handling should be skipped and the
 79        `handle` function will no longer be called for this emitted event.
 80
 81        As advanced usage, it can return a `deferred` which will delay
 82        the execution of the event, without keeping a worker process busy.
 83        This mechanism can also be used for implementing a wait condition
 84        based on which the event is handled or not.
 85        """
 86        if event.id == '1234' or event.account.name == 'fail-user':
 87            # Any exception raised here will stop the handling of this
 88            # specific event instance by the extension.
 89            raise RuntimeError('Rejected event.')
 90
 91        if event.account.name == 'skip-user':
 92            # When `None` is returned the handling is skipped and the
 93            # `handle` function will not be called.
 94            return None
 95
 96        if event.account.name == 'skip-emit':
 97            # When skipping, you can trigger emitting an event with custom
 98            # message and attached data.
 99            return None, {'message': 'Handling skipped.', 'extra': 'skip-emit'}
100
101        if event.account.name == 'error-user':
102            # You can skip and emit an event ID dedicated to errors.
103            return None, {
104                'event_id': '20202',
105                'message': 'Can be a generic description for the error case.',
106                'details': (
107                    'Can contain details specific to this error. '
108                    'Example a path to a file.'
109                ),
110                'tb': 'Can include option traceback info as text.',
111            }
112
113        if event.account.name == 'delay-user':
114            # For username `delay-user` we delay processing of the event
115            # for 0.5 seconds.
116            return self._parent.delay(
117                0.5, lambda: ['delayed-configuration', []]
118            )
119
120        # Events can be triggered as part of the event handling configuration.
121        # You can have one for more events.
122        # Event can have custom ID or use default ID.
123        events = [
124            {'event_id': '20201', 'message': 'Handling started.'},
125            {'message': 'Default ID is 20200 as informational.'},
126        ]
127        # There is also the option of returning just the configuration,
128        # without any extra events.
129        return self._configuration, events
130
131    @staticmethod
132    def handle(event, configuration):
133        """
134        Called in a separate process when it should handle the event.
135
136        This is a static function that must work even when
137        onStart and onStop were not called.
138
139        `configuration` is the Unicode value returned by
140        getConfiguration(event).
141
142        If an exception is raised the processing is stopped for this event.
143        Future events will continue to be processed.
144        """
145        # Output will overlap with the output from other events as each
146        # event is handled in a separate thread.
147
148        if event.account.name == 'inactive-user':
149            # The extension can return a text that is logged as an event.
150            return 'Extension is not active from this user.'
151
152        if event.account.name == 'test@proatria.onmicrosoft.com':
153            # The extension has access to the Entra ID OAuth2 token.
154            return 'Entra ID token: {}'.format(event.account.token)
155
156        if event.account.name == 'ignored-user':
157            # Don't handle events from a certain username.
158            # The extension can return without any value, and no
159            # event is emitted.
160            return
161
162        # Here we get the full event, and then we sample a few fields.
163        message = (
164            'Received new event for DemoEventHandler\n'
165            '{event_json}\n'
166            '-----------------\n'
167            'configuration: {configuration}\n'
168            '-----------------\n'
169            'id: {event.id}\n'
170            'account: {event.account.name}\n'
171            'at: {event.timestamp.timestamp:f}\n'
172            'from: {event.component.name}\n'
173            'data: {event_data_json}\n'
174            '---\n'
175        )
176        output = message.format(
177            event=event,
178            event_json=json.dumps(event, indent=2),
179            event_data_json=json.dumps(event.data, indent=2),
180            configuration=configuration,
181        )
182
183        # Inform the handler to emit several events at the end.
184        # For a single event, it is recommended to pass only a dictionary.
185        return [
186            # The "message" attribute is required.
187            {'message': 'A simple message.'},
188            # Other attributes are allowed.
189            {'message': 'state', 'value': 'OK'},
190            # Explicit Event ID is also supported
191            # For this case the attributes should match the attributes
192            # required by the requested Event ID.
193            # Event '20201' requires the `message` attribute.
194            # Any extra attributes are allowed.
195            {'event_id': '20201', 'message': output, 'extra': configuration},
196        ]

This event handler can be configured as:

[event-handlers/56df1d0a-78c6-11e9-a2ff-137be4dbb9a8]
enabled = yes
type = extension
name = python-extension

entry_point = python:extensions.demo_event_handler.DemoEventHandler

configuration = some-free-text-configuration

Execution queue#

SFTPPlus will only handle in parallel N events, where N is based on the number of CPUs available to the OS.

All the other events required to be handled by the extensions are placed into a queue.

The extension is called to handle the event only when there are free CPUs.

To prevent misconfiguration, there is a hard limit of 10 minutes for how long an event can stay in the queue and for processing the event.

Vault items access#

Extensions can load PEM certificates that are stored as trusted-certificates vault items.

The parent.getVaultItemContent(vault_item_uuid) is used to retrieve the concatenated PEM content of the certificates.

The method will validate that the vault item has content.

It will also return expired certificates.

Event data members#

Event values are captured when the event is generated. Changes to an account or component after that point do not change the event's public values. The event and its nested dictionaries and lists passed to getConfiguration(event) in the main process are read-only. The handle(event, configuration) method receives its own mutable copy, decoded from JSON in the worker process.

Create working copies of the fields that your extension needs to modify. For example, to add a value to the event data for your own processing:

data = dict(event.data)
data['custom_value'] = 'example'

This is a shallow copy. Copy any nested dictionary or list before modifying it as well. Changing a working copy does not change the original event or the values seen by another event handler. Changes to the event in handle() also stay within that worker invocation. Credential access through event.account.token remains available in getConfiguration().

The event object received in the handler has the following structure.

The overall structure of the event object is presented below.

The following variables (case-insensitive) are provided as context data containing information about the event being triggered:

  • id

  • uuid

  • message

  • msg

  • loglevel

  • peer.ip

  • peer.port

  • account.name

  • account.email

  • account.peer.address

  • account.peer.port

  • account.peer.protocol

  • account.peer.family

  • account.peer.country_code

  • account.peer.asn_number

  • account.peer.asn_name

  • account.uuid

  • component.name

  • component.type

  • component.uuid

  • server.name

  • server.uuid

  • server.instance_name

  • data.DATA_MEMBER_NAME

  • data_json

  • tb

  • timestamp.cwa_14051

  • timestamp.iso_8601

  • timestamp.iso_8601_fractional

  • timestamp.iso_8601_local

  • timestamp.iso_8601_basic

  • timestamp.iso_8601_compact

  • timestamp.timestamp

  • timestamp.year_decimal

  • timestamp.year_no_century

  • timestamp.month_decimal

  • timestamp.month_padded

  • timestamp.day_padded

  • timestamp.day_decimal

  • timestamp.day_of_year_padded

  • timestamp.hour_padded_24

  • timestamp.hour_decimal_24

  • timestamp.hour_padded_12

  • timestamp.hour_decimal_12

  • timestamp.hour_am_pm

  • timestamp.minute_padded

  • timestamp.minute_decimal

  • timestamp.second_padded

  • timestamp.second_decimal

msg is an alias for message. loglevel is the event severity: trace, info, warn, error, or critical. It is specified in the event definition and follows the group semantics described in Events, event handlers and the audit trail. The peer field contains only ip and port, copied from account.peer.address and account.peer.port. Events generated by the server process use IP address 0.0.0.0 and port 0. loglevel is included in default event JSON serialization. The msg and peer aliases are excluded. Use {msg}, {loglevel}, {peer.ip}, and {peer.port} in event handler templates.

data_json contains an indented JSON serialization of the event data, excluding tracebacks. It is generated on first use and cached for subsequent requests. tb contains the traceback, or an empty string when there is no traceback. Both are separate from data and available in all event handler templates. Account information is available through account, rather than data.avatar. Tracebacks can contain sensitive information and are omitted from default email, log, and webhook output. To include a traceback, explicitly add {tb} to the template.

DATA_MEMBER_NAME can be any of the supported data attributes specific to each event ID. You can discover the available data attributes as part of the Events reference documentation or the Web Manager Activity log.

Peer information includes an estimated country and the autonomous system (AS) that announces the IP address. country_code is a two-letter country code, asn_number is the AS identifier represented as a string with the ASN prefix (for example, ASN15169), and asn_name is the AS organization name. These values are empty strings for private addresses and addresses without matching information. Country and AS information is resolved locally using a database bundled with SFTPPlus and refreshed for each release. IP geolocation is an estimate and does not identify a person's location. For details, see IP geolocation database.

The members of data are specific to each event. See Events page for more details regarding the data available for each event.

Many events have data.path and data.real_path, together with the associated data.file_name, data.directory_name, data.real_file_name, and data.real_directory_name.

Below is the description for the main members of the event object.

name:

id

type:

string

optional:

No

description:

ID of this event. See Events page for the list of all available events.

name:

message

type:

string

optional:

No

description:

A human readable description of this event.

The timestamp contains the following attributes:

name:

timestamp

type:

string

optional:

No

description:

Date and time at which this event was created, as Unix timestamp with milliseconds.

name:

cwa_14051

type:

string

optional:

No

description:

Date and time in CWA 14051 at which this event was emitted.

The account contains the following attributes:

name:

uuid

type:

string

optional:

No

description:

UUID of the account emitting this event. In case no account is associated with the event, this will be the special process account. In case the associated account is not yet authenticated this will be the special peer account.

name:

name

type:

string

optional:

No

description:

Name of the account emitting this event.

name:

email

type:

string

optional:

yes

description:

The primary email, as text associated to this account.

name:

emails

type:

string

optional:

yes

description:

A list of 2 value tuples (name, email) for the emails associated to this account.

name:

token

type:

string

optional:

Yes

description:

For Windows local or domain accounts a token that can be use to impersonate the account. For Azure AD accounts, when extra api_scopes are configured, this is the latest OAuth2 token that can be use to obtain access to an extra API or refresh a token.

name:

peer

type:

JSON Object

optional:

No

description:

Address of the peer attached to this account. This might be a local or remote address, depending on whether the account is used for client side or server side interaction.

The peer contains the following attributes:

name:

address

type:

string

optional:

No

description:

IP address of this connection.

name:

port

type:

integer

optional:

No

description:

Port number of this connection.

name:

protocol

type:

string

optional:

No

description:

OSI Layer 4 transport layer protocol used for this connection in the form of either TCP or UDP.

The component contains the following attributes:

name:

uuid

type:

string

optional:

No

description:

UUID of the component (service or transfer) emitting this event.

name:

type

type:

string

optional:

No

description:

Type of the component emitting this event.

The server contains the following attributes:

name:

uuid

type:

string

optional:

No

description:

UUID of the server emitting this event.

name:

instance_name

type:

string

optional:

Yes

description:

Name of the node instance emitting this event. It is only set for events emitted from nodes in a cluster pool.

name:

type

type:

string

optional:

No

description:

Type of the server emitting this event.

OAuth proxy value for extensions#

For OAuth2 based authentication methods, use event.account.token['proxy'] to check whether API calls from extensions should use a proxy.

This value is the resolved proxy_value (authentication method proxy with fallback to the global server proxy).

The event.account.token['configuration']['proxy'] member only stores the raw authentication method proxy option and can be empty when the proxy is inherited from the global server configuration.