Session Recording

Session-Recording Setup

When session recording is enabled, connection-specific audit events also provide:

Video playback. With SSH sessions you can search for keyword occurrences.
Transferred files.
Clipboard (RDP only).
Channel logs (SSH only).
Protocol stream (SSH exec channels, SFTP commands, Database connections).

To enable session recording for connections to a host:

On the Administration→Hosts page, Edit the host.
Under Options, enable the setting Session Recording.
You may optionally omit clipboards and/or file transfers from recordings. This can be useful for example if such content contains sensitive information like user credentials.
Click Save to apply your changes.
Subsequent sessions to the host are recorded. You can view the playback and transferred files from the connection-specific audit events, available from Monitoring→Connections.

Session recordings should not be stored on PrivX servers as they may consume lots of disk space; you should configure PrivX to store session recordings on an external share instead (such as NFS or EFS). To set up external storage share for PrivX session recordings:

On your external storage server, create a share for storing PrivX session recordings. The share must be a directory that satisfies the following:

The share must be mountable by all PrivX servers.
The share must be readable and writable by the privx system user of every PrivX server.

On each PrivX server, install any extensions required for mounting the external-storage share. For example, to mount NFS shares you will likely need to install nfs-utils; for SMB shares you will likely need cifs-utils. These extension packages are available from the RHEL/CentOS public repositories.
On each PrivX server, mount the external share to a local directory. The directory path must be the same on all PrivX servers. To enable mounting the share on system startup, we recommend adding the mount directive to /etc/fstab.
To allow the GUI to display other connection logs when the NFS server is unavailable, mount the share with options like the following:
```
soft
timeo=10
retry=1
```

To enable live monitoring it is required that NFS clients do not buffer write operations. All PrivX nodes (in an HA setup) should mount the NFS mount point with noac option. This will likely increase the load on the NFS server due to heavier traffic.

To configure PrivX with the new storage location, access the PrivX GUI, go to Administration→Settings→Global, and specify the location in Data folder.
Save your changes, then restart PrivX services to apply the changes.

PrivX generates keyframe data when opening RDP session recordings for the first time. Note that this may take up to several minutes for large RDP and web-connection trails. Also note that RDP session recording takes considerable storage space. For some rough estimates about space requirements, see Data Encryption.

PrivX indexes session recordings when they are searched for the first time. Depending on the duration of the recording, the first search may take some time. SSH transcripts require roughly ten times the storage space compared to the original video recording.

While an RDP session is open, the file transfers and the session recording are temporally stored in PrivX servers' /tmp directory. For this reason you will need to ensure sufficient disk space on PrivX servers even when external trail storage is configured.

Inspecting Session Recordings

Session recordings for interactive SSH shell sessions and RDP / VNC desktop sessions are provided as session recording playbacks. The playback shows the shell session / desktop as the end user sees it.

Transferred files are provided as downloadable files.

Protocol streams for SSH exec channels, SFTP commands and Database connections are provided as downloadable log files. These log files are provided in two alternative formats:

hex: each protocol message is represented in the log as a combination of a header line - consisting of timestamp, connection ID, channel type, message direction - and the message payload in hexdump format
jsonl: each protocol message is represented as a json object containing properties for timestamp, connection ID, channel type, message direction and base64 encoded message payload.

Real-Time Auditing SSH Connections

Generally, session recordings are available only after the session is closed. Recordings of certain channels can be monitored live. See next chapter for details. For real-time auditing SSH connections, you can configure PrivX to output audit events from SSH connections to syslog. You can then integrate the output with your SIEM for automatic event handling.

Before setting up real-time auditing, you should have a SIEM that is accessible from PrivX.

To enable real-time audit events from SSH connections to syslog:

Configure logging.
- RPM Deployments: Audit events from SSH connection are audited via SSH-PRIVX-SENSITIVE-AUDIT and need to be forwarded to SIEM.
  For example, this can be configured for rsyslog with the following setting (replace @@192.0.2.8:9010 with the address of your SIEM listener/forwarder):
```
:msg, contains, "SSH-PRIVX-SENSITIVE-AUDIT" @@192.0.2.8:9010
```
  Note that by default, audit events from SSH connections are discarded by the following rule in /etc/rsyslog.d/privx-syslog.conf:
```
:msg, contains, "SSH-PRIVX-SENSITIVE-AUDIT" /dev/null
```
- Kubernetes Deployments: Audit events from SSH connection can be enabled by settings the following values in the PrivX Helm Chart (replace @@192.0.2.8:9010 with the address of your SIEM listener/forwarder):
```
privx.syslog.enabled = true
privx.syslog.audit.sensitive.to = @@192.0.2.8:9010
```

🚧 Caution

SSH-PRIVX-SENSITIVE-AUDIT events contain sensitive data that should never be stored on file or disk. Always ensure that all SSH-PRIVX-SENSITIVE-AUDIT events go to your SIEM, or are discarded.

Real-time audit events from SSH connections are logged only to syslog (if syslog is enabled). PrivX by default discards the audit events related to the SSH connection sessions: PrivX on RPM has a default syslog configuration at /etc/rsyslog.d/privx-syslog.conf that discards all SSH connection audit events. PrivX on Kubernetes comes with its syslog pod that has a similar default rule to discard SSH connection events.

On Administration→Settings→Global, Edit the SSH common settings:
- Enable Send SSH events to audit log
- Select the SSH channels that are to output audit events.
Save your settings, then Restart PrivX to apply your changes.

Audit events from SSH connections are now output to the syslog in real time.

Live Connection Monitoring

You can view real-time video of ongoing connections.

To enable live monitoring:

Enable session recording for hosts where you need live monitoring.
Enable live monitoring on Administration→Settings→Global Settings, under Live connection monitoring. You can enable/disable live monitoring per connection type.

Once enabled, you can live monitor ongoing connections from their connection details:

On Monitoring->Connections, click a Connected connection with REC symbol.
Under Channels, click a channel to view it.

Each applicable channels can be monitored live independent from each other.

Enabling live recording affects PrivX performance with increased filesystem read/write calls and encryption/decryption. This cost is active, as long as session recording is enabled, even if the channel is not being monitored live. RDP live monitoring is particularly resource-intensive.

You should review your PrivX resources (RAM, CPU) and trail storage setup if live monitoring is to be used at scale. Ensure that PrivX has sufficient RAM/CPU, a fast filesystem, and a fast network (in case of remote trail storage).

For improved performance, we recommend disabling live monitoring if you are only interested in playing back recordings after the connections have ended.

Access Control

Live monitoring access control is basically the access control of the underlying recordings. Users can access the recordings if:

The user is a privx-admin.
The user has a role with connection-playback permission. The role must be in the same access group as the target. The user can access all recordings in this access group.
The user has a role and this role is specified as access role to a connection. The user can only access recordings in this connection.

Typical access control setup is described as follows:

Scenario 1: internal sub-admins wishes to live-monitor connections to specific targets.

A sub group of internal users are assigned a task of monitoring connections to highly classified targets. A role should be created for this user group that belongs to the same Access Group as the targeted hosts. If the hosts are scattered among many Access Groups then as many roles should be created targeting each of them. In (each of) the role, at least the connection-playback permission should be enabled.

When logged in, the sub-admins can select an ongoing connection from Monitoring->Connections. They can click a channel to view its live playback.

Usually for such sub-admins the connections-view, connections-manage, connections-trail, connections-terminate and logs-view are also enabled so that they can see the connection details, monitor it live or not, as well as see audit events and terminate the connection if needed.

Scenario 2: a sub-admin wishes to share a single connection (view-only) with an external user as part of a demo session.

The external user has a role in an isolated Access Group. There is no target hosts in this isolated Access Group.

The sub-admin starts a connection to the target host (with session recording enabled). The sub-admin then browses to this connection in Monitoring->Connections and click on it to open the connection details view. In the "Access roles" the sub-admin adds the role in the isolated Access Group, thereby granting (the external user) ad-hoc access to this connection.

Finally, the sub-admin retrieves the channel URL by clicking ☰ corresponding to the opened channel. The sub-admin delivers this URL to the external user and the latter can go directly to the live playback.

Playback Control in Live Session

When you click an ongoing channel, the playback will jump to the latest recording position. A "Live" flag is displayed next to the timer in the playback progression bar.

During the live playback session no playback controls (rewind, fast-forward, pause/unpause) are possible. Data is served passively in one direction to the monitoring user. The monitoring user, however, can resize the browser as needed.

Data arrives to the live monitoring session on best-effort basis: there is always some delay from processing and dispatching the data.

The monitoring user can terminate the live playback at any time. Closing the live monitoring session has no impact on the original connection. If, however, the user in the original connection closes the channel (by closing the connection, or closing the channel tab) then the monitoring session will eventually transform into a normal playback with all controls being again enabled.

Live-Monitoring Audit Events

There are 2 new audit events corresponding to the start and end of a live monitoring session. The audit events have exactly the same content as session-added and session-removed audit events except for the "monitoring-user-id" which indicates the UUID of the user who monitored the connection.

Monitoring Constraints

There is no limit for the total number of monitoring sessions, for the number of live monitoring sessions by any users and for any single channel. This is consistent with playing back the recordings after the connection has ended. The playback is not counted towards license's concurrent connection limit in all cases.

Within an SSH connection each channel is independently monitored.

For SSH only shell and exec (with PTY) channels can be played back and therefore monitored live. Requests to playback other channels, live or not, will be rejected.

When SSH live monitoring flag is set to false PrivX will reject requests to playback the recording while the channel is ongoing. Only after the channel has ended it is possible to playback the recording.

Other Notes

The user in the original connection is not informed in the UI if another user starts or stops monitoring the connection.

The monitoring user is not informed if the user in the original connection opens new channels.

HA setup with NFS external storage: live monitor feature requires trail data to be pushed to NFS server as soon as possible. NFS clients typically buffer file-write operations thus making it hard for the monitoring session to follow updates in the actual connection. Mount the NFS mount point (on client side) with noac to disable write buffering on all PrivX nodes.