initial guide on fapolidy and firewalld

source/administration-guide/configure/calls-deployment.md

@@ -127,6 +127,26 @@ Mattermost Calls can function in air-gapped environments. Exposing Calls to the
 - Configuring a STUN server is unnecessary, as all connections occur within the local network.
 - The [ICE Host Override](https://docs.mattermost.com/configure/plugins-configuration-settings.html#ice-host-override) configuration setting can be optionally set with a local IP address (e.g., 192.168.1.45), depending on the specific network configuration and topology.
 
+### RHEL deployments with firewalld and fapolicyd
+
+When deploying Mattermost Calls or the rtcd service on RHEL systems with firewalld or fapolicyd enabled, additional firewall and security configuration is required.
+
+**Required ports:**
+- **Calls plugin (integrated)**: Port 8443 UDP/TCP for RTC media traffic
+- **rtcd service (standalone)**: Ports 8443 UDP/TCP for RTC media traffic + Port 8045 TCP for API communication with Mattermost
+
+**Configuration steps:**
+
+For complete firewalld and fapolicyd configuration instructions, including troubleshooting steps and example rules, see the [RHEL deployment guide](https://docs.mattermost.com/deploy/server/deploy-rhel.html). The guide includes:
+- firewalld port configuration for Mattermost, Calls plugin, and rtcd service
+- fapolicyd rules for Mattermost server and Calls plugin
+- Separate fapolicyd configuration for standalone rtcd service
+- Troubleshooting guidance for "operation not permitted" errors
+
+```{note}
+The Calls plugin runs as part of Mattermost and is covered by the standard Mattermost fapolicyd rules. Only the standalone rtcd service requires separate fapolicyd configuration.
+```
+
 ## Limitations
 
 - All Mattermost customers can start, join, and participate in 1:1 audio calls with optional screen sharing.

source/deployment-guide/server/linux/deploy-rhel.rst

@@ -234,6 +234,235 @@ The final step, depending on your requirements, is to run sudo ``systemctl enabl
       - `A Sysadmin's Guide to SELinux: 42 Answers to the Big Questions <https://opensource.com/article/18/7/sysadmin-guide-selinux>`_
       - `Mastering SELinux: A Comprehensive Guide to Linux Security <https://srivastavayushmaan1347.medium.com/mastering-selinux-a-comprehensive-guide-to-linux-security-8bed9976da88>`_
 
+.. important::
+
+  **Configure firewalld for government hardened environments**: When deploying Mattermost on RHEL systems with firewalld enabled, you'll need to configure firewall rules to allow access to the Mattermost Server.
+
+  Firewalld is the default firewall management tool on RHEL systems and provides a dynamically managed firewall with support for network zones. By default, it may block incoming connections to Mattermost's port (8065).
+
+  Check if firewalld is running:
+
+  .. code-block:: sh
+
+    sudo systemctl status firewalld
+
+  If firewalld is active, configure it to allow Mattermost traffic:
+
+  **Configure firewalld for Mattermost Server:**
+
+  1. Open port 8065 for Mattermost Server:
+
+    .. code-block:: sh
+
+      sudo firewall-cmd --permanent --add-port=8065/tcp
+
+  2. If you're using a reverse proxy (e.g., nginx or Apache) on the same server, also open HTTP and HTTPS ports:
+
+    .. code-block:: sh
+
+      sudo firewall-cmd --permanent --add-service=http
+      sudo firewall-cmd --permanent --add-service=https
+
+  3. Reload firewalld to apply the changes:
+
+    .. code-block:: sh
+
+      sudo firewall-cmd --reload
+
+  4. Verify the configuration:
+
+    .. code-block:: sh
+
+      sudo firewall-cmd --list-all
+
+    You should see port 8065/tcp (and optionally http/https services) in the output.
+
+  .. note::
+
+    If you're using a custom port for Mattermost, replace ``8065`` with your configured port number. The ``--permanent`` flag ensures the rules persist after system reboots.
+
+  **Configure firewalld for Mattermost Calls (optional):**
+
+  If you're deploying the Mattermost Calls plugin or the standalone rtcd service, additional ports need to be opened. See the :doc:`Calls deployment </administration-guide/configure/calls-deployment>` documentation for details on Calls architecture and deployment modes.
+
+  For integrated Calls plugin (running alongside Mattermost):
+
+  .. code-block:: sh
+
+    sudo firewall-cmd --permanent --add-port=8443/udp
+    sudo firewall-cmd --permanent --add-port=8443/tcp
+    sudo firewall-cmd --reload
+
+  For standalone rtcd service (requires additional API port):
+
+  .. code-block:: sh
+
+    sudo firewall-cmd --permanent --add-port=8443/udp
+    sudo firewall-cmd --permanent --add-port=8443/tcp
+    sudo firewall-cmd --permanent --add-port=8045/tcp
+    sudo firewall-cmd --reload
+
+  .. note::
+
+    - Port 8443 (UDP/TCP) is used for RTC media traffic (audio/video/screen sharing)
+    - Port 8045 (TCP) is used for API communication between Mattermost and rtcd service
+    - TCP support for RTC requires Calls v0.17+ and rtcd v0.11+
+    - If you're using custom ports, replace the default port numbers with your configured values
+
+.. important::
+
+  **Configure fapolicyd for government hardened environments**: When deploying Mattermost on RHEL systems with fapolicyd enabled, you may encounter "operation not permitted" errors in the Mattermost logs.
+
+  Fapolicyd (File Access Policy Daemon) is a userspace daemon that determines access rights to files based on trust. It's commonly enabled in secure environments to prevent execution of unauthorized binaries. By default, fapolicyd may block Mattermost binaries and plugins from executing.
+
+  Troubleshoot fapolicyd issues:
+
+  1. If you're seeing "operation not permitted" errors in your Mattermost logs, first check if fapolicyd is the cause by temporarily stopping it:
+
+    .. code-block:: sh
+
+      sudo systemctl stop fapolicyd
+
+  2. Test your Mattermost deployment. If it works now, fapolicyd is blocking Mattermost execution and you need to configure it properly.
+
+  3. Start fapolicyd in debug mode to identify what's being denied:
+
+    .. code-block:: sh
+
+      sudo fapolicyd --debug
+
+  4. Look for denial messages in the output. They will look similar to:
+
+    .. code-block:: text
+
+      rule=15 dec=deny_audit perm=execute auid=-1 pid=19735
+        exe=/opt/mattermost/bin/mattermost : path=/opt/mattermost/plugins/focalboard/server/dist/plugin-linux-amd64
+        ftype=application/x-executable trust=0
+
+    Note the rule number that's denying execution - your rule file must be numbered to come before this rule.
+
+  Configure fapolicyd to allow Mattermost:
+
+  1. Create a rule file for Mattermost. The naming convention is critical - it must come before the rule that's denying Mattermost. If you're using a stock fapolicyd configuration, ``80`` works fine:
+
+    .. code-block:: sh
+
+      sudo touch /etc/fapolicyd/rules.d/80-mattermost.rules
+
+  2. Add the allow rules to the file:
+
+    .. code-block:: sh
+
+      sudo vi /etc/fapolicyd/rules.d/80-mattermost.rules
+
+    Add the following rules to allow Mattermost execution:
+
+    .. code-block:: text
+
+      allow perm=execute exe=/usr/bin/sudo trust=1 : dir=/opt/mattermost/ all trust=0
+      allow perm=execute exe=/opt/mattermost/bin/mattermost : dir=/opt/mattermost all trust=0
+      allow perm=execute exe=/usr/lib/systemd/systemd trust=1 : dir=/opt/mattermost/ all trust=0
+
+  3. Check that the rules are valid and will be applied:
+
+    .. code-block:: sh
+
+      sudo fagenrules --check
+
+    You should see "Rules have changed and should be updated" in the output.
+
+  4. Load the new rules:
+
+    .. code-block:: sh
+
+      sudo fagenrules --load
+
+  5. Restart the fapolicyd service:
+
+    .. code-block:: sh
+
+      sudo systemctl restart fapolicyd
+
+  6. Restart Mattermost to verify it works correctly:
+
+    .. code-block:: sh
+
+      sudo systemctl restart mattermost
+
+  7. Verify that Mattermost is running without errors:
+
+    .. code-block:: sh
+
+      curl http://localhost:8065
+      sudo systemctl status mattermost
+
+  .. note::
+
+    The rule file numbering is critical. Rules are processed in order, and your allow rules must come before any deny rules. If you continue to see denials after following these steps, check your deny rule number (from the debug output) and ensure your rule file number is lower.
+
+  .. note::
+
+    If you have plugins that execute binaries (like Playbooks or the Calls plugin), the fapolicyd rules above allow execution of any binaries within the ``/opt/mattermost`` directory. If you need more restrictive rules, you can create specific rules for each plugin binary.
+
+  **Configure fapolicyd for rtcd service (optional):**
+
+  If you're deploying the standalone rtcd service for Mattermost Calls, you'll need separate fapolicyd rules for the rtcd binary. The rtcd service is deployed separately from Mattermost and runs its own binary. See the :doc:`Calls deployment </administration-guide/configure/calls-deployment>` documentation for details on when and why to use rtcd.
+
+  1. Create a rule file for rtcd. The naming convention is critical - it must come before the deny rule (typically 80 works):
+
+    .. code-block:: sh
+
+      sudo touch /etc/fapolicyd/rules.d/80-rtcd.rules
+
+  2. Add the allow rules for rtcd:
+
+    .. code-block:: sh
+
+      sudo vi /etc/fapolicyd/rules.d/80-rtcd.rules
+
+    Add the following rules (adjust paths based on your rtcd installation directory):
+
+    .. code-block:: text
+
+      allow perm=execute exe=/usr/bin/sudo trust=1 : dir=/opt/rtcd/ all trust=0
+      allow perm=execute exe=/opt/rtcd/bin/rtcd : dir=/opt/rtcd all trust=0
+      allow perm=execute exe=/usr/lib/systemd/systemd trust=1 : dir=/opt/rtcd/ all trust=0
+
+  3. Check that the rules are valid and will be applied:
+
+    .. code-block:: sh
+
+      sudo fagenrules --check
+
+    You should see "Rules have changed and should be updated" in the output.
+
+  4. Load the new rules:
+
+    .. code-block:: sh
+
+      sudo fagenrules --load
+
+  5. Restart fapolicyd and the rtcd service:
+
+    .. code-block:: sh
+
+      sudo systemctl restart fapolicyd
+      sudo systemctl restart rtcd
+
+  6. Verify that rtcd is running without errors:
+
+    .. code-block:: sh
+
+      sudo systemctl status rtcd
+
+  .. note::
+
+    - The paths shown above assume rtcd is installed in ``/opt/rtcd/``. Adjust these paths based on your actual rtcd installation directory.
+    - If you continue to see denials after following these steps, run ``sudo fapolicyd --debug`` to identify the specific paths being denied and adjust your rules accordingly.
+    - See the :doc:`rtcd deployment documentation </administration-guide/configure/calls-deployment>` for installation and configuration details.
+
+  See the `Mattermost and fapolicyd <https://support.mattermost.com/hc/en-us/articles/12167526545172-Mattermost-and-fapolicyd>`_ support article for additional troubleshooting steps.
+
 Step 6: Update the server
 ~~~~~~~~~~~~~~~~~~~~~~~~~