Troubleshooting authentication and authorization

When authentication or authorization fails it usually is due to a mismatch in the configuration of the provider and SUSE Observability. To make troubleshooting easier it is possible to enable debug logging on SUSE Observability for authentication and authorization specifically.

Disable the debug logging again as soon as your are done with troubleshooting, because it is very likely debug logging contains secrets and/or personal information.

To enable debug logging copy/paste the following yaml snippet into a debug-auth.yaml file.

stackstate:
  components:
    server:
      additionalLogging: |
        <logger name="org.pac4j.core.engine" level="DEBUG"/>
        <logger name="org.pac4j.oidc.profile.creator" level="DEBUG"/>
        <logger name="org.pac4j.oidc.credentials.authenticator" level="DEBUG"/>
    api:
      additionalLogging: |
        <logger name="org.pac4j.core.engine" level="DEBUG"/>
        <logger name="org.pac4j.oidc.profile.creator" level="DEBUG"/>
        <logger name="org.pac4j.oidc.credentials.authenticator" level="DEBUG"/>

Now run the helm upgrade command you used before but include this one extra yaml file (so helm upgrade .... --values debug-auth.yaml) to enable debug logging. No pods will be restarting, the logging configuration changes will be loaded automatically after about 30 seconds.

To disable the debug logging run the helm upgrade .... command again but omit the --values debug-auth.yaml. After 30 seconds the updated logging configuration is loaded and the debug logging stops.

Known issues

Rancher 2.15 and later require the groups scope to return group claims. Without it, users can lose permissions assigned through groups while direct user assignments still work. The groups scope is available from Rancher 2.14. For SUSE Observability installations whose Rancher preset cannot configure scopes, use the following workaround.

  1. Update the existing OIDCClient in the Rancher local cluster to allow the required scopes.

    kubectl patch oidcclients.management.cattle.io oidc-observability --type=merge -p '{"spec":{"scopes":["openid","profile","offline_access","groups"]}}'

    An empty spec.scopes defaults to a list without groups.

  2. Remove stackstate.authentication.rancher from your SUSE Observability values and replace it with the generic OIDC configuration below. Keep your existing client ID and secret, and use your Rancher URL in discoveryUri. Preserve any custom redirect URI, logout options, and custom parameters under oidc. Keep existing role and group bindings.

    stackstate:
  authentication:
    sessionLifetime: "16h"
    oidc:
      clientId: "<oidc-client-id>"
      secret: "<oidc-secret>"
      discoveryUri: https://<rancher-host>/oidc/.well-known/openid-configuration
      scope: [openid, profile, offline_access, groups]
      jwsAlgorithm: RS256
      jwtClaims:
        usernameField: sub
        groupsField: groups

  3. Apply the values with your usual Helm upgrade procedure. After the rollout, log out of SUSE Observability and log in again. Check that the new session contains the expected group identities and grants access through an existing group binding.

Troubleshooting issues with permissions