|Index|Troubleshooting SUSE AI Factory|What troubleshooting principles should I follow?
Applies to SUSE AI Factory

1 What troubleshooting principles should I follow?

Before diving into component-specific issues, consider the following general principles.

Check logs

Logs are the primary source of information. Most of the time, errors are self-explanatory and contain hints on what failed.

Check clocks

Having clock differences between systems can lead to all kinds of different errors. Ensure clocks are in sync.

Boot issues

If the system is stuck during boot, note down the last messages displayed. Access the console (physical or via BMC) to observe boot messages.

Network issues

Verify network interface configuration (ip a), routing table (ip route), test connectivity from/to other nodes and external services (ping, nc). Ensure firewall rules are not blocking necessary ports.

Verify component status

Use kubectl get and kubectl describe for Kubernetes resources. Use kubectl get events --sort-by='.lastTimestamp' -n <NAMESPACE> to see the events on a particular Kubernetes namespace. Use kubectl get events --sort-by='.metadata.creationTimestamp' -n <NAMESPACE> to sort events by creation time on a particular Kubernetes namespace.

Verify services status

Use systemctl status <SERVICE> for systemd services.

Check syntax

Software expects certain structure and syntax on configuration files. For YAML files, for example, use yamllint or similar tools to verify the proper syntax.

Isolate the problem

Try to narrow down the issue to a specific component or layer (for example, network, storage, OS or Kubernetes).

Documentation

Always refer to the official SUSE AI Factory documentation and also upstream documentation for detailed information.

Known issues

For each SUSE AI Factory release, the release notes contain a "Known issues" section detailing issues that will be fixed in future releases.