1 What troubleshooting principles should I follow? #
Before diving into component-specific issues, consider the following general principles.
- Check logs
Logs are the primary source of information. Most of the time, errors are self-explanatory and contain hints on what failed.
- Check clocks
Having clock differences between systems can lead to all kinds of different errors. Ensure clocks are in sync.
- Boot issues
If the system is stuck during boot, note down the last messages displayed. Access the console (physical or via BMC) to observe boot messages.
- Network issues
Verify network interface configuration (
ip a), routing table (ip route), test connectivity from/to other nodes and external services (ping,nc). Ensure firewall rules are not blocking necessary ports.- Verify component status
Use
kubectl getandkubectl describefor Kubernetes resources. Usekubectl get events --sort-by='.lastTimestamp' -n <NAMESPACE>to see the events on a particular Kubernetes namespace. Usekubectl get events --sort-by='.metadata.creationTimestamp' -n <NAMESPACE>to sort events by creation time on a particular Kubernetes namespace.- Verify services status
Use
systemctl status <SERVICE>for systemd services.- Check syntax
Software expects certain structure and syntax on configuration files. For YAML files, for example, use
yamllintor similar tools to verify the proper syntax.- Isolate the problem
Try to narrow down the issue to a specific component or layer (for example, network, storage, OS or Kubernetes).
- Documentation
Always refer to the official SUSE AI Factory documentation and also upstream documentation for detailed information.
- Known issues
For each SUSE AI Factory release, the release notes contain a "Known issues" section detailing issues that will be fixed in future releases.