Invalid Authentication Credential
If you reboot your fluid-slurm-gcp cluster, you may run into issues where Slurm commands no longer work from the login node. Most often this is associated with system clocks on the controller and login node being out of sync when the munge service starts on each node. In this event, running commands like sinfo will fail with errors that look like :
To solve this problem,
SSH into your cluster's controller
Restart the munge service
sudo service munge restart
If you continue to experience issues, reach out to firstname.lastname@example.org for community support.