Email Relay Setup

Email Relay setup is necessary for users to receive emails regarding job status updates from the Slurm controller daemon. On Fluid-Slurm-GCP images (v2.4.0 and greater), postfix and mailx/mailutils (CentOS/Ubuntu) are available for sending mail from the controller instance. Additionally, the cluster-services configure mail-relay command set is available for quickly configuring your postfix main.cf and sasl_passwd files (if SMTP auth is required).

We recommend you review the SMTP relay service limits so that you are aware of the limitations of this system.

To use the email relay, you must complete the following steps

1. Enable Static IP Address on your controller Instance
2. Create an A Record for a subdomain of your G Suite domain to map to your controller static external IP address
3. (Optional, if SMTP Required) Create a user for SMTP Authentication within your subdomain
4. Add and Verify your subdomain in G Suite Admin panel
5. Configure SMTP-Relay settings in G Suite Admin panel
6. Configure the postfix server on your fluid-slurm-gcp controller instance

Enable Static IP Address on your Controller Instance

A static IP address ensures that your controller’s IP address remains the same, even if it is turned off. If the controller is deleted, the static IP address remains available to assign to future instances. Reserving a static IP address makes management of your SMTP Relay settings in G Suite easy to manage.

To reserve a Static IP address for your controller instance

1. Navigate to https://console.cloud.google.com/networking/addresses/list and make sure you are in the GCP project that hosts your fluid-slurm-gcp cluster
2. Find your controller instance in the “In use by” column. For that row, change “Ephemeral” to “Static” in the “Type” column. You can click on the word “Ephemeral” to bring up a dropdown menu to select “Static”
3. Wait for the changes to propagate. There is no need to restart the instance.

Create an A Record

In order to use the G Suite SMTP Relay service, you will need to create an A Record with your DNS host that maps a subdomain of your organization to your controller’s static external IP address.

To learn more about A Records, see “About A Records” from G Suite Documentation.

Google Domains

To create a new A Record for your domain, if it is managed by Google Domains, you can follow these instructions. In step 6.a use the following settings:

6.a.i In the first field, enter "fluid-slurm-gcp" or your desired subdomain.

6.a.ii In the dropdown menu, select A.

6.a.iii In the TTL field, enter "1H."

6.a.iv In the data field, enter the Static External IP address of your Controller

Other providers

For other providers, the “Configure A Records” G Suite documentation for guidance. Use the following settings

Set the Name/Host/Alias to “fluid-slurm-gcp” or your desired subdomain for your controller.

Set the Record Type to A

Set the TTL to 3600

Set the Value/Answer/Destination to the static external IP address of your controller.

Create an Email Address (Optional)

SMTP Authentication requires that an authenticated user email address and password from your organization is provided when mail is sent from your controller. This extra layer of security gives you the ability to quickly shut down email service on the controller from G Suite, in the event that your system becomes compromised.

Further, if 2-factor authentication is enabled on the account (Recommended), temporary App Passwords are used that can further limit the scope of access given to the account via the postfix and mail utilities on the controller instance.

Add and Verify your Subdomain

Once you have the A Record configured, you need to add the subdomain to your organization within G Suite. You can follow STEP 2 in the “Add multiple domains or domain aliases” documentation from G Suite. When you add a domain use the subdomain you created with the A Record in the previous step.

For example, if your organization domain is organization.com, and your A Record maps fluid-slurm-gcp to your controller static external IP address, then the domain you want to create is fluid-slurm-gcp.organization.com

If you are using Google Domains, the verification process is often immediate, but can take up to 24 hours. For other DNS hosts, you may need to wait up to 48 hours.

Configure SMTP-Relay in G Suite

Email from the controller will be routed through your organization’s SMTP Relay. Once your controller’s static external IP address is mapped to a verified subdomain in your organization, the SMTP Relay service can be configured to allow email to be routed from your controller.

You can follow the “SMTP relay: Route outgoing non-Gmail messages through Google” documentation from G Suite.

Use the following settings to configure your SMTP Relay

Step 6. Set the Allowed Senders to Only Addresses in my domain

Step 7. In the Authentication section, select Only accept mail from the specified IP addresses. If you want to use SMTP Authentication, also select Require SMTP Authentication

Step 8.a. For the Description enter “Fluid-Slurm-GCP Controller : Slurm Job Scheduler Relay”

Step 8.b. For the IP Address or Range, enter the static external IP address of the controller.

Step 9. You must enable TLS Authentication - the controller’s postfix settings use Port 587 for TLS encryption.

Once you save these settings, it can take anywhere from 1 hr - 24 hr for the settings to propagate.

Configure the Postfix server on the controller instance

With the GSuite SMTP relay configured in G Suite and the controller external IP address associated with a verified subdomain in your organization, you are now ready to set up your fluid-slurm-gcp cluster to send email notifications to users regarding job status updates.

1. Start by logging into your controller instance and go root $ sudo su -

2. Create a temporary cluster-configuration file root# cluster-services list all > config.yaml

3. Add the mail_settings definition to your temporary cluster-configuration file

a. If you are not using SMTP authentication

mail_settings:

    domain: VERIFIED-GSUITE-DOMAIN

    smtp_authentication: false

b. If you are using SMTP authentication

mail_settings:

    smtp_authentication: true

4. Use cluster-services to configure the postfix server. If SMTP authentication is enabled, follow the prompts to provide email and password for a valid account in your organization.

root# cluster-services configure mail-relay --config=config.yaml

5. Test the mail service by sending a message to yourself, replacing you@domain.com with your email address. You should see an email in your inbox from root@subdomain.domain.com (where subdomain is the subdomain in your A record, and domain.com is your domain name)

root# echo “message” | mail -s “Test” you@domain.com

If you did not receive an email, you can check the mail logs at /var/log/maillog.