SMTP Server

[MrJohnson-SixNinesIT]

Howdy,

How is the mail server functionality intended to work? Does all it require is that those email addresses exist, and the MX records for the external host (I.E. something like GoDaddy for) are copied into the primary hosted zone in Route53?

If the above is true, would adding the records in Route53 and running the Github Actions update things accordingly?

Lastly, in the deploy-{env}.yml there is the “Configure SMTP” step. In the “Configure AWS credentials” step, we were able to use our session token to authenticate and get things working. In the “Configure SMTP” step, the session token is not recognized and shows a warning. Is the session token variable required in this step? If we exclude attempting to use a session token, we do NOT see the warning. Wondering if this will cause an issue with the emails down the line.

An example of our credentials step that works (in our case it will NOT work without it):

Here is the warning we receive when we attempt to use the session token in the “Configure SMTP” step:

[lpm0073]

there's a bit of apples-and-oranges in your question, so i'll respond in pieces to try to unwind.

configuring SMTP: the short answer is that you only need to set 4 Python variables in the LMS:

• https://github.com/openedx/edx-platform/blob/master/lms/envs/production.py#L151
• https://github.com/openedx/edx-platform/blob/master/lms/envs/production.py#L495
  there potentially are additional details, like say tls and ssl, depending on your email provider. I prefer AWS SES but also occasionally use Google mail services at clients' requests. The hook that i provide in the deployment work flow is simply some scaffolding to grab the username and password values from Github Actions secrets. please see the README in your openedx_devops repo for more info, and screen shots.

more ....

aws-actions/configure-aws-credentials: the purpose of this Github Action, created and maintained by AWS, is to configure AWS credential and region environment variables for use in other GitHub Actions. The environment variables will be detected by both the AWS SDKs and the AWS CLI to determine the credentials and region to use for AWS API calls. Thus, it generates a session token based on an AWS IAM key pair that you provide. hence, the framing of your question above is at odds w what this Action actually does. More generally, and to your broader question about configuring SMTP: this AWS Action is unrelated to configuring SMTP for open edx. you can read more here: https://github.com/marketplace/actions/configure-aws-credentials-action-for-github-actions

hope that helps.

[MrJohnson-SixNinesIT]

Okay, so that makes sense.

So the host and port look fine. We added the EMAIL_HOST_PASSWORD and EMAIL_HOST_USER secrets to the workflow, but it does not appear to be setting the EMAIL_HOST_USER.

When looking at the workflow, I don't see it there in the list of variables at all:

So I guess my question is, where does the value go for the EMAIL_HOST_USER, if not the Actions Secrets?

[xavierperez06]

Hello, i'm having the same problem. I've deployed everything and it seems to be working fine except the email. If i go the AWS SES i don't see anything configured there, should i have anything in there, or do any extra step?

Also, regarding the 4 Python variables for the LMS you @lpm0073 you pointed out, those variables should be in the file settings_merge.yml? Because i don't see EMAIL_HOST or EMAIL_HOST_PASSWORD anywhere.

[lpm0073]

i need to improve the documentation. responding to questions from you both:

• the short answer is that you either set a bash environment variable (does not matter how you do this) of the form TUTOR_EMAIL_HOST_PASSWORD, which tutor will then consume and use on the next "tutor config save" operation, or, you explicitly set the value using a call such as "tutor config save --set EMAIL_HOST_PASSWORD='YADDAYADDA'". The Github Actions that i wrote do this for you, but, you can also create these variables yourself.
• you can read values from Actions Secrets, but you cannot write to them. You set Actions Secrets from the screen in the screen shot, below.

[MrJohnson-SixNinesIT]

Hello, so we were able to get credentials to work by passing in the values into the tutor config save command:

Although this appears to work, emails are not being sent out. We are using AWS Simple Email Service with the following configurations:

In SES:

• Domain Identity has been verified.
• Email addresses such as “support@domain”, “registration@domain” etc… have all been verified.
• The email host is the is SMTP endpoint value value in SES.

We’ve confirmed in SES that emails could be sent out. And just to be sure, we registered at courses.{ourDomainName}.com using an SES verified email, but still did not see any emails go out from SES.

I feel like there is something else we are missing here, but not quite sure what that is.

[MrJohnson-SixNinesIT]

@lpm0073 Oh, for our code, we had caught that and have been using the upper-casing in our code, much like your screenshots depict:

Per your suggestion, we have also started using the password reset functionality for testing, and do not see any emails being sent in SES. The IAM user that we have configured for this also has SendRawEmail permissions.

And for testing purposes, we have even hardcoded the values for the aws-ses-id and secret, but still do not see any emails going out.

Is there some way to see logs or something as to why this would be failing to send?

[lpm0073]

yes, you can view the lms logs directly from inside of k9s by using the "L" hotkey on an lms pod. alternatively, there's a cookiecutter command-line script available in the newest version of the ec2_bastion -- klog.sh that will download pod log data to your console.

Either way, keep in mind that the email processing will be sent to a random lms pod and so you'll need to keep an eye on all lms pods that are running.

With all of that in mind, i'm usually able to see and diagnose all SMTP send problems from the lms log data.

[MrJohnson-SixNinesIT]

Okay, so thanks for the awesome advice on using k9s to view the logs. I wasn't able to see anything pertinent to this on the LMS pods, but I was able to view logs on the SMTP server.

On the SMTP server, we get timeouts:

Just to be sure, we tested sending emails using AWS CLI and that worked without issue. Sending emails worked from the Sandbox as well.

Just in case, we also tried different email providers, and we still never see emails being sent.

[lpm0073]

The following is a general purpose smart bomb for trouble shooting purposes. you can use the following from inside the shell of an lms pod to dump EVERY django setting, including any and all SMTP email settings:

./manage.py lms --settings=tutor.production diffsettings --all

[MrJohnson-SixNinesIT]

@lpm0073 Okay, we were able to dig further into the logs and found the following:

Above, we get the error of "SMTP AUTH extension not supported by server".

With the following settings being passed in:

[MrJohnson-SixNinesIT]

When we look online about the "SMTP AUTH extension not supported by server" error, people who receive that message in other contexts mention a port issue, but port 587 seems to work for them.

[lpm0073]

any update on this?

[MrJohnson-SixNinesIT]

@lpm0073 unfortunately we are still unable to figure out why the emails don't send.

[lpm0073]

ok. i have to deploy a large platform this coming saturday. so, one way or another i'll figure this out.

[MrJohnson-SixNinesIT]

@lpm0073 Awesome! Thank you. Looking forward to seeing what you turn up.

[MrJohnson-SixNinesIT]

@lpm0073 Howdy. Did you have any luck this past weekend?

[MrJohnson-SixNinesIT]

I'm wondering if this functionality can be overridden with a plug perhaps. Or if that isn't the intended purpose of the plugins.

[MrJohnson-SixNinesIT]

@lpm0073 Ultimately, we got things working by adding those port, email_host_user and email_host_password directly to the settings_merg.yml.

Once that worked, we went back and created secrets with those same names and that got email working for us via AWS SES.