The alarm and email sending functions of superset are unavailable #29541

hswlxc · 2024-07-10T13:01:56Z

Bug description

superset_config.py.zip
This is my profile, and according to this configuration, the email cannot be sent

How to reproduce the bug

Screenshots/recordings

No response

Superset version

4.0.1

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

I have searched Superset docs and Slack and didn't find a solution to my problem.
I have searched the GitHub issue tracker and didn't find a similar bug report.
I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

hswlxc · 2024-07-10T13:05:00Z

The task to send the mail is not scheduled，No task error information is displayed in the log

dosubot · 2024-07-10T13:29:47Z

Hey @hswlxc, I'm here to assist you with any bugs, questions, or contributions you may have. Let's tackle this issue together!

To resolve the issue where the alarm and email sending functions of Superset are unavailable, you need to ensure that the ALERT_REPORTS configuration is set to True in your superset_config.py file.

From the provided context, it appears that the ALERT_REPORTS setting might be set to False. You should change this setting to True:

# Enables Alerts and reports new implementation
"ALERT_REPORTS": True,

This change will enable the alerts and reports functionality, which includes the alarm and email sending features ^[1].

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T01:32:19Z

"ALERT_REPORTS": True, This value is already set to true，But it still doesn't trigger an email

hswlxc · 2024-07-11T04:59:07Z

The celery_worker and beat logs are normal with no errors, the chrome driver is also normal and can take thumbnails normally, but the mail and alarms are not working and there are no execution logs ，Please look at this problem. Thank you @mistercrunch，@villebro，@michael-s-molina，@betodealmeida，@john-bodley @rusackas @dosu

dosubot · 2024-07-11T05:04:30Z

Since the ALERT_REPORTS flag is already set to True and there are no errors in the Celery worker and beat logs, let's check a few other configurations and steps to ensure everything is set up correctly:

SMTP Configuration: Ensure your SMTP settings in superset_config.py are correct and that EMAIL_NOTIFICATIONS is set to True. Here is an example configuration:

EMAIL_NOTIFICATIONS = True
SMTP_HOST = "your_smtp_host"
SMTP_STARTTLS = True
SMTP_SSL = False
SMTP_USER = "your_smtp_user"
SMTP_PORT = 587  # or the port your SMTP server uses
SMTP_PASSWORD = "your_smtp_password"
SMTP_MAIL_FROM = "[email protected]"

Celery Beat Schedule: Ensure that the Celery beat schedule includes the reports.scheduler task. Here is an example configuration:

from celery.schedules import crontab

beat_schedule = {
    "reports.scheduler": {
        "task": "reports.scheduler",
        "schedule": crontab(minute="*", hour="*"),
    },
    "reports.prune_log": {
        "task": "reports.prune_log",
        "schedule": crontab(minute=0, hour=0),
    },
}

Check Celery Worker and Beat Logs: Even though you mentioned there are no errors, ensure that the Celery worker and beat are running and processing tasks correctly. You can check the logs for any relevant information.
Browser Installation for Screenshots: Ensure that the headless browser (e.g., Chrome) is correctly installed and configured. Since you mentioned that the Chrome driver is working and can take thumbnails, this step might already be covered.
Review Superset Logs: Check the Superset logs for any additional information or errors that might indicate why the emails and alarms are not being triggered.
Test SMTP Configuration: Manually test the SMTP configuration to ensure that emails can be sent using the provided credentials and settings.

If all these configurations are correct and the issue persists, it might be helpful to provide more specific details about the current configuration and any additional logs or error messages that might be available.

For further reference, you can check similar issues and discussions:

@rusackas, this one's a bit out of my wheelhouse, could you step in and provide your guidance?

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T05:57:13Z

The configuration check above was fine, I checked the task list and found that no scheduled tasks were started @dosu

dosubot · 2024-07-11T06:30:45Z

Since the configuration checks are fine and no scheduled tasks are being started, let's ensure a few more things:

Dry-Run Mode: Ensure that the ALERT_REPORTS_NOTIFICATION_DRY_RUN setting is set to False to disable dry-run mode and start receiving notifications.
```
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
```
Manual Trigger: Manually trigger the reports.scheduler task to verify if it is configured correctly and can run as expected. You can do this by calling the scheduler task directly using Celery's send_task method:
```
from superset.extensions import celery_app

# Trigger the scheduler task manually
celery_app.send_task('reports.scheduler')
```

Celery Configuration: Ensure that the Celery configuration is correctly set up, including the beat_schedule for reports.scheduler:

from celery.schedules import crontab

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"
    imports = ("superset.sql_lab", "superset.tasks.scheduler")
    result_backend = "redis://localhost:6379/0"
    worker_prefetch_multiplier = 10
    task_acks_late = True
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

Celery Worker and Beat: Ensure that both the Celery worker and beat are running. You can start them using the following commands:
```
celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4
celery --app=superset.tasks.celery_app:app beat
```
Superset Logs: Check the Superset logs for any additional information or errors that might indicate why the emails and alarms are not being triggered.

By following these steps, you should be able to identify and resolve the issue with the scheduled tasks not being started ^[1]^[2].

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T08:09:49Z

from superset.extensions import celery_app
celery_app.send_task('reports.scheduler')

When manually triggered, the message was that I could not connect, but my redis service started normally, and I could take screenshots and use the cache normally
This error occurs, where the configuration is incorrect @dosu

dosubot · 2024-07-11T08:18:18Z

The ConnectionError: [Errno 111] Connection refused indicates that Celery is unable to connect to the Redis broker. Here are some steps to troubleshoot and resolve this issue:

Verify Redis Connection:
Ensure that Redis is running and accessible. You can test the connection to Redis using the redis-cli tool:
```
redis-cli ping
```
You should receive a PONG response if Redis is running and accessible.

Check Redis Configuration:
Ensure that the Redis configuration in your superset_config.py or celeryconfig.py is correct. The broker_url should point to the correct Redis host and port:

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"  # Ensure this matches your Redis configuration
    result_backend = "redis://localhost:6379/0"
    # Other configurations...
CELERY_CONFIG = CeleryConfig

Firewall and Network Issues:
Ensure that there are no firewall or network issues preventing Celery from connecting to Redis. Check that the Redis port (default is 6379) is open and accessible.
Redis Logs:
Check the Redis logs for any errors or warnings that might indicate why the connection is being refused. The logs are usually located in /var/log/redis/redis-server.log or a similar location depending on your setup.
Celery Worker and Beat:
Ensure that both the Celery worker and beat are running and configured to use the correct Redis broker URL. Start them using the following commands:
```
celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4
celery --app=superset.tasks.celery_app:app beat
```

Superset Configuration:
Double-check the Superset configuration to ensure that all necessary settings are correctly applied. Here is an example configuration for reference:

from celery.schedules import crontab

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"
    result_backend = "redis://localhost:6379/0"
    imports = ("superset.sql_lab", "superset.tasks.scheduler")
    worker_prefetch_multiplier = 10
    task_acks_late = True
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }
CELERY_CONFIG = CeleryConfig

By following these steps, you should be able to identify and resolve the connection issue between Celery and Redis ^[1]^[2].

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T08:28:24Z

Manually triggered, you can print out the task ID, indicating that the connection is fine, but this task ID is not available from flower

@dosu

dosubot · 2024-07-11T08:33:04Z

Since the task ID is printed, indicating that the connection to Redis is fine, but the task is not visible in Flower, let's focus on a few additional checks and configurations:

Celery Task Registration:
Ensure that the reports.scheduler task is correctly registered with Celery. Verify that the task is imported and recognized by Celery.
```
from superset.tasks.scheduler import scheduler

@celery_app.task(name="reports.scheduler")
def scheduler_task():
    scheduler()
```
Celery Worker Configuration:
Ensure that the Celery worker is configured to recognize and execute the reports.scheduler task. Check the worker logs for any errors or warnings related to task registration.

Celery Beat Configuration:
Ensure that the Celery beat schedule includes the reports.scheduler task and that it is correctly configured to run at the desired intervals.

from celery.schedules import crontab

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"
    result_backend = "redis://localhost:6379/0"
    imports = ("superset.sql_lab", "superset.tasks.scheduler")
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

Check Celery Logs:
Review the Celery worker and beat logs for any errors or warnings that might indicate why the task is not being executed. Look for messages related to task registration, scheduling, and execution.
Flower Configuration:
Ensure that Flower is correctly configured to monitor the Celery tasks. Sometimes, tasks might not appear in Flower due to configuration issues. Restart Flower and check if the tasks appear.
Task Visibility in Flower:
Verify that the task is not filtered out in Flower. Check the Flower interface for any filters or settings that might hide certain tasks.

By following these steps, you should be able to identify and resolve the issue with the reports.scheduler task not appearing in Flower and ensure it is correctly scheduled and executed by Celery ^[1]^[2].

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T08:54:38Z

celery -A superset.tasks.celery_app inspect registered

Through the above command, reports.scheduler has registered Celery, but still has no celery sending task @dosu

dosubot · 2024-07-11T09:07:46Z

Since the reports.scheduler task is registered with Celery but still not sending tasks, let's ensure a few more configurations and checks:

Verify Celery Beat Schedule:
Ensure that the Celery beat schedule includes the reports.scheduler task and that it is correctly configured to run at the desired intervals. Here is an example configuration:

from celery.schedules import crontab

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"
    result_backend = "redis://localhost:6379/0"
    imports = ("superset.sql_lab", "superset.tasks.scheduler")
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

Check Celery Beat Logs:
Review the Celery beat logs to ensure that the reports.scheduler task is being scheduled. Look for log entries indicating that the task is being added to the schedule.
Manual Trigger:
Manually trigger the reports.scheduler task to verify if it can run as expected. You can do this by calling the scheduler task directly using Celery's send_task method:
```
from superset.extensions import celery_app

# Trigger the scheduler task manually
celery_app.send_task('reports.scheduler')
```
Check Task Execution Logs:
Check the Celery worker logs for any errors or warnings related to the execution of the reports.scheduler task. Ensure that the task is being picked up and executed by the worker.
Superset Logs:
Review the Superset logs for any additional information or errors that might indicate why the emails and alarms are not being triggered. Look for log entries related to the reports.scheduler task.

Feature Flags:
Ensure that the necessary feature flags are enabled in your superset_config.py:

FEATURE_FLAGS = {
    "ALERT_REPORTS": True,
    "EMAIL_NOTIFICATIONS": True,
}

Dry-Run Mode:
Ensure that the ALERT_REPORTS_NOTIFICATION_DRY_RUN setting is set to False to disable dry-run mode and start receiving notifications:
```
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
```

By following these steps, you should be able to identify and resolve the issue with the reports.scheduler task not sending tasks and ensure it is correctly scheduled and executed by Celery ^[1]^[2].

To continue talking to Dosu, mention @dosu.

hswlxc · 2024-07-11T09:18:03Z

from celery.schedules import crontab

class CeleryConfig:
broker_url = "redis://localhost:6379/0"
result_backend = "redis://localhost:6379/0"
imports = ("superset.sql_lab", "superset.tasks.scheduler")
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="", hour=""),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}

CELERY_CONFIG = CeleryConfig
FEATURE_FLAGS = {
"ALERT_REPORTS": True,
"EMAIL_NOTIFICATIONS": True,
}

ALERT_REPORTS_NOTIFICATION_DRY_RUN = False

The above configuration is correct, but I found a problem. I registered multiple tasks, but only the cache_dashboard_thumbnail task was executed, and none of the other tasks were executed properly. Could you please confirm the reason for this @dosu

dosubot · 2024-07-11T11:15:59Z

Given that only the cache_dashboard_thumbnail task is being executed and other tasks like reports.scheduler are not, let's ensure a few more configurations and checks:

Task Registration:
Ensure that all tasks are correctly registered with Celery. Verify that the tasks are imported and recognized by Celery.

from superset.tasks.scheduler import scheduler, prune_log

@celery_app.task(name="reports.scheduler")
def scheduler_task():
    scheduler()

@celery_app.task(name="reports.prune_log")
def prune_log_task():
    prune_log()

Celery Worker Configuration:
Ensure that the Celery worker is configured to recognize and execute all tasks. Check the worker logs for any errors or warnings related to task registration.

Celery Beat Configuration:
Ensure that the Celery beat schedule includes all tasks and that they are correctly configured to run at the desired intervals.

from celery.schedules import crontab

class CeleryConfig:
    broker_url = "redis://localhost:6379/0"
    result_backend = "redis://localhost:6379/0"
    imports = ("superset.sql_lab", "superset.tasks.scheduler")
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

Check Celery Beat Logs:
Review the Celery beat logs to ensure that all tasks are being scheduled. Look for log entries indicating that the tasks are being added to the schedule.

Manual Trigger:
Manually trigger the tasks to verify if they can run as expected. You can do this by calling the tasks directly using Celery's send_task method:

from superset.extensions import celery_app

# Trigger the scheduler task manually
celery_app.send_task('reports.scheduler')

# Trigger the prune_log task manually
celery_app.send_task('reports.prune_log')

Check Task Execution Logs:
Check the Celery worker logs for any errors or warnings related to the execution of the tasks. Ensure that the tasks are being picked up and executed by the worker.
Superset Logs:
Review the Superset logs for any additional information or errors that might indicate why the tasks are not being executed. Look for log entries related to the tasks.

Feature Flags:
Ensure that the necessary feature flags are enabled in your superset_config.py:

FEATURE_FLAGS = {
    "ALERT_REPORTS": True,
    "EMAIL_NOTIFICATIONS": True,
}

Dry-Run Mode:
Ensure that the ALERT_REPORTS_NOTIFICATION_DRY_RUN setting is set to False to disable dry-run mode and start receiving notifications:
```
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
```

By following these steps, you should be able to identify and resolve the issue with the tasks not being executed and ensure they are correctly scheduled and executed by Celery ^[1]^[2].

To continue talking to Dosu, mention @dosu.

dosubot bot added the alert-reports Namespace | Anything related to the Alert & Reports feature label Jul 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The alarm and email sending functions of superset are unavailable #29541

The alarm and email sending functions of superset are unavailable #29541

hswlxc commented Jul 10, 2024 •

edited

Loading

hswlxc commented Jul 10, 2024

dosubot bot commented Jul 10, 2024

hswlxc commented Jul 11, 2024 •

edited

Loading

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024 •

edited

Loading

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

The alarm and email sending functions of superset are unavailable #29541

The alarm and email sending functions of superset are unavailable #29541

Comments

hswlxc commented Jul 10, 2024 • edited Loading

Bug description

How to reproduce the bug

Screenshots/recordings

Superset version

Python version

Node version

Browser

Additional context

Checklist

hswlxc commented Jul 10, 2024

dosubot bot commented Jul 10, 2024

hswlxc commented Jul 11, 2024 • edited Loading

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024 • edited Loading

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 11, 2024

dosubot bot commented Jul 11, 2024

hswlxc commented Jul 10, 2024 •

edited

Loading

hswlxc commented Jul 11, 2024 •

edited

Loading

hswlxc commented Jul 11, 2024 •

edited

Loading