Comments (10)
tirkarthi
commented on June 18, 2024
1
Can you please post the dag or a minimal example to reproduce this?
from airflow.
tirkarthi
commented on June 18, 2024
1
I am unable to reproduce this locally on main branch with the given dag code. The dag is scheduled hourly for me. Since this is a common cron expression I am wondering if there is something else wrong specific to the installation.
python -m airflow dags details gh39435
property_name | property_value
================================+=======================================================================================
dag_id | gh39435
dag_display_name | gh39435
root_dag_id | None
is_paused | False
is_active | True
is_subdag | False
last_parsed_time | 2024-05-06T16:26:38.541571+00:00
last_pickled | None
last_expired | None
scheduler_lock | None
pickle_id | None
default_view | grid
fileloc | /home/karthikeyan/airflow/dags/gh39435.py
file_token | Ii9ob21lL2thcnRoaWtleWFuL2FpcmZsb3cvZGFncy9naDM5NDM1LnB5Ig._hmr8O0KN7-ndOioQCJjf0dqsfo
owners | ggomiz
description | None
schedule_interval | {'__type': 'CronExpression', 'value': '0 * * * *'}
timetable_description | Every hour
tags | {'name': 'dbservers'},{'name': 'redis'},{'name': 'backups'}
max_active_tasks | 16
max_active_runs | 16
max_consecutive_failed_dag_runs | 0
has_task_concurrency_limits | False
has_import_errors | False
next_dagrun | 2024-05-06T16:00:00+00:00
next_dagrun_data_interval_start | 2024-05-06T16:00:00+00:00
next_dagrun_data_interval_end | 2024-05-06T17:00:00+00:00
next_dagrun_create_after | 2024-05-06T17:00:00+00:00
from airflow.
GabrielGomiz
commented on June 18, 2024
1
@Taragolis, setting TZ env var to "America/Buenos_Aires", as described in #37869 worked perfectly.
I've had already in docker compose file set the following env vars:
AIRFLOW__CORE__DEFAULT_TIMEZONE: 'America/Buenos_Aires'
AIRFLOW__WEBSERVER__DEFAULT_UI_TIMEZONE: 'America/Buenos_Aires'
but didn't know that also must set TZ env var.
Weird thing is that on airflow version 2.7.3 worked ok without setting TZ env var.
Thanks for your help!
from airflow.
GabrielGomiz
commented on June 18, 2024
#!/usr/bin/python3
from airflow import DAG
from datetime import datetime
from airflow.operators.bash import BashOperator
from airflow.hooks.base import BaseHook
default_args = {
'owner': 'ggomiz',
'depends_on_past': False,
'start_date': datetime(2024,5,6),
'email': ['[email protected]'],
'email_on_failure': True,
'email_on_retry': False
}
# hourly redis dag
dag_hourly = DAG(dag_id='redis_test', default_args=default_args, schedule='0 * * * *', catchup=False, tags=["dbservers", "backups", "redis"])
task_args = {
'retries': 0,
'trigger_rule': 'all_done'
}
# hourly backup task
task_daily = BashOperator(task_id="test_redis_master",
bash_command="cronjob.sh -q -u root /srv/redis/scripts/redisbackup.sh master",
dag=dag_hourly, queue = "galadriel", default_args=task_args)
This minimal DAG to make an hourly backup of our redis database is getting incorrect schedule.
from airflow.
GabrielGomiz
commented on June 18, 2024
As additional information, I've several other DAGs with hourly schedule definition, but also are incorrectly scheduled on a 4 hour interval, but they are not executing. Last run time is before the upgrade.
from airflow.
Taragolis
commented on June 18, 2024
@GabrielGomiz Could you provide also information about your timezones configurations: in OS and in Airflow Configuration?
from airflow.
GabrielGomiz
commented on June 18, 2024
@GabrielGomiz Could you provide also information about your timezones configurations: in OS and in Airflow Configuration?
Ok. We're in Argentina, so timezone is -3 ART as you can see in the timestamps above: "2024-05-06T12:00:00-03:00"
Inside the docker containers TZ is -3 ART also:
[/opt/airflow] [docker] airflow > date
Mon May 6 18:52:14 -03 2024
from airflow.
Taragolis
commented on June 18, 2024
I more interested in values which are setting up, not a result. Invalid configuration may lead to some errors, e.g. Airflow expected to work with IANA timezones, if OS misconfigured it also might return invalid values.
from airflow.
Taragolis
commented on June 18, 2024
This might be similar to this one #37869 (comment) or it could be something new and different, without understand what is your setup it is hard to guess what is going wrong here, because I could also confirm the fact, that with default configurations it is works without any issues.
from airflow.
GabrielGomiz
commented on June 18, 2024
Ok, I'll look into timezone configuration and report back asap. Thanks @Taragolis
from airflow.
Related Issues (20)
- Starttls error when it's disabled in config HOT 2
- @hookimpl on_dag_run_running, on_dag_run_success, on_dag_run_failed do not find Connections and Variables HOT 1
- Params Fail to Evaluate In Dag Body HOT 6
- The Contributor's Quick Start isn't quick enough and needs some TLC HOT 5
- Airflow 2.8.1: awscli installation hindered by version conflicts in requirements.txt HOT 2
- Bad rendering of an inline code in the documentation HOT 1
- Add median duration markline to dag run duration page HOT 1
- DAG list: filter FAILED/RUNNING HOT 3
- Output tasks doc in the UI HOT 3
- GenericTransfer - Warning: "Placeholder defined in Connection..." HOT 4
- Dynamically Mapped Tasks: DB performance issues HOT 16
- KPO - get_logs - logs missing HOT 6
- EksPodOperator error with deferrable=True HOT 1
- Difference between logs which stores in files and logs in UI HOT 5
- KubernetesPodOperator with multiple containers hangs if container other than base container is still running HOT 3
- Unable to open AirFlow webserver UI HOT 1
- data to long when using s3 to gcs HOT 1
- Audit task failed-deps for later investigation HOT 2
- Executor reports task instance (...) finished (failed) although the task says it's queued HOT 12
- Deployment: Airflow 2.9.1 Installed on Kubernetes via Helm Chart - Opensearch Connection Issue (Error 406) HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.