加载中...
Skip to main content
11,944 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

We are planning airflow upgrade from 2.7 to 3.0.6. Here we are triggering external dag it is refreshed and having the expected tasks. with 2.6 version, there was direct access to database but 3.0.7 ...
Sudhanshu Prakash's user avatar
  • 9
1 vote
1 answer
9k views

We recently upgraded from Airflow 1.10.12 to Airflow 2.1.0 and are getting issues errors like this in the UI for each of our dags. We have upgraded the version from 1.10.12 to 1.10.15 and it was ...
Community's user avatar
  • 1
0 votes
2 answers
2k views

I want to connect from composer to a VM with ssh --internal-ip, but I get a permission error. I tested ssh with VM and composer using the same VPC / subnet and it looks like this. What are the ...
Peter Mortensen's user avatar
  • 31.2k
7 votes
1 answer
4k views

I am looking for support to debug this Airflow KubernetesPodOperator Issue. We randomly get this error when the Airflow task executes. The job is almost finished and at the end of the job execution, ...
Andrei_N's user avatar
  • 436
0 votes
0 answers
34 views

I'm trying to figure out how to successfully run dag with SparkSubmitOperator on Airflow 3.1.5, I have a wrapper which sets pod config: self.executor_config = { "pod_override": ...
twierdzenie twierdzenie's user avatar
  • 1
1 vote
4 answers
3k views

I need to poll DB2 database table until a record is created and once it is successful,i need to trigger a task in dag. For this I have to do below: Create a DB2 connection in airflow Use SQL sensor ...
Ebrahim Karam's user avatar
  • 1,001
0 votes
0 answers
54 views

I have created custom Airflow operator to trigger some of my api when the job submits. Everything works fine but one problem is in Airflow UI we have an option to mark state as success, when a user ...
Alex Man's user avatar
  • 4,946
2 votes
1 answer
548 views

I want to periodically delete old XCom records (e.g. older than 15 days) from inside a DAG. In Airflow 2 I could connect directly to the metadata DB, but in Airflow 3 direct DB access is no longer ...
ale's user avatar
  • 227
1 vote
0 answers
71 views

I am migrating a dag and its queries to a new DB using DBT. The problem is that one of the existing tables has an EVENT_ID column, defined as autoincrement, but when the queries try to insert data, ...
marc_s's user avatar
  • 760k
0 votes
1 answer
49 views

Airflow Docs say that the LocalExecutor supports an "unlimited Parallelism". Source They do not specify exactly how to run LocalExecutor in this mode, but I assume it is through the core....
raphaelauv's user avatar
  • 1,054
3 votes
3 answers
3k views

I have created an AWS MWAA environment and correctly pointed it to an S3 bucket as well as an "dags" folder inside the S3 bucket. I have created a simple dag file (python) and added it to my ...
tomcheney's user avatar
  • 1,821
5 votes
3 answers
5k views

I cannot find any information on what the owner variable is used for in the docs. The only mention of owner I found was in this Stack Overflow question and in the Security section of the docs, both of ...
Jeremy Caney's user avatar
  • 7,797
0 votes
1 answer
40 views

Is there anyone who has configured LDAP or Active Directory in Airflow? When I try to set things inside webserver_config.py, it doesn’t work. [core] auth_manager = airflow.providers.fab.auth_manager....
Duurenjargal Davaadorj's user avatar
  • 1

15 30 50 per page
1
2 3 4 5
…
797