Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 69%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKN%3C/text%3E%3C/svg%3E)
Hello @Saurabh Sharma, I'm experiencing the same problem as THIMMAIAH GARI,PRASHANTH. Can you help me?
Pipeline Activities are running very slow
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETG%3C/text%3E%3C/svg%3E)
THIMMAIAH GARI,PRASHANTH,,
201
Reputation points
Hi..Earlier pipelines were running an average of 3min...Now same pipelineS and same data load taking hours to execute. I have tested procedures in SQL DB they are running fine. I think the issue might be with self-hosted IR. Please find the below IR's Report ID and check the logs.
copyacitvty, log update procedure, procedures running for long time
Report ID: b6db227f-f910-4817-8619-d562ebae4662
Azure Data Factory
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 44%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator2021-01-26T01:01:37.1+00:00 @THIMMAIAH GARI,PRASHANTH,, Thanks for using Microsoft Q&A !!
Can you please provide pipeline run id as we do not have any access to logs as those logs are accessible by Azure Support team. I can try though with Pipeline Run id for any clues. Thanks.
-
THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
2021-01-27T05:54:40.167+00:00 Pipeline Runid : f4292b82-2b87-49da-935a-91e3a3b7edb2
-
Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
2021-01-28T05:30:09.053+00:00 From backend activity logs, your lookup which uses your shared integration runtime is taking more time.
I can see the below activities took more time -
- Activity name: LOOKUP_ON_TABLE_METADATA Duration: 00:19:05.7636600 (which is using your shared integration runtime)
- Activity Execute_STG_TO_TRGT_DB pipeline took approximately 2 hours for 3 times
- Activity: LOOKUP_SET_LAST_LOAD_DTTM took approximately 41 minutes, 3 times
I will check further and update you if able to find more details from the backend. Thanks
-
THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
2021-01-28T06:58:45.343+00:00 please share the details of what was the issue with IR?...so I can check with my team
-
Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
2021-01-28T18:58:14.613+00:00 @THIMMAIAH GARI,PRASHANTH,, I believe it is network issue as logs has multiple instance of "Unable to connect to remote server". I am sending you IP separately of the host for which it looks like failing.
DEBUG:
Unable to connect to the remote server
Unable to connect to the remote server
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond <IP Address> -
Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
2021-02-01T16:09:48.107+00:00 @THIMMAIAH GARI,PRASHANTH,, I have not heard back from you. Are you able to fix your pipeline issue.
-
THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
2021-02-02T08:50:06.18+00:00 I have informed to my netwrok team....they are working on that I will let you know if any additional help required.
-
Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
2021-02-10T18:59:19.757+00:00 @THIMMAIAH GARI,PRASHANTH,, Is your issue resolved or do you need any further help ?
-
THIMMAIAH GARI,PRASHANTH,, 201 Reputation points
2021-02-11T05:30:35.587+00:00 my team is still investigating and will get back to you..If we need more help
Sign in to comment