Prefect 2 - current working directory changes

[chandruexf]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the Prefect documentation for this issue.
• I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

os.getcwd() I used this line in the task , what my expectation is /home/chandru/prefectver2/ed-2-app/app/ to give this path, but I got the result in logger /tmp/tmpa97l021qprefect some other path and not able to overcome this with the deployment.

Reproduction

deployment = Deployment.build_from_flow(
        flow=create_new_flow,  # replace with flow name
        name=user_tenant,  #replace with deployment name
        work_queue_name="test-queue",  # replace with agent name
        work_pool_name = "test-pool",
        path='/home/chandru/prefectver2/ed-2-app/app/',
        entrypoint='/home/chandru/prefectver2/ed-2-app/app/project/components/exfinsights/data_quality/prefect_layer2/tasks.py:create_new_flow', # replace with "flow file name : flow name"
        # schedule=IntervalSchedule(interval=timedelta,anchor_date=datetime.datetime.now(),timezone="UTC"),
        # parameters={"job_params":{
        #     "group_name":test_group_name,
        #     "user_tenant":user_tenant,
        #     "selected_version_data":selected_version_data
        # }}
    )

my deployment

Error

os.getcwd() I used this line in the task , what my expectation is /home/chandru/prefectver2/ed-2-app/app/ to give this path, but I got the result in logger /tmp/tmpa97l021qprefect some other path and not able to overcome this with the deployment.

Versions (prefect version output)

my op need to be /home/chandru/prefectver2/ed-2-app/app/  system path

Additional context

No response

[zhen0]

Hi @chandruexf - thanks for the issue. Can I ask that you provide a full MRE and prefect version for us to be able to look into this further? It really helps us know if you are on a version where an issue has already been fixed and makes sure we're looking into the same thing as you are reporting.

You also look to be using the older build_from_flow way of creating a deployment - it might be worth checking out flow.deploy to create your MRE.

[chandruexf]

Hi Jenny, name : prefect version : 2.19.6 description : Workflow orchestration and management. I am working on this prefect version and /tmp/somename setting has its current working directory,

…

On Mon, 1 Jul 2024 at 20:30, Jenny G ***@***.***> wrote: Hi @chandruexf <https://github.com/chandruexf> - thanks for the issue. Can I ask that you provide a full MRE and prefect version for us to be able to look into this further? It really helps us know if you are on a version where an issue has already been fixed and makes sure we're looking into the same thing as you are reporting. You also look to be using the older build_from_flow way of creating a deployment - it might be worth checking out flow.deploy <https://docs.prefect.io/latest/guides/upgrade-guide-agents-to-workers/> to create your MRE. — Reply to this email directly, view it on GitHub <#14433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7KFZUY7XTBLCCIPVGGKJP3ZKFVH3AVCNFSM6AAAAABKFQXG6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBQGM4TQNJTG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[zhen0]

Hi @chandruexf - Here's some guidance for what I'm looking for when I ask for an MRE.

For the prefect version, can you please provide all of the information given when running the prefect version command?

As we say in our issue template:
Provide information about your Prefect version and environment. The easiest way to retrieve all of the information we require is the prefect version command. If using Prefect 1.x, it is useful to also include the output of prefect diagnostics. Please do not just include your Prefect version number. The command provides additional context such as your operating system, Prefect API type, Python version, etc. that we need to diagnose your problem.

[chandruexf]

Version: 2.19.7
API version: 0.8.4
Python version: 3.10.12
Git commit: 60f0512
Built: Fri, Jun 28, 2024 11:27 AM
OS/Arch: linux/x86_64
Profile: default
Server type: ephemeral
Server:
Database: sqlite
SQLite version: 3.37.2

I have provided full info of the prefect version

[chandruexf]

Hi Jenny, Some how I am able to all the tasks using the prefect 2. I need your help on. - When I try to use os.getcwd() I am getting default path which should not happen. /tmp/tmpz2orrx79prefect dyname=dynamic_flname(prefect_flow_name) deployment = Deployment.build_from_flow( flow = dyname, name=user_tenant, work_queue_name="test-queue", work_pool_name = "test-pool", path=f'{str(os.getcwd())}/', # entrypoint=f'{str(os.getcwd())}/project/components/exfinsights/data_quality/prefect_layer2/tasks.py:exfdq_app_wf', # replace with "flow file name : flow name" # schedule=IntervalSchedule(interval=timedelta,anchor_date=datetime.datetime.now(),timezone="UTC"), parameters = { "group_name" : test_group_name, "user_tenant" : user_tenant, "selected_version_data" : selected_version_data, "user_name" : user_name } ) deployment_id = deployment.apply(upload=True,) My deployment method that is used What should i include to overcome this current directory. Thanks chandru

…

On Tue, 2 Jul 2024 at 19:11, Jenny G ***@***.***> wrote: Hi @chandruexf <https://github.com/chandruexf> - Here's some guidance <https://stackoverflow.com/help/minimal-reproducible-example> for what I'm looking for when I ask for an MRE. For the prefect version, can you please provide all of the information given when running the prefect version command? As we say in our issue template: *Provide information about your Prefect version and environment. The easiest way to retrieve all of the information we require is the prefect version command. If using Prefect 1.x, it is useful to also include the output of prefect diagnostics. Please do not just include your Prefect version number. The command provides additional context such as your operating system, Prefect API type, Python version, etc. that we need to diagnose your problem.* — Reply to this email directly, view it on GitHub <#14433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7KFZUZKJSL7LYIYN647BB3ZKKUYRAVCNFSM6AAAAABKFQXG6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTGIYDCMJZGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[zhen0]

Hi @chandruexf - I've converted to a discussion. It sounds like you want to get the path of your flow code? Can you use path? I'm sorry I can't help further but I'm not fully understanding your request.

[chandruexf]

Sorry if I am not clear, I Deployed a flow. And inside a task I utilized os.getcwd() to fetch the current running directory but it returns me /tmp/tmpa97l021/
which is wrong , my home directory is different I am trying to figure that out how to set os.getcwd to my home path.

Example :
prefect give me /tmp/tmpa97l021/
Required output
/home/chandru/prefectver2/

How to achieve this.