Ideal Folder Structure for Composer Bucket
26 views
Skip to first unread message
Vishal Verma
Apr 1, 2025, 4:21:25 AMApr 1
to cloud-composer-discuss
I'm working on organizing the ideal folder structure for Composer bucket. Currently, I have YML config files, utils, and several PySpark code files that I run on Dataproc Serverless through DAGs.
Do you have any recommendations on whether I should distribute these files between dags and plugins folders, or if another structure would be more appropriate considering performance?
I gone through this doc - https://cloud.google.com/composer/docs/composer-3/cloud-storage
Do you have any recommendations on whether I should distribute these files between dags and plugins folders, or if another structure would be more appropriate considering performance?
I gone through this doc - https://cloud.google.com/composer/docs/composer-3/cloud-storage
Reply all
Reply to author
Forward
0 new messages