C2C || Azure Databricks Architect || Onsite in Plano
Vishal Kumar
Hope you are doing well!!
Greetings from Arkhya Tech Inc. My name is Vishal Kumar and I’m a Technical Recruiter at Arkhya
Please find the job requirements below and let me know if you are available for the same.
Title: Azure Databricks Architect
Location: Plano , TX (Onsite)
Job Type: Contract
Job Summary:
We are seeking an experienced Azure Databricks Architect to lead the design, development, and implementation of scalable and high-performing data solutions using Azure cloud services, Databricks, and the broader Azure ecosystem. The ideal candidate will have a strong background in big data architecture, Spark-based ETL pipelines, cloud security, and performance tuning.
Key Responsibilities:
· Architect and design end-to-end data pipelines on Azure Databricks, including ingestion, transformation, and serving layers.
· Lead migration of legacy data platforms to Azure-based lakehouse architectures using Delta Lake, ADLS Gen2, and Synapse.
· Implement and optimize large-scale ETL/ELT workflows using Apache Spark, PySpark, and Databricks notebooks.
· Define and enforce data governance, lineage, and security best practices using Unity Catalog, Purview, RBAC, and ABAC.
· Collaborate with business stakeholders, data engineers, data scientists, and DevOps teams to translate business requirements into scalable solutions.
· Enable CI/CD pipelines and automation for Databricks using Terraform, GitHub Actions, or Azure DevOps.
· Guide performance tuning, cost optimization, and monitoring strategies for large-scale data processing jobs.
· Create architecture diagrams, documentation, and knowledge-sharing artifacts.
Required Skills & Experience:
· 10+ years of experience in data engineering, with 4+ years in Azure cloud ecosystem.
· Strong expertise in Azure Databricks, Delta Lake, Spark (PySpark/Scala).
· Deep understanding of Azure Data Lake (Gen2), ADF, Synapse Analytics, Azure SQL, Key Vault, and Azure Monitor.
· Experience with DevOps practices: CI/CD pipelines, Infrastructure as Code (Terraform, ARM, Bicep), Git.
· Hands-on with Unity Catalog, data lineage, audit logging, and access control models.
· Familiarity with data modeling, data mesh, lakehouse, and MPP-based architectures.
· Strong problem-solving, leadership, and communication skills.
Preferred Qualifications:
· Azure or Databricks certifications (e.g., Azure Data Engineer, Databricks Certified Data Engineer Professional).
· Experience integrating with Power BI, MLflow, Azure ML, or Snowflake.
· Familiarity with event-driven architecture (Kafka/Event Hub) and streaming data pipelines.
· Exposure to ML/AI workloads on Databricks.
|
Sent with Mailsuite · Unsubscribe |
08/07/25, 09:00:37 PM |
