Hiring! -- Databricks Lead/Architecture - Washington DC ( Remote)

[D Rao]

Please share resumes to durgara...@allwyncorp.com 

EXP = 14+ years (Must)

Job Title: Databricks Lead/Architecture

Location: Washington DC ( Remote)

 

Architecture & Design

 

• Define and design end-to-end data lakehouse architecture leveraging Databricks, Delta Lake, and cloud-native services.
• Create reference architectures for batch, real-time, and streaming data pipelines.
• Architect data ingestion, curation, storage, and governance frameworks.
• Ensure the platform is scalable, secure, and optimized for performance and cost.
• Establish standards for data lineage, metadata management, and compliance.
• Work with enterprise architects to align Databricks solutions with overall cloud strategy (AWS).
• Leadership & Delivery
• Lead and mentor a team of data engineers in building robust data pipelines.
• Collaborate with data scientists, BI teams, and business stakeholders to enable advanced analytics and AI/ML use cases.
• Drive adoption of DevOps and CI/CD practices for data engineering.
• Review designs and solutions to ensure adherence to architectural principles.
• Implementation & Optimization
• Build, optimize, and manage large-scale PySpark/SQL pipelines in Databricks.
• Enable real-time data processing through Kafka, Kinesis, or Event Hubs.
• Implement security best practices including RBAC, data masking, and encryption.