Hiring Senior Data Engineer
Abhinav Mohanty
Hello Everyone,
Please share suitable profiles.
Don’t call me, Once I review the profile will give you a call.
|
If you are sharing any profile, please mention: Location – Work Authorization – "Before submitting any candidates please share the visa back and front copy must and LinkedIn id " |
Role: Senior Data Engineer
Location: Milpitas, CA (Hybrid)
Job Description:
These are skills we are looking
Specialization: Azure | Databricks | PySpark | Python | SQL | Data Warehousing | Data Products
Core Technical Expertise
Azure Cloud Platform:
Microsoft Azure (ADLS Gen2, ADF, Synapse, Azure SQL, Key Vault, App Services)
Cloud-native architecture design & cost optimization
Security & governance (RBAC, Managed Identity, Private Endpoints)
Databricks Engineering:
Azure Databricks end-to-end development
Lakehouse architecture implementation
Delta Live Tables (DLT) & Unity Catalog
Performance tuning & cluster optimization
CI/CD for Databricks workloads
Big Data & Processing Frameworks:
Apache Spark (PySpark advanced transformations, optimization)
Structured Streaming & batch pipelines
Delta Lake architecture & optimization techniques
Programming & Querying:
Python (advanced data engineering & automation)
PySpark (distributed data processing)
SQL (complex query optimization, performance tuning, analytics engineering)
Data Architecture & Engineering
Enterprise Data Warehouse Architecture (EDW)
Dimensional modeling (Star/Snowflake schemas)
Data Vault 2.0 modeling
Metadata-driven ingestion frameworks
CDC implementation (Change Data Capture)
Medallion architecture (Bronze/Silver/Gold layers)
Data lineage, governance, and cataloging
Master Data Management (MDM)
Data Products & Analytics Enablement
Design & delivery of scalable data products
Business-aligned semantic layer design
KPI frameworks & enterprise reporting enablement
Integration of ERP, SaaS, and operational systems
Lakehouse + EDW hybrid architecture
DevOps & Engineering Practices
CI/CD pipelines (Azure DevOps, GitHub Actions, Bitbucket)
Infrastructure as Code (Terraform, ARM templates)
Automated testing (unit, integration, data quality)
Monitoring & observability (logging, alerting)
Agile & Scrum Methodologies
Strategic Impact
Principal-level solution architecture design
Cross-functional stakeholder engagement
Technical roadmap Strategies and delivery
Advanced Competencies
Performance optimization for large-scale distributed systems
Data governance frameworks (GDPR, SOX compliance alignment)
Cost optimization strategies in cloud data platforms
Migration from legacy EDW (e.g., Teradata, Oracle, SQL Server) to Lakehouse