Hi,
Please share relevant Profiles Only NO H1B NO GC GCEAD USC
LOCALS ONLY H4 TN OPT IS ALSO FINE BUT NO H1B
Sr. Data Engineer – MS Fabric, Azure
- Job Title: Sr. Data Engineer – MS Fabric, Azure
- Location: Hybrid (Chicago Office 2 days a week)
Required Skills & Qualifications
Microsoft Fabric (Must-Have)
- Fabric Data Engineering workloads
- Lakehouse & Warehouse
- OneLake architecture
- Fabric pipelines & notebooks
- Capacity planning and performance optimization
- Advanced PySpark (joins, windows, UDFs, optimization)
- Strong SparkSQL
- Strong MLOps & Feature Engineering
- Partitioning strategies, shuffle tuning, caching
- Large-scale data processing (TB+)
Azure Data Platform
- Azure SQL Database / SQL Server
- Azure Data Factory (ADF)
- SSIS / Azure-SSIS Integration Runtime
- ADLS Gen2
Data Lineage & Security (Microsoft Purview)
- Purview data catalog & scanning
- Automated lineage across ADF, Fabric, SQL, ADLS
- Business glossary management
- Integration with Azure RBAC & security policies
Role Overview
Design and build end-to-end data platforms using Microsoft Fabric, develop scalable ETL/ELT pipelines, implement MLOps for advanced analytics, and enforce data security and governance using Microsoft Purview.
Technical Core & Responsibilities
- Design and build end-to-end data platforms using Microsoft Fabric (Lakehouse, Warehouse, OneLake, Dataflows Gen2)
- Develop and optimize Spark workloads using PySpark and SparkSQL
- Develop MLOps pipelines for Advanced Analytics & AI
- Build scalable ETL/ELT pipelines using Azure Data Factory (ADF) , MS Fabric Data pipeline, Dataflow Gen2, and SSIS (on-prem, Azure-SSIS IR, and migration scenarios)
- Implement data modeling patterns: Medallion (Bronze/Silver/Gold), Dimensional modeling (Star/Snowflake), and manage data file formats (Parquet, JSON, XML)
- Integrate Microsoft Purview for data cataloging & classification and automated data lineage (ADF, Fabric, SQL, ADLS)
- Enforce data security and access controls: RBAC, column-level security, masking, and Fabric & Purview policy alignment
- Optimize performance, reliability, and cost across Fabric capacities
- Implement CI/CD and IaC for data pipelines and governance artifacts
- Partner with security, compliance, and BI teams to ensure trusted data delivery
Thanks,
_______________________________________
Aditya Jain | New York Technology Partners
120 Wood Avenue S | Suite 504 | Iselin NJ 08830
Email:
adity...@nytpcorp.com
LinkedIn
| www.nytp.com
We respect your online privacy. If you would like to be removed from our mailing list please reply with
"Remove" in the subject and we will comply immediately. We apologize for any inconvenience caused. Please let us know if you have more than one domain. The material in this e-mail is intended only for the use of the individual to whom it is addressed and may
contain information that is confidential, privileged, and exempt from disclosure under applicable law. If you are not the intended recipient, be advised that the unauthorized use, disclosure, copying, distribution, or the taking of any action in reliance on
this information is strictly prohibited.