Need 10 years Senior Data Engineer with strong in Databricks and Azure

[Ashwini Shukla]

Hi Vendor partner,

Highly closable position, please share the resume at your earliest convenience to - Mridul Gupta-Mri...@ustechsolutionsinc.com; Mukesh Choudhary-Muke...@ustechsolutionsinc.com; Md Shahid-Perwez Sha...@ustechsolutionsinc.com;

SR. Data Engineer (10+ Years)

Deerfield Illinois

Long Term Contract 

Description :

Client is looking for Data Engineers with Databricks experience: who have experience in Databricks. Should be capable of programming in Python, Pyspark, Scala and Spark SQL.

 

As a Data Engineer you:

• Should be able to work with other professionals to ensure successful implementation of proposed designs and completion of projects.
• Should be responsible for the development, and integration of computer programs.
• Should understand the business needs and works strictly in compliance with the needs of the client.

 

Requirements :

• Capability or programming in one of the following languages.
• Python/Pyspark
• Scala
• Spark SQL
• Deep understanding of distributed computing and Spark Architecture
• Experience with Delta Lake/Tables within Databricks
• Experience with optimizing Spark Queries and analyzing Spark DAGs
• Understanding of both Batch and Streaming Jobs in Spark
• Familiarity with different file types such as CSV, Parquet, JSON, etc.
• Experience with other Azure data services such as Azure Data Lake Gen2, Event Hubs, Synapse etc.
• Deep understanding of Azure data storage technologies such as Azure Blob Storage, Azure Data Lake Gen2, Synapse, Azure SQL, Azure Event Hub and the frameworks applied to each.
• Understanding of ETL/ELT practices including data cleansing, validation, and transformation activities.
• Data modelling experience shaping and transforming data into third normal form and dimensional models.
• Advanced understanding of best practices for structuring and organizing Data Lake file systems for large volumes of data.
• Knowledge of big-data ecosystems such as Hadoop, Hive, Sqoop, Cloudera Data Platform.