Reg: SR. Data Engineer || Min 10 years of exp
Srikanth HCL Global
Role: SR. Data Engineer
Location: Deerfield, Illinois
Looking for Data
Engineers with Databricks experience: who have experience in Databricks. Should
be capable of programming in Python, Pyspark, Scala and Spark SQL.
Responsibilities:
As a Data Engineer you:
· Should be able to work with
other professionals to ensure successful implementation of proposed designs and
completion of projects.
· Should be responsible for the development, and
integration of computer programs.
· Should understand the business needs and works
strictly in compliance with the needs of the client.
Requirements:
· Capability or programming in one of the
following languages.
o Python/Pyspark
o Scala
o Spark SQL
· Deep understanding of distributed computing and
Spark Architecture
· Experience with Delta Lake/Tables within
Databricks
· Experience with optimizing Spark Queries and
analyzing Spark DAGs
· Understanding of both Batch and Streaming Jobs
in Spark
· Familiarity with different file types such as
CSV, Parquet, JSON, etc.
· Experience with other Azure data services such
as Azure Data Lake Gen2, Event Hubs, Synapse etc.
· Deep understanding of Azure data storage
technologies such as Azure Blob Storage, Azure Data Lake Gen2, Synapse, Azure
SQL, Azure Event Hub and the frameworks applied to each.
· Understanding of ETL/ELT practices including
data cleansing, validation, and transformation activities.
· Data modelling experience shaping and
transforming data into third normal form and dimensional models.
· Advanced understanding of best practices for
structuring and organizing Data Lake file systems for large volumes of data.
· Knowledge of big-data ecosystems such as
Hadoop, Hive, Sqoop, Cloudera Data Platform.
Thanks & Regards,
Srikanth | Technical Recruiter
HCL Global Systems, Inc.
Direct: 248-473-3868
Desk: 248-473-0720 Ext: 179
Email ID: S...@hclglobal.com
Certified Minority Business Enterprise (MBE)