Hadoop Developer with Azure @ Deerfield, IL
Sekhar S
Hi ,
Please share your profile to adi...@torquetek.com
Hadoop Developer with Azure
Deerfield, IL
Long Term
Above 8+ Years profile required
Mandatory:
Python, Pyspark, Scala and Spark SQL.
Requirements :
· Capability or programming in one of the following languages.
o Python/Pyspark
o Scala
o Spark SQL
· Deep understanding of distributed computing and Spark Architecture
· Experience with Delta Lake/Tables within Databricks
· Experience with optimizing Spark Queries and analyzing Spark DAGs
· Understanding of both Batch and Streaming Jobs in Spark
· Familiarity with different file types such as CSV, Parquet, JSON, etc.
· Experience with other Azure data services such as Azure Data Lake Gen2, Event Hubs, Synapse etc.
· Deep understanding of Azure data storage technologies such as Azure Blob Storage, Azure Data Lake Gen2, Synapse, Azure SQL, Azure Event Hub and the frameworks applied to each.
· Understanding of ETL/ELT practices including data cleansing, validation, and transformation activities.
· Data modelling experience shaping and transforming data into third normal form and dimensional models.
· Advanced understanding of best practices for structuring and organizing Data Lake file systems for large volumes of data.
· Knowledge of big-data ecosystems such as Hadoop, Hive, Sqoop, Cloudera Data Platform.
Responsibilities :
As a Data Engineer you:
· Should be able to work with other professionals to ensure successful implementation of proposed designs and completion of projects.
· Should be responsible for the development, and integration of computer programs.
· Should understand the business needs and works strictly in compliance with the needs of the client.
Thanks & Regards
Aditya
IT Recruiter
|
Torque Technologies LLC A |
Torque Technologies LLC is an Equal Opportunity Employer (EOE). Qualified applicants are considered for employment without regard to age, race, color, religion, sex, national origin, sexual orientation, disability, or veteran status.
Applicants
In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and complete required employment eligibility verification documentation upon hire.