REQ: AZURE DATA ENGINEER
Philip phili
Position: AZURE DATA ENGINEER
Location: Des Moines, IA
The Azure Data Engineer will work with the
business in understanding data requirements and will serve as a technology
& platform expert in designing and building technical solutions focused on
Microsoft Azure Data Factory, Azure SQL Datawarehouse etc. with a focus on
understanding data need for data initiatives in building scalable, distributed,
and high performance computing cloud based platform solutions on Azure then be
tasked with designing and implementing solutions for data management
improvements.
Responsibilities:
• Create functional design specifications, Azure
reference architectures, and assist with other project deliverables as needed.
• Design and Develop Platform as a Service
(PaaS) Solutions using different Azure Services
• Create a data factory, orchestrate data
processing activities in a data-driven workflow, monitor and manage the data
factory, move, transform and analyze data
• Design complex enterprise Data solutions that
utilize Azure Data Factory Create migration plans to move legacy SSIS packages
into Azure Data Factory
• Build conceptual and logical data models
• Design and implement big data real-time and
batch processing solutions
• Design, build, and scale data pipelines across
a variety of source systems and streams (internal, third-party, as well as
cloud-based), distributed / elastic environments, and downstream applications
and/or self-service solutions.
• Develop and document mechanisms for
deployment, monitoring and maintenance
Skills and Experience:
• Bachelor's degree or higher in Computer
Science Engineering/ Information Technology, Information Systems
• 3+ years experience with Microsoft Cloud Data
Platform: Azure Data Factory, Azure Databricks, Python, Scala,Spark SQL, SQL
Data Warehouse
• 3+ years of experience in developing data
ingestion, data processing and analytical pipelines for big data, relational
databases, NoSQL, data lake solutions
• Expertise with SQL, database
design/structures, ETL/ELT design patterns, and datamart structures (star,
snowflake schemas, etc.)
• Functional knowledge of programming scripting
and data science languages such as JavaScript, PowerShell, Python, Bash, SQL,
.NET, Java, PHP, Ruby, PERL, C++, R, etc
• Creation of descriptive, predictive and
prescriptive analytics solutions using Azure Stream Analytics, Azure Analysis
Services, Data Lake Analytics, HDInsight, HDP, Spark, Databricks, MapReduce,
Pig, Hive, Tez, SSAS, Watson Analytics, SPSSA
• Experience in Azure Data Factory (ADF)
creating multiple pipelines and activities using Azure for full and incremental
data loads into Azure Data Lake Store and Azure SQL DW
• Experience for Azure Data Lake Storage and
working with Parquet files and partitions
• Experience managing Microsoft Azure
environments with VM’s, VNETS, Subnets, NSG’s, Resource Groups, etc.
• Experience in Creation & Configuration of
Azure Resources & RBAC
• Experience with Git/Azure DevOps
• Azure certification would be desired
• Must have an ability to communicate clearly
and be a team player
Regards
Philip