As a Data Engineer, you will be working on a number of high-profile projects and collaborating with key partners to design and build solutions that enable the business to gain insights needed for decision-making. You will be instrumental in our digital transformation by building the foundations for our cloud solutions and providing enterprise solutions to common problems. Projects will focus mainly on new development in the areas of data discovery, data quality, data integration and data movement.
Responsibilities
• Experience in Data Warehousing & Snowflake Computing platform
• Experience in AWS Glue for Creating the Ingestion process
• Experience on Python for Lambda function. Python library and data sets, Data frame exp is must. Python programing, libraries, NumPy Pandas etc.
• Exp in Data Pipeline, Step function and EMR
• Exp in Snow pipe, SnowSql
• Experience working on migrating data from on premise (Hadoop) data lake to Cloud – AWS or, Snowflake
• Experience working on different source data - RDBMS, Flat files, XML, JSON
• Expertise in SQL, especially within cloud-based data warehouses like Snowflake
• Strong understanding of ETL concepts and work experience in AWS Glue
• Should have experience on Agile methodology
• Experience on CI/CD is must
• Experience in implementation CDC, SCD-2, Data warehousing exp
• Experience on orchestration tool like Airflow is must
• Proven analytical and problem-solving skills
• Exp in Python, Spark
• Ability to work independently and on multiple tasks/initiatives with multiple deadlines
• Effective oral, presentation, and written communication skills
• Data modelling and data integration
• Support/Troubleshooting
Qualifications/Minimum qualifications
• Bachelor’s degree in Computer Science, Information Systems, Engineering, related fields
• Hands on experience with AWS Services
o AWS S3, Data Pipeline, Lambda, Step function, EMR
o SnowSql Development
o Python programing is must
• Snowflake Certification / AWS Certification preferred-Good to have
Preferred Qualifications
• Hands on experience in python & related frameworks on Linux environment
• The ability and desire to learn and adapt in complex work environment
• Outstanding communication and critical thinking skills
• Good analytical and problem solving skills and ability to balance team and client discussions.