Lead Data Engineer (AWS Big Data Kafka) in Santa Clara, CA

1 view
Skip to first unread message

Sanjiv Singh

unread,
Jun 30, 2020, 9:39:21 AM6/30/20
to ssa...@andeogroup.net

Lead Data Engineer (AWS Big Data Kafka)

6 Months Contract

Santa Clara, CA

USC, GC, TN only

 

Key Skills: AWS & Kafka

 

Primary Need-

A candidate who has worked on Data Pipelines on AWS.

Used AWS Big Data Stack - Kinesis, MKS, Redshift, Athena etc

Exp in Kakfa a must.    

 

About the role: 

Looking for experienced Lead Data Solution Engineer who has built data pipelines and data systems at scale using the Apache Open Source stack and the Hadoop ecosystem. They should have strong familiarity working in a AWS cloud environment. Comfortable working with data engineers, product managers and product delivery teams. 

Key Responsibilities

  • Provide technical solution leadership in data engineering team, driving technology decisions, mentoring others, and contributing significantly on an individual level
  • Build frameworks to handle data at high scale using Apache Spark and data cataloging tools like Apache Hive, AWS Glue on top of a multi-tiered data lake storage
  • Use exploration and analytic tools like AWS Athena/Presto to probe and validate data
  • Build robust data processing pipelines using AWS Services and integrate with multiple data sources
  • Collaborate with product owners and stakeholders to plan and define requirements

Experience with the following software/tools is highly desired

    1. AWS Services: RDS, AWS Lambda, AWS Glue, Apache Spark, Kafka, Hive, etc
    2. SQL and NoSQL databases like MySQL, Postgres, Elasticsearch
    3. AWS EMR 
    4. Familiarity with Spark programming paradigms (batch and stream-processing)
    5. Strong programming skills in at least one of the following languages: Java, Scala. Familiarity with a scripting language like Python as well as Unix/Linux shells
    6. AWS Athena
    7. Strong analytical skills and advanced SQL knowledge, indexing, query optimization techniques.
    8. Good to have ETL skills
    9. Ability to translate data needs into detailed functional and technical designs for development, testing and implementation
    10. Ability to serve as a liaison between technical, quality assurance and non-technical stakeholders throughout the development and deployment process

Qualifications & Experience

Candidates with 8+ years’ experience in data engineering, who have either obtained a Graduate degree in the field of Computer Science or related field, or Bachelor's degree with 8+ years of relevant experience in the above fields.



Regards,

Sanjiv Singh | ssa...@andeogroup.net

Business Development Manager | Recruitment Specialist

T: (301) 804-0308

JOIN MY GROUP:-
https://groups.google.com/forum/#!forum/c2cc2h-reqs-available
JOIN HANGOUT:-
https://hangouts.google.com/group/wku13Sw4GcvqSEuM7

Reply all
Reply to author
Forward
0 new messages