Title: Big Data/Data Engineer
Location: Plano, TX
Duration: 6-12+ Months
Minimum Requirements:
At least 3 years of experience developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python
At least 2 years experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC etc..)
At least 3 years of developing applications with Monitoring, Build Tools, Version Control, Unit Test, TDD, Change Management to support DevOps
At least 3 years of experience with SQL and Shell Scripting experience
At least 2 years of experience with software design and must have an understanding of cross systems usage and impact
Nice to Have qualifications: 2+ years of experience working with Dimensional Data Model and pipelines in relation with the same
2+ years experience with Amazon Web Services (AWS), Microsoft Azure or another public cloud service
2+ years of experience working with Streaming using Spark or Flink or Kafka or NoSQL Intermediate level experience/knowledge in at least one scripting language (Python, Perl, JavaScript) Hands on design experience with data pipelines, joining data between structured and unstructured data