Hiring for Spark/Kafka Bigdata Engineer in Iowa City, IA

[kailash singh]

Role                       :               Big Data Engineer (Spark, Kafka, Java)

Location               :               Iowa City, IA

Duration              :               12 Months

Interview            :               Skype

No H1B

 

Skill: Kafka, Spark, Python, Java

 

**REAL TIME DATA STREAMING EXPERIENCE VIA KAFKA, BIG DATA, SQL, AND SPARK**

 

Focusing on the Following Technologies:

1.       Spark

2.       Python

3.       Java/Kafka – will be using Kafka for pulling Data

4.       ETL Environments

5.       Database Knowledge – Relation and Non-Relational – Mongo, Postgres and MySQL

6.       JSON

They are in the midst of transformation  of their Information Architecture.  They’re moving to more streaming of data Architecture compared to Batch oriented and Flat Files.

 

These resources will be working on a project called Blue Lake.

                -This is based on Data Bricks – Spark based platform

                -Majority of work is within Python

                                -Primarily focused on Statistic Generation in a Live Streaming sort of way

                                -Consolidating dating statistical generation and automation

                                -A lot of it will be ETL, Info will be coming in from Streaming    

                -Main candidate would be with Python and Spark Platform experience and Data Brick (if possible).  ETL and Analytical and Statistical Work

                -Will be using Integration and Regression Testing, Performance and Scale Testing

 

They’re using Kafka for the live streaming of the data and also using Kubernetes as well.

They have Data Bricks as a Platform and are looking for resources who need to look at how they take the data streaming and put it into a data Structure to be able to move it back and forth.

They are using multiple Databased that include MySQL or a Postgres or Mongo.

Using Scaled Agile with 2 week sprints. 

Use DevOps practices as well, Use Jenkins and have an Automated Pipeline, and all on AWS as a Cloud Platform.

They want to bridge a Path from getting the Data from the repositories and into the Analytics Framework.

Java Development is what the development is built on and they would prefer to have someone with Kafka experience. 

Working on project called Blue Lake. Job Posting has been posted by Nick Parker.

                -This is based on Data Bricks – Spark based platform

                -majority of work is within Python

                -next Program Increments – 6+ Months

                                -Primarily focused on Statistic Generation in a Live Streaming sort of way

                                -Consolidating dating stat generation and automation

                                -A lot of it will be ETL, Info will be coming in from Streaming    

                -Main candidate would be with Python and Spark Platform experience and Data Brick (if possible).  ETL and Analytical and Statistical Work

                -Will be using Integration and Regression Testing, Performance and Scale Testing

 

 

 

 

Regards ,

Kailash Negi |  KPG99,INC

3240 E State, St Ext |  Hamilton, NJ 08619 
609-681-2603  ||   k...@kpgtech.com

Certified Minority Business Enterprise (MBE)

Working together is a Success!!!

NOTE: Email is the best way to reach me if i am not available at my number.