Big
Data Development Lead
NYC, NY
2-3
days on site
Any
visa is fine but not H1B and CPT
Long Term Consulting Assignment
JD:
As a Big
Data Platform Engineer you will be responsible for our Big Data platform's
stability and the design and development of core functionality and strategic
solutions that utilize big data technologies in order to meet our constantly
changing business requirements. This includes the development of reusable
tooling / APIs, an efficient entitlements models, and software stack to
accelerate our current big data development and analytics solutions.
Responsibilities
include:
Designing
and developing strategic technology solutions on Hadoop based on business
requirement
- Collaboration
within the organization on current and future state architecture
- Hands-on
development working within a global development
- Monitoring
platform processes and activities across various analytics teams to ensure
platform stability
- Coordination
with our DevOps teams, infrastructure engineers, and production support to
triage and resolve issues and efficiently manage resource capacity of
our Big Data platform
Skills Required:
- 5-7
years of appropriate technical experience
- Proficiency
with developing software with Scala and Python
- Experience
with Spark for data processing
- Experience
maintaining Cloudera Hadoop infrastructure such as HDFS, YARN, Spark,
Impala and edge nodes
- High
quality software architecture and design methodologies and patterns
- Strong
SQL skills with commensurate experience in a large database
platform. Complete SDLC process and Agile Methodology (Scrum)
- Strong
oral and written communication skills
Skills Desired:
- Experience
with Core Java
- Experience
with XML/JSON technologies
- Experience
with Data Federation or Virtualization technologies
- Experience
with developing Cloud-based Big Data solutions
- Experience
with tools / platforms / technologies like Dataiku, ElasticSearch, etc.
- Experience
in development on other application types (Web applications, batch, or
streaming)