Position

Hadoop and Spark Lead Developer with Python

Location

Clinton, NJ

Position Type

CTH

Rate

Open  

Job Description

Job Description:

• 8+ years of Data warehousing Experience
• 5+ years' experience with Hadoop stack – Cloudera or Hortonworks distribution
• 3+ years' experience with Apache Spark (PYSPARK) and Python 3.x is desired.
• 3+ years' experience with any ETL tool (Datastage is preferred) with Data warehousing concepts
• Experience in building utilities using python for data analysis.
• 3+ years of Linux or Unix conversant with bash/Unix shell scripting experience 
• Processing large volumes of data using Data Frames, HIVE tables and ETL in PySpark
• Responsible for Analysis, Design, Coding, Unit Testing and other SDLC activities.
• Experience in Spark performance tuning and process optimization methodologies
• Agile/Scrum methodology experience is required.
• Strong communication skills
• Ability to prioritize and handle multiple assignments
• Hands on development is mandatory, with a willingness to troubleshoot and solve complex problems
• Experience working in Onsite/Offshore Development model

Good to Have:

• Spark streaming experience
• Insurance domain experience
• Hands on experience on Cloud
• Life insurance Industry Experience a huge plus