Previous Job
Ref No.: 20-00789
Location: Wayne, New Jersey


Hadoop and Spark Lead Developer with Python


Clinton, NJ

Position Type




Job Description

Job Description:


  • 8+ years of Data warehousing Experience
  • 5+ years' experience with Hadoop stack – Cloudera or Hortonworks distribution
  • 3+ years' experience with Apache Spark (PYSPARK) and Python 3.x is desired.
  • 3+ years' experience with any ETL tool (Datastage is preferred) with Data warehousing concepts
  • Experience in building utilities using python for data analysis.
  • 3+ years of Linux or Unix conversant with bash/Unix shell scripting experience 
  • Processing large volumes of data using Data Frames, HIVE tables and ETL in PySpark
  • Responsible for Analysis, Design, Coding, Unit Testing and other SDLC activities.
  • Experience in Spark performance tuning and process optimization methodologies
  • Agile/Scrum methodology experience is required.
  • Strong communication skills
  • Ability to prioritize and handle multiple assignments
  • Hands on development is mandatory, with a willingness to troubleshoot and solve complex problems
  • Experience working in Onsite/Offshore Development model


Good to Have:

  • Spark streaming experience
  • Insurance domain experience
  • Hands on experience on Cloud
  • Life insurance Industry Experience a huge plus