Previous Job
Previous
Data Engineer - San Francisco, CA
Ref No.: 17-03566
Location: San Francisco, California
Job Title : Data Engineer
Location : San Francisco, CA
Position Type : Long Term Contract

US Citizen, Green Card, TN, GC EAD and H4 EAD only No Third-party agencies crop to corp.

Job Description

Responsibilities:

Skills:
  • Mid-level, Data Engineer with experience in Tools Apace Kafka, Apace Spark. This project is about Data Integration not for streaming ( Usually people think the tools used for data streaming)
  • Who can - Build scalable, high- performance, and efficient pipelines and workflows that are capable of processing billions of transactions and real-time customer activities,
  • Work with big data, Fluent in Pig and/or Hive with experience in building UDFs, Pig and Hadoop streaming,
  • Build automated reports that can help the team to proactively identify quality and/or coverage problems in releases or new versions of our models,
  • Apply knowledge of Azkaban, Oozie or Hamake for workflow management and job scheduling, Work on Data Warehousing architecture and data modeling best practices.
  • Deep understanding of Map Reduce framework & Hadoop, Fluent in Pig and/or Hive with experience in building UDFs, strong scripting ability,
  • Proven expertise and understanding of ETL techniques, Knowledge of Azkaban