Previous Job
Previous
Big data Engineer
Ref No.: 19-02632
Location: Totowa, New Jersey
ngestion Data Engineer / Developer
  • ETL Background *
  • Informatica
  • Active Job, Power Center
  • SQL Server *
  • AWS
  • Python *
  • Involving in all phases of development life cycle from gathering Business Requirements till the successful implementation of the project.
  • Work closely with the Business Analysts for requirements gathering, preparing low level and high-level design documents based on Business requirements.
  • Responsible for building big data solutions using Informatica, Talend and AWS Technologies.
  • Work on Informatica cloud, Talend, Python, REST API to collect and load the data on Cloud and on-premise systems and Informatica Power Center to load the data from relational databases
  • Responsible for loading Data Vault models on S3/Hive to reduce data hops and to address inserts/updates on Hive in type-2 fashion
  • Involved in designing, developing and production support of applications, by leveraging technologies like, Cloud Computing, Data Lake AWS
  • Responsible for resolving all the queries and concerns the application team will be having and providing the correct and optimum solutions for complex scenarios
  • Involve in designing and optimizing Redshift SQL queries, Data frames, import data from various Data sources, performing transformations, read/write operations and saving the results to output directory into S3/Hive/Presto
  • Participate in sprint planning meeting and retrospective meetings reviews to provide input on functional requirements, product designs, schedules, or potential problems and suggesting solutions to overcome these problems.