|Reference # : ||18-01114
||Title : ||Big Data Software Engineer|
|Position Type : ||Right to Hire|
|Experience Level : ||
||Start Date / End Date : ||06/25/2018 / 12/28/2018
| Description |
Senior Data Engineer
· Overall Purpose
This position designs, develops, tests and maintains distributed data processing applications on a Big Data Platform.
o Design, develop, document, test, and debug new and existing software systems and applications.
o Serve as technical expert on development projects.
o Drives full development life cycle including requirements analysis and design.
o Write technical specifications based on conceptual design and stated business requirements.
o Support, maintain, and document software functionality.
o Identify and evaluate new technologies for implementation.
o Analyze code to find causes of errors and revise programs as needed.
o Drive software design meetings and analyze user needs to determine technical requirements.
o Consult with end user to prototype, refine, test, and debug programs to meet needs.
o Complies with all security policies and procedures, to ensure that the highest level of system and data confidentiality, integrity and availability is maintained.
o Ability to use Unix/Linux command line programs and create/edit scripts.
o Able to analyze results of automated deployment scripts and makes modifications.
o Experience with Spark, Map/Reduce distributed processing
o Able to prepare detail design documents from requirements.
o Demonstrable understanding of distributed, component based Real-Time and Batch architecture
o A solid understanding of event driven / asynchronous programming and multi-threading
o Demonstrable technical design skills
o Familiar with TDD/BDD.
o Common Developer Skills
- Competent (5+ years) Java or Scala
- Experience with Kafka messaging system
- Experience with Hbase and/or Solr
- Competent with SQL or PL/SQL and relational database concepts.
- Experience with source code control systems.(GIT)
- Experience with secure programming techniques is highly desirable
- Able to perform code reviews to ensure the code meets the acceptance criteria.
- Scala/Java (5+ Years)
- Spark (1+ Years)
- Kafka (1+ Years)
- Hbase and/or Solr
- Real-Time and Batch Architecture