Previous Job
Automation Engineer (Python, Java) III
Ref No.: 19-00435
Location: Moorestown, New Jersey
Title: Automation Engineer (Python, Java) III
Location: Moorestown, NJ
Type: Contract

Our client is seeking an Automation Engineer (Python, Java) who will be responsible for designing, deploying, and maintaining mission critical security analytics data environment that process data quickly at large scale.

Key Accountabilities:
  • Responsible for designing, deploying, and maintaining mission critical security analytics data environment that process data quickly at large scale.
  • Build new applications and tools to better analyze, visualize, and present complex data. Maintains and extends existing analytics web application.
  • Contributes design, configuration, deployment, and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation, enrichment, and loading of data into a variety of data platforms, including (Hadoop, AWS S3, Spark, Redshift, etc.)
  • Identifies gaps and improves the existing platform to improve quality, robustness, maintainability, and speed.
  • Evaluates new and upcoming big data solutions and makes recommendations for adoption to extend our platform to meet advanced analytics use cases, such as predictive modeling and recommendation engines.
  • Performs development, QA, and dev-ops roles as needed to ensure total end to end responsibility of solutions.
  • Mentors junior and senior engineers, provides code reviews, feedback, and enables professional growth.

Required Skills:
  • 10+ years working on data processing environments.
  • 5+ year working in a Linux environments, scripting and Python.
  • 2+ years working with Kafka, Hadoop, or similar large-scale data platforms.
  • Experience building, maintaining, and improving Data Processing Pipeline / Data routing in large scale environments.
  • Fluency in common query languages, API development, data transformation, and integration of data streams.
  • Strong experience with large dataset platforms such as (Spark, Cassandra, Hadoop, MongDB, Elasticsearch, etc.)
  • Fluency in multiple programming languages, such as Python, Shell Scripting, Regex, SQL, Java, or similar languages and tools appropriate for large scale data processing.
  • Experience with acquiring data from varied sources such as: API, data queues, flat-file, remote databases.
  • Must have basic Linux administration skills and Multi-OS familiarity (Windows / OSX).
  • Experience in design and implementation of Data Pipeline and Data processing experience using common big platforms such as Hadoop/Spark.
  • Understanding of traditional DW/BI components (ETL, Staging, DW, ODS, Data Marts, BI Tools).
  • Creativity to go beyond current tools to deliver best solution to the problem.
  • Experience in producing and consuming topics to/from Apache Kafka.

Additional Relevant Experience:
  • Familiarity of information/network security detection tools, logging systems, network architecture, and threat intelligence a plus.
  • Familiarity with IP networking concepts (TCP/IP, pcap analysis).
  • Experience with AWS environment.
  • Experience with Dev-Ops functions such as Ansible, Chef, Puppet, Cloud Foundry.
  • Experience in system and Database administration.