Previous Job
Developer Big Data ETL
Ref No.: 19-01305
Location: Dallas, Texas
Job Description:
The Big Data Architect is responsible for providing technical leadership to short term complex fast paced projects. The architect is expected to have an in-depth knowledge of the Big Data, Data Warehousing and Data Integration areas, which includes knowledge of applicable processes, methodologies, standards, products and frameworks. Should be responsible for defining and documenting architecture, capturing and documenting non-functional (architectural) requirements, preparing estimates and defining technical solutions to proposals (RFPs). You should provide technical leadership to project teams in the design, development and deployment of Big Data Warehouse solutions. Should be able to provide guidance to the team, perform reviews, prevent and resolve technical issues.

Minimum work experience:
  • Over 10 years of engineering and/or software development experience and demonstrable architecture experience in a large organization.
  • Experience should contain 5+ years of experience of architecture support combined of these environments: warehouse, DataMart, business intelligence, and big data.
  • 5+ years of consulting experience desired
  • Hands-on experience in Big Data Components/Frameworks such as Hadoop, Spark, Storm, HBase, HDFS, Pig, Hive, Scala, Kafka, PyScripts, Unix Shell scripts
  • Experience in architecture and implementation of large and highly complex projects
  • Deep understanding of cloud computing infrastructure and platforms
  • History of working successfully with cross-functional engineering teams
  • Experience in business domains like Retail, Communications, Finance, Consumer goods and Supply Chain
  • Demonstrated ability to communicate highly technical concepts in business terms and articulate business value of adopting Big Data technologies
Bachelor's degree or foreign equivalent required from an accredited institution. Master's Degree preferred. Will also consider years of progressive experience with a leadership role in Big Data and Data Warehousing at fortune 50 companies in lieu of a Master's Degree.
Job Responsibilities
  • Perform architecture design, data modeling, and implementation of Big Data platform and analytic applications for clients
  • Analyze latest Big Data Analytic technologies and their innovative applications in both business intelligence analysis and new service offerings; bring these insights and best practices to Professional Services Practice
  • Architect and implement complex Big Data Warehouse and Analytic solutions
  • Understand competing technologies and eloquently demonstrate the leadership in this space to customers
  • Develop highly scalable and extensible Big Data platforms which enable collection, storage, modeling, and analysis of massive data sets including those from IoT and streaming data
  • Drive architecture engagement models and be an ambassador for partnership with IT delivery and external vendors.
  • Effectively communicate complex technical concepts to non-technical business and executive leaders
  • Lead large and varied technical and project teams
  • Assist with scoping, pricing, architecting, and selling large project engagements
  • Review, analyze, and evaluate the scope of the business problems presented and help identify viable technical solutions
  • Develop ETL and data warehousing solutions with the customer product and services
  • Drive data warehouse architectural decisions and development standards
  • Create detailed Architectural design documents and presentation for delivery to Executives and Senior Management
  • Multi-task across several ongoing projects and daily duties of varying priorities as required
  • Work with Project teams to review design and development activities and proactively address risks while collaborating with Product Engineering team
  • Ensure adherence to published development standards and Hadoop best practices, resulting in consistent and efficient implementation of project components
  • Ability to make good judgments, decisions, negotiate and good problem solver
Preferred Skills:
  • Cloud platform technologies such as Microsoft Azure, Amazon Web Services and Google Cloud.
  • On premises Big Data platforms such as Cloudera, Hortonworks and MapR
  • Big Data Analytic frameworks and query tools such as Spark, Storm, Hive, Impala
  • Streaming data tools and techniques such as Kafka, AWS Kinesis, Microsoft Streaming Analytics
  • ETL (Extract-Transform-Load) tools such as Pentaho or Talend or Informatica); also experience with ELT
  • Continuous delivery and deployment using Agile Methodologies.
  • Data Warehouse and DataMart design and implementation
  • NoSQL environments such as MongoDB, Cassandra and HBase
  • Data modeling of relational and dimensional databases
  • Metadata management, data lineage, data governance, especially as related to Big Data
  • Structured, Unstructured, Semi-Structured Data techniques and processes
  • Hands on Knowledge of working with Teradata, Netezza, Oracle, DB2 databases
  • Strong understanding of File Formats – ORC, Parquet, Hadoop File formats.
  • Strong understanding and hands-on programming/scripting experience skills – UNIX shell, Python, Perl, and JavaScript.
  • Should have worked on large data sets and experience with performance tuning and troubleshooting.
  • Experience to Retail, Consumer Goods, Finance and Telco domain is a plus
  • Experience and desire to work in a fast-paced delivery environment
  • Should be a proven leader in this space with active participation in industry forums
Travel Requirements:
  • Regular and expeditious travel throughout the United States is required to meet client needs and timetables
  • Available to be stationed at and work from an out-of-town client site as needed