Previous Job
Software Engineering, Big Data Architect
Ref No.: 13-00457
Location: New York, New York
Position Type:Direct Placement
Start Date: 06/06/2014
New York, NY
Digital Media Powerhouse: As a member of the Client Tech Team you will start launching new software ASAP. We use a real-world agile development process that includes rapid development of new products designed by small teams working closely with our tech-savvy data-driven editors. We are innovating on the web with partners North America, South America, Europe, Asia, and Africa; on mobile with new and ground breaking iOS and Android applications and responsive websites; and with our realtime video network .
Client Tech Team members are hardcore software development and design professionals who take user interface design and coding seriously but have a great time while doing so. The Client Tech Team is small enough that you get know know everyone but large enough that you'll find a great work-life balance.


  • Build a next generation global stats infrastructure and architecture for both real-time and historical data analytics for all of the Huffington Post product suites including mobile products
  • Build a world class reporting suite to complement the new stats architecture for realtime and historical analysis
  • Enhance and optimize existing big data architecture components using Hadoop, Pig, RabbitMQ and Redis
  • Work closely with Editorial and Business teams to optimize our data and reporting needs to provide important and timely business intelligence
  • Lead the efforts to architect our next generation data science needs such as recommendation systems, A/B Testing and other Big Data projects
  • Taming huge amounts of text/unstructured data using not only programming paradigms like functional programming (map reduce) but the skillset in hacking scripts to deliver logic that you need to productize and operationalize

General Requirements

  • 3 or more years experience building large , scalable multi-datacenter big data architectures using technologies such as Hadoop, Cassandra and Hbase
  • Familiarity with data mining algorithms such as K-Means, SVM, PageRank, etc
  • Experience with Agile Development
  • Experience with MySQL, Unix, NoSQL databases and Open Source Software as well as Unix system administration skills, especially scripting in Bash or Shell
  • Experience operationalizing a large data infrastructure by working closely with operations teams and integrating proper monitoring tools into the architecture
  • Sense of humor
  • Strong interest in journalism, media, or any of the liberal arts
  • Expertise and experience with with almost any of these languages: Scala, Java, C/C++, R, Shell, Pig.