|Reference # : ||17-00052
||Title : ||Data Engineer|
|Position Type : ||Direct Placement|
|Experience Level : ||
||Start Date : ||08/31/2017
| Description |
|At Trillium, we assist our clients in finding just the right candidates to fit their needs! They are currently seeking a Data Engineer to join their team on a full-time basis. The ideal candidate will help build and architect databases, data sets and systems to support our client’s products. You will have the opportunity to use established and new technologies to help accomplish these goals.|
- You work with relational database technologies (e.g. PostgreSQL, Oracle, MySQL, or SQL Server) and write performant queries on large data sets.
- You have worked with moving lots of data from one system to another and transforming it along the way (e.g. JSON to relational data sets). You have developed full load and incremental load ETL processes using tools such as Microsoft SSIS, Pentaho Data Integration, or custom scripts. You have also worked on enhancing existing data sets such as adding additional attributes.
- You get things done using established tools and at the same time, enjoy exploring new technology for better/faster ways of processing data.
- You are comfortable with the software development workflow: ie., moving and deploying code from development to QA to production.
- You have a basic understanding of bottlenecks that may occur such as disk IO, CPU, or memory. You have improved the performance of a database system or query by making either DDL or DML changes or both.
- You are familiar with ETL workflows which progress through several different database technologies.
- We make heavy use of PostgreSQL for our OLTP databases and even some small OLAP databases.
- Occasionally, the ability for database administration tasks such as management of database configuration, replication, access, backups, and so-on.
- We use cloud technologies such as BigQuery and Amazon AWS RDS for some of our needs.
- We use git, maven, and Jenkins, and Docker. You do not need to be an expert in these, but familiarity helps.
- We practice sprints and use JIRA for project management.
- Bachelor's degree in Computer Science or a related field of Engineering, or equivalent combination of education and experience.
- Significant background in creating complex functions/stored procedures in SQL
- Experience with database modeling tools, database normalization, logical and physical database design and implementation
- Familiarity with database internals such as binary logging, indexes, statistics, query planning, transactions, concurrency management, and high-performance loading.
- Strong analytical skills
- 5-8 years of experience in database development with RDBMS (PostgreSQL, MySQL, Oracle, etc.)
- ETL operations
- Big Data and Agile experience is a plus.
- Experience preferred with Linux, Web Services, Postgres, Lucene, ElasticSearch and MapReduce frameworks such as Hadoop and Hive
- Experience with Cloud platforms (eg., AWS, GCP, Azure, or similar)
- Experience with higher-level languages such as Groovy or Python, particularly for authoring ETL.
- Experience authoring webservices and webservice clients.
- Bonus points are awarded if you have worked with append-only or distributed technologies.
- You are also familiar with what data modeling approaches work well for transactional needs vs. analytical needs.