Role: ETL Developer
Location: Jersey City, NJ (Onsite)
Duration: Contract
Job Description:
· Design and develop scalable ETL pipelines using AWS services such as:
· AWS Glue for serverless data integration
· AWS Lambda for lightweight transformations
· Amazon S3 for data lake storage
· Amazon Redshift or RDS for data warehousing
· Integrate data from diverse sources including APIs, databases, and flat files into AWS-based data platforms.
· Implement data transformation logic using PySpark, Python, or SQL within AWS Glue or Lambda.
· Monitor, schedule, and orchestrate ETL workflows using AWS Step Functions, Glue Workflows, or Apache Airflow on Amazon MWAA.
· Ensure data quality, consistency, and lineage using AWS Glue Data Catalog and AWS Lake Formation.
· Optimize ETL performance and cost-efficiency through partitioning, parallelism, and resource tuning.
· Implement security best practices including encryption, IAM roles, and VPC configurations.
· Collaborate with data engineers, analysts, and DevOps teams to support analytics and reporting needs.
· Document ETL processes, data flows, and architecture using tools like AWS Architecture Diagrams or Confluence.