AWS Data Engineer

JD- AWS Data Engineer

Location – Bangalore(WFO)

Exp:6-10yrs

Working time zone - 12 noon to 8:30 PM

Mandate Skills - AWS Lambda, Glue, S3, IAM, SNS, Step Functions, SQS, DataSync, Python, REST API

Good to have - Agile Development,Data Quality, Metadata Management

Key Responsibilities:

Design, develop, and implement a scalable data quality monitoring pipeline within AWS Infrastructure

Design, develop, and implement data feeds for systems that intend to consume data from Collibra

Centralize job run metadata from IDMC and Collibra to feed into an operational dashboard

Streamline regression testing for IDMC and Collibra jobs

Required Skills & Experience:

Strong understanding of Agile methodologies and project management tools like Jira and Confluence.

Experience in building CI/CD pipelines and automation using Jenkins & Terraform

Experience in code version control and best practices

Hands-on experience with AWS Lambda, Glue, S3, IAM, SNS, Step Functions, SQS, and DataSync

Experience in calling REST APIs

Intermediate development experience with Python

Nice to Have:

Knowledge of IDMC REST APIs

Knowledge of Collibra REST APIs

Working experience with IDMC Cloud Data Governance and Catalog, Cloud Data Profiling, Metadata Command Center, and Cloud Data Quality

Working experience with Collibra

Working experience with Tableau or Amazon Quick Sight

Pay: ₹1,700,000.00 - ₹2,700,000.00 per year

Benefits:

Paid sick time

Paid time off

Application Question(s):

Notice Period

Expected CTC

Willing to Work from Office from 12pm-8.30pm in Bangalore

Experience:

Python & AWS development: 6 years (Required)

Work Location: In person