Data and Cloud Engineer

Job Description: Design, implement and deliver large scale enterprise applications using big data open-source solutions such as Apache Hadoop, Apache Spark, Kafka, and Elastic Search. Implement data pipelines and data driven applications using Python on distributed computing frameworks like Hadoop, Apache Spark, etc.

Responsibilities: Use AWS services and GCP services to build data pipelines and migrate on-prem data pipelines and data applications to Cloud infrastructure.

Proficient in using AWS cloud-based services to implement batch and online steaming applications.

Work closely with Data science teams to integrate data, algorithms into data lake systems and automate different Machine Learning workflows and assist with data infrastructure needs.

Experience in working both, On-Prem and Cloud.

Design and implement efficient data pipelines (ETLs) in order to integrate data from a variety of sources into Data Warehouse.

Design and implement data model changes that align with warehouse standards.

Design and implement backfill or other warehouse data management processes.

Develop and execute testing strategies to ensure high quality warehouse data.

Provide documentation, training, and consulting for data warehouse users.

Perform requirement and data analysis in order to support warehouse project definition.

Excellent database troubleshooting skills.

Working technical knowledge of PowerShell.

Strong object-oriented design and analysis skills.

Verbal and written communication skills and the ability to interact professionally with a diverse group, executives, managers, and subject matter experts.

Working in agile methodology, involve in Grooming, Sprint Planning, and daily Scrum meetings.

Position may work at various and unanticipated worksites throughout the United States. Telecommuting permitted.

Qualifications: Requires a Bachelor’s degree Computer Science, Engineering, or a directly related field plus five (5) years of software engineering experience. Experience must include: Two (2) years of experience in: SQL scripting; Big data technologies: HDFS, YARN, Spark, Hive, Sqoop, Impala, Airflow, Zookeeper, and Kafka; Enterprise GitHub: branch, release, DevOps, and CI/CD pipeline; and Data engineering or ETL development. Three (3) months of experience in: Snowflake or other cloud-based data platforms (AWS, GCP, Databricks) and Python web frameworks: Django or Flask. Employer will also accept a Master’s degree plus three (3) years of engineering experience in lieu of Bachelor’s plus five (5) years of software engineering experience.

40 hours/week, 9:00am-5:00pm, Salary range: $190,000 to $193,000 per year.