Data Engineer II

Job Description: Responsibility: • Support the design, implementation, and maintenance of enterprise ETL processes for data platforms, for a global client base. • Develop scalable and efficient code to process data, ensuring availability and accessibility in a timely manner. • Leverage big data processing frameworks such as Apache Spark and Hadoop to build and optimize data pipelines. • Collaborate with senior engineers to address data challenges, contributing to solutions that maintain high data quality. • Assist in the data delivery process, working alongside Data Engineers and Analysts to support accurate, high-value data solutions across various clients and industries. • Build strong working relationships with team members and clients, contributing to both local and global projects. • Learn and apply industry best practices, including version control, code reviews, and data validation, to ensure quality in data processes. • Use SQL and other database technologies to help optimize data processing and reduce the time required to handle large data sets. • Design, implement, and maintain data pipelines using ETL frameworks, orchestration tools, and distributed data processing engines. • Participate in efforts to automate routine data tasks and streamline processes. • Comply with all Mastercard internal policies and adhere to external regulations. Required Skills: • Experience as a Data Engineer or in a similar role, with a strong understanding of data engineering concepts and methodologies. • Strong knowledge of writing and optimizing SQL queries to retrieve, manipulate, and analyze data efficiently. • Hands-on experience with big data technologies such as: • Apache Spark (PySpark, Spark SQL, Spark Streaming) • Hadoop ecosystem (HDFS/ Ozone, Hive, YARN) • Familiarity with ETL frameworks and the ability to design, implement, and maintain data pipelines. • Understanding data modeling concepts and database design to support scalable data solutions. • Familiarity with Python. • Ability to analyze and troubleshoot data issues and provide solutions with minimal supervision. • Basic knowledge of testing and validating data to ensure accuracy and consistency in data pipelines. • Excellent verbal and written communication skills, with the ability to articulate complex ideas clearly and concisely to both technical and non-technical stakeholders. • Bachelor's degree in quantitative discipline such as Engineering, Mathematics, Finance, Business, or a related field. Equivalent practical experience may also be considered.