Chief Manager - Information Technology
Skills
About the role
What are we looking for?
We are looking for a passionate and experienced Data Engineer to join our team and help build scalable, reliable, and efficient data pipelines on cloud platforms like Primarily on Google Cloud Platform (GCP) and secondary on Amazon Web Services (AWS). You will work with cutting-edge technologies to process structured and unstructured data, enabling data-driven decision-making across the organization.
What does the job entail?
Design, develop, and maintain robust data pipelines and ETL/ELT workflows using PySpark, Python, and SQL.
Build and manage data ingestion and transformation processes from various sources including Hive, Kafka, and cloud-native services.
Orchestrate workflows using Apache Airflow and ensure timely and reliable data delivery.
Work with large-scale big data systems to process structured and unstructured datasets.
Implement data quality checks, monitoring, and alerting mechanisms.
Collaborate with cross-functional teams including data scientists, analysts, and product managers to understand data requirements.
Optimize data processing for performance, scalability, and cost-efficiency.
Ensure compliance with data governance, security, and privacy standards.
Work Experience
6-12 years of experience in data engineering or related roles.
Essential Qualification
Bachelors
Preferred Skills
Strong programming skills in Python and PySpark.
Proficiency in SQL and experience with Hive.
Hands-on experience with Apache Airflow for workflow orchestration.
Experience with Kafka for real-time data streaming.
Solid understanding of big data ecosystems and distributed computing.
Experience with GCP (BigQuery, Dataflow, Dataproc
Ability to work with both structured (e.g., relational databases) and unstructured (e.g., logs, images, documents) data.
Familiarity with CI/CD tools and version control systems (e.g., Git).
Knowledge of containerization (Docker) and orchestration (Kubernetes).
Exposure to data cataloging and governance tools (e.g., AWS Lake Formation, Google Data Catalog).
Understanding of data modeling and architecture principles.
Soft Skills:
Strong analytical and problem-solving abilities.
Excellent communication and collaboration skills.
Ability to work in Agile/Scrum environments.
Ownership mindset and attention to detail.
Questions about this role
Want AI Applyd to auto-apply to roles like this?
We tailor your resume per posting, fill the forms, and track replies for you.