Data & Analytics Data Pipelines Lead
Skills
About the role
Key Responsibilities
Data Pipeline Leadership
Lead the design, development, deployment, and support of enterprise data pipelines and integration solutions.
Establish standards, patterns, and best practices for data ingestion, transformation, orchestration, and delivery.
Oversee data movement across the full analytics lifecycle:
o Source systems and external data providers
o Landing/staging databases
o Enterprise data warehouses
o Data lake environments
o Analytics platforms
o Reporting databases and data marts
Ensure scalable, secure, and high-performing data integration architectures.
Drive automation and operational efficiency within data pipeline environments.
Data Integration Engineering
Manage batch, near-real-time, and streaming data ingestion processes.
Coordinate data onboarding and integration activities for new source systems and external data suppliers.
Define and maintain ETL, ELT, and data replication standards.
Support cloud and on-premises data integration platforms.
Collaborate with enterprise architects to align data movement solutions with strategic architecture standards.
Data Quality Governance
Establish and maintain enterprise data quality controls and monitoring processes.
Define data validation, reconciliation, exception handling, and alerting frameworks.
Monitor pipeline performance and proactively identify data integrity issues.
Partner with data governance teams to ensure compliance with organizational standards.
Implement and maintain data lineage documentation across data movement processes.
Support audit, compliance, and regulatory reporting requirements related to data traceability.
Data Lineage Metadata Management
Document end-to-end data flows across all pipeline stages.
Maintain lineage mapping between source systems, transformation processes, and downstream reporting assets.
Ensure metadata accuracy and availability for analytical consumers.
Support impact assessments related to upstream and downstream changes.
Drive adoption of data catalog and metadata management capabilities.
Production Support Operations
Lead operational support for production data integration and analytics pipelines.
Manage ServiceNow incidents, service requests, problem records, and change activities related to data pipeline operations.
Coordinate incident triage, root cause analysis, issue resolution, and stakeholder communications.
Establish and monitor service level agreements (SLAs) and operational metrics.
Ensure rapid resolution of critical data availability and quality issues.
Coordinate production releases and change management activities.
Monitoring Reliability
Implement pipeline monitoring, observability, and alerting solutions.
Track pipeline health, throughput, latency, failure rates, and data quality metrics.
Develop operational dashboards and reporting for platform performance.
Lead efforts to improve platform reliability, resiliency, and recoverability.
Support disaster recovery and business continuity processes for critical data assets.
Stakeholder Team Leadership
Serve as the primary point of contact for data pipeline operations and support.
Partner with business intelligence, analytics, reporting, application, and infrastructure teams.
Mentor data engineers and analysts on integration standards and best practices.
Facilitate prioritization of enhancements, technical debt reduction, and operational improvements.
Communicate risks, issues, and performance metrics to leadership.
________________________________________
Required Qualifications
Bachelor's degree in Computer Science, Information Systems, Engineering, Data Analytics, or related field.
7+ years of experience in data engineering, data integration, ETL/ELT development, or data platform operations.
3+ years of experience leading enterprise-scale data pipeline and integration initiatives.
Experience supporting production data environments and operational processes.
Experience managing incident, problem, and change management processes within ServiceNow or similar ITSM platforms.
Strong understanding of data warehousing, dimensional modeling, and data lake architectures.
Experience implementing data quality and data governance practices.
Strong analytical, troubleshooting, and problem-solving skills.
________________________________________
Preferred Qualifications
Experience with cloud data platforms such as Oracle, Azure, AWS, or Google Cloud.
Experience with the Teradata platform and Teradata Data Mover (TDM)
Experience with modern data engineering platforms including:
o Azure Data Factory
o Databricks
o Synapse Analytics
o Snowflake
o Informatica
o Talend
o SSIS
o Kafka
o Fivetran
o dbt
Experience with metadata management and data lineage tools.
Knowledge of DevOps, CI/CD, infrastructure automation, and data observability platforms.
Familiarity with Agile delivery methodologies and product operating models.
________________________________________
Key Competencies
Technical Competencies
Data Engineering
ETL/ELT Architecture
Data Warehousing
Data Lake Architectures
Data Quality Management
Metadata Management
Data Lineage
Production Support Operations
Monitoring and Observability
ServiceNow Administration and Workflow Processes
Leadership Competencies
Operational Excellence
Stakeholder Management
Team Leadership
Incident Management
Strategic Planning
Continuous Improvement
Risk Management
Communication and Influence
Questions about this role
Want AI Applyd to auto-apply to roles like this?
We tailor your resume per posting, fill the forms, and track replies for you.