Quantexa Data Engineer

Job Summary

We are looking for an experienced Quantexa Certified Data Engineer / Data Architect to join our growing data engineering team. The ideal candidate will have strong expertise in Quantexa, Apache Spark, Scala, Hadoop, Elasticsearch, OpenShift (OCP), and DevOps technologies. You will be responsible for designing, developing, and optimizing enterprise-scale data processing pipelines while supporting Compliance, AML, and Financial Crime solutions.

This role requires hands-on experience with large-scale distributed systems, data integration, containerized deployments, and CI/CD practices.

Key Responsibilities

Design, develop, and maintain scalable data engineering solutions using Apache Spark, Scala, Hadoop, and Quantexa.

Build high-performance data pipelines to process structured and unstructured datasets.

Develop Spark applications utilizing RDDs, DataFrames, Datasets, and Spark SQL.

Design and implement data transformation, enrichment, aggregation, and validation processes.

Integrate Elasticsearch with Spark applications for indexing, search, and analytics.

Optimize Spark jobs and Elasticsearch queries for maximum performance and scalability.

Develop and maintain robust, fault-tolerant distributed data processing applications.

Deploy and manage applications on OpenShift Container Platform (OCP) using Kubernetes.

Collaborate with DevOps teams to implement CI/CD pipelines and automate deployments.

Support SIT, UAT, deployment activities, and production cutover.

Implement monitoring, logging, and performance optimization using industry-standard tools.

Ensure data quality, governance, lineage, metadata management, and regulatory compliance.

Troubleshoot complex data processing and system integration issues.

Prepare technical documentation and support knowledge transfer activities.

Work closely with Solution Architects, Technical Leads, and Application Delivery Managers to deliver enterprise-grade solutions.

Required Qualifications

Bachelor's or Master's Degree in Computer Science, Information Technology, Software Engineering, or related discipline.

Minimum 5+ years of Data Engineering experience.

Experience working in Banking, Financial Services, Compliance, AML, or Financial Crime projects is highly preferred.

Strong experience working in Agile development environments.

Mandatory Skills

Quantexa Certified Data Engineer or Data Architect

Hands-on experience with Quantexa Platform

Apache Spark

Scala

Hadoop Ecosystem (HDFS, Hive, Pig)

Elasticsearch

OpenShift Container Platform (OCP)

Kubernetes

DevOps & CI/CD

Docker

Jenkins

Git / BitBucket

Data Integration

Distributed Data Processing

Technical Skills

Big Data

Apache Spark

Hadoop

HDFS

Hive

Pig

Spark SQL

RDD

DataFrames

Datasets

Programming

Scala

Python

Java

Search & Analytics

Elasticsearch

Data Indexing

Query Optimization

DevOps & Containers

OpenShift (OCP)

Kubernetes

Docker

Jenkins

Ansible

BitBucket

CI/CD Pipelines

Monitoring

Grafana

Prometheus

Splunk

Cloud (Preferred)

AWS

Microsoft Azure

Google Cloud Platform (GCP)

Preferred Experience

Banking & Financial Services

Compliance & AML Solutions

Financial Crime Analytics

Quantexa Implementation Projects

Large-scale Enterprise Data Platforms

Distributed Computing

Performance Tuning

Data Governance & Metadata Management

Key Competencies

Excellent analytical and problem-solving skills

Strong debugging and troubleshooting capabilities

Experience with enterprise system integration

Strong understanding of software architecture and design principles

Excellent communication and stakeholder management skills

Ability to work independently and within cross-functional teams

Strong documentation and technical writing skills