Sr. Developer

Job Summary

Sr Developer role in a global organization focusing on data engineering solutions for life and annuities insurance using Databricks SQL Databricks Workflows and PySpark. The role involves building scalable data pipelines optimizing analytical workloads and ensuring high quality data products in a hybrid work model with day shift and no travel.

Responsibilities

Design robust data pipelines in Databricks using Databricks SQL and PySpark to transform complex life and annuities insurance data into reliable analytics ready datasets

Develop reusable PySpark modules that standardize ingestion cleansing and enrichment of policy claims and actuarial data while meeting performance expectations and coding standards

Configure and manage Databricks Workflows to orchestrate end to end batch and near real time data processing jobs supporting reporting and analytics needs for insurance stakeholders

Optimize Databricks SQL queries and PySpark jobs by tuning partitions caching strategies and cluster configurations to achieve efficient resource usage and faster insights

Implement data quality checks validation rules and reconciliation processes that improve the accuracy and completeness of life and annuities insurance datasets used across business functions

Collaborate with business analysts and actuaries to translate insurance product rules policy events and financial calculations into clear and maintainable data engineering logic

Document technical designs job dependencies transformation logic and operational procedures to ensure transparency maintainability and smooth knowledge transfer within the team

Partner with testing and operations teams to define test cases support defect resolution and enable stable deployment of Databricks solutions into production environments

Monitor production workflows in Databricks investigate failures or performance regressions and implement corrective actions that protect data reliability and service commitments

Apply secure coding practices implement access controls and follow compliance guidelines relevant to financial and insurance data handling across all data engineering activities

Contribute to continuous improvement by identifying automation opportunities refactoring legacy code and adopting emerging Databricks features that deliver better outcomes for the company

Support business reporting and advanced analytics initiatives by exposing curated data models reusable views and standardized metrics for life and annuities insurance use cases

Work effectively in a hybrid environment by coordinating with on site and remote team members ensuring timely communication and consistent progress on shared deliverables

Qualifications

Possess professional experience between six and eight years in data engineering or related roles with a strong focus on large scale analytics solutions

Demonstrate deep proficiency in Databricks SQL including development of complex queries views and performance optimized data models for analytical consumption

Show hands on expertise in PySpark programming including data frame operations user defined functions and performance tuning techniques on distributed datasets

Exhibit practical experience configuring and managing Databricks Workflows including job scheduling dependency handling and monitoring of production pipelines

Bring solid domain knowledge in life and annuities insurance including familiarity with policies riders premiums benefits and related data structures

Display strong understanding of data warehousing concepts dimensional modeling and best practices for building curated layers supporting reporting and analytics

Apply sound knowledge of software engineering practices such as version control code review and modular design to maintain high quality data engineering solutions

Communicate effectively with cross functional partners by explaining technical designs in business friendly language and incorporating feedback into implementation plans

Adhere to organizational standards for security compliance and documentation while contributing ideas that enhance efficiency and long term maintainability of solutions

Certifications Required

Preferred certifications include Databricks Data Engineer Associate or Professional and cloud platform certifications relevant to data engineering