Senior Engineer, Data Engineering

Job Description :

Lead Data Engineer One of the most innovative and high-prole teams in News Corp is looking for a lead data engineer to help accelerate its vision. The role will involve combining all of News Corp’s data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The Dow Jones Lead Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.

Responsibilities

Design, implement and maintain high performance big data infrastructure/systems & big data processing pipelines scaling to billions of structured and unstructured events daily

Design, implement and maintain deep integration with up-stream systems

Implement data pipelines to automate the ingestion, transformation, and augmentation of data sources, and provide best practices for pipeline operations.

Advanced SQL writing and experience in data mining (SQL, ETL, data warehouse, etc.) and using databases in a business environment with complex datasets

Monitor and optimize the performance of the data platform to ensure reliability, scalability, and efiency.

Ensure data quality across systems (accuracy, consistency, completeness) by providing transparency and proactive monitoring.

Support products with the overall roadmap and ensure updates to senior leadership are 100% technically correct.

Data analysis, understanding of business requirements and translation into logical pipelines & processes

Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)

Evaluate and prototype new technologies in the area of data processing

Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams

Data Engineering & Data Modelling: Creating data architectures and logical/physical data models for databases and data warehouses.

Providing training on data engineering concepts and techniques to other members of the team

Writing documentation for data pipelines and data models

Mentor and guide a team of data engineers, fostering a culture of continuous improvement, innovation, and knowledge sharing to enhance technical expertise and collaboration across the team.

Promote a high-performance team environment, ensuring strong work ethic, accountability, and enthusiasm to drive successful project delivery.

Technologies we use

AWS (EMR, EC2, Glue, Athena, Redshift, Lambda, S3, DynamoDB, SNS)

Python, Spark (PySpark)

Kubernetes, Docker

Airow

Git for source code management

Jira

Qualifications

BS in Computer Science or other technical discipline

8+ years of experience designing and developing big data processing systems using distributed computing

6+ years experience in SQL

5+ years experience in Python and Spark

6+ years of experience working with cloud platforms such as AWS

4+ years experience in Apache Airow

Expert knowledge in optimizing complex SQL queries and Hands on experience with NoSQL (DynamoDB)

An affinity for automation

Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs

Experience with at least one object oriented language (ex: Java)

Strong OO design, data structure, and algorithm design skills

Strong interest in emerging technologies: Spark, EKS

Nice to Have

Experience in the digital advertising and marketing industry

Contribution to Open Source projects

Reasonable Accommodation

We are committed to providing reasonable accommodation for qualified individuals with disabilities in our job application and/or interview process. If you need assistance or accommodation in completing your application or participating in an interview due to a disability, email us at humanresources@newscorp.com . Please put "Reasonable Accommodation" in the subject line and provide a brief description of the type of assistance you need. This inbox will not be monitored for application status updates.

Please refer to the privacy notice at the bottom of this page for submitting any data access, deletion, or other data subject rights requests, where permitted under your local laws and regulations.

This job is posted with NTS Technology Services Pvt. Ltd.

News Corp (Nasdaq: NWS, NWSA; ASX: NWS, NWSLV) is a global, diversified media and information services company focused on creating and distributing authoritative and engaging content and other products and services. The company comprises businesses across a range of media, including: information services and news, digital real estate services and book publishing. Headquartered in New York, News Corp operates primarily in the United States, Australia and the United Kingdom, and its content and other products and services are distributed and consumed worldwide. More information is available at: http://newscorp.com.