Sr Databricks Certified Data Engineer
Skills
About the role
Company Description
Derex Technologies Inc specializes in providing IT consulting, staffing solutions and software services. Globally headquartered in Harrison New Jersey since 1996 Derex delivers the highest quality technology professionals and an array of customized IT talent solutions designed to improve productivity and drive results to global clients throughout North America.
With over two decades of unparalleled experience, Derex provides supports to its clientele, across such industries as Systems Integration, Banking and Finance, Telecommunications, Pharmaceutical and Life Sciences, Energy, Healthcare, Technology, Transportation, and local and federal Government agencies.
Job Description
Position: Sr Databricks Certified Data Engineer
Location: Remote
Duration: Full Time
Job Description
Sr Databricks Certified Data Engineer
Must Have: Databricks Certified Data Engineer Professional Certification.
Strongly preferred: Prior experience working in the Databricks Partner Program Databricks Partner Champion Certificate
10+ years of experience in data engineering, data platforms & analytics.
Comfortable writing code in either Python or Scala.
Extensive Working knowledge of Databricks is required.
Experience with advanced Databricks concepts such as Genie Spaces, Agents, and Genie Code.
Experience handling or leading large scale projects/customers.
Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one.
Deep experience with distributed computing with Apache Spark™️ and knowledge of Apache Spark™️ runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps Capable of design and deployment of highly performant end-to-end data architectures Experience with technical project delivery – managing scope and timelines Documentation and white-boarding skills Experience working with clients and managing conflicts Experience in building scalable streaming and batch solutions using cloud-native components
Here is some feedback I got for one of the consultant that took the interview:
1) Not strong in Data engineering and advanced concepts.
2) No Experience with advanced Databricks concepts such as Genie Spaces, Agents, and Genie Code.
3) No experience handling or leading large scale projects/customers.
Key Skills:
They want someone with. Spark fundamentals - Spark architecture, DataFrames, partitions, shuffles, joins, and performance behavior.
Delta Lake - ACID tables, schema evolution, merges, optimization, and reliable lakehouse storage patterns.
ETL / pipeline design - batch and incremental pipelines, medallion-style thinking, and production pipeline design.
Lakeflow /SDP / Jobs / workflows - orchestration, declarative pipelines, and workload deployment on Databricks.
Ingestion patterns - landing data from databases, files, streams, and connectors into bronze/silver/gold pipelines.
Performance tuning - cluster sizing, file layout, pipeline optimization, and Spark tuning best practices.
Data modeling and warehousing - dimensional modeling, warehouse migration patterns, and serving BI/reporting use cases.
CI/CD and deployment - asset bundles, deployment workflows
Regards,
Manoj Goud
Derex Technologies INC
Contact : 973-834-5005 Ext 206
Additional Information
Questions about this role
Want AI Applyd to auto-apply to roles like this?
We tailor your resume per posting, fill the forms, and track replies for you.