Senior/Lead Databricks Engineer

Malta Island
Permanent
Full-time

1 month ago

We are looking for a Senior/Lead Databricks Engineer with expertise in cloud infrastructure, big data processing, data integration and analytics.In this role, you will play a crucial part in advancing our client's Data-Driven Company initiative. Your role will involve designing and building robust data architecture to support various aspects of the client's business. This position is integral to one of our projects in the client's IT area, focusing on the integration component of their landscape. Join us at our Malta office, which offers a flexible hybrid work setup.Would you like to be a part of this ambitious and innovative project? We look forward to receiving your application.ResponsibilitiesData integration, transformation and modellingServe large amounts of structured & semi-structured financial dataAnalyze existing process logicImplement best practices in data architecture, including security, governance, compliance and performance optimizationOptimize the data storage across Delta Lake for data managementDevelop and maintain scalable data pipelines and build out new API integrations to support continuing increases in data volume and complexityCollaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision-making across the organizationRequirementsAt least 5+ years of professional experience in the data engineering fieldProficiency in Azure Databricks, Delta Lake, Python and PySpark for data manipulationProficiency in designing and optimising pipelines using Azure Data Factory, including performance tuning and optimising data pipelines with Databricks NotebooksExperience with Azure data platforms and related servicesGood understanding of SQL and experience with relational database systems (e.g. MSSQL, PostgreSQL, Oracle DB)Understanding data structures, data modelling and data architecture patterns (Data Lake, DWH, Data mesh, etc.)Ability to work in an Agile environment and eager to learn and develop new skillsGood problem-solving skills and attention to detailStrong written and spoken EnglishNice to haveCertifications in Azure data or Cloud Services (e.g., Azure Data Engineer Associate)Business understanding of the insurance or financial domainExperience with BI Tools (Power BI)We offer/Benefits

Private health insurance (including dental insurance)
EPAM Group life assurance
Wellness allowance
Employee assistance program
Corporate Programs including Employee Referral Program with rewards
Learning and development opportunities including in-house training and coaching, professional certifications, over 22,000 courses on LinkedIn Learning Solutions and much more
*All benefits and perks are subject to certain eligibility requirements

EPAM Systems

Apply Now