Senior/Lead Data Engineer.
Australia 1+ months ago
Job Description
Location: Gurugram,Haryana,India
About us
We turn customer challenges into growth opportunities.
Material is a global strategy partner to the world’s most recognizable brands and innovative companies. Our people around the globe thrive by helping organizations design and deliver rewarding customer experiences.
We use deep human insights, design innovation and data to create experiences powered by modern technology. Our approaches speed engagement and growth for the companies we work with and transform relationships between businesses and the people they serve.
Srijan, a Material company, is a renowned global digital engineering firm with a reputation for solving complex technology problems using their deep technology expertise and leveraging strategic partnerships with top-tier technology partners. Be a part of an Awesome Tribe
Role- Senior/Lead Data Engineer (Azure, Databricks, Data Factory, Data Fabric)
Job Responsibilities:
Design and Develop Data Pipelines:Development and optimisation of scalable data pipelines on Azure Databricks, utilising both batch and real-time processing techniques. Ensure seamless integration with Azure Data Factory for orchestration and workflow management.
Cloud Data Architecture:Collaborate closely with the Data Architecture team on development and implement robust data architectures that align with business needs and optimise performance, scalability, and cost-efficiency in the Azure environment.
Data Pipeline Optimisation:Continuously monitor, enhance, and optimise data pipelines for performance, cost, and reliability, ensuring the efficient processing and storage of large datasets.
Collaboration with Cross-functional Teams:Work with data engineering teams, analysts, and business stakeholders to understand their data requirements and ensure that the data platform meets those needs. Enable self-service analytics through well-designed data models and solutions.
Documentation and Knowledge Sharing:Contribute to internal documentation and knowledge-sharing practices. Provide technical mentorship to junior engineers to improve team skills and performance.
Azure Data Fabric Familiarity:Leverage your knowledge of Azure Data Fabric (or willingness to quickly learn) to manage end-to-end data movement, orchestration, and governance across various cloud platforms and on-prem systems. Ensure seamless data integration across hybrid environments.
Databricks Platform Expertise:Use your deep expertise in Azure Databricks to build and manage data models, create data processing pipelines, and integrate with other Azure services such as Azure Data Lake and Synapse for data storage and analytics.
Required Skills and Qualifications:
Experience in Azure Databricks:5+ years of hands-on experience with Azure Databricks, including Spark, Delta Lake, and other relevant Databricks tools. Expertise in optimising Databricks clusters and managing big data workloads efficiently.
Proficiency in Azure Data Factory:Strong experience in designing and orchestrating data pipelines using Azure Data Factory, including integrating with other Azure services for seamless data flow.
Familiarity with Azure Data Fabric:Working knowledge oreagernessto learn Azure Data Fabric to manage cross-platform data orchestration, governance, and security.
Advanced Data Engineering Skills:Extensive experience in data engineering, includingdata pipelinedevelopment,ETLprocesses, and working withlargedatasets. Proven ability to implement best practices in data quality, monitoring, and testing.
Cloud Architecture Design:Experience designing and implementing cloud-based data architectures that leverageAzure’s ecosystem, including Data Lake, Synapse, and Azure Storage.
Strong SQL and Data Modelling:Expertise in SQL and data modelling techniques to create optimised data structures, tables, and views. Understanding of both transactional and analytical data modelling.
Collaboration and Communication:Strong ability to collaborate with cross-functional teams and effectively communicate complex technical concepts to both technical and non-technical stakeholders.
Cost Optimisation:Proven experience optimising data engineering processes and Azure resources for cost reduction and performance improvements, particularly in large-scale cloud environments.
Preferred Skills:
Familiarity withData Lakehousearchitectures and working with tools likeDelta Lake,One Lake, etc.
Familiarity with theAzure ecosystemfor data engineering, specifically forend-to-end data integrationand ETL pipelines.
Knowledge ofPySpark, strongSQLskills, and basicPythonknowledge is mandatory.
Familiarity withDatabricksand its integration with the broader Azure data platform.
Who We Are& What We Care About
Material is a global company and we work with best-of-class brands worldwide. We also create and launch new brands and products, putting innovation and value creation at the center of our practice. Our clients are in the top of their class, across industry sectors from technology to retail, transportation, finance and healthcare.
Material employees join a peer group of exceptionally talented colleagues across the company, the country, and even the world. We develop capabilities, craft and leading-edge market offerings across seven global practices including strategy and insights, design, data& analytics, technology and tracking. Our engagement management team makes it all hum for clients.
We prize inclusion and interconnectedness. We amplify our impact through the people, perspectives, and expertise we engage in our work. Our commitment to deep human understanding combined with a science& systems approach uniquely equips us to bring a rich frame of reference to our work.
A community focused on learning and making an impact. Material is an outcomes focused company. We create experiences that matter, create new value and make a difference in people's lives.
Apply to this job
E-7A Development, Product Support Block Execution Lead
Brisbane, Queensland 17 hours ago
Job Type: Permanent Category: Business Support Location: Brisbane Date Advertised: 24-APR-25 Job Reference: BOE/1856920B2 Job Title: E-7A Development, Product Support Block Execution Lead Business Unit: Division: Summary: About Boeing Defence...
Product Support Analyst - Support Equipment
Brisbane, Queensland 18 hours ago
Job Type: Permanent Category: Business Support Location: Brisbane Date Advertised: 23-APR-25 Job Reference: BOE/1856914B Job Title: Product Support Analyst - Support Equipment Business Unit: Division: Summary: About Boeing Defence Australia Boeing...
Technology Senior Consultant (Melbourne/ Sydney/ Brisbane) (Southbank (Melbourne), VIC, AU, 3006)
Melbourne, Victoria 19 hours ago
We help the world run better At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. How? We focus every day on building the foundation for tomorrow and creating...
(AUS-Scoresby) Business Partnering and Reporting Manager APAC
Australia 7 days ago
**DESCRIPTION** **1. Business Partnering and Advisory:** + Act as a trusted advisor to the MD SPAC, providing strategic financial guidance and recommendations to meet and exceed financial targets. + Collaborate closely with the MD and senior...
Senior Associate - Global Data Collection
Australia 7 days ago
Location: Gurugram,Haryana,India About US We turn customer challenges into growth opportunities. Material is a global strategy partner to the world’s most recognizable brands and innovative companies. Our people around the globe thrive by helping...
Configuration, Data&Cross Services Manager
Adelaide, South Australia 7 days ago
Job Type: Permanent Category: Engineering / Technical Location: RAAF Base Edinburgh Date Advertised: 17-APR-25 Job Reference: BOE/1855144A Job Title: Configuration, Data & Cross Services Manager Business Unit: Division: Summary: The Opportunity -...
Senior ICT and Business Systems Support Technician
Brisbane, Queensland 7 days ago
Job Type: Permanent Category: MIS / IT Location: Brisbane Date Advertised: 14-APR-25 Job Reference: BOE/1855742 Job Title: Senior ICT and Business Systems Support Technician Business Unit: Division: Summary: About Insitu Pacific Insitu Pacific...
Senior Analyst - Private Markets
Australia 9 days ago
Job Type: Temporary Category: Location: Multiple locations Date Advertised: 14-APR-25 Job Reference: ASIC/1852668 Job Title: Senior Analyst - Private Markets Business Unit: Division: Summary: At ASIC, you can be the change that ensures a fair, strong...
Senior Test Automation Engineer - Web / Mobile
Australia 9 days ago
Are you passionate about quality assurance and eager to make a significant impact? We are seeking a Senior Test Automation Engineer to lead the management of a large project a...
Senior Specialist - Cyber Identity and Access Management
Victoria 9 days ago
Job Type: Ongoing - full time Category: IT and Telecommunications Location: Blackburn North Business Centre Date Advertised: 15-APR-25 Job Reference: AV/1855849 Job Title: Senior Specialist - Cyber Identity and Access Management Business Unit:...