Manager, Data Integration

Job Description:

We are seeking a dynamic and hands-on Manager of Data Integration Engineering to lead a small team focused on building robust, scalable, and high-performance data integration solutions. This individual will play a critical role in the architecture, development, and optimization of our data infrastructure, with a strong emphasis on Microsoft SQL Server, Azure Synapse Analytics, and API integrations.

The ideal candidate will have hands-on expertise in SQL, Azure Data Factory, Databricks, Python, Azure Functions, Azure DevOps, and data governance.   The Manager of Data Integration will lead a small team of both domestic and global resources in data integration efforts of strategic corporate projects, working with business stakeholders, tech services and vendors.

Responsibilities

Team Leadership & Development

• Lead, mentor, and grow a small team of data integration specialists
• Define team goals, manage performance, and foster a culture of technical excellence and collaboration
• Handle multiple projects simultaneously, ranging from small departmental to strategic corporate initiatives, ensuring timely and efficient completion.

Architecture & Development

• Design and implement scalable data architectures and ETL/ELT pipelines using Azure Data Factory, Databricks and other cloud-native tools.
• Oversee the development and optimization of Microsoft SQL Server and Azure Synapse databases.
• Ensure the reliability, security, and scalability of data systems and integrations.
• Manage data ingestion, processing, and storage, ensuring scalability and performance.

Data Quality and Governance

• Implement data quality checks and data validation to ensure data accuracy and integrity.
• Collaborate with data governance teams to ensure data compliance and regulatory requirements.

Hands-On Engineering

• Develop and maintain integration components, including data mapping, transformation, and validation.
• Write, review, and optimize complex SQL queries and stored procedures.
• Participate in the design and development of REST APIs for data exchange across systems.
• Drive performance tuning and query optimization in Azure Synapse.
• Troubleshoot integration issues, perform root cause analysis, and implement fixes.
• Collaborate with internal stakeholders and vendor teams to resolve integration challenges.

Strategic & Cross-Functional Collaboration

• Partner with business stakeholders, analysts and vendors to understand data needs and deliver actionable insights, in support of ongoing operations and strategic corporate initiatives.
• Contribute to enterprise data strategy, data governance, and best practices for integration and engineering.
• Collaborate with data governance and security teams to ensure data compliance and regulatory requirements.

Operational Excellence

• Establish monitoring and alerting workflows for pipelines and APIs
• Lead incident resolution and root cause analysis for integration issues
• Maintain accurate and up-to-date documentation of integration designs, architectures, and technical specifications.
• Share knowledge and expertise with junior team members, contributing to the growth and development of the team.

   

Required Qualifications
A bachelor's degree in engineering, computer science, or a related field.

• 7+ years of experience in data engineering or data integration, with at least 2 years in a leadership or management role.
• Deep hands-on experience with Microsoft SQL Server and Azure Synapse Analytics.
• Advanced proficiency in SQL for data manipulation, transformation, and performance tuning.
• Strong experience with data integration technologies, such as ETL tools (e.g., Azure Data Factory, Databricks), APIs (e.g., REST, SOAP), and AzureDevOps, Azure API management and Azure Functions.
• Proven track record of designing and deploying data pipelines and ETL/ELT processes in Azure.
• Knowledge of data security and compliance best practices
• Experience in agile environments with Scrum or Kanban methodologies, and Azure DevOps
• Strong experience developing and consuming REST APIs.
• Scripting: batch, PowerShell: creating new scripts and manipulating files in bulk.
• Proficiency in programming languages, such as Python
• Analytical Skills: Strong analytical thinking and problem-solving abilities to address complex integration challenges.
• Project Management Skills: Ability to manage multiple projects simultaneously, ranging from short-term to long-term durations.
• Communication Skills: Excellent communication skills to collaborate with internal teams and external clients.
• Leadership: Capable of coalescing multiple levels of client and internal resources to solve problems.