Design, implement, and maintain the cloud infrastructure required for data and analytics solutions in Azure.
Configure and manage virtual machines, networks, storage, and other resources as needed.
Establish secure access controls, networking configurations, and compliance measures.
Set up and maintain CI/CD pipelines for data pipelines, analytics workflows, and applications.
Automate build, test, and deployment processes to ensure a consistent and efficient release cycle.
Monitor and enhance pipeline performance, reliability, and scalability.
Manage code repositories using version control systems (e.g., Git) and collaborate with development teams to ensure code quality and version management.
Implement branching strategies, code reviews, and merge processes to maintain a structured development environment.
Create and manage infrastructure as code templates using tools like ARM templates or Terraform to enable consistent and repeatable provisioning.
Update and maintain infrastructure configurations as application requirements evolve. Design, build, and maintain data pipelines for ELT/ETL processes using Azure Data Factory, Databricks, or other appropriate tools.
Monitor pipeline performance, troubleshoot issues, and optimize data flow and transformation processes.
Implement monitoring solutions using Azure Monitor, Application Insights, or similar tools to ensure application and system health.
Analyze performance metrics, identify bottlenecks, and optimize applications and services for scalability and efficiency.
Implement security measures such as identity and access management, encryption, and network security groups to protect data and applications.
Ensure compliance with relevant data protection regulations and industry standards. Containerize applications using Docker and manage them within a Kubernetes cluster if applicable.
Set up Kubernetes-based deployment and scaling for containerized applications. Diagnose and resolve technical issues, system failures, and performance bottlenecks across data and analytics solutions.
Collaborate with cross-functional teams to troubleshoot and implement solutions.
Create and maintain documentation for infrastructure, processes, and deployment configurations.
Ensure documentation is up-to-date and easily accessible to team members.
Collaborate with writing requirement specifications and design documents for a variety of development tasks including feature development, design specifications and system integrations.
Collaborate closely with data engineers, data analysts, data scientists, developers, and operations teams to ensure successful deployment and maintenance of data and analytics solutions.
Communicate effectively with team members, stakeholders, and management about project status, challenges, and solutions.
Monitor cloud resource usage and optimize costs by right-sizing resources, utilizing reserved instances, and implementing cost-effective strategies.
Stay updated with the latest Azure services, DevOps best practices, and trends in the data and analytics field.
Conduct blameless postmortems, find problems, root causes, and solutions to mitigate them. Use failures and the opportunity to learn and improvise the existing system.
Skills Required
Machine Learning
Python
Find more jobs at Fusemachines
Unleash your AI Transformation with AI Products and AI Solutions.
