Python/Databricks/AI Developer-

New: Python/Databricks/AI Developer-

This assignment is 12 months with opportunity to convert. Candidate must be working onsite Tuesdays - Thursdays; virtual on Mondays and Fridays. There will be 2 round interviews; 1. Round of virtual for 45 min. 2. In person interview at SV7 if selected in Round 1. In person interview is Mandatory with no exceptions and Local candidates are preferred. There will be a coding assignment during interview.

Candidate must have 7 - 10 years of experience for this role. Key Responsibilities: - - Design Architecture: Participate contribute in end-to-end design and implementation of scalable data and application design using Python and Azure ML. - ETL/ELT Development: Develop and implement complex ETL/ELT processes to process and transform large-scale datasets from diverse sources using Spark and Python - Performance Optimization: Optimize Python jobs for performance and cost-efficiency using advanced techniques such as partitioning caching cluster configuration tuning and troubleshooting bottlenecks. - Collaboration Leadership: Collaborate with data scientists analysts and business stakeholders to translate complex business problems into technical solutions. - Data Governance Security: Implement data security strategies compliance standards and governance frameworks like Unity Catalog including managing access controls encryption and data lineage. - DevOps Automation: Champion code quality and implement DevOps CI/CD pipelines using tools like Azure DevOps GitHub Actions or Jenkins to automate deployment and management of data assets and infrastructure. - Troubleshooting Monitoring: Develop and implement robust monitoring logging and alerting systems for production-grade pipelines to proactively identify and resolve issues. Top 3 skillsets: Python Databricks Azure Cloud spark Azure ML and good Application development skills. Nice to have skillsets: Java Spring Good knowledge of different services in Azure. Excellent communication skills Describe a great candidate that you are looking for and what skills and experience they will have: - Should have 7 years of experience in Python ETL data handling SQL. - Should have experience in using AI tools for better productivity. - Advanced proficiency in Python Scala and/or SQL for data processing and application development. - 5 years of experience in Spark and Databricks programming - Should have very good experience in software development life cycle - Must be able to design and develop CICD pipelines that can push the code to different development and prod environments preferably to Azure using git actions - Must have 5 years of experience in Logging Monitoring and Alerting systems. What is the team environment and structure like?: Team is divided into pods with each of the pod handling different projects. The resource should be able make all the pods follow Agile methodologies and track deliverables.Candidate will be helping with All pods project planning and deliverables.