Engineer

Java Nifi

Technical Skills : Strong NiFi Expertise : Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities. Programming Languages : Proficiency in Java, Spring boot & API Microservices. Database and SQL : Familiarity with databases, SQL, and data manipulation techniques. Big Data Technologies : Knowledge of big data technologies like Kafka ( Event driven & streaming architecture) Airflow Workflow orchestration

Responsibilities : Design and Implementation : Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes. Ingestion and Transformation : Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption. Routing and Enrichment : Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location). Scalability and Performance : Ensuring data flows are scalable and performant to handle varying data volumes and velocity. NiFi Cluster Management : Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting. Microservices Integration : Developing and integrating microservices to support data conditioning, format validation, and transformation processes. Troubleshooting and Issue Resolution : Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes Airflow Expertise : Possessing deep knowledge of Airflow's architecture, including schedulers, executors (Celery, Kubernetes), and plugin development. Workflow Design and Development : Designing and developing complex, modular, and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines. Performance Optimization : Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling

Salary Range- $100,000-$120,000 a year

#LI-SP3 #LI-VX1