Talent.com
PortPro
Director of Software Engineering (Node.js & Web Scraping Expert)PortPro • Los Angeles, CA, United States
Director of Software Engineering (Node.js & Web Scraping Expert)

Director of Software Engineering (Node.js & Web Scraping Expert)

PortPro • Los Angeles, CA, United States
15 hours ago
Job type
  • Full-time
Job description

Director Of Software Engineering

We are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.

Key Responsibilities - Software Engineering & Web Scraping Leadership

  • Architect, develop, and maintain scalable and distributed web scraping systems using Node.js.
  • Design and implement data extraction pipelines to process large volumes of structured and unstructured data.
  • Develop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotation.
  • Optimize scraping processes for performance, reliability, and efficiency while managing proxy services (residential, datacenter, rotating). Oversee data storage and processing strategies, ensuring high availability and consistency.
  • Collaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applications.
  • Implement best practices for microservices, API integrations, and real-time data streaming.

Key Responsibilities - Scalability, Security & DevOps

  • Lead the transition to cloud-native, containerized, and serverless architectures for web scraping.
  • Ensure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.). Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scraping.
  • Manage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecks.
  • Work closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalability.

Key Responsibilities - Engineering Team Management & Strategy

  • Lead, mentor, and grow a high-performance engineering team.
  • Define and execute the technology roadmap, aligning with business objectives.
  • Foster a culture of continuous learning, collaboration, and innovation.
  • Implement agile development methodologies (Scrum, Kanban) to optimize project execution.
  • Ensure code quality, security, and best practices across all engineering efforts.

Qualifications & Experience - Technical Expertise

  • 10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extraction.
  • Strong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automation.
  • Extensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniques.
  • Deep knowledge of proxy management (residential, datacenter, rotating, and VPNs). Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.).
  • Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.). Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure).

Qualifications & Experience - Leadership & Soft Skills

  • Proven track record of scaling engineering teams and leading complex projects.
  • Strong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecks.
  • Excellent communication and stakeholder management skills.
  • Passion for mentorship, team development, and continuous learning.

Preferred Qualifications:

  • Experience with machine learning for data extraction and NLP.
  • Knowledge of browser fingerprinting and bot detection mechanisms.
  • Familiarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.).
  • Prior leadership experience in data-driven businesses or web scraping startups.
Create a job alert for this search

Director of Software Engineering (Node.js & Web Scraping Expert) • Los Angeles, CA, United States

Similar jobs

Director of Engineering, Android

KaavLos Angeles, CA, United States
Full-time

Director Of Engineering, Android.Location: San Francisco, Los Angeles, CA or Dallas, TX.As the Director of Engineering for Android Platforms, you will join an existing team that builds mobile, tabl... Show more

 • Promoted

Manager, Software Engineering BI & Analytics

The Walt Disney StudiosGlendale, CA, United States
Full-time

Manager, Software Engineering, Focused On Bi & Analytics.Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data... Show more

 • Promoted

VP Product

UltralyticsLos Angeles, CA, United States
Full-time

At Ultralytics, we commit to relentless innovation in the AI space and seek team members who resonate with our ambition to produce the world's best YOLO AI models.If you're obsessed with AI, eager ... Show more

 • Promoted

Director of Engineering

LinktreeLos Angeles, CA, United States
Full-time

We're looking for an Engineering Director to lead multiple product engineering teams responsible for building and evolving Linktree's core user-facing experiences.This role is for a product-first e... Show more

 • Promoted

Manager Software Engineering 2

Northrop GrummanLos Angeles, CA, US
$161,100.00–$241,700.00 yearly
Full-time

RELOCATION ASSISTANCE: Relocation assistance may be available.CLEARANCE REQUIRED FOR START: No.At Northrop Grumman, our employees have incredible opportunities to work on revolutionary systems th... Show more

 • Promoted

Manager, Regulatory Systems Engineering

RadiantEl Segundo, CA, United States
Full-time

Manager, Regulatory Engineering.Radiant is seeking an experienced Manager, Regulatory Engineering who loves to think outside the box and challenge convention; in this role you will lead the authori... Show more

 • Promoted

Director, Shopify Engineering

RitualLos Angeles, CA, United States
Full-time

Ritual is a personal health brand that is raising the bar for daily wellness.Known for our best-selling, clinically backed prenatal, we create science-backed supplements with transparency at their ... Show more

 • Promoted

Head of User Operation, Robotics

Faraday FutureEl Segundo, CA, United States
Full-time

Head Of User Operation, Robotics.El Segundo, California, United States.Faraday Future is a California-based technology company focused on the design, engineering, and development of intelligent, co... Show more

 • Promoted

Sr Manager, Enterprise Acquisition Ecosystems

Palo Alto NetworksBurbank, CA, United States
Full-time

Sr Manager, Ecosystems Enterprise Acquisition.As the Sr Manager, Ecosystems Enterprise Acquisition, you will be responsible for driving growth across our partner ecosystem specifically targeted a... Show more

 • Promoted • New!

Supervisor of AI Software Engineering

Latham & WatkinsLos Angeles, CA, United States
Full-time

Supervisor of AI Software EngineeringThe Supervisor of AI Software Engineering is an integral part of Latham's Technology & Information Services team.This role will be responsible for leading a... Show more

 • Promoted

Director of Website and Ecommerce

Georgia IT, Inc.Los Angeles, CA, United States
Full-time

Director Of Website And Ecommerce.Strong ownership of P/L and optimizing it in order to deliver the business profit.Develop and execute the eCommerce plan in line with the enterprise planning proce... Show more

 • Promoted

Robotics Engineering Manager: Systems & Applications

GrayMatter RoboticsCarson, CA, United States
Permanent

Robotics Engineering Manager: Systems & Applications.GrayMatter Robotics is a well-capitalized Physical AI company headquartered in Los Angeles, building Factory SuperIntelligence (FSI) through sys... Show more

 • Promoted

Senior Software Engineering Manager

ServiceTitanGlendale, CA, United States
Full-time

Senior Software Engineering Manager.ServiceTitan is a mobile, cloud-based software platform that helps home services companies streamline operations, improve customer service, and grow their busine... Show more

 • Promoted

Senior Director of User Acquisition and Growth (Remote)

Huckleberry LabsLos Angeles, CA, United States
Remote
Full-time

Senior Director Of User Acquisition And Growth.At Huckleberry, our purpose is simple yet powerful: to build life-changing products and experiences that help all families unlock everyday magic.Today... Show more

 • Promoted

Exec Director, Viral Vector Process Development

GileadSanta Monica, CA, United States
Full-time

Viral Vector Development Leader.We're here for one reason and one reason only to cure cancer.Every moment is dedicated to developing treatments and every action moves us one step closer to our goal... Show more

 • Promoted

Director of Ecommerce

Jackson JamesSylmar, CA, United States
Full-time

Director of eCommerce - Full Time Hybrid - San Fernando, CA.This is an opportunity to join an exciting Women's fashion brand in a huge growth period.Around 95% of the $10Millionrevenue comes throug... Show more

 • Promoted

Director of Web Development

HexcladLos Angeles, CA, United States
Full-time

Join HexClad, driving the hybrid revolution to elevate kitchens across the globe.From bootstrap roots to Gordon Ramsay's endorsement, HexClad thrives as a pre-IPO, best-in-class cookware brand.Our ... Show more

 • Promoted

Head of User Acquisition, Robotics

Faraday FutureEl Segundo, CA, United States
Full-time

Head Of User Acquisition, Robotics.El Segundo, California, United States.Faraday Future is a California-based technology company focused on the design, engineering, and development of intelligent, ... Show more

 • Promoted

Director, AI VFX Engineering

AdobeLos Angeles, CA, United States
Full-time

Role summaryLead Firefly Foundry's GenAI Field Architecture practice, with a focus on VFX, building and managing a team of senior VFX engineers and architects proficient in Generative AI.Drive exec... Show more

 • Promoted

Director of R&D Engineering Aesthetic Medical Devices - (Remote, U.S.)

Beauty HealthLong Beach, CA, United States
Remote
Full-time

Hydrafacial is a global category-creating company focused on bringing innovative products to market and delivering beauty health experiences by reinventing our consumer's relationship with their sk... Show more