Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

EPAM Systems

NewBe an early applicant

Lead Data Engineer (PySpark, statistics, MLflow)

Posted Updated

Description

We are looking for a Lead Data Engineer to join our team and take ownership of complex data engineering challenges, from migrating legacy codebases to building robust, production-grade pipelines. In this role, you will work independently on large-scale PySpark systems, apply statistical methods to real-world sensor data, and leverage MLflow to manage the full lifecycle of machine learning models.

Responsibilities

  • Migrate existing pandas, Koalas or pandas-on-Spark code to native PySpark implementations
  • Decompose and refactor large legacy notebooks without relying on existing documentation
  • Apply statistical techniques to detect outliers and events in sensor time series data
  • Package and manage machine learning models using MLflow, including experiment tracking, Model Registry and versioning
  • Implement model explainability techniques such as SHAP for existing models
  • Develop unit and regression tests for PySpark

Skills

What Lead Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →