Lead Data Engineer

[Vikki Sai]

Role : Lead Data Engineer

Remote

Role Overview

We are looking for a highly skilled Data Engineer with strong expertise in Python-based optimization, graph data structures, and rule-based decision systems. The ideal candidate will design and build scalable data pipelines and intelligent optimization frameworks leveraging graph modeling, semantic data, and cost-based optimization techniques.

A key aspect of this role involves working with Excel spreadsheets as primary data inputs and outputs, transforming structured business data into optimized solutions and actionable insights.

Key Responsibilities

Design, build, and maintain robust data pipelines and processing systems using Python

Ingest, clean, and transform data from Excel spreadsheets into structured formats for processing

Generate and export optimized results and reports back into Excel for business users

Model complex relationships using graph-based approaches (e.g., NetworkX)

Work with semantic data structures and knowledge graphs using RDF frameworks (e.g., RDFLib)

Develop and implement rule-based optimization systems and decision engines

Apply cost models and optimization logic to improve system efficiency and performance

Build and maintain optimization solutions using:

Rule engines (e.g., rule-engine, durable_rules)

Optimization libraries (e.g., PuLP, linear/integer programming)

Translate business rules defined in spreadsheets into automated rule-based systems

Collaborate with AI/ML teams to integrate machine learning models into data pipelines and optimization workflows

Ensure scalability, reliability, and performance of data systems

Perform data validation, quality checks, and monitoring

Document system architecture, data models, and optimization logic

Required Skills & Qualifications

Strong programming experience in Python

Hands-on experience with Excel data processing using libraries such as:

pandas, openpyxl, or xlrd/xlsxwriter

Experience with:

NetworkX (graph modeling and analysis)

RDFLib (semantic web / RDF data handling)

Experience building rule-based systems using tools like:

rule-engine

durable_rules

Practical experience with optimization techniques using libraries such as:

PuLP (linear programming / optimization)

Solid understanding of:

Data structures and algorithms

Graph theory and network modeling

Cost-based optimization and decision systems

Experience handling structured tabular data workflows (especially Excel-based systems)

Familiarity with AI/ML concepts and integrating models into production systems

Strong problem-solving and analytical skills

Preferred Qualifications

Experience designing Excel-driven decision-support tools

Experience with knowledge graphs and ontology design

Exposure to distributed data systems (e.g., Spark, Kafka)

Understanding of constraint programming or operations research

Experience deploying models/services in cloud environments (AWS, GCP, Azure)

Familiarity with MLOps practices

Nice to Have

Experience with real-time decision systems

Background in supply chain, logistics, or optimization-heavy domains

Exposure to hybrid systems combining rules + ML models