Azure Databricks Architect (Retail / CPG)-Chicago, IL

[David Miller]

Job Title: Data Solutions Architect (Azure Databricks / Retail / CPG)

Location: Chicago, IL (Hybrid)

Duration: Long Term  

 We are seeking an experienced Data Solutions Architect to lead the design and delivery of enterprise-scale cloud data platforms for Retail and Consumer Packaged Goods (CPG) clients. This role combines deep technical expertise with strong consulting and stakeholder management skills to architect scalable, secure, and business-aligned data solutions. The ideal candidate will have extensive experience designing modern data platforms using Microsoft Azure and Databricks, with strong proficiency in Python, SQL, Apache Spark, and Data Modelling. The successful candidate will work closely with business leaders, enterprise architects, engineering teams, and client stakeholders to define data strategies, solution architectures, and implementation roadmaps that enable advanced analytics, AI/ML, customer insights, and operational excellence.

Key Responsibilities

Solution Architecture & Strategy

• Design end-to-end cloud-native Data & AI platforms leveraging Azure, Databricks, Snowflake, AWS & GCP & Gemini Enterprise.
• Define target-state data architecture aligned to business objectives and enterprise technology strategy.
• Develop architecture blueprints, reference architectures, solution patterns, and technology roadmaps.
• Lead architecture discussions with client executives, enterprise architects, and engineering teams.
• Translate business requirements into scalable technical solutions.

Technical Leadership

• Architect enterprise data lakes, lakehouses, data warehouses, and analytical platforms.
• Define scalable ingestion, transformation, storage, and serving patterns for data & analytical products.
• Lead architectural decisions around batch, streaming, real-time analytics, and AI-ready data platforms.
• Establish reusable design patterns, standards, governance, and engineering best practices.
• Guide engineering teams throughout implementation and provide technical leadership.

Retail / CPG Domain Consulting

• Design data platforms supporting Retail and CPG use cases, such as Customer 360 & MDM, Sales & Commercial Analytics, Demand Forecasting, Supply Chain Visibility, Inventory Optimization, Trade Promotion Management, Marketing Analytics etc.

• Collaborate with business stakeholders to identify opportunities for data-driven transformation.

Governance & Quality

• Design secure, governed, and compliant data architectures.
• Define metadata management, lineage, master data, and data quality frameworks.
• Ensure architectures align with enterprise security, privacy, and regulatory standards.

Delivery Leadership

• Support pre-sales activities including solution design, proposals, estimates, and technical presentations.
• Conduct solution architecture reviews and design governance.
• Mentor engineers and architects while fostering engineering excellence.
• Drive successful project delivery across cross-functional teams.

Required Technical Skills

• Cloud & Data Platforms: Microsoft Azure, Databricks, Azure Data Factory, ADLS Gen2, Azure Key Vault, Azure DevOps
• Programming & Data Engineering: Python, SQL, Apache Spark (PySpark), Delta Lake, ETL / ELT, Data Modelling (Conceptual, Logical & Physical)
• Architecture: Enterprise Data Architecture, Modern Data & AI Platform Design, Data Acquisition & Integration, API-based Integration, Batch & Streaming Architectures, Data Security & Identity, Performance Optimisation, Solution Design Documentation

Preferred Skills

Databricks Unity Catalog, Federation Design, Snowflake, AWS (S3, Glue, Lambda, EMR, Redshift), Data Governance & Data Management frameworks (e.g., DAMA-DMBOK), Lakeflow Designer, Genie, Lakebase, Master Data Management (MDM), Data Quality Frameworks, Terraform, CI/CD using Azure DevOps, REST API integration, AI/ML platform architecture, GenAI and Large Language Model (LLM) integration patterns, Gemini Enterprise integration etc.

Experience

• 12+ years of experience in Data Analytics & AI including 8+ years relevant designing enterprise cloud data platforms.
• Proven experience architecting Azure Databricks solutions including new Databricks offerings
• Strong consulting and client-facing experience.
• Experience leading geographically distributed engineering teams.
• Experience delivering enterprise-scale transformation programs.
• Retail or Consumer Packaged Goods (CPG) industry experience is essential.

Preferred Certifications

• Microsoft Certified: Azure Solutions Architect Expert (AZ-305) or (DP-203)
• Databricks Certified Solution Architect (preferred)
• TOGAF Certification (desirable)