Snowflake DBT Lead Engineer in New York, NY Onsite with LTIMindtree - Need Local Candidates Only
Neha Chaudhary
<neha.chaudhary95.nlb@gmail.com>Snowflake DBT Lead Engineer
LTIMindtree
Location New York, NY Onsite
"Key Responsibilities
Design develop and maintain robust ETLELT pipelines using SQL DBT and Snowflake
Perform data matching processes using various methodologies Union Union Precision Columns Joined Precision Columns
Configure data loads and creating data pipeline workflows for customer
Write optimize the DMLDDL Operations in snowflake
Enabling and validating match process to Blu id at client level
Match process automation Snowflake data ingestion
Converting multi cloud social data into structured data in Snowflake
Onboard data files and segments into Snowflake tables
Migrate data processes eg from EMR to Snowflake
Investigate data issues including low match rates unusual record countsspikes truncated data and discrepancies after adding fields
Convert Snowflake views to CSV files for external useanalysis
Manage Snowflake roles and grant access to views
Modify data consumption procedures
Research and implement new data onboardingprocessing workflows
Generate synthetic IDs and map them to client data eg for partner matching like Yahoo
Split and compress data files eg using bzip
Use vendorspecific APIs for data upload and download eg Yahoo API
Perform impact analysis on backfill process
Own the technical roadmap including performance tuning cost optimization and platform scalability
Build and optimize modular reusable dbt models for analytics and reporting
Implement and enforce data governance quality checks and documentation standards
Guide and mentor a team of developers and analysts ensuring adherence to best practices
Collaborate with data analysts product teams and business stakeholders to understand data requirements
Implement data quality checks testing frameworks and documentation standards
Monitor and troubleshoot data workflows ensuring reliability and performance
Participate in code reviews CICD processes and version control using Git
Contribute to the architecture and design of the data platform and analytics ecosystem
Monitor and troubleshoot production data pipelines and ensure SLAs are met
Required Skills Qualifications
5-10 years of experience in data engineering ETL development or analytics engineering
Expertlevel proficiency in SQL and Snowflake
Snowflake Databases Schemas Tables Views Procedures Datashare Roles Staging Production Work area Sandbox Partition Date SQL SELECT WHERE ORDER BY qualify rownumber MAX etc
Handson experience with DBT Data Build Tool in enterprise environments
Experience with Airflow Jobs Orchestration AutomationOrchestration Dag DailyWeekly Cadence scripting
Experience in Python scripting
File Formats Compression Parquet CSV GZ bzip
Exposure and Experience in matching methodologies Union Union Precision Columns Joined Precision Columns
Strong understanding of data architecture data modeling and pipeline orchestration
Strong proficiency in SQL for data transformation analysis and performance tuning
Handson experience with DBT in production environments
Experience in performing DMLDDL operations in snowflake
Data Processing Techniques Deduplication Blacklist Filtering Data Merging Data Copying Backfilling
API Interaction Fetching data via API using Yahoo API for partner match uploaddownload
Deep understanding of Snowflake architecture including virtual warehouses data sharing and performance optimization
Experience with data modeling starsnowflake schema data governance and metadata management
Familiarity with cloud platforms AWS Azure or GCP Esp AWS Cloud Storage s3
Proficiency in Git Bit bucket CICD pipelines and Agile methodologies
Excellent problemsolving communication and stakeholder management skills
Excellent leadership communication and stakeholder management skills"
|
Neha Chaudhary |