Senior Data Engineer
Bhargavi
Location: Remote, USA
Department: Data & Analytics Engineering
Job Type: Contract
Visa: USC
Role Overview
We are seeking a highly skilled Senior Data Engineer to lead the architectural design and hands-on implementation of our next-generation data platform on Google Cloud Platform (GCP). The ideal candidate is a subject matter expert in BigQuery and possesses deep experience building robust, scalable data pipelines using Dataform, Dataflow, and Dataproc.
In this role, you will bridge the gap between high-level architectural strategy and technical execution, ensuring our data ecosystem is performant, cost-effective, and capable of supporting advanced analytics and machine learning initiatives.
Key Responsibilities
Architectural Design & Leadership
● Design and implement end-to-end data architectures on GCP, ensuring high availability, security, and scalability.
● Establish best practices for data modeling (Medallion architecture, Star Schema) specifically optimized for BigQuery's columnar storage.
● Evaluate emerging GCP technologies and provide strategic recommendations for platform evolution.
● Mentor junior engineers and conduct rigorous code reviews to maintain high engineering standards.
Data Pipeline Engineering
● Develop complex ELT workflows using Dataform to manage SQL-based transformations, ensuring data lineage and quality through automated testing.
● Build and maintain large-scale batch and streaming data processing pipelines using Dataflow (Apache Beam).
● Preferred to have Dataproc knowledge for distributed processing of massive datasets using Spark or Hadoop where specialized compute is required
● Integrate data from various sources (SaaS APIs, RDBMS, NoSQL) into the central BigQuery data warehouse.
Performance & Governance
● Optimize BigQuery performance through partitioning, clustering, and materialized views while managing cost through efficient slot utilization.
● Implement CI/CD pipelines for data infrastructure and code deployments using Cloud Build or similar tools.
● Ensure data governance standards are met by implementing Identity and Access Management (IAM), data masking, and encryption.
Technical Requirements
Category
Required Expertise
Data Warehousing
Advanced BigQuery (Storage optimization, BQML, Analytics Functions)
Transformation
Expert-level Dataform (SQLX, JavaScript, Project Configuration), understand Cortex CDC and Merge logical code logic and flow
Category Required Expertise
Stream & Batch
Hands-on Dataflow (Apache Beam in Java or Python)
Big Data Compute
Dataproc (Spark, Hive, Hadoop ecosystem)
Programming
Proficiency in Python, Java, or Go; Expert SQL skills
Orchestration
Dataform code generation and custom namespace configuration. Cloud Composer (Airflow) or Vertex AI Pipelines
Infrastructure
Terraform for GCP Resource Provisioning (IaC)
Vijay Yadav
Available C2C candidate Hotlist
1. Manikantha---Sr. lead Java developer---10 yrs—H1b—Texas—open
2. Sweta---lead data analyst—15 yrs---h4ead—MD—fully remote
3. Karthick---lead data engg----18 yrs---GCead---NC---remote
4. Manoj---lead data engg—12 yrs—GC—TX----Open
5. Harshvardhani----lead data engg—10 yrs---h4ead—TX---remote/TX
6. Shravani----service now—10 yrs----h1b—OH---open
7. Surya---Sr. lead dot net Developer—12 yrs---H1b---TX—fully Remote
8. Bharath—lead dot net—14 yrs----h4ead—GA---open
9. Manoj—QA manager---20 yrs---USC---CA--open
10. Himabindu---MS Dynamic crm developer---10 yrs---h4ead---CA---Remote
11. Kalyani----Sr. Sap Fico—16 yrs—GC ead----TX----Remote/Tx only
12. Mohan---Sap SD/4 HANA/OTC---10 yrs—h1b----MN--open
13. Sai Krishna---pega developer—10 yrs---h1b—tx---open
14. Narmada---pega developer---14-yrs---h4ead---TX---Remote/ TX onsite
15. Ashwini---Salesforce Developer---10yrs—h4ead---PA—remote/PA onsite
16. Jyoti—Senior lead SQl/SQl server DBA—13 yrs—Usc—GA---remote
17. Bharat kumar---lead Sql DBA/principal engg—15 yrs—GC----TX---remote/TX onsite
18. Venketesh—Sr UI angular developer---15 yrs---GC—CO---remote/CO onsite
19. Sowmya Donapati--Devops engg/sre engg—13 yrs----h4ead—-CA---Remote/CA onsite
20. Surekha----lead BA----15 yrs—USC----FL----remote/Fl onsite
21. Sheetal----lead BA/scrum master---14 yrs---GC----KS—fully remote
22. Uma—cloud engg/solution architect—17yrs---h1b---TX----Fully Remote
23. Veeranjayyulu----Workday application development lead---21 yrs—USC---TX—Remote/ TX, WI, OH and MO
24. Arun---Scrum master/pm/product/release manager—USC—20 yrs---IL----remote
share me c2c requirement vij...@innovatechstaffing.com or connect me –201-292-7487 ext—847
share##c2c##Requirements##