Role: Snowflake Data Engineer
Job Summary:
QBrainX is looking for an experienced Data Engineer to join our dynamic team. The successful candidate will be responsible for designing, building, and maintaining our data pipelines and infrastructure, ensuring that our data is accurate, reliable, and easily accessible to our users. This individual will work closely with our data scientists, domain experts, and leadership teams to deliver high-quality data solutions that meet our business needs.
Key Responsibilities:
• Design, build and maintain our data pipelines and infrastructure using Snowflake, Python, Spark, Scala, and Airflow.
• Develop, optimize and maintain complex SQL queries for structured and unstructured data.
• Operate in a cloud environment such as AWS or Azure to develop and maintain data solutions.
• Implement Continuous Integration and Continuous Deployment (CI/CD) processes using CircleCI, Git, and GitHub pull requests.
• Implement real-time data replication/streaming tools such as Kafka.
• Use SDLC methodologies and defect tracking tools such as JIRA, Azure DevOps, ServiceNow, etc.
• Communicate effectively with the client and leadership teams to deliver high-quality data solutions that meet business needs.
• Collaborate with domain experts such as marketing teams, sales managers, and finance teams to understand business requirements and deliver data solutions that support their needs.
• Work with data scientists to develop machine learning (ML) models and statistical analysis.
Requirements:
• Bachelor's degree in Computer Science, Engineering, or related field. • 3+ years of experience as a Data Engineer or related field. • Strong expertise in Snowflake, Python, Spark, Scala, Airflow, and SQL. • Experience with cloud environments such as AWS or Azure. • Experience with CI/CD processes using CircleCI, Git, and GitHub pull requests. • Familiarity with real-time data replication/streaming tools such as Kafka. • Familiarity with SDLC methodologies and defect tracking tools such as JIRA, Azure DevOps, ServiceNow, etc. • Strong communication skills and ability to work collaboratively with clients and domain experts. • Knowledge of machine learning (ML) models and statistical analysis.