Urgent Req: Very Strong AWS Kubernetes with AI DC area ( Onsite) CTC

[Murali Krishna Tummala]

Urgent Req: Very Strong AWS Kubernetes with AI DC area ( Onsite) CTC

Need Senior 14+ years experience

AI Tool Proficiency:
* Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.)

Big Data Technologies:
* Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
* Understanding of common issues like data skew and strategies to mitigate it, working with massive data volumes in PetaBytes, and troubleshooting job failures due to resource limitations, bad data, and scalability challenges.
* Real-world experience with debugging and mitigation strategies.

Container Orchestration & Kubernetes:
* Strong experience with Kubernetes architecture, concepts, and operations (pods, services, deployments, namespaces, ConfigMaps, Secrets)
* Hands-on experience with Amazon EMR on EKS (Kubernetes) for running Apache Spark workloads
* Experience with Kubernetes resource management, scheduling, and auto-scaling
* Knowledge of Helm charts for deploying and managing applications on Kubernetes
* Understanding of Kubernetes networking, storage (PVs, PVCs), and security best practices
* Experience with kubectl and Kubernetes YAML manifests
* Ability to troubleshoot Kubernetes cluster issues, pod failures, and resource constraints
* Experience integrating Spark with Kubernetes operators and dynamic allocation

Apache Spark (Development, Internals & Tuning):
* Deep understanding of Spark's core architecture - executors, tasks, stages, DAG
* Expertise in Spark performance tuning techniques: partitioning, caching, broadcast joins, etc.
* Experience troubleshooting slow running/stuck jobs or resource issues in Spark
* Proven ability to optimize Spark jobs for large-scale datasets
* Experience running Spark on Kubernetes and understanding Spark-on-K8s architecture

Cloud Technologies:
* Experience with AWS services like S3, EMR, EMR on EKS, Glue, Lambda, Athena, etc.
* Hands-on experience using S3 with Spark (e.g., dealing with file formats, consistency issues)
* Strong experience with Amazon EKS (Elastic Kubernetes Service) architecture and best practices
* Experience with AWS IAM roles for service accounts (IRSA) for Kubernetes workloads
* Knowledge of AWS networking for EKS (VPC, subnets, security groups)
* Experience with AWS monitoring and logging tools (CloudWatch, CloudTrail) for Kubernetes workloads
* Serverless knowledge (Lambda, Fargate)

Programming - Python or Scala:
* Ability to write clean, modular, and performant code
* Experience with functional programming concepts (e.g., immutability, higher-order functions)
* Real-world use cases where scalable data processing code was implemented
* Strong understanding of collections, concurrency, and memory management

SQL Skills (Window Functions, Joins, Complex Queries):
* Proficiency with SQL window functions, multi-table joins, and aggregations
* Ability to write and optimize complex SQL queries
* Experience handling edge cases like NULLs, duplicates, and ordering