Artificial Intelligence Data Engineer

Описание

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Build reliable batch and real-time data processing systems to support AI/ML applications.
• Develop data ingestion, transformation, integration, and orchestration frameworks.
• Optimize data workflows for performance, scalability, and cost efficiency.
• Build and maintain data infrastructure supporting machine learning, deep learning, and Generative AI applications.
• Develop data pipelines for model training, validation, testing, and deployment.
• Work with vector databases, embeddings, and retrieval-augmented generation (RAG) architectures.
• Prepare, clean, transform, and validate datasets for AI model development.
• Support feature engineering, feature stores, and model data management.
• Design scalable data architectures using modern data platforms and distributed computing frameworks.
• Work with cloud environments such as AWS, Microsoft Azure, or Google Cloud Platform.
• Implement data lakes, data warehouses, and lakehouse architectures.
• Leverage technologies such as Apache Spark, Kafka, Airflow, Databricks, Snowflake, and Delta Lake.
• Establish data quality checks, monitoring, validation, and observability frameworks.
• Ensure data consistency, integrity, security, and compliance.
• Implement access controls, metadata management, and data governance practices.
• Identify and resolve data pipeline bottlenecks and infrastructure issues.
• Collaborate with AI engineers, ML engineers, data scientists, and product teams.
• Lead technical discussions, architecture decisions, and engineering best practices.
• Mentor junior engineers and drive continuous improvements in data engineering processes.
• Evaluate emerging AI and data technologies to improve platform capabilities.

Requirements

• 7–18 years of experience in Data Engineering, AI Data Engineering, or related engineering roles.
• Strong proficiency in Python, SQL, and data processing frameworks.
• Hands-on experience with Apache Spark, Kafka, Airflow, and distributed data systems.
• Experience building large-scale ETL/ELT pipelines and modern data architectures.
• Strong understanding of AI/ML data workflows and Generative AI infrastructure.
• Experience with cloud platforms (AWS, Azure, or GCP).
• Knowledge of vector databases, embeddings, RAG pipelines, and unstructured data processing.
• Experience with data warehouses, data lakes, and lakehouse platforms.
• Strong understanding of database technologies, data modelling, and optimization techniques.
• Familiarity with Docker, Kubernetes, CI/CD, and infrastructure automation.

Nice to have

• Experience working with LLMs and AI-powered applications.
• Exposure to LangChain, LlamaIndex, and model-serving infrastructure.
• Knowledge of MLOps, feature stores, and model lifecycle management.
• Experience with Databricks, Snowflake, BigQuery, or Redshift.
• Familiarity with data observability and governance tools.

We offer

• Salary range: Rs 4500000 - Rs 6500000 (ie INR 45 - 65 LPA)

Hiring process

• Apply via the provided link.