Senior Data Engineer
1 500 000 – 2 000 000 KZTв месяц
NJSC “Social Health Insurance Fund”AstanaВ офисеот 4 летПолная занятость
PythonSQLPostgreSQLClickHouseData EngineeringETLAirflowDWHELTAI PlatformsDagsterPrefect
Описание
The compulsory medical insurance system operator ensures the provision of medical care to the population.
Responsibilities
- Building and developing the data layer for a unified AI platform of the Fund
- Integrating DWH, internal information systems, documents, and other data sources
- Developing ETL/ELT pipelines, data marts, and data quality control mechanisms
- Preparing reproducible training datasets and features for ML models
- Preparing and maintaining data delivery pipelines in production in collaboration with MLOps/LLMOps
- Forming sources for RAG: regulations, internal documents, knowledge bases, metadata, updating and indexing documents
- Organizing access control, data storage, and auditing
Requirements
- 4+ years of experience as a Data Engineer
- Proficient in SQL and Python
- Experience in designing DWH, data marts, and integration solutions
- Practical experience in building ETL/ELT pipelines using Airflow, Dagster, Prefect or similar
- Knowledge of PostgreSQL, ClickHouse, MS SQL, Oracle or similar DBMS
- Experience working with APIs, message queues, and file storage
- Understanding of data quality, data lineage, metadata management, and data access
- Experience processing unstructured documents and preparing data for search/RAG
Nice to have
- Experience with Kafka, Debezium, dbt, Spark, MinIO/S3
- Experience with Qdrant, OpenSearch/Elasticsearch or other search and vector databases
- Experience working with government, medical, or insurance data
- Understanding of information security requirements and working in closed environments
We offer
- 1,500,000 to 2,000,000 KZT Gross per month
- Astana, on-site
- Full-time