Thoughtworks 3 дня назад
Lead Data Engineer
- Bangalore, India
- Гибрид
- Опыт от 10 лет
SQLData EngineeringNoSQLData ModelingApache SparkData QualityBig DataData GovernanceData SecurityData Privacy
Описание
Responsibilities
- You will lead and manage data engineering projects from inception to completion, including goal-setting, scope definition and ensuring on-time delivery with cross team collaboration.
- You will collaborate with stakeholders to understand their strategic objectives and identify opportunities to leverage data and data quality.
- You will design, develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions.
- You will be responsible to create, design and develop intricate data processing pipelines, addressing clients' most challenging problems.
- You will collaborate with data scientists to design scalable implementations of their models.
- You write clean and iterative code based on TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines.
- You will lead and advise clients on how to use different distributed storage and computing technologies from the plethora of options available.
- You will develop data models by selecting from a variety of modeling techniques and implementing the chosen data model using the appropriate technology stack.
- You will be responsible for data governance, data security and data privacy to support business and compliance requirements.
- You will define the strategy for and incorporate data quality into your day-to-day work.
Requirements
- You have 10+ years of experience in leading the system design and implementation of technical solutions.
- Working with data excites you; You have created Big Data architecture, can build and operate data pipelines, and maintain data storage, all within distributed systems.
- You have a deep understanding of data modelling and experience with modern data engineering tools and platforms.
- You have experience in writing clean, high-quality code using Pyspark programming language.
- You have built and deployed large-scale data pipelines and data-centric applications using any of the distributed storage platforms and distributed processing platforms in a production setting.
- You have experience with data visualisation techniques and can communicate the insights as per the audience.
- You have experience with data-driven approaches and can apply data security and privacy strategy to solve business problems.
- You have experience with different types of databases (i.e.: SQL, NoSQL, data lake, data schemas, etc.).
- You understand the importance of stakeholder management and can easily liaise between clients and other key stakeholders throughout projects, ensuring buy-in and gaining trust along the way.
- You are resilient in ambiguous situations and can adapt your role to approach challenges from multiple perspectives.
- You don’t shy away from risks or conflicts, instead you take them on and skillfully manage them.
- You coach, mentor and motivate others and you aspire to influence teammates to take positive action and accountability for their work.
- You enjoy influencing others and always advocate for technical excellence while being open to change when needed.
- You are a proven leader with a track record of encouraging teammates in their professional development and relationships.
- Cultivating strong partnerships comes naturally to you; You understand the importance of relationship building and how it can bring new opportunities to our business.
We offer
- Learning & Development
- There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you.
- But we also balance autonomy with the strength of our cultivation culture.
- This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow.
- We see value in helping each other be our best and that extends to empowering our employees in their career journeys.
Hiring process
- Responsible Use of AI in Recruitment
- At Thoughtworks, we use AI tools to support our recruitment team with administrative tasks such as drafting communications, scheduling interviews and writing job descriptions.
- Crucially, our AI tools do not screen, assess, rank or make hiring decisions.
- Every application is reviewed by our team and all selection decisions are made exclusively by our interviewers and hiring managers.
- We are committed to fairness and responsible AI.
- We actively manage our AI systems by testing, monitoring for biased outcomes and implementing mitigation measures.
- We hold our third-party vendors to these same high standards through a rigorous governance process.
- For additional information, please see our full Thoughtworks AI Policy for Recruitment.
Other things to know
- About Thoughtworks
- Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech.
- As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work.
- For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator.
- Bring your brilliant expertise and commitment for continuous learning to Thoughtworks.
- Together, let's be extraordinary.
- #LI-Hybrid