Brazil
GFT Technologies SE
Zup Innovation
Senior Data Engineer
Brazil
São Paulo, São Paulo, Brazil
PythonSQLAWSTerraform
Automatic translation. Check the original.
Why be a Zupper?
Let's get straight to the point: what we offer is an environment of exponential growth. More than just pretty words, this means we provide all the opportunities for you to lead your own development and all the necessary support.
Since leadership without support doesn't lead to accelerated development, in addition to opportunities, every Zupper has a career development plan, close guidance from a People team member, and access to a proprietary platform that makes the feeling of progress tangible.
And why do we do all this?
Because we believe that only an environment of exponential growth will get us where we want to go: using technology to transform people's lives, empower talents wasted due to lack of access, and position underprivileged regions as hubs of quality technology.
Oh, and we can't forget: at Zup, we believe in #Liberdade! With this, Zuppers can work from wherever they want, whether it's home office, in our offices, or in a hybrid model.
What you'll do here
- Participate in the process of understanding business needs, comprehending and ensuring the delivery of value;
- Design the architecture and evaluate available technologies to solve the right problem;
- Develop and evolve scalable data pipelines prepared to feed Generative AI models and RAG solutions;
- Implement Data pipelines for data democratization in a Data Mesh architecture, creating semantic layers and structured data for consumption by AI;
- Combine architecture and technology to develop and deliver reusable components such as connectors, APIs, SDKs, CLIs, integrating with APIs of LLMs (OpenAI, Claude, Gemini);
- Prototype intelligent agents using frameworks like LangChain, LangGraph, and CrewAI, evaluating cost, latency, and integration quality;
- Test and automate everything possible;
- Validate and adapt components according to user feedback;
- Collaborate on the quality and technological innovation of the product and work environment.
What we expect you to know
- Knowledge in large-scale data processing, parallel and distributed processing (Apache Spark, Hadoop, Hive, etc);
- Hands-on experience in ELT using dbt, Data Lake/Lakehouse, and orchestration with Airflow;
- Experience with AWS Data & Analytics services (Glue, EMR, Athena, Quicksight);
- Knowledge of Python programming language;
- Proficiency in core AWS services (S3, Lambda, SQS, DynamoDB, API Gateway);
- Proficiency in using LLMs, practical application of embeddings, RAG (Retrieval-Augmented Generation), and Vector Databases (Pinecone, Milvus, pgvector, Qdrant, etc.);
- Skill in Prompt Engineering and orchestration with frameworks like LangChain or LangGraph;
- Consistent hands-on experience with relational (SQL) and NoSQL databases (document and key-value);
- Proficiency in DevOps practices and infrastructure as code (Terraform, CodePipeline, etc).
What would be great if you knew
- Previous experience in data democratization initiatives with Data Mesh;
- Experience in benchmarking different market LLMs considering cost-benefit and latency;
- Experience with generative AI or automation tools applied to data pipelines, such as assisted code generation, automated documentation, or intelligent data quality monitoring;
- Knowledge of Observability (Cloudwatch, CloudTrail, Prometheus, AppDynamics, Splunk, ElasticSearch, Grafana, etc.);
- Knowledge in advanced caching techniques for scalable AI systems;
- Having experience in developing solutions for financial institutions.
What we offer you
Here, not only your career matters, we want you to have health and physical and mental well-being for the best professional and personal experience journey of your life.
Career
- Freedom to work from wherever you want
- Flexible hours*
- Education Assistance
- Proprietary career development tool
- Internal guilds and other study and interest groups
Health and well-being
- Health plan
- Dental plan
- Partnership for medication purchase
- Telemedicine: medical assistance available 24x7
- Free online therapy
- Wellhub
- Extended maternity leave
- Extended paternity leave
- CAZ – Zupper Assistance Center
Financial comfort
- Meal and food voucher
- Life insurance
- Transportation voucher
- Home office assistance
- Childcare assistance
- Phone plan assistance
- Profit and Results Sharing
If you are ready to transform data into strategic decisions and want to be part of an environment that values innovation and growth, come be a Zupper! 🚀
More opportunities with one profile
Want to be considered for more roles without repeating the whole process? Create your profile and be found by companies using Nort.
Create my profile