LUPA: A Data & MLOps Platform

Siloed teams

Data teams operated in silos, each with their own structure, technologies, and custom tweaks that accumulated technical debt

‍

Lack of scalable systems

No framework was in place to build reliable, uniform, and reproducible pipelines at scale

‍

Slow time-to-market

Delivery cycles were far longer than the business expected, limiting investment and reducing confidence in the department

LUPA or Lightweight Universal Profiles Accelerator, is an MLOps platform designed to empower the work of both Data Scientists and Data Engineers. It manages the necessary infrastructure and implements a unified framework to reduce a project’s time-to-market, helping eliminate project’s hidden technical debt.

‍

LUPA was built in a market orientation fashion, empowering data teams to take ownership of their products. At its core, LUPA uses only open-source components such as Spark, Airflow, Soda and MLflow. Its agnostic design allows the adoption of other open source tools.

‍

A cloud version, CLUPA, integrates with Google Cloud Platform through Terraform, enabling developers to set up infrastructure in just two days.

LUPA has become the backbone of the advanced analytics department, serving as the foundation for more than 20 data science projects and involving over 80 developers - an adoption rate of 80%. Onboarding new projects now takes just two days thanks to the platform’s standardized framework.

‍

Business owners are experiencing a new reality: time-to-market has decreased by approximately 75%. This encourages them to iterate on existing projects and explore new business opportunities, rather than being constrained by lengthy timelines.

‍

Beyond reducing time-to-market, LUPA supports large-scale projects that represent million-euro impact on the company’s revenue. For example, it powers the company’s recommendation system that selects personalized offers for up to 1.5 million clients.