A powerful database in the Azure cloud in a project for Respect Energy

We have designed and implemented a highly efficient solution based on Microsoft Azure and Snowflake for our client, a green energy supplier. The combination of these cloud technologies allowed us to shorten data operations from a few hours to mere minutes and reduce costs by almost 30%.

Business Intelligence tool in a cloud environment

In order to obtain key data for making decisions at the operational and strategic level, the client obtained data from many sources, which then went to Cassandra, a Key Value Store database. This was not a satisfactory solution, as it did not allow for full control of the costs of data operations (the client paid for the maintenance of unused virtual machines, ineffective queries sent to the database, etc.). Furthermore, they used the Digital Ocean cloud, which did not meet expectations and did not offer all the services required.

The database was inadequate in terms of queries and was slow, and the overload generated additional costs. Therefore, the client needed an efficient Business Intelligence tool in a cloud environment that would meet their expectations and allow for cost optimization. At the same time, they had previously used a number of different tools and preferred to avoid retraining the team.

Project challenges

• Checking the possibilities of available cloud solutions.
• Organization of infrastructure – transferring the entire database to the cloud.
• Creation of a data warehouse enabling the integration of data from various levels and sources.
• The continued use of previously used tools and scripts without the need to retrain the client’s specialists.

Architecture based on MS Azure cloud

We wanted the client to be able to regain control over costs, which is why we decided to present the possibilities of cloud-based architecture. As part of PoC, we recreated some of the data and reports from the Cassandra database. Based on our previous experience, and at the same time encouraged by the positive attitude of the client, we proposed solution architecture based on the tried and tested Microsoft Azure cloud.

We confirmed that Azure was a suitable environment for embedding the Snowflake database technology as well as the possibility of using the existing scripts and tools in the Azure cloud. Subsequently we conducted an audit and performance tests, confirming Snowflake’s superior performance over Cassandra.

A high-performance Business Intelligence tool

With data security foremost in our minds, we transferred further parts of the Cassandra database and built the foundations of a data warehouse which will ultimately integrate data from various sources.

The client has gained modern reporting and analytical tools which allow for the optimization of costs. The solution allows the user to create in-depth analyses through the use of Machine Learning algorithms. The new tools do not require employees to be retrained. The client is now able to use components such as Docker or Kubernetes independently, which leads to higher efficiency while maintaining an optimal level of costs.

Customer benefits

• • Reduction of the process time from several hours to mere minutes.
  • Replacement of the Cassandra database with a tool from a similar price range, but with much greater efficiency.
  • An ordered data structure and having a data warehouse that enables the integration of data from many other systems.
  • Support and knowledge of the official Microsoft partner, Snowflake, and a leader in Business Intelligence implementations in Poland.

Azure cloud services used in the project

Data Factory, Functions, Logic Apps, DevOps, SQL Server, Virtual Machines, Blob Storage, Key Vault