Statistics Finland is the national statistics office for the government of Finland, responsible for producing publicly available statistics on everything from the country’s economic status to public health and the environment. Nearly 160 different sets of statistics are derived from existing registers and new surveys conducted by the bureau, each with its own home page and tables that link to a database. Employing 800 people, it’s a rigorous and expertly honed operation, providing an invaluable window into the state of the nation.
Within Statistics Finland, the National Accounts team produces statistics that describe Finland’s national economy systematically and according to international recommendations. The National Accounts team collects, combines, and analyses an immense mass of data that reports crucial facts about the Finnish economy.
As a whole, Statistics Finland runs on Microsoft databases, with a local SQL Server with Analysis Services providing the analytical data engine. For the National Accounts team, which also runs on this infrastructure, reliability is paramount. The goal is always to produce high-quality and consistent statistics to support decision-making, research, and democratic social debate. To achieve this, the team has to be confident that data modeling processes are consistent.
The National Accounts team needed a BI solution that could help identify anomalies that might reflect a flaw in the coding which could subsequently be fixed. An international tender was run to find a solution. At the end of an exacting government procurement process, an early version of the Pyramid Platform, BI Office, won out over competitors because of functionality, usability, and cost. So successful was the investment that the National Accounts team at Statistics Finland has continued with Pyramid through version updates and new product iterations
The bureau opted for 100 user licences, divided between administrators and statisticians who use Pyramid to validate their work, analyse the data for errors and unexpected variations, or identify sources of imbalance. Errors exposed by Pyramid prompt micro-level corrections at the data source, or a re-evaluation of methods used earlier in the pipeline.
By slicing through line charts overlaid on top of averages or totals, Pyramid enables unusual numbers in a subset of a bigger data set to be identified – a small sector of businesses, for example, from a large industry database. When an anomaly has been detected, statisticians can check the original data, or the model used to process the data, and see if mistakes in coding explain the outlier.
“When you are working with so much data; when you’re holding so many strands of information in your hands, you want to have a visual understanding if something’s wrong,” explained Lappalainen. “Most of the work carried out with Pyramid is about looking at year-on-year data to make sure it is consistent.”
Using the Pyramid Decision Intelligence Platform has encouraged ongoing dialogue between statisticians and administrators, where the goal is to continually improve the accuracy of published data. Statisticians will ask the administrators to set up new discovery views that allow them to compare statistics automatically, side-by-side with visual representations, rather than having to dissect numbers manually in spreadsheets.
Pyramid plays an important role in ensuring Statistics Finland produces reliable and impartial statistics on Finnish society, for the benefit of citizens, decision-makers, researchers, and other information users. Not only is it is improving accuracy, but it also helps stay on top of growing volumes of data from different sources in a timely and efficient way.
“The biggest benefit for us has been the flexibility of the UI [User Interface] in the discovery phase, where we can track various statistics and how they change over various dimensions,” said Aaro Lappalainen. “The ability of Pyramid to mix and match sources from all the data we consume in a year has made our daily tasks faster and easier. It’s very intuitive.”
For statisticians, the UI in the data modeling stage has been a big win, making light work of comparing data from multiple sources, along with self-service capabilities that allow them to add their own piece of code to try something out and transform data in different ways.
For administrators, vendor support from Pyramid has been pivotal. “When we have a new requirement, they have been quick to respond. When I wanted to test something, I got an email confirming they would try it within the same day. They have been terrific to deal with,” said Lappalainen. “The formatting tools get better and easier to use with every version, which helps us to look at data in different ways.”
There have even been occasions when Lappalainen and other members of the team have requested adjustments that have gone into the Pyramid product and helped further automate data modeling processes. In the future, the plan is to use more Pyramid functionality and harness its dashboards and machine learning components.
The formatting tools get better and easier to use with every version, which helps us to look at data in different ways.