How to navigate through the ocean of dark data
The growing amount of data is creating a nightmare for companies. In some cases, too much data can even be a major roadblock for digital transformation.
The growing amount of data is creating a nightmare for companies. In some cases, too much data can even be a major roadblock for digital transformation.
99.5% of all collected data is never used.
To improve safety, operational efficiency, and reduce emissions, big data is becoming a trendy topic in the shipping industry. While a large amount of information is collected, the majority of it ends up as "dark data", ie, the type of data organizations collect and store but fail to use.
This normally happens because companies collect data without a strategy and alignment. In the maritime industry specifically, it's common to see low-quality data due to reasons like lack of context, complex phenomena, non-automated data entry, lack of interface standards, etc.
Examples of dark data in the maritime industry:
● Routine inspection data
● Maintenance records
● Fuel consumption logs
● Cargo tracking data
● Deficiency recordsIncident reports
● Checklists
With the upside of automation and better decision support, shipping companies are investing in technology and talent to capture more data. However, the growing amount of data is creating a nightmare for companies - processing data is far slower than accumulating data, and storing a large amount of data can be expensive. This makes it easy for companies to get drowned in the dark data ocean before any business value is generated. This is a major roadblock on the way to digitization.
1. Track what matters
Big data is often defined by attributes like high volume, high speed, and/or high variety. This means the conventional data processing techniques are insufficient to process data efficiently. Especially nowadays the majority of shipping companies still have limitations on computer capacity and processing power. This makes it even more important to decide on the objectives of leveraging data before getting drowned in the data ocean.
In short, getting big insights from big data is not about buying technologies to collect data, but firstly and more importantly about an aligned strategy and clearly defined outcome goals.
2. Choose technologies that work today
The priority for the maritime industry should be to possess a high-quality dataset. This means that data needs to correctly represent the real-world situation. Only reliable and standardized data can produce insights that resolve real-world problems. However, collecting high-quality data is one of the most challenging factors to be tackled in the maritime industry.
Due to market turbulence, it's challenging for shipping companies to invest a large amount of money in connectivity or sensors on every ship of the fleet.
Though it is said that the shipping industry is currently on its way into its fourth technical revolution, innovation doesn't have to be revolutionary. In the foreseeable future, humans will still be the center of shipping. Technology, therefore, needs to empower humans to collect reliable data - instead of replacing them.
3. Establish a data culture
There are three types of data culture: Data-informed, data-driven, and data-inspired. Each of them supports a different part of the decision-making process.
● Being data-informed means that while the company is collecting and organizing data, the staff knows how to access them. This allows an understanding of the current performance based on KPIs. To make it easier for staff to be data-informed, visualized reports and/or dashboards are necessary.
● Data-driven means that companies are using data to make decisions. This means that the right data must be collected to assist decision-making. The types of questions that can be answered are for example: “Which types of paint coating work the best?”, “What should we budget for this type of vessel in the future?”
● Instead of coming up with ideas based on experience and intuition, being data-inspired helps to come up with ideas that are supported by facts already.
Having a good data culture means consistently referring to data while collecting information, making decisions, and coming up with new ideas. A tool that automates reporting, insight generation, and trend exploration makes it easy to establish a proper data culture.
About Kaiko Systems
Kaiko Systems empowers manual vessel health data collection for a safer, smarter, and more collaborative maritime industry.
