New research from Hitachi Vantara has revealed that data storage needs are expected to increase by 150% in the next few years. Currently, large organisations store an average of 150 petabytes (PB) of data, but by the end of 2026, this is projected to exceed 300PB. Alongside this surge, investments in data storage are also set to more than double, increasing by 224%. Similarly, investments in AI and processing power will see a significant rise.
The study shows that 31% of IT leaders are concerned about data storage capacity as the rapid growth puts pressure on resources, with many also noting the increasing complexity of managing data. Over a third (76%) of respondents said that more than half of their data is now unstructured.
Despite this growing demand for larger, more complex data storage, the storage industry is not prepared for this increase. Seagate CCO B.S Teh commented that as AI grows, the value of data will rise, requiring more storage for longer periods. However, storage infrastructure is expected to grow at just 17% annually. This mismatch could disrupt the global storage supply, and businesses will need to develop long-term strategies to meet future storage demands, especially with the strategic rise of generative AI.
AI is the main driver behind the looming data storage challenge. The technology not only requires larger amounts of data but also demands long-term storage. The data must be available long term, not just to meet evolving legal requirements, but also to ensure AI's decisions are explainable.
The growing volume of data presents different challenges for organisations, depending on their storage approach. Outsourcing storage comes with its own security, compliance, and risk concerns, while keeping the data in-house for AI can become prohibitively expensive.
To manage the rising demand for data storage, businesses will need to focus on improving their data management strategies and architectures. Chris Harris, VP of global field engineering at Couchbase, points out that in order to fully leverage generative AI, organisations must modernise their infrastructure while developing a strong data management strategy.
As AI becomes more embedded in applications, data architectures will have to adapt to support new workloads. Companies should implement advanced data architectures that not only store records but also capture the "intelligence history" and decision-making processes of AI systems.
At Penta, we have worked hard to design and build the most comprehensive suite of IT solutions for data security, regulatory compliance, and support to suit businesses of all types and sizes. Contact us today to find out more.