Blockchain and big Data: A great mariage
| 12-2-2019 | Carlo de Meijer | treasuryXL
Blockchain and Big Data are among the emerging technologies that are high on many companies’ agendas. Both are expected to radically transform the way businesses and organizations are run in the upcoming years. Long-time developing in a separate way, at first sight one might assume that these technologies are mutually exclusive. But that idea is rapidly changing.
There are growing expectations that distributed ledgers will help enterprises finally get to grips with Big Data, which thus far is struggling with a number of challenges. They are both powerful on their own, however when combined they may bring a large number of opportunities. Some even say that blockchain and Big Data are made for one another.
“Big Data is an incredibly profitable business, with revenues expected to grow to $203 billion by 2020. The data within the blockchain is predicted to be worth trillions of dollars as it continues to make its way into banking, micropayments, remittances, and other financial services. In fact, the blockchain ledger could be worth up to 20% of the total big data market by 2030, producing up to $100 billion in annual revenue.” Chris Neimeth, COO of NYC Data Science Academy.
In this blog I will look at what the interception of these two innovations may bring. Could blockchain be the solution for the existing Big Data issues and challenges?
Big data and data science/analytics: present challenges
Big Data is one of the fastest growing sectors in the world. Every business wants to get insights into usage patterns of their consumers. Massive datasets are thereby analysed using advanced statistical models and data mining. These Big Data sets will become even more prevalent over the coming years.
“It’s not the amount of data that’s important. It’s what organizations do with the data that matters. Big data can be analysed for insights that lead to better decisions and strategic business moves.” Data Analytics Company SAS
“Data analytics has become the key to corporate competitive advantage because of its role in identifying emerging market trends. In turn, companies can use this information to make quicker and better decisions that help them drive profitability”.EY
The rise of Big Data has presented a slew of issues for both big businesses and everyday consumers. With the growth in data good analytics is becoming all the more problematic. Some major problems to data management and analytics include so-called dirty data, inaccessible data, and privacy issues. And as Big Data increases in size and the web of connected devices explodes, it exposes more of companies data to potential security breaches..
With the advent of Big Data, data quality management is both more important and more challenging than ever. Companies that are dealing with large datasets should ensure that the data are clean, secure and not been modified and come from an authentic source. They have to make sure that the latest version is synchronized among all of the data centres in real time. It should also be ensured that these data are accessible. For most, however, the data silos are still a major issue and a full company-wide digital transformation is still more concept that reality.
Blockchain and Big Data: two sides of the same coin
Main question is: how do both technologies relate to each other, if any? Notwithstanding blockchain has not been explored extensively in aspects of Big Data management and analytics, both technologies could and should be seen as two sides of the same coin.
While blockchain is focused on recording validating data (data integrity), data science analyses data for actionable insight, making predictions from large amounts of data (prediction). While blockchain is changing data management, the latter is transforming the nature of transactions. Or said in another way: “If Big Data is the quantity, blockchain is the quality”.
Read the full article of our expert Carlo de Meijer on LinkedIn

Economist and researcher





Last year I wrote a blog on the Hyperledger project and what that could mean for blockchain acceptance (see my blog: Hyperledger Project: collaboration pays off, 9 April 2017). We are now almost a year later and I am wondering if they are meeting my expectations. “2017 was a milestone year for Hyperledger both for new members and for new technical breakthroughs. In 2017 we doubled our membership, gaining companies like American Express, Cisco, Daimler and Baidu, and we’re expecting more companies and organizations to join in 2018.” Brian Behlendorf, Executive Director, Hyperledger.
Blockchain technology enables real-time settlement finality in the securities world. This may mean the end of a number of players in the post trade area. For a long time, central securities depositories (CSDs), as intermediators in the post-trade processing chain, were expected to become obsolete. CSDs, but also other existing players in the post-trade environment, are however changing their mind on these new technologies and on their future position in the blockchain world. Increasing regulation, legacy systems and costs pressures, are drivers for CSDs to at least embrace some aspects of blockchain. They are increasingly considering them as enabler of more efficient processing of existing and new services, instead of a threat to their existence. It is interesting to see that some of these actors – who could be potentially big losers in a distributed ledger technology (DLT) or blockchain system – are open to innovation with blockchain and willing to invest in DLT. Last January SWIFT and seven CSDs worldwide agreed on a Memorandum of Understanding to explore the use of blockchain technology in the post trade process esp. e-proxy voting.
Almost a year ago I wrote my blog “Blockchain and the Ripple effect: did it Ripple?”. Now twelve months later we may conclude it did. And even more than that. Ripple is making many waves. A lot happened both in broadening their offerings and in enlarging their network. A growing number of banks and payment providers, increasingly join RippleNet, Ripple’s decentralized global network, to “process cross-border payments efficiently in real time with end-to-end tracking and certainty”. By using the growing set of Ripple solutions they are able to expand payments offerings into new markets that are otherwise too difficult or too expensive to reach. The focus of Ripple therefor has especially moved towards emerging markets.