Toshiba in Japan has announced it has created an algorithm that can crunch data faster than the world’s fastest supercomputer – apparently using everyday desktop PC technologies.
“This news highlights the pivotal role algorithms play in machine learning and future business competitiveness,” says Brett StClair, CEO of Teraflow, a data engineering firm with offices in London, Johannesburg and Cape Town.
What the algorithm does
Toshiba’s Simulated Bifurcation Algorithm is a breakthrough in solving combinatorial optimisation. This involves a class of problems in which the addition of each new element to a set of items increases the possible combinations exponentially, until calculating the best combination becomes impossible using traditional computers.
Real-world examples include selecting optimal logistic routes, reducing traffic congestion, selecting lowest cost/highest return financial investments, or designing molecules for drug development. It is this type of problem to which quantum computers will one day be applied, once they are sufficiently advanced.
“The ability to now solve these problems using classical computers shows how the right algorithm can make all the difference,” says StClair.
Why it’s important
Many organisations still handle data science and machine learning in an inefficient and repetitive manner, instead of developing an automated and integrated approach to data preparation, the machine learning process, and the deployment of AI models to their live business operations.
“When the data science team focuses strictly on the creation (or effective selection) and application of the right algorithms, it can solve business problems faster and more efficiently,” says StClair. “Every other consideration, like data cleaning, should be subordinated to that goal.” He refers to this as the “operationalisation of data and productionisation of AI”.
StClair likens the approach to DevOps, where software developers focus on writing code and solving business problems, while their code is compiled, tested, integrated and deployed by a background system that is typically both centralised and automated. “This is how organisations can reap the greatest benefits and maximize the return on their machine learning investment,” he says.
Algorithms in the Cloud
Even though the Toshiba algorithm can be run on desktop machines, accelerated computation is achieved by parallelising computers. “Our algorithm thus highly resonates with the technology trends in parallel computing,” states the company’s website.
This highlights another implication of data science. As their data grows, it is becoming increasingly difficult for organisations to develop automated decision-making under the computing and storage constraints of their corporate systems.
“Many are turning to the Cloud, not only for unlimited compute and storage but also unlimited parallelised processing,” says StClair. “There is no way to become an AI-driven organisation without it.”
Calculated competitiveness
The competitiveness of future organisations will hinge not only on how well they employ AI within their business but the agility with which they can evolve their algorithm portfolio to meet new business challenges.
“Companies cannot keep rolling their models manually on a corporate system without eventually running out of resources,” warns StClair. “Toshiba’s algorithm shows us what’s possible and that investment in a Cloud-based, systematized approach is worth it.”