“Digital information is growing at such a rapid rate and in such dramatic volumes that traditional storage systems used to house and manage it will eventually run out of runway,” Tom Rosamilia, senior vice president of IBM's systems and technology group, said in a mid-May presentation to industry analysts. References were made to Watson, which had access to 200 million pages of structured and unstructured data (including the full text of Wikipedia) in its Jeopardy television match . Watson was using “elastic storage” that enabled it to load around five terabytes of “knowledge” into its memory in minutes.
Now IBM Research has demonstrated that elastic storage technology can scan 10 billion files on a single cluster in just 43 minutes. Rosamilia says that software-defined elastic storage is “application aware.” It virtualizes storage and allows multiple systems and applications to share storage pools. Storage allocations are performed transparently without modifying applications or manually managing storage. Instead, the technology can reference business rules and learn about how applications function and the types of storage they will require. “Customers can have continuous and highly-available access to data in the event of software or hardware failures,” IBM said in a press release.
Elastic storage that is virtual and software-defined is already making a difference in semiconductor integrated circuit design, production, and time to market.
“Total cycle time is key in the semiconductor integrated circuit design-to-production process,” said Alan Malek, director of IT at Cypress Semiconductor. “The goal is to do everything possible to reduce costs while improving productivity.”
Storage is always a problem for the design organization, Malek said. “One of the issues we have contended with is that we have always addressed storage in the past by adding to storage when we have needed to on an ad hoc basis. In essence, we were meeting our integrated circuit production needs in a piecemeal fashion.”
That process, though imperfect, continued to rule in the datacenter — until Cypress experienced an explosion in data originating from its design process and from the burgeoning requirements it was receiving from its customers. “As integrated circuit designs grew more complex, there was also more to worry about. This is because the design process is now more complicated and more time-consuming.”
Malek's CEO phoned him one day. “The call was prompted by a particular incident where inefficient storage in the datacenter caused product to be late to market. The CEO said that this situation just can't happen, and the goal was to fix this.”
He and his staff looked at all the options. “We had been using an open-source distributed file solution for our storage that had passed our performance tests and had also kept up with our product designs through the test phases, but when we took the product out of the design phase and wanted to move it to the plant for production, the system broke. This prompted the call from the CEO.”
Cypress IT reviewed its storage process and identified several problems. The team surveyed the market for a solution and eventually chose IBM's elastic storage.
We performed a proof of concept of the elastic storage for a 10-day period, and our findings were remarkable. Rigorously testing elastic storage on the same hardware that we had been using for our distributed file system, we found around 8.5 to 13 times improvement in processing time. This really raised our eyebrows, so we ran the tests again and got similar results. Since then, we have rolled the solution out over our existing hardware, and it has delivered great flexibility without creating any storage bottlenecks. Design-to-production cycle times have been significantly reduced. Best of all, the CEO isn't calling me.
Let us know in the comment section below how you see emerging technology improving the electronics industry and the supply chain.