DEEP Preview-Testbed exposing enhanced DEEP-Rosetta services is now available for users to experience the ease of building, developing, training and deploying machine learning models and exploiting the new DEEP training dashboard.
Although called a testbed, in reality it is a small scale production infrastructure, all software components and services are running the versions released in the DEEP-2 – coded named Rosetta – and are operated and managed as any other production service or platform. A diagram of the Pilot Preview is shown in the figure below.
Resources made available by project partners are nonetheless significant, one can exploit about 30 high end NVIDIA GPUs distributed across 3 cloud e-infrastructures and a data/storage management system that is federated between 3 providers with about 80TB of total storage. One of the main features is the data locality to the computing resources, allowing a more efficient computation.
Other features worthy to mention are: the cloud resource providers are part of the production EGI Fedcloud infrastructure, the data/storage management system is a “result” of a tight collaboration between DEEP-HybridDataCloud and eXtreme DataCloud – XDC projects, where storage resources from both projects are federated through the Onedata service (bottom of the diagram); users are authenticated and authorized through the Federated AAI service called DEEP-IAM (right side of the diagram).
Finally, and the most important highlight, the users can execute ML/AI applications in a production mode with long training of the models and using large datasets (some cases of the order of TBs).
