- National Audiovisual Institute
- ICM UW
A modern concert hall and cinema in combination with fast access to enormous archives of Polish culture are the greatest attractions of the new seat of the National Audiovisual Institute. At the end of May the new building hidden among Służewiec apartment blocks welcomed the first visitors.
A large red cube, next to the building, is an immediate signal that something extraordinary is waiting inside. The red cube is an entrance to the new seat of the National Audiovisual Institute at ul. Wałbrzyska. The institution, also known under a simpler and nicer name of NInA, has so far been involved in the archivisation of the works of Polish culture, mainly films, performances and music, as well as publishing them both in traditional forms (a series of the best Polish documentaries) and online (e.g. ninateka.pl and dwutygodnik.com). NInA will continue with this mission, but now, in the new space, it can extend its activities.
The heart of the Institute are still digital archives hidden in the basement: server rooms with a 5-petabyte disk array. This corresponds to about 5 million DVDs. “Just to compare: the whole archives of the public television would take up about 1.5 petabytes”, said Michał Merczyński, NInA director, in a meeting with journalists.
To ensure the security of the collected data the servers were placed in two separate rooms. Visitors can access the resources on workstations one floor above in room 16.
“We will make available films or performances which for different reasons, usually of formal and legal nature, cannot be accessed online. The archives hold about 10 thousand items and will be updated daily”, said Filip Kwiatek, the head of the archive department. This part will be open to film and theatre lovers, as well as enthusiasts of old TV shows or radio programmes every day, except for Sundays, including two days when the place will be open till late hours.
Interdisciplinary Centre for Mathematical and Computational Modelling (ICM, www.icm.edu.pl) was set up in 1993 at Warsaw University in Poland. ICM’s mission is to support a wide range of research areas whose success depend on the development of computing sciences. This includes: biomedical sciences, physics, chemistry, theoretical biology, technology of materials, earth sciences, astronomy and modelling of non-linear processes in large-scale systems. ICM provides both high capacity computing infrastructure and support for Polish research circles, being at the same time a part of the European Infrastructural Network and partnership for advanced computing in Europe (PRACE).
Established in 1816, Warsaw University (www.uw.edu.pl) has over 53,500 students and a staff of over 6,300. It has 20 faculties and 30 other educational units. Additionally, it is among 4% of the best universities in the world.
ICM is one of the leading Polish high-performance computing (HPC) centres. Together with HPC centres in Kraków and Poznań, ICM plays a key role in the POWIEW project (HPC Infrastructure for Grand Challenges of Science and Engineering). The project under the auspices of the Ministry of Science and Higher Education, co-financed by the European Union, was launched in 2011. The project covers the installation and operation of the leading computing infrastructure between three cities, which is open to all researchers and supports most engineering applications.
After 20 years of operation, ICM’s computational and storage environment consists of products of six major vendors (according to ICT) such as NetApp and IBM. IBM Blue Gene/P and IBM Power 775 systems provide part of their computing power to the POWIEW infrastructure. When the centre was running out for storage capacity for stored files, the team of engineers decided to implement a central storage system for temporary data sets created with the use of HPC, and long-term storage of data such as home directories.
“We needed an effective high-end solution with a sequential data throughput of at least 6 GB/s with scalable capacity and high density, because space in the Data Centre is at a premium”, says Arkadiusz Niegowski, senior systems engineer at ICM, who was also responsible for defining the present requirements. “As a government organisation we held a tender, and used the best price combined with high density to select the winning solution”.
ADT Group Sp. z o.o. won the tender and beat four competitors. ADT Group was selected as the team with the best offer, also due to the highest data density. The proposed system, NetApp E5460, has 2.1 PB of RAW capacity on 720 SAS disks occupying only 12 shelves. That is why disk resources in the data processing centre take up the area of only 1.2 square metre. The project implementation was excellent. ADT Group configured and tuned the system to meet all the requirements. After all capacity tests had been completed, NetApp E5460 was integrated with the production environment. In the next stage of the project the E5460 disk array was integrated with the complex environment made up of components bought from other producers. “We had a lot to do”, says Maciej Filocha, PhD, HPC project manager and system coordinator at ICM. “Effective wiring, design for the set-up of NFS and GPFS partitions, defining user groups, servers and data export paths. The environment is very demanding, but NetApp E5460 flexibility is fully sufficient”.
High performance, maximum security
NetApp E5460 meets stringent requirements for performance and capacity without compromising the ease of management and efficiency. ICM can now easily monitor and manage the device with NetApp SANtricity software. The project team chose RAID 6 to protect the disk array if two disks from each array fail at the same time. Importantly, the bandwidth can be increased to 9 GB/s for writing and to 12 GB/s for reading.
Arkadiusz Niegowski, senior systems engineer, is very satisfied with the implemented solution: “The storage solution operates without any problems. The system needs checking only once a week. The disk array has been used for nine months and only one hard disk failed. Because the utilisation of the disk array as compared with other solutions available in the market was very high, this is a very good result”. Additionally, system recovery takes up eight times less time as compared to other, traditional RAID solutions. This is possible thanks to the Dynamic Disk Pools technology created by NetApp.
Designed for further expansion
The disk array solution has been created to meet the requirements for fast changing parameters for resource demand for temporary files created by different computing software. ICM forecasts a huge increase in the demand for resources within the next few months, when the setting up and implementation of the POWIEW infrastructure is completed. This challenge can be easily met by adding additional disk shelves. The scalability of the implemented system is sufficient so that too large an increase in the volume of generated data will not be any problem.
“Big data” solutions for science, commerce and public institutions.
Performance, availability and reliability are key features in the world of HPC environments. Some processes take up a whole month to be processed, and file sizes range from 20 GB to 70 TB and more. The ICM team has to do careful planning of how to provide access to the POWIEW infrastructure and other HPC resources to researchers. In the worst-case scenario, if the disk array fails, ICM will be able to operate without any delays for the next two hours. The reliability of the implemented data storage solutions helps to maintain high system performance while at the same time providing widely popular services, such as numerical weather forecast for Europe. Every 6 hours ICM publishes new weather forecasts for Central Europe and the Baltic Sea. Access to the service is free of charge and has earned a good reputation in the world of sport, business and sailing. Moreover, a few wind farms in Poland use the data from weather forecasts provided by ICM for daily predictions of electricity generation capacity.
“For our infrastructure high performance is a must because our main objective is to provide services to our community. So in fact this is a question of our reputation, and the field-tested reliability of the E5460 disk array is an added value. Now instead of spending time on managing the system, we can support the users so that they can make the most of our resources, from the HPC entry level to high-end level. This is what counts the most and helps us to make our mission a reality”, sums up Maciej Filocha.