Category: Science Cloud

Serious vulnerability in pwnkit (CVE-2021-4034)

Pwnkit is installed by default in most linux distributions, there is no permanent fix yet but there is a workaround, you can remove the suid bit from the binary using chmod 0755 /usr/bin/pkexec and that will make it impossible to exploit this bug.

  • Pkexec is installed by default on all major Linux distributions.
  • Pkexec has been vulnerable since its creation in May 2009.
  • Any unprivileged local user can exploit this vulnerability to get full root privileges.

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4034

Serious vulnerability in sudo (CVE-2021-3156)

Make sure to install the latest security updates in your instances to fix a Serious vulnerability in sudo (CVE-2021-3156) that will let any user run any command as root without entering a password.

In combination with other less severe security exploits this can in some cases be used to compromise your instances remotely.

Read more about it: https://www.openwall.com/lists/oss-security/2021/01/26/3

Scheduled downtime for the HPC2N region of SSC – 3/9 2018

One of the central network switches in the storage-infrastructure will be replaced due to a failed fan and to do this we must shut down all running instances in the HPC2N region on 3/9 2018 between 08:00 and 12:00.

This will only effect the HPC2N region and will not have any impact on instances running in the other SSC regions.

However login via the web dashboard to the other regions will not work during this maintenance period.

Information Security Workshop

The work on SNIC Science Cloud is progressing rapidly. Today we are operating two fully functional regions (at HPC2N and C3SE), with capacity to host over 2000 VCPUs. That capacity will soon double with the addition of our third region provided by the UPPMAX centre.

As part of assessing the readiness of the project to start delivering production resources, we gathered last week for an Information Security Workshop organized by the SNIC office. It was two intensive days filled with interesting potential scenarios, and corresponding mitigation options.

One major outcome of the workshop was a more extensive list of topics that we need to carefully explain to the users of the IaaS systems, in a structured manner. This is important input that we will bring with us into the work on finalizing a draft service description for SSC, and a security guideline, that we are currently working on.

SSC in pre-production

It has been a busy spring in the SSC team. Following instructions from SNIC to start converging to a production-level national service, we have worked hard to refine and automate our deployment and management of the cloud control planes (in the process standardizing on e.g. operating systems and management software across all regions). We now have a common operations base, contributed to from all participating centra and publicly available on GitHub.

We are happy to announce that we open up for project requests again, in what we call the “pre-production stage”. To get access to the resources, simply make a new project request in SUPR, following the instructions here.

During pre-production, we have all intention to provide a stable, well supported infrastructure mature for real use-cases. We offer SNIC’s normal best offer support. However, we acknowledge the fact that there might be unforeseen modifications needed when then number of users ramps up again. Hence, we reserve the right to have frequent service windows, sometimes with short announcement times during the pre-production stage. All service descriptions, policies etc. regarding usage and quotas should also be considered in draft-stage (we need this pre-production time to verify that our models hold up in practice). By using SSC in the pre-production stage,  we also ask you to be helpful to report any issues you encounter swiftly, and we might reach out to you as a user for feedback on critical functionality  and documentation to help us harden the setup further.

If you were a pilot user of SSC in 2016/2017, you will now notice the following major changes:

  1. In SUPR the cloud resource is now associated with the SSC-metacenter rather than UPPMAX.  There is a dedicated round associated with SSC.
  2. When requesting the project, the unit for the resource is “Coins”. We are in the process of implementing an accounting system that resembles commercial clouds, to better support the “cloud economic model” of usage. In pre-production, you can ignore this number (just write e.g. 1000 in the box), but it will become used in production stage.
  3. Account management has been reworked, and now uses SUPR as the identity provider. This means that in order to log into the dashboard, you will log into SUPR first to prove your identity. This means that SWAMID is supported (and the preferred mechanism for authentication to Horizon).
  4. There are two independent (and hence resilient) but harmonized regions,  HPC2N and C3SE. Each currently offer the same set of services. You are welcome to use either, or both. Since the regions have HW of different quality, later there will be a differentiated cost.

During the pre-production stage we will continue the work on:

  • Finalizing the cost/quota model and implementing accounting with SAMS.
  • Improve our automated monitoring of systems.
  • Scaling regions with more HW.
  • Finalizing an offical service description.
  • Finalizing end-user policies, incl.  security considerations.

The following timelines now apply:

  • March 21-August 30: Pre-production stage opens.
  • April 4: The old “SMOG” cloud is decommissioned.
  • April 5-August 30, “Dis”, a new large region based on UPPMAX-Tintin resources is added to SSC.
  • August 30 – Production stage (pending final review).

Welcome back to SSC!

The SNIC Science Cloud Team

 

Towards production in 2017

It has been a busy 2016 for the SSC team. We have served more than 60 pilot projects, conducted both beginning and more advanced level training at several locations in Sweden, and started working on a hardened infrastructure. Since competency renewal on OpenStack operations is expected to be one key challenge for SSC long-term, we have taken measures to standardize operations across regions to facilitate a joint, national responsibility for operations. At the end of the year, SNIC conducted a thorough evaluation of the project, looking specifically at whether the project had succeeded in creating services of value to the research community.

We are happy to announce that the outcome of this processes is a decision to converge towards production resources in 2017. This is great news for end-users, since this will mean a higher level of service and support.

SSC closes for new project request in Q1 order to transition to a production service

Early in 2017 we will upgrade the control planes at UPPMAX and C3SE with new hardware capable of supporting a larger amount of users and projects. The bulk of the compute nodes will continue to come from second generation HPC clusters but they will be modernized and expanded. Our two regions at C3SE and HPC2N will become available for general project requests. We will accelerate the work to integrate SSC into the SNIC ecosystem. In particular, we will redesign our temporary project and account handling. Security policies will also be documented and communicated to end users.

To free up time in the project to make this transitions as rapidly as possible, we will not accept any new pilot project requests until we are ready to announce the production services (the goal is early Q2 2017). During the transition period, we will keep supporting our existing pilot users on the same levels as they are now. When we reopen the services, it will be with the same best effort support levels as other SNIC resources.

Glenna 2

Glenna is a Nordic e-Infrastructure Collaboration (NeIC) initiative, with focus on knowledge exchange and Nordic collaboration on cloud computing. The first Glenna project has now concluded, and from January 2017 a new phase of the project, Glenna 2, starts. Glenna 2 will focus on four main aims:

  1. Supporting national cloud initiatives to sustain affordable IaaS cloud resources through financial support, knowledge exchange and pooling competency on cloud operations.
  2. Using such national resources to establish an internationally leading collaboration on data intensive computing in collaboration with user communities.
  3. Leveraging the pooled competency to take responsibility for assessing future hybrid cloud technology and communicate that to the national initiatives.
  4. Supporting use of resources by pooling national cloud application expert support and create a Nordic support channel for cloud and big data. The mandate is to sustain a coordinated training and dissemination effort, creating training material and providing application level support to cloud users in all countries.

In short, aim 1 ensures the availability of IaaS, aim 2 seeks to establish PaaS and SaaS services for Big Data Analytics, aim 3 investigates future emerging technology and HPC-as-a Service and aim 4 will provide advanced user support for research groups transitioning into cloud computing infrastructure. The project directive for Glenna 2 can be found here.

SNIC Science Cloud is a cloud computing infrastructure run by SNIC, the Swedish National Infrastructure for Computing. SNIC Science Cloud provides a national-scale IaaS cloud and associated higher-level services (PaaS), for the research community

SNIC Science Cloud Workshop (Fall 2016)

Overview:

Instructor: Salman Toor.
Level: Basic.

Location: Chalmers University of Technology, Room Raven & Fox, Fysik forskarhus 5th floor.

Visiting address: Chalmers Campus Johanneberg, Room Raven & Fox, Fysik forskarhus, 5th floor entrance Fysikgränd 3.

Infrastructure: SNIC Science Cloud (OpenStack based Community Cloud).

Date & duration: 25:th November, (10:00 – 16:00).

Audience: Users and potential users of SNIC Science Cloud resources with no previous cloud experience.

Registration:

Register here.

Topics:

  • Brief overview of Cloud Computing.
  • Cloud offerings: Compute, Storage, Network as a Service (*aaS).
  • Brief description of IaaS, PaaS, SaaS etc.
  • How to access Cloud resources?
  • Introduction to SNIC Science Cloud initiative.

Hands-on session topics:

1 – How does the Horizon dashboard work?
2 – How to start a virtual machine (VM)?
3 – Instance snapshots.
4 – Access to cloud storage (volumes and Object store)
5 – Storage snapshot
5 – Network information
6 – Basic system interaction with APIs

Lab-Document

Schedule:

First half (10:15 – 12:00): Lectures
Second half (13:00 – 16:00): Lab session

 

Who are the pilot users of the SNIC Science Cloud?

The SNIC Science Cloud is a project run by the Swedish National Infrastructure for Computing. The goal is to investigate if and how cloud resources should be provided as a complement to the more traditional HPC-resources. An important part of this investigation is our pilot users.  The project has run in two main phases:

1. 2013-2014: A small-scale pilot system and a small number (3-5) of predefined pilot use-cases, chosen to highlight the utility of cloud resources. Advanced user support in the form of SNIC Application Experts (AEs) were available to assist the pilot users.

2. 2015-2016: A scaled up infrastructure with more resources, and an open process for user-initiated project requests. In this phase, we have concentrated on training workshops and creating open source tutorials instead of targeted advanced support.

In the summer 2016 we reported on the growth of project requests and promised to follow up with a more in-depth analysis of who these users are and what they are doing. We have looked in our project database at all the projects that registered for SSC resources in 2015-2016 (as of Oct. 11). During the period, we have had 57 project requests.

As can be seen, Life Science users dominate. Of these, a large fraction are affiliated with SciLifeLab/NBIS. This is expected since the Bioinformatics community was an early adopter of service-oriented computing, and since their applications often have the need to integrate multiple software. Many Swedish universities are represented, but Uppsala University (UU), The Royal Institute of Technology (KTH) and the Karolinska Institute (KI) dominate. This is likely a consequence of the fact that these institutions’ involvement in SciLifeLab, and that both KTH and UU has served as hosts for the SSC project, presumably increasing awareness of SSC amongst the scientists.  

In the period Sep 11-Oct 11, a total of 47000 instance hours were deployed in 31 different projects. Using a reference instance type (flavor) with 4 VCPUs and 8GB RAM, this corresponds to an average of 130 instances continuously deployed during the month. We will follow up on the resource usage patterns over time in future posts, when we have more fine-grained data.

So what are the users doing with the SSC OpenStack resources? It appears that development and testing of software and services, as well as exploring the cloud computing paradigm for old and new types of applications are still the dominating use case. In common to most projects is the need for flexible customization of the computing environment, made possible by virtualization. Many projects also want to provide their solutions as services to serve their own specific community.

Some projects are making more substantial use of the IaaS resources, making use of advanced tools for contextualization, automation and orchestration to achieve quite a diverse range of objectives. In common to all these projects is that they have access to own expertise on distributed and cloud computing in the project groups. To serve as an inspiration to new users, we have during 2016 highlighted some of them as user success stories:

Elastic proteomics analysis in the Malmstroem Lab.

Processing ozone data from the Odin satellite at Chalmers University of Technology.

Estimation of failure probabilities with applications in underground porous media flows.  

Virtual Research Environment for Clinical Metabolomics.  

So what will happen in 2017? A projection is hard to provide, but given the global trend that private/community IaaS becomes more and more common also in academia, observations made by our partners in the Nordic Glenna project, and with the momentum created via the European OpenScienceCloud initiative, we believe that the interest in cloud resources will keep increasing rapidly. 

Fortunately, in the SSC project we are in a good position to meet an increased demand due to our architectural design based on regions, in which we can leverage previous generation HPC hardware at multiple geographic locations to quickly add compute hosts at low cost. We have now integrated resources at three HPC-centra, UPPMAX, C3SE and HPC2N and can if needed scale resources to over 5000 physical cores and 1PB of storage during 2017. This model also opens up for substantial user communities to enter SSC with their own dedicated regions. We also hope to start looking into public-private partnerships to secure a larger variety of SLA-backed resources and to allow for users to burst outside of the allocated quotas.

Scalable processing of ozone data from the Odin satellite

The Odin satellite has measured ozone and related gases in the stratosphere and mesosphere for more than 15 years. This is one of the longest data series with global coverage that exists in atmospheric science. To make sure we can provide the best possible reference data set to the science community ESA has funded a total overhaul of the data set. This includes everything from review of the calibration algorithm of the sensor, to a complete review of the algorithms that provide concentrations of the different species at different heights and locations in the atmosphere.

The instrument is a passive sub millimeter radiometer, measuring emitted energy from different molecules through the limb of the atmosphere. The atmosphere is scanned over different heights around 60 times an orbit and the satellite makes about 15 orbits a day. That way measurements quickly cover most of the Earth’s atmosphere. As an example we have been able follow the development of the ozone depletion over the polar regions. We can also follow the circulation of air masses globally.

Technology

The Odin science community is using the SNIC Science Cloud. This enables scalable processing of Odin data both for testing of alternative algorithms and standard products.

To be able to process the dataset as quickly as possible we have packaged a set of “processors” in docker images ready to be deployed at any docker enabled computer. These docker images are self-contained with auxiliary datasets and code ready to be fed with measurements from the Odin satellite. Once the docker images are deployed the container asks the Odin-API, a REST service, for the next available measurement and starts immediately to crunch the data. When the process is done it delivers the results back to the Odin-API and starts over with a new measurement.

Users can browse and download data from the user interface http://odin.rss.chalmers.se and power-users can communicate directly with the REST-API to analyse data programmatically or start new processing campaigns.