In this series of articles we present the technology behind the iEx.ec Distributed Cloud. In the previous article we discussed the architecture. This article will introduce our security considerations. It is assumed that the reader is already familiar with the architecture and the main principles of the system as described in previous articles.
Introduction
iEx.ec ensures security at different levels. In this article we detail security in the client-scheduler-worker interaction. In this context, security services and protocols can be introduced as follows :
- communication
- authentication
- authorization
- access rights
- confinement
- data
In the iEx.ec network, workers are managed similarly to miner pools in classical blockchain network. Thus, several iEx.ec schedulers may manage different subnetworks of workers, each secheduler being itself connected to the Ethereum blockchain. Each iEx.ec pool is managed by a set of clients, which can perform the administrative work such as task submission and user management.
The iEx.ec testbed network since Décember 2016, when we installed the ganglia cluster monitoring software. To perform scalability and reliability test, we use one worker per core (not CPUs). We successfully used more than 3600 workers for a single scheduler using the DSP application. The ganlia view of the iEx.ec network can be monitored alive at http://xw.iex.ec/ganglia
Communication
As sensitive data may be transferred over the network, the XWHEP middleware systematically ensures communications privacy by using TLS protocol. This ensures communication security by using encryption and by identifying communicating entities.
Scheduler authentication is implemented by way of an electronic key pair: the scheduler private key is securely stored on server side, and the scheduler public key, which can be freely distributed, is enclosed inside the installation packages of the client and the worker. The use of an electronic key pair permits certification of a scheduler identity: decentralized XWHEP components establish a session with a scheduler only if the server proves that it owns the private key corresponding to the public key. On the other side, the scheduler cannot accept connection without receiving its own public key.
Communication encryption is ensured by the use of an electronic key as defined by the TLS protocol.
Authentication
Being connected to the scheduler is not sufficient to execute a command on the platform. Each client and worker must present valid credentials so that the scheduler can securely identify who is connecting. Different credential types are accepted by the server:
login/password: login/password is a key pair that uniquely defines an identity.
X509 certificate: this is an IETF standard ensuring certificate owner identity. It is beyond the scope of this article to detail certificates in this document. The middleware authenticates the communication initiator presenting an X509 certificate by verifying the certificate authority (CA) that signed the certificate. The scheduler can manage its own list of trused CA certificates.
OpenId and OAuth: the middleware may delegate the authentication process to known and trusted distributed identity servers, such as your google account for example. On successful authentication, the middleware checks if the user is allowed to use the deployment. It is the highest responsibility to manage the list of trusted servers as well as users allowed to authenticate in the platform using their remote identity. Using this authentication mechanism, there are two choices: -1- automatically accept new users that can immediately start working, or -2- use an internal validation process so that getting a new registered user is easy but not automatic. This last behaviour is the default for security purposes.
Authorization
Being authenticated on the server side is not sufficient to execute a command on the platform. Each authenticated entity has a usage level called “user rights”, that defines authorization. Authorization allows or denies execution of a requested action.
Authorizations are defined as enumerations from least to most privileged. The super_user authorization allows full access to all objects. In IEx.ec, we call “administrator” an identity associated to a super_user authorization. The standard_user authorization allows to do actions such as insert_job, insert_data etc, but not — let’s say- insert_user.
The general case is as follows: user rights R allows action A if and only if:
0 < A <= R
The only specific case is: worker_user. Presenting credentials associated with this user rights, each worker can:
never list anything (list_app, list_data etc.)
never insert anything (insert_app, insert_data etc.)
never delete anything (delete_app, delete_data etc.)
ask for a pending job
send a heartbeat signal
retrieve an application by providing the application unique identifier (UID)
download data by providing the data UID
upload results for the job it has computed. This is strictly checked by the server. One should understand that uploading a result is then somehow different than a “simple” insert_data.
Access rights
In iEx.ec, each object has an owner. By default the owner has full access to their objects. Any user may have access to an object it does not own, if this object permits it. For example, any user belonging to a user group may have access to an object owned by a member of the same user group, if this object permits group access.
Being allowed to execute a command on the server is not sufficient to execute this command on a specific object. All objects (applications, data, jobs etc, even user objects) have associated access rights implemented “à la” linux file system access rights. Access rights allow or deny access to the object they are associated with.
IEx.ec assigns “Read”, “Write” and “Execute” access rights separately for:
- the owner of the object;
- members of the group the owner belongs to (if any);
- others.
Access rights of an object can be modified by its owner or by an iEx.ec administrator but it is never possible to increase access rights, except for an administrator.
There is one exception where access rights are fully bypassed so that access is always allowed: an iEx.ec administrator is always allowed to access (read, write, execute) any object. This exception permits to modify access rights in any manner (even increasing).
Access rights are defined as follows:
Ur Uw Ue Gr Gw Ge Or Ow Oe
Where:
Ur = User (owner) read access rights
Uw = User (owner) write access rights
Ue = User (owner) execute access rights, if applicable
Gr = Group read access rights
Gw = Group write access rights
Ge = Group execute access rights, if applicable
Or = Other read access rights
Ow = Other write access rights
Oe = Other execute access rights, if applicable
Examples in octal notation:
755 defines
- full access for the owner;
- read and execute accesses to the user group, if any;
- read and execute accesses to others.
750 defines
- full accesses for the owner;
- read and execute accesses to the user group, if any;
- no access to others.
700 defines
- full accesses for the owner;
- no access to the user group, if any;
- no access to others.
Confinement
Since XWHEP assigns an authorisation to each identity and access rights to each object, XWHEP effectively confines all objects it manages. Inside XWHEP, a combined setup of authorisations and access rights is call a confinement. All applications, all data, all jobs are confined but also all users, all clients and all workers.
Confinements aim to restrict access so that distributed entities (clients and workers) are allowed or denied access (read, write, execute) to confined objects. The XWHEP middleware helps to define as many confinements as possible as combinations of authorizations and access rights. The major “public”, “group”, and “private” confinements are shown in next figure “Confidentiality”
The figure “Confidentiality” shows by the example the paradigm of confinement applied to applications. We can see the three application confinements:
- public application are inserted by administrator only. It can be accessed by all users: any user can submit a task for this
- application. It can be run on public workers only.
- group application are inserted by administrator group. It can be accessed by all group users: any group user can submit a job for this application. It can be run on group workers only.
- private application can be inserted by any user. It can be accessed by its owner only: the owner can submit a task for this
- application. It can be run by a private worker only.
On the other hand, this figure shows another interesting paradigm: the worker (i.e the volunteer resources) confinements:
- public worker can download and run public applications only.
- group worker can download and run group application only.
- private worker can run private application only.
Data
Data may circulate over the network, from client to server, from server to workers, and back. Data transit security is ensured by communication security detailed previously in the section ”Communication”.
Confidentiality
The middleware enforces data confidentiality according to the authorization and access rights mechanisms described above, defining the following confinements:
- private data are sent to private clients and workers only;
- group data are sent to group clients and workers only;
- public data are freely accessible.
XWHEP provides absolutely no guarantee to integrity and privacy as soon as an object is sent to a distributed entity. It is the users responsibility to properly use confinement mechanisms to ensure confidentiality and restricted access to sensitive data.
The middleware defines a specific case for X509 certificates. X509 certificates are automatically private data. This ensures that very sensitive data can never be downloaded by anyone but the owner. The middleware is written so that workers can never access certificates. It is the users responsibility to ensure its X509 certificate is registered into the platform as such, and to not bypass the default behaviour by registering X509 certificate with another data type (e.g “text” only).
Integrity
The middleware ensures data integrity by checking data size and MD5 checksum on each access. It is the responsibility of the iEx.ec administrator to ensure integrity of data stored on the server side.
Persistence
The XWHEP middleware does not propose anything on this field. It is the end users responsibility to ensure persistence of their data. The administrator may remove any data from the server without any formal notification to the data owner. It is the administrator responsibility to avoid removing any data that might lead to problems regarding registered applications and submitted jobs.
Proof-of-Contribution or verifying the correctness for off-chain computation ? How will it work?
The goal is of iEx.ec is to develop a new protocol, called PoCo, that provides consensus for off-chain computation, e.g making sure that the data are correctly transferred and the computation ran without errors. In short, it will associate result checking mechanisms (à la majority voting, spot checking) with a reputation and reward system to ensure that people have incentives to behave correctly and avoid over replications.
But one has to understand that it will take time to develop something sound and secure, with incentives that actually help developing the network. Nevertheless, XWHEP already features task replication, so it is absolutely trivial to implement majority voting, which is the baseline algorithm for doing result certification à la BOINC.
Conclusion
This article introduced security features of the iEx.ec platform by detailing several configuration settings of the XWHEP middlware. We will introduce the application deployment in a next article.
Media
- You are kindly invited to:
- join our Slack: https[Suspicious link removed]c-team.slack.com
- follow us on Twitter: https://twitter.com/iEx_ec
- interact with us on Reddit : https[Suspicious link removed]c/
Resources
iEx.ec Github
XtremWeb-HEP Github
XWHEP documentation
Video introducing XWHEP
I presented at CrowComputing 2014, in Almere, NL.
https://vimeo.com/113122296
Relevant Research Papers
Caillat, G., Lodygensky, O., Urbah, E., Fedak, G., & He, H. (2008, August). Towards a security model to bridge internet desktop Grids and service Grids. In European Conference on Parallel Processing (pp. 247–259). Springer Berlin Heidelberg.
Cheikh, A. B., Abbes, H., & Fedak, G. (2014, September). Towards privacy for MapReduce on hybrid clouds using information dispersal algorithm. In International Conference on Data Management in Cloud, Grid and P2P Systems (pp. 37–48). Springer International Publishing.