Onedata Storage

Onedata is a global data access solution for science developed by INDIGO-DataCloud project. With Onedata you can:

  • perform heavy computations on huge datasets
  • access your data in a dropbox-like fashion regardless of its location
  • publish and share your results with public or closed communities

Basic concepts and full user documentation is available at this link.

Onedata for West-Life users

INFN is providing a Onedata experimental storage space for West-Life users. The storage consists of 14.6 TB.

West-Life users can obtain a storage space by login at the Onezone server hosted at INFN-CNAF centre: onezone.cloud.cnaf.infn.it

Thanks to the collaboration with the INDIGO-DataCloud project, West-Life users can at first register with the IAM service instance provided by INDIGO, and the use it as authentication method for automatically obtaining an account on the Onedata system. Username/password, SAML and Google indentities are supported by IAM service.

A tutorial on how to use Onedata is available here.

Once obtained the account in Onedata, create your space and get support for it sending the generated token (see slide 7 of the tutorial) to the West-Life team of INFN-Padova.

Onedata REST API

It is possible to use Onedata's REST API's as well as oneclient command line tool for mounting virtual Onedata filesystem on the local machine.

For official reference about Onedata's REST API's please look at:

As a common use case, an user wants to copy files to or from Onedata space (e.g. form a grid WN where oneclient tool is not available). In this case he can simply use curl commands. Before running the curl command, It can be useful to export the following environment variables:

export $ACCESS_TOKEN=token
export $ONEPROVIDER_HOST=https://one-data-01.pd.infn.it:8443

where "token" can be retrieved from the Onedata dashboard under the ACCESS TOKENS menu, and the space is supported by INFN-Padova Oneprovider.

To copy a local file to a specific Onedata space you can use curl command with the options:

curl -k -H "X-Auth-Token: $ACCESS_TOKEN" -T "/path/filename" "ONEPROVIDER_HOST/cdmi/space-name/subdir/"

To copy a file from Onedata space to a local path, use instead:

curl -k -H "X-Auth-Token: $ACCESS_TOKEN" -o "/path/filename" "ONEPROVIDER_HOST/cdmi/space-name/subdir/filename"

or

curl -k -H "X-Auth-Token: $ACCESS_TOKEN" -O "ONEPROVIDER_HOST/cdmi/space-name/subdir/filename"

in the second case the destination path is not specified so the file is copied in the current directory.