Data Storage

Your YNiC IT account gives you access to multiple storage areas. Some of these are personal storage areas whilst some are shared with the various groups (usually projects) you will be associated with whilst working at YNiC. Areas to which you can write will be subject to a quota. Once this quota is exceeded, you will no longer be able to write to this area until you clear it out; if this area is your home space, you may not be able to login and will need to see the IT staff to fix this.

Another important distinction is between scratch and non-scratch space. Scratch space is an area which is backed up to tape less often (although it is still stored on highly resilient systems). The rationale behind scratch space is that it should store data which can be regenerated if there were a catastrophic failure. To take some examples, when using the YNiC NAF software, the .yaml configuration file stores all of the information needed to re-run the analysis from the raw data. Therefore, in the absolutely worst case, should we ever need to restore everything, by storing the .yaml file in your home space, and the results from the beamformer in your group scratch space, you are still safe. For FSL, if you are running a FEAT analysis, the directory which contains the .fsf and other small configuration files contains everything needed to replicate the analysis. More details on scratch space can be found below.

The data stored on IT systems and in home spaces at YNiC is much more secure than on standard PCs. All files are stored on RAID arrays, which means that even if a hard disc should fail data is not lost. In addition all raw data and essential files, that cannot be reproduced accurately or at all, are backed up to tape so they can be recovered in the case of a complete system failure. This backing up however takes some considerable time. It is unnecessary and impractical to back up every file on the systems to tape every night. Also, please note that scratch space is not stored on less reliable machinery than the normal data stores, it is simply that we do not back the scratch areas up to tape every night, as this is not feasible.

The quotas which are set reflect the recommended usage of that area.

Area Name

Path

Use

Quota

Home

/home

Personal scripts and documents

20Gb

Home Scratch

/scratch/home

Personal analysis results

30Gb

Groups

/groups

Project scripts and documents

30Gb

Groups Scratch

/scratch/groups

Project analysis results

200Gb