Skip to content

Data Portal

Researchers utilizing Monsoon's compute power now have the ability to share project data with colleagues via a web browser. Any individual files or directories in your /projects/<project_name>/ folder can be shared. No file is too large, or too small.

A long-term storage space is required for this service to work. To obtain a long-term storage space, please request one on the long-term storage request.

Publishing data

To share data using the data portal, perform the following steps while SSH'd into Monsoon:

  1. Change directory to your project directory. The publish script must be ran there. For example, if your project is called hpc_project, you'd run:

cd /projects/hpc_project

  1. Run the publish_data script. When provided with no files to publish, it will print some usage information. If one or more files and/or directories are specified, those files and/or directories will be published to the web.For example, if you wanted to publish the following files and directories:
- /projects/hpc_project/january_results
- /projects/hpc_project/computational_data/dataset1.csv
- /projects/hpc_project/computational_data/dataset2.txt

You would run:

publish_data january_results computational_data/dataset1.csv computational_data/dataset2.txt

  1. Your data can now be accessed at https://rcdata.nau.edu/<project_name>. Using the previous examples, the URL would be https://rcdata.nau.edu/hpc_project.

Unpublishing data

If you wish to unpublish files, the unpublish_data script works similarly:

  1. Change directory to your project directory.
    For example, if your project is called hpc_project, you'd run:
    cd /projects/hpc_project

  2. Run the unpublish_data script. For example, to unpublish the following files:

- dataset1.csv 
- january_results/bad_results.out

you would run:

unpublish_data january_results/bad_results.out computational_data/dataset1.csv
if you wanted to unpublish all of the january_results directory, you'd run:
unpublish_data january_results

Once done, the data will still be in your project directory, but will no longer appear on the web.

Optionally securing your data

To secure your data so only users with authorized credentials can access your published data, use the secure_published_data script to secure your published site.

secure_published_data /projects/hpc_project

The script will create a file under /projects/hpc_project/public_data/ called .htaccess and will automatically fill it with the following contents:

AuthType Basic
AuthName "Restricted Area"
AuthUserFile "/projects/hpc_project/.htpasswd"
require valid-user

The secure_published_data script will also ask you if you would like to add your first user. To add your first user, navigate to your projects folder and run the following command:

htpasswd -cB .htpasswd <username>

To add subsequent users, navigate to your projects folder and run the following command:

htpasswd -B .htpasswd <username>

To change the password of a user, you can re-run the command above for the target user.

FAQs

What happens when I delete files in my project area that have been published?

The files remain published.

What happens when I manually delete files from the public_data area?

The files are unpublished effectively. The files that were published still remain in your project area.