Overview

The ImmPort File Download Tool is a ZIP package which contains two directories:

The bin directory contains a bash shell script downloadImmportData.sh, referred to as the File Download Tool is designed to automate the authentication, authorization and downloading of files from the ImmPort Data Browser web site. Details on how to use the downloadImmPortData.sh tool are are available below. In addition a Python file immport_download.py file is included to demonstrate how you can use Python to automate downloading files and using the Download and Query API.

The aspera directory contains software made available from [Aspera] (http://asperasoft.com/). Aspera has developed software to transfer large files at high rates of speeds, faster than normal FTP, and is used by many scientific sites to support downloading of files. Aspera has provided executable programs for both the Mac and Linux operating systems, and support for authentication and authorization.

Download Tool

The File Download Tool distribution can be obtained from HERE. Then unzip the contents of the zip package and cd to the immport-data-download-tool directory.

If you execute the bin/downloadImmPortData.sh script without any parameters it will display useful information on the parameters available and other options. (check the Response tab)

unzip immport-data-download-tool.zip

cd immport-data-download-tool

./bin/downloadImmportData.sh

usage: downloadImmportData.sh username password file | --manifest-file=filename [ --verbose | --debug ]
       downloadImmportData.sh --clean

where

  username = ImmPort username
  password = ImmPort password
  file     = Name of file or directory to be downloaded (e.g. /ALLSTUDIES/ALLSTUDIES-DR22_table_count.txt). If this parameter is not specified, then the --manifest-file option must be specified.

  --manifest-file=filename = A manifest file containing the list of files and directories to be downloaded. The filename can be either a text file or a JSON file.  If it is a text file, it must contain one file or directory name per line.  If it is a JSON file, it must have a .json file extension AND conform to the output of the ImmPort Shared Data Query API.  If this option is not specified, then parameter file must be specified.

  --verbose = run in verbose mode
  --debug   = run in debug mode (for troubleshooting purposes)
  --clean   = clean output, error, and log files
  --version = version of immport-data-download-tool

Aspera Software

The downloadImmportData.sh shell script provides a command line wrapper to make the use of the Aspera executables much easier for the user. In addition, Python and R programs can also take advantage of the Aspera executables.

Reminder the Aspera executables are only available for the Mac and Linux operating systems.

Prerequiste

Access to all shared ImmPort data requires you to become a registered user, registration is simple and you can register here . Because access is limited to registered users, the File Download and Data Query API methods require an authorization token to be used as part of the request. The File Download Tool hides this complexity, but you need to be aware of the need to obtain a token when using the File Download Tool and API's.

Download Tool Examples

Run the Download Tool using the name of a file/directory to be downloaded

./bin/downloadImmportData.sh "REPLACE_WITH_USERNAME" "REPLACE_WITH_PASSWORD" "/SDY478/ResultFiles/MBAA_result/5-22-14 FLU P-1 STUDY 15 2013_rcsv|5-22-14 FLU P-1 STUDY 15 2013_rcsv|5-22-14 FLU P-1 STUDY 15 2013_P1_A10.514277.csv"

Run the Download Tool by passing a file which contains the name of the file/directory to be downloaded

The file can contain multiple file/directory names separated by the newline character.

echo "/SDY478/ResultFiles/MBAA_result/5-22-14 FLU P-1 STUDY 15 2013_rcsv|5-22-14 FLU P-1 STUDY 15 2013_rcsv|5-22-14 FLU P-1 STUDY 15 2013_P1_A10.514277.csv" > manifest.txt

./bin/downloadImmportData.sh "REPLACE_WITH_USERNAME" "REPLACE_WITH_PASSWORD" --manifest-file=manifest.txt

Run the Download Tool by passing a json file created from the Immport Data Query Tool

#!/bin/bash

#
# Get an ImmPort token to access the ImmPort Shared Data Query API
#
export token=`curl -X POST https://auth.immport.org/auth/token -d username="REPLACE_WITH_USERNAME" -d password="REPLACE_WITH_PASSWORD" 2>&1 | fgrep '"token"' | sed -e 's/^.*"token" : "//;s/".*$//'`

#
# Create a JSON file containing the output of the ImmPort Shared Data Query API
#
curl -k -H "Authorization: bearer $token" "https://api.immport.org/data/query/result/filePath?studyAccession=SDY208" > manifest.json

#
# Call the Download Tool and pass the JSON file created above as the manifest file
#
./bin/downloadImmportData.sh "REPLACE_WITH_USERNAME" "REPLACE_WITH_PASSWORD" --manifest-file=manifest.json

To create the manifest.json file mentioned in the above script refer to the ImmPort Data Query documentation