HTTP Communication Overview

It is highly advisable to use HTTP for all communication with a Mascot server. One reason is that the HTTP API is uniform across different platforms (Windows, Linux). A second reason is that even if you are prepared to write platform-dependent code, direct access to the server by users may not be permitted by the IT department.

The HTTP Client classes provided a high level interface to allow access to the Mascot server.

There are three main levels at which to access the Mascot server:

HTTP Client: Remote access to the server is made available by ms_http_client. This holds the base URL to the server (e.g. "http://myserver/mascot/cgi/") that is the root address for all the server applications, typically ending in "/cgi/". Queries allow you to find out such things as the server version and operating system. This is also where you login to get a session.
HTTP Client Session: ms_http_client_session gives access to a single Mascot session. The client session is created by logging into the HTTP Client. You must login to create the session, even if security is not enabled on the server. The username and password can be left blank if security is disabled or the guest account is to be used. It holds the Mascot session ID and allows access to functions that require you to be logged into the server, such as submitting new searches or accessing old searches.
HTTP CLient Search: ms_http_client_search gives access to a single search on the Mascot server. With this new searches can be submitted, their progress monitored and results downloaded. The new search is created by submitting a search with the Client Session. An old search is accessed by creating a new ms_http_client_search using the Client Session and the search task id (the one used when the search was originally run).

Lower level access is available using ms_http_helper and its associated classes (ms_http_helper_handles, ms_http_helper_progress, ms_http_helper_response and ms_http_helper_return). The higher level HTTP client classes are implemented using these low level classes. On Windows, these low level classes are implemented using the WinHTTP 5.1 API and as such requires at least Windows 2000, Service Pack 3 or later.

Obtaining Parameter Lists

Some of the search parameters are chosen from controlled lists of terms that are server specific. For example, the name of the sequence database to be searched. Mascot is very configurable, and you should not make any assumptions about what choices will be available. It is possible for people to define their own databases, instrument types, modifications, and enzymes.

The most flexible way to obtain these lists is using Mascot Parser. The URL of the Mascot server is passed to the relevant constructor. See the example code in the Mascot Parser documentation for config_enzymes, config_fragrules, config_mascotdat, config_masses, config_config_modfile and config_taxonomy.

Creating Searches

The MIME format is described in Chapter 8 of the Mascot Installation and Setup manual and RFC's 2045 - 2049. A file of the required format is created automatically, when the search form is submitted. You can capture this output by setting the option SaveLastQueryAsc in mascot.dat to 1. See Chapter 6 of the manual for further information.

The data passed to the server is a concentenation of the prologue, the file and the epilogue. It is often more convenient to save just the peak list in a file. The prologue and epilogue can then be created for the search and sent directly to the server without having to save them to a file.

If the peak list is relatively small, then it can be stored in memory and passed in one parameter (e.g. as the prologue) along with all the other informaion. The other parameters (e.g. the file and epilogue) can be left blank.

When creating MIME format information (e.g. the prologue), a ms_http_client_mime provides a simple mechanism for formatting the data correctly.

Connect to a Server

Create a new ms_http_client, passing the base URL and suitable settings.

matrix_science::ms_connection_settings settings;
settings.setUserAgent("your_application_name");
matrix_science::ms_http_client server("http://your_mascot_server/mascot/cgi/", settings);

Get a Session

Login, passing a username and password if required, to obtain a new ms_http_client_session (not to be confused with ms_session, which is used locally on the server only and does not communicate via HTTP). If security has not been enabled, then the username and password can be blank.

matrix_science::ms_http_client_session session;
matrix_science::ms_http_client::LoginResultCode loginResult = server.userLogin("your_username", "your_password", session);
// ... check success

Create a Search

Get a new ms_http_client_search from the session.

std::string httpHeader = "Content-Type: multipart/mixed; boundary=---MascotDistillerMascotSearch";
matrix_science::ms_http_client_search search;
matrix_science::ms_http_helper_progress progress;
bool isOk = session->submitSearch(search, httpHeader, "", "your_source_mime_filename", "", progress);
// ... check success

Follow its Progress

Follow the progress of the search on the server until is is complete.

matrix_science::ms_http_client_search::SearchStatusCode code;
int progressPercent;
do {
    bool isOk = search.getStatus(code, progressPercent);
    // ... check success
    // ... waiting for search to complete
    double x = progressPercent;
} while (code == matrix_science::ms_http_client_search_status::TS_RUNNING);

Download the Results

Download the results.

bool isOk = search.downloadResultsFile("your_target_download_filename");
// ... check success

Previous Search Results

To retrieve the results from a previously run search you will need to know its search task identifier.

Connect to a Server

Create a new ms_http_client.

matrix_science::ms_connection_settings settings;
settings.setUserAgent("your_application_name");
matrix_science::ms_http_client server("http://your_mascot_server/mascot/cgi/", settings);

Get a Session

matrix_science::ms_http_client_session session;
matrix_science::ms_http_client::LoginResultCode loginResult = server.userLogin("your_username", "your_password", session);
// ... check success

Attach to the Search

Get a new ms_http_client_search using the ms_http_client_session and the task identifier.

matrix_science::ms_http_client_search search(session, "search_task_identifier");

Download the Results

Download the results.

bool isOk = search.downloadResultsFile("your_target_download_filename");
// ... check success

Sequence Retrieval using getseq.pl

To retrieve the sequence for a particular accession:

Connect to a Server

Create a server object, passing the base URL and suitable settings.

matrix_science::ms_connection_settings settings;
settings.setUserAgent("your_application_name");
matrix_science::ms_http_client server("http://your_mascot_server/mascot/cgi/", settings);

Get a Session

matrix_science::ms_http_client_session session;
matrix_science::ms_http_client::LoginResultCode loginResult = server.userLogin("your_username", "your_password", session);
// ... check success

Get sequence

Query the sequence data for an accession number (e.g. "041R_FRG3G") from the database (e.g. "SwissProt").

const std::vector<std::string> & accessionNames;
accessionNames.push_back("041R_FRG3G");
const std::vector<int> accessionFrames; // not using NA frames so leave empty
bool isOk = session.getSequenceFile("SwissProt", accessionNames, accessionFrames, "target_file_name");
// ... check success

Error handling and logging

All the classes inherit from ms_errors. A logging level and logging file can be set on the initial ms_http_client object, for example:

my $objHttpClient = new msparser::ms_http_client($url, $objSettings);
if ($objHttpClient->isValid) {
  $objHttpClient->getErrorHandler->setLoggingFile("log.txt", $msparser::ms_errs::sev_debug3);
}

When ms_http_client::userLogin is called, the logging values are copied to the ms_http_client_session object passed to the function.

When ms_http_client_session::submitSearch is called, the logging values are copied to the ms_http_client_search object passed to the function.

to HTTPS websites

It is possible to connect to websites using https on Linux or Windows systems. To do so, the target website must have a valid certificate. The certificate may be refused by Parser for three main reasons.

The certificate is valid and signed by a recognised authority, but Parser uses an outdated certificate bundle.
The certificate is self-signed, the root certificate of the target server must be added to Parser.
The certificate is missing or invalid, a valid connection cannot be established over https.

To update the list of certificates or to add a self-signed certificate on Linux, two methods are possible.

Note that on Linux, all certificates to be added must be in a valid PEM format. Namely, the certificates must be between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- in the file. There are some tools to convert from other formats to the PEM format, for example on this website : https://www.sslshopper.com/ssl-converter.html, Linux packages also contain similar tools.

Add a root certificate to a Linux system

This method is preferred when it is required to update the system's bundle of certificates. For example, if it not possible to connect to https://www.google.com/, it may be due to the system's bundle. It is however not recommanded to use this method for a self-signed certificate, unless you want to grant access to this website for all other programs on your system.

An up-to-date list of certificates can be downloaded from any of the following links, either over http or https.

Debian and Ubuntu systems To add a root certificate to a Debian-based Linux system, type the following commands.

sudo cp newCertificate.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

Fedora, RedHat and CentOS systems To add a root certificate to a Fedora-based Linux system, type the following commands as root.

yum install ca-certificates
update-ca-trust force-enable
cp newCertificate.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust extract

Add a certificate to Parser on Linux

This method is preferred for a self-signed certificate. To download the root certificate of a website, you can do so from your browser or ask its administrator, and convert it to PEM format as mentioned above. The extension of root certificates are usually .crt, although it does not matter for this method to work.

Once the certificate has been downloaded on your system, type the following commands, the access rights depend on your installation.

mkdir -p /usr/local/mascot/cert
chmod 755 /usr/local/mascot/cert
cp selfSignedCertificate.crt /usr/local/mascot/cert/
chmod 644 /usr/local/mascot/cert/selfSignedCertificate.crt

It is also possible to add a self-signed certificate on Windows. Note that, on Windows, the certificate must be in one of the following formats : .PFX, .P12, .P7B, .SST .

Add a certificate to a Windows system

To download the root certificate of a website, you can do so from your browser or ask its administrator, and make sure it is in a correct format as mentioned above.

Once the certificate has been downloaded on your system, do the following

Click Start, click the text box Search Program and files
Type mmc and type Enter
Click File -> Add/Remove Snap-in...
Click Certificates, click Add >, select Computer account, click Next, click Finish, click OK
On the left panel, Navigate to Console Root/Certificates (Local Computer)/Personal/Third-Party Root Certification Authorities, Certificates
Right click on Certificates, click on All tasks -> Import...
Click Next, click Browse..., change the file filter to match your file type, click your file, click Open, click Next, click Next, click Finish, click OK, close the Console.

HTTP Communication to a Mascot Server [Accessing a Mascot Server using HTTP]

HTTP Communication Overview

Obtaining Parameter Lists

Creating Searches

Previous Search Results

Sequence Retrieval using getseq.pl

Error handling and logging

to HTTPS websites

HTTP Communication to a Mascot Server
[Accessing a Mascot Server using HTTP]