Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note

This page describe How to easily deploy the latest version latest version of Cytomine (for installation procedure of V1.0 (January 2016), see this page)

...

Cytomine uses many libraries and services but this page describe how describes how to have a production instance of Cytomine ready to use with only four steps.

Once Cytomine's server is installed, one can access Cytomine through a modern web browser or through clients using RESTful API.

Please, before further reading, pay attention to the requirementsThis installation documentation is for an Ubuntu 14.04 LTS & 04LTS / 16.04 LTS.04LTS / 18.04LTS.  

Please note we use Docker which is a kind of lightweight virtualization platform, so you do not need to create virtual machines to install Cytomine (it's better not do so for better performances).

If you experiment any issue, please contact us by e-mail or describe your problem precisely in our ticket system on Github.

Table of Contents
outlinetrue

Install Cytomine

Step 1 - Retrieve Cytomine-bootstrap

First of all, we retrieve the Cytomine-bootstrap installation procedure.

1.1 Install latest official release from Cytomine cooperative

If you want to install the last official release, copy/paste these commands:

Code Block
languagebash
linenumberstrue
mkdir Cytomine_src/
cd Cytomine_src/
wget https://github.com/cytomine/Cytomine-bootstrap/archive/master.zip -O bootstrap.zip
unzip bootstrap.zip
mv Cytomine-bootstrap-master Cytomine-bootstrap

1.2 Install

...

latest release with last developments from Cytomine ULiege

To install Cytomine with the last features from ULiège research team, copy/paste these commands:

Code Block
languagebash
linenumberstrue
sudo apt-get install git
mkdir Cytomine_src/
cd Cytomine_src/
git clone https://github.com/Cytomine-ULiege/Cytomine-bootstrap.git
cd Cytomine-bootstrap

 


Step 2 - Install Docker

Then, we need to install the Docker engine.

...

If needed, you will find a complete documentation on the official site.

Step 3 - Configure Cytomine installation

Now, it is required to edit the Cytomine file configuration.sh in the Cytomine-bootstrap/ directory to prepare the installation of Cytomine Docker containers, using e.g.:

Code Block
languagebash
linenumberstrue
cd Cytomine-bootstrap/
emacs configuration.sh

You have two deployment options:

3.1 If you want to deploy on a local host :

First of all, the variables "XXX_URL" will not be visible outside of your local server. So, due to the Docker architecture (isolation of the Docker's containers), we need to apply some changes to authorize network communication between our IMS and CORE containers (see below for further details). To do so, add these URL values (CORE_URL, IMS_URLS, UPLOAD_URL, RETRIEVAL_URLto your /etc/hosts file with the following format : "127.0.0.1       XXX_URL".

...

NB : Please be careful that some problems might appear if you use localhost instead of CORE_URL for a connection to Cytomine. So, keep using CORE_URL.

3.2 If you want to make your Cytomine instance accessible from anywhere :

Contact your institutional system/network administrator before installing Cytomine so that they create DNS entries and make their HTTP port (80) accessible for CORE_URL, IMS_URLS and UPLOAD_URL.

 

Detailed explanation of configuration options

The following tables explain how to configure your installation. 

For a basic, simple installation, you need to configure only the mandatory variables.

URLS

Please use URLs that are not already present in your /etc/hosts file to avoid conflicts.

VariableImportanceExplanationExample value
CORE_URLMandatorythe URL dedicated to the Core of Cytominelocalhost-core
IMS_URLSMandatory

the URL(s) dedicated to one or more image server

Be careful that their value are space free even between two values

[localhost-ims]
UPLOAD_URLMandatorythe URL for image upload

localhost-upload

RETRIEVAL_URLAdvancedthe URL of retrieval serverlocalhost-retrieval
IIP_CYTO_URLAdvancedthe URL of Cytomine IIP serverlocalhost-iip-cyto
IIP_JP2_URLAdvancedthe URL of IIP server for JPEG2000localhost-iip-jp2

 

Storage Paths

You have to choose existing paths on an available filesystem where image files will be stored. Please note you have to specify paths without a "/" at the end (e.g. "/data", not "/data/").

VariableImportanceExplanationExample value
IMS_STORAGEMandatoryA pre-existing filesystem path to store images (withtout ending /)/data (but not /data/)
IMS_BUFFER_PATHMandatoryA pre-existing filesystem path to IMS buffer (without ending /)/data/buffer (but not /data/buffer/)
ALGO_PATHMandatoryA pre-existing filesystem path to store Cytomine softwares (without ending /)/data/algo
RETRIEVAL_PATHMandatoryA pre-existing filesytem path to store retrieval data (without ending /)/data/thumb
BACKUP_PATHIf backupA pre-existing filesystem path to store auto backup 

Email
VariableImportanceExplanationExample value
SENDER_EMAIL

Optional to test the app, mandatory otherwise

email params of the sending account 
SENDER_EMAIL_PASSOptional to test the app, mandatory otherwiseemail params of the sending account 
SENDER_EMAIL_SMTP_PORTOptional to test the app, mandatory otherwiseemail params of the sending account 
SENDER_EMAIL_SMTP_HOSTOptional to test the app, mandatory otherwiseemail params of the sending account 
RECEIVER_EMAILIf backupemail adress of the backup reports receiver 

 

IRIS
VariableImportanceExplanationExample value
IRIS_ENABLEDAdvanced  Enable IRIS (see IRIS project) 
IRIS_URLAdvanced  
IRIS_IDAdvanced Usefull if we are admin of multiple IRIS version 
IRIS_ADMIN_NAMEAdvanced  
IRIS_ADMIN_ORGANIZATION_NAMEAdvanced  
IRIS_ADMIN_EMAILAdvancedThe email adress where user will ask permissions for IRIS projects  

 

Other advanced configuration
VariableImportanceExplanationExample value
BACKUP_BOOLAdvancedTo enable an automatic daily backup, set to truefalse
IS_LOCALAdvancedTo deploy on distinct machines, set to falsetrue
RETRIEVAL_ENGINEAdvanced memory, redisredis
RETRIEVAL_PASSWDAdvanced  
RABBITMQ_LOGINAdvanced  
RABBITMQ_PASSWORDAdvanced  
NB_IIP_PROCESSAdvancedNumber of simultaneous IIP processes20
MEMCACHED_PASSAdvanced  
BIOFORMAT_ENABLEDAdvancedEnable the bioformat convertor to support more formats (VSI, OME-TIFF,...) 

BIOFORMAT_ALIAS

Advanced  
BIOFORMAT_PORTAdvanced  


Step 4 - Deploy Cytomine

Then, you can deploy Cytomine by launching these two following commands in the Bootstrap directory (the installation of Cytomine can take up to 1 hour depending on network and server speed):

Code Block
languagebash
linenumberstrue
sudo sh ./clean_docker_keep_data.sh #(the first time, you don't need to run this line)
sudo sh ./start_deploy.sh

 

Step 5 - Install test data (optional)

At the end of the installation we recommend to install test data (which takes roughly 30 additional minutes) in order to follow examples from our user guide.

Step 6 - Contact us and cite us (optional)

We kindly ask you to cite our (Marée et al., Bionformatics 2016) paper and website (http://www.cytomine.be/) when using Cytomine in your work.

Please also consider contacting us to let us know who you are and why you are using Cytomine.

...

Troubleshooting

In case of problems, see page Known problems of v1.0 (and specially the entry "Error : postgis_data not found/not started").

...

Please note that server components might take one or two minutes to boot the first time (which will generate a "502 Bad Gateway message" or the browser to wait for a reponse). 

 

...

How to test your installation.

You can log into Cytomine through the http://CORE_URL (by default: http://localhost-core/) in your web browser using the admin session (the password was asked during installation). If you did not install the "test data", you will have an empty instance. You have to create a project, users, ...

...

We provide a script (in the Cytomine-bootstrap directory) that outputs logs of the different containers and packs them in an reporting.tgz file for further analysis:

Code Block
languagebash
linenumberstrue
sudo sh ./reporting.sh

If you want to analyze more deeply an issue, depending on your problem (e.g. issue with image viewing), you can enter into the corresponding Docker container (e.g. ims) using the docker enter command and analyze log files.

First, install once the docker enter tool using (or use the docker exec command):

Code Block
languagebash
linenumberstrue
docker run -v /usr/local/bin:/target jpetazzo/nsenter

 Then enter into the corresponding docker (e.g. image server) and analyze the log:

Code Block
languagebash
linenumberstrue
sudo docker exec -it ims /bin/bash/
tail -n 100 -f var/lib/tomcat7/logs/catalina.out

...

Code Block
#######################
# Root logger option
log4j.rootLogger=DEBUG, stdout
 
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
#######################

 

...

How to update your installation.

Docker is also used to easily update your Cytomine installation.

...

all you need to do is to restart the containers to have an up-to-date version of Cytomine with your own customization.

Note
Of course, restarting all the containers will make you lost all your data. To avoid this problem, some containers are specifically designed for your data and must always be running (if you need to stop them, do a backup first) ! The "data containers"  will have to be updated only if you made fundamental changes on the DBMS (e.g. update version, change postgres to mysql, ...)

The restart_keep_data.sh script (in Cytomine-boostrap directory) is made to re-deploy an up-to-date app without erasing your data.

So, as much as possible(*), only run the this script to redeploy your app.

Code Block
cd Cytomine-bootstrap
sudo sh restart_keep_data.sh

...


Tip
NB : If you add a custom container to your installation, you have to update this script.

* : The "data containers"  will have to be updated only if you made fundamental changes on the DBMS (e.g. update version, change postgres to mysql, ...)

 

...

How to uninstall Cytomine.

Please let us know why Cytomine does not meet your expectations.

...