A software is an external application that is able to interact with the Cytomine server. The software may get, add update or delete data according to the user right. The integration with Cytomine is done thanks to the client (Java, Python,...).
In this document, we will take the example of a simple scripts computing annotations area with term T inside an annotation (ROI). We call it: "ComputeAnnotationStat".
This software has 4 steps:
- Get the annotation ROI from Cytomine (the location properties is needed),
- Search on Cytomine all annotations with the term T inside the ROI location,
- Compute the sum of the area for all annotations,
- Add a property to the annotation ROI. The key is "AREA_OF_TERM" (where TERM is the term T name) and the value is the areas sum.
The 2 main parameters are the id of the annotation ROI and the id of term T.
The script will be available on Cytomine server in directory algo/ (relative to Cytomine home directory in the server).
The software needs two dependencies: algo/computeAnnotationStats/Cytomine-Java-Client.jar (Cytomine client) and algo/computeAnnotationStats/jts-1.13.jar (JTS for GIS ops).
How to create software
Here is the basic steps to add a new software to Cytomine:
- Create a software instance,
- Create a software parameter list,
- Create a service for this software (Optional),
- Adapt your script,
- Add the software to a project
- Adapt your software/algo
Create a software
First create a new software with a client (Java or Python).
Here is the properties description:
Software service in Cytomine. More details in next sections.
If you want to display result information on the web client, you can provide a template name.
|executedCommand||String||Command to execute the software|
Here is the code to create or new software with the Java client:
Examples are also available in Python e.g. in our Cytomine-DataMining repository in the applications/ directory (e.g. file add_software.py with the add_software function).
Create software parameters
The software command may contains some hard-corded parameters (e.g: script name, "-cp dep.jar",...) but you may need some extra parameters.
In this case, the 2 main parameters will be the annotation ROI id and the term T id.
Parameter type. Usefull to provide a "auto-generate" form in cytomine to launch the software.
"Number", "String", "Domain",...
|software||Long||Software id for this parameter||The software|
|mandatory||boolean||The parameter is mandatory (auto-check in the form)||"annotation" is mandatory and "term" too|
|index||Long||Parameters will be send to the server with a specific order||The software will take "annotation" before "term" param, so annotation index must be lower than term index.|
|default_value||String||Default value if mandatory is false|
|set_by_server||Boolean||The parameter value is automatically set by the server. The parameter will not be print the launch form.|
If type = Domain/ListDomain, we may add a service URI to get the choice for the select input in the launch form.
$currentOntology$ will be replace by the current project ontology id. A software may be link with multiple project, so we cannot directly set the ontology id.
|uri_print_attribut||String||Define the property from the uri service to show in the selectbox. In this case, we want to show the "name" of the term.||name|
|uri_sort_attribut||String||Define the property from the uri used to sort the item in the select box. In our case, we sort on "name" too.||name|
Examples are also available in Python e.g. in our Cytomine-DataMining repository in the applications/ directory (e.g. file add_software.py with the add_software_parameter function).
Create the service
When a software is triggered on Cytomine, it creates a Job that executes the application. A service in Cytomine must be defined to run this program.
There are some predefined services:
Launch a software local to the platform.
If executedCommand is "java myApp.jar -cp client.jar" and the job parameters are "idImage=5" and "level=hight", the command could be:
Write/Adapt your script
Our example, use a basic Groovy script. You can see a complete example in our Github repository in Java.
We first need to retrieve all parameters:
Retrieve Cytomine data
We can open Cytomine connection and we can call some Cytomine services:
Update progression and status
During the script execution, its possible to update the job status and progression.
In the web client, the user will see a progress bar and a status label.
Here is an example in Java:
Similarly this can be done in Python using the update_job_status function (see also the CytomineJob class in the Python client utilities)
Upload algo/script results to Cytomine
A lot of software need to add, edit or remove data on Cytomine. These data may be domain instance (new annotations, add new term to an annotation, add a property to a domain...) or simply a file that will be attached to the job.
- For a domain instance: simply use the client to add/update or delete data on the server (e.g. function addAnnotation to add an annotation to an image; function addAnnotationTerm to add an ontology term to an existing annotation, ...),
For a binary/text file, you need to create a job data and to upload the file to this job data thanks to these methods:
You can do this:
In our example, the job simply add a property to the ROI annotation and add the result as the value.
Add the software to a project
A software needs to be linked with a project before using it. You can simply do that in the web client (Project configuration > Software) or with a client.
In Java you can use the addSoftwareProject function.
Examples are also available in Python e.g. in our Cytomine-DataMining repository in the applications/ directory (e.g. file add_software.py with the add_software_project function)
How to run a software (command-line and/or Cytomine-WebUI)
A software can be run through command-line on a remote computer, or through the Cytomine-WebUI using our Software router package (using RabbitMQ broker): See this guide to learn how to made a job executable from the Cytomine-WebUI "Run Job" Panel.
When running an algorithm with certain parameter values (e.g. a threshold value when performing a intensity-based segmentation; or machine learning parameters to build a recognition model), it is useful to store these values for reproducibility. Any job (instance of a software) can have its parameter values stored in the Cytomine-Core database. With such information, it is possible to compare different execution of an algorithm and choose the best parameter values for the task. These parameter values can be checked in the Jobs tab, in the Details section of each job, as illustrated below.
In Python, they can be added once the job is running using the add_job_parameters function.