Migrate OmicSoftServer v11 to v12

From Array Suite Wiki

This page has been migrated, find the latest information here

Overview

OmicSoft recommends migrating all installations of OmicSoft Server on version 11.7 or earlier to version 12.1, to new server instance(s), running on Ubuntu 20, RHEL 8, CentOS 8 or Amazon Linux 2, or Windows Server 2019.

Before Migration

1. Make an inventory/list with all places where data is stored by OmicSoft Server

  • In the ArrayServerLinuxBeta.exe installation directory, check ArrayServer.cfg, make a copy
  • Folders.cfg: Double-check the configuration file that defines where other custom relative paths can be added
  • Copy default.template and sample.template: Default.template; Sample.template
  • Other configurations in ArrayServer.cfg to be aware of: Database backup, Analytic Servers, LDAP section, SearchProject section, Filter section, CloudFolder section, CloudInstanceTag, CloudVolumeTag, CFRPart11 section.

2. Inform the users

Announce all users well in advance for a planned downtime, and ask them to complete any pending jobs before this.

Setup OmicSoft Server on a new server instance

If you are installing on AWS, we recommend using one of our pre-built AMIs


If required by your IT policy, you can Install OmicSoft Server on your own, using any of the following operating systems


Make sure the hardware characteristics (processor, storage, memory) are at least equal or better than the old machine. Please consult Hardware/Software Requirements for OmicSoft Server.

Please also make sure at this point that you have obtained a valid TLS1.2 Server Certificate from a public certificate authority.

Perform data and configuration migration

  • Make sure there are no running jobs or connected users (In Studio > Server tab > Manage > Manage sessions: Sessions, Users, Log)
  • Stop the existing OmicSoft Server program and all remaining ojob processes and cloud instances connected to it (see Manage Cloud Instances)
  • Stop the OmicSoft Server program on the new server instance, if it is running.
  • Copy all data identified at step (1) (the server's and users' data) from the old server on the new one, in folders having the same path/name.
    • If possible, make sure the old data is backed-up
    • do NOT copy the mono or sqlite folders
  • Copy the old ArrayServer.cfg and other config files (from step 1.) from the old OmicSoft Server installation folder: default.template, sample.template, Folders.cfg (if it exists) to the new server (backup the existing files first).
    • Modify ArrayServer.cfg in order to conform with the requirements of OmicSoft Server v12. Please see section "Update ArrayServer.cfg".
  • Attach and mount all existing volumes/devices, that contain data and were not copied, with the same names, on the new server(e.g., for Amazon EBS: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-attaching-volume.html;

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html)

  • If needed, re-create the local users, ex. - the user used to run Array Server service


Run Omicsoft Service Updater

If the mono process instance running the Omicsoft Server is started, make sure it is stopped first. 

$ cd /opt/arrayserver

# If not downloaded recently, get the latest version of OmicSoftServiceUpdater: wget -c https://resources.omicsoft.com/software_update/OmicSoftServiceUpdater.exe
$ mono ./OmicSoftServiceUpdater.exe

Update ArrayServer.cfg

The downloaded template contains the ArrayServer.cfg, which is the key configuration file. The following sections and fields are mandatory:

  • License section - server license code for your company
  • Port2 and Port3 - define the port number for data communication (gRPC and SFTP data transfer) between OmicSoft Studio client and OmicSoft Server. Make sure to open these ports on firewall.
  • CertificateThumbprint - specifies the server certificate thumbprint
  • MasterServerUrl - identify the DNS name and the gRPC port on which the server can be accessed
  • OmicSoft directories
    • BaseDirectory - this will be the working directory of the array server, storing all the raw and possessed data. Depending on the projects, it can take huge amount of disk space.
    • OmicsoftDirectory - This directory can be defined locally or on a network drive. All the reference genomes, gene models, Affymetrix CDF files, log files, etc. are stored in this folder.
    • TempDirectory - This should be a local directory (i.e. NOT a network drive) for fast read and write access. It can take twice the size of an unzipped fastq file in some NGS tasks (we would suggest use a drive with at least 100GB storage).
    • LandDirectory - directory for Land data storage. Depending on the number of Lands installed, this can take several hundred GB.
  • Locations of Mono 6 binaries installed on your Oshell instance(s), which are required by jobs started through the server:
  • The [Folder] section defines additional local or network folders monitored and available for access by array server users.

To enable cloud-based NGS analysis with the Array Server Cloud add-on, please follow the instructions from ArrayServer_Configuration_with_Cloud. The following properties must be configured within ArrayServer.cfg.

For a full list of options, see ArrayServer.cfg for more details. For master-analytic server setup, please read Master Server and Analytic Server for more details.

Update metadata template files

  • default.template: it defines project level meta data, such as requiring project title, and list organism. Admin can customize the project meta data based on this template file and even enforce controlled vocabulary. For first time setup, the default values we prepared should be good to go. Admin can further customized based on the understanding of default.template wiki article.
  • sample.template: it defines sample level meta data, such as requiring users to fill organism and tissue for each samples in sample registration. Admin can customize the sample meta based on this template file and even enforce controlled vocabulary. For first time setup, the default values we prepared should be good to go. Admin can further customize based on the understanding of sample.template wiki article.

Update Network settings

In addition to the networking requirements described at OmicSoft_Suite_IT_Requirements, as OmicSoft Server jobs are spawning new processes (ojob.exe processes) which will connect to the OmicSoft Server, please make sure access to the server, over the server's own public IP, is allowed.

In that case, modify/add a security group that allows all TCP/IP traffic, for that particular EC2 instance, inbound and outbound for the public IP assigned to the EC2 instance. You can restrict this, for the inbound rule, by allowing only the ports on which OmicSoft Server is listening (Port2 and Port3).

Final steps

  • Make the necessary DNS changes to point to the new server's IP, if necessary.
  • Announce the users about the change, including the fact that they must update OmicSoft Studio to the latest version, before using the software.
Retrieved from "https://wiki.arrayserver.com/wiki/index.php?title=Migrate_OmicSoftServer_v11_to_v12&oldid=31582"