McIDAS-X User's Guide
Version 2022.1

[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]

Preparing the Workstation

Before installing McIDAS-X, verify that the workstation is configured to the specifications listed in the table on the previous page. Then continue with the section below for information about memory usage in McIDAS-X.

Memory Usage

Each McIDAS-X session can have up to 9,999 frames. However, not all workstations can reasonably accommodate one or more users running sessions with thousands of frames. This is because system performance is reliant on several factors, including how much memory the workstation has, how many frames are created, how large the frames are, and how many graphics are drawn on the frames. McIDAS-X uses shared memory (a subset of system memory) to store image and graphics data for quick retrieval, while the X11 display subsystem uses a separate allocation of system memory for its own frame buffers. Shared memory usage in bytes is approximately equal to the number of pixels of imagery plus pixels of graphics and is allocated on the machine running the McIDAS-X commands.

The number of pixels of imagery is LINES x ELEMENTS x FRAMES, while the number of pixels of graphics is usually far less than the number of image pixels, with each graphics pixel on screen using eight bytes of memory. Graphics covering 10% of your imagery pixels will approximately double the shared memory requirements. Erasing graphics does not free up memory, though the space is reused when new graphics are drawn. Note that when using the RGBDISP command to display color imagery, three color planes (for RGB) are used and thus image frames displaying RGBDISP output use three times as much shared memory as regular grayscale image frames. System memory usage for the X11 display subsystem is about LINES x ELEMENTS x FRAMES x 4 due to the high bit-depth of the frame buffers, and is allocated on the computer being used to display the McIDAS-X session.

For example, a McIDAS-X session consisting of 2000 frames at 960 x 1280 pixels each, with a total of 10% of the frames' content covered by graphics command output, will require approximately 4.9 GB (2000 x 960 x 1280 for imagery + 2000 x 960 x 1280 for graphics) of shared memory (a subset of system memory) on the workstation running the session. The computer that is being used to display the McIDAS-X session will require approximately 9.8 GB (2000 x 960 x 1280 x 4) of system memory for X11. If the workstation running the session is also the display machine, the total system memory requirement will be about 15 GB. Similarly, if the McIDAS session has 1000 RGB frames, they will require approximately 4.9 GB (1000 x 960 x 1280 x 3 for imagery + 1000 x 960 x 1280 for graphics) of shared memory on the workstation running the session, and 4.9 GB (1000 x 960 x 1280 x 4) of system memory for X11.

Port 112 Configuration

ADDE data requests from clients to remote servers and back require use of port 112. Upon receiving a data request from a client, the remote server gathers the requested data, packages it, then sends it to the client back through the same open connection. Therefore, sites with firewalls or other screening mechanisms must open port 112 on their remote servers if they want to allow their clients to access the servers' data. And sites must ensure that outbound requests on port 112 are allowed on their client workstations and from their local network.

Now go to the section below that applies to your configuration and complete its instructions.

• Linux Workstations
• macOS Workstations

Linux Workstations

Read the five sections below and complete the tasks that apply to your configuration.

Installing the Required Packages

On Linux systems, a set of packages must be installed in order for McIDAS-X to build and run successfully. Run the commands below appropriate for your system as user root to check for the packages and install them if they are not already present.

If you will also be installing McIDAS-XCD on the workstation, the mysql and mysql-devel packages must also be installed. Run the command below as user root to check for the packages and install them if they are not already present.

If the compilation fails due to a missing library or header file, you can run the command below (where "*/file" is a regular expression identifying the missing library or header file enclosed in quotes, e.g., "*/Intrinsic.h") as user root to have yum report the required package name.

Then run this command to install the required package:   yum install package

Enabling Inter-Process Communication

On Linux systems, the system administrator should use the command ipcs to verify that your workstation configuration has Inter-Process Communication (IPC) enabled in the system kernel. If IPC is not running, rebuild your system kernel with all IPC functions enabled.

Allocating Sufficient Shared Memory

McIDAS-X uses shared memory for frame allocation. On some operating systems, the shared memory facility is activated by default and has a sufficient maximum segment size for most McIDAS-X sessions. However, this is not always true for Linux systems. Therefore, on Linux systems the system administrator must complete the steps below to check and, if necessary, force the kernel to activate the shared memory system at boot time and set the maximum segment size to 512 MB or greater the first time McIDAS-X is installed on a Linux workstation. Note: The 4.9 GB of shared memory in the Memory Usage example above does not have to be allocated in one large block. Shared memory is made up of multiple segments, whose minimum size is determined by the system memory requirement of one frame.

1. Edit the file /etc/sysctl.conf.

2. Check for lines similar to the three below. If the lines do not exist then add them to the end of the file. If the lines exist and are set to the values below (536870912 for the first two, 131072 for the third) or greater you can leave them as is. If the lines exist but are set to smaller values, change the values to those below.

   kernel.shmmax = 536870912 kernel.shmall = 536870912 kernel.shmmni = 131072

3. Run the following command from the Unix prompt.

   Type:  /sbin/sysctl -p

Allocating Sufficient Swap Space

If your system has less than the sum of the memory requirements described in the Memory Usage section above plus overhead for the OS and other programs, you may experience slower performance as memory is swapped to disk. This is normal behavior and can be alleviated by adding additional physical memory. If both memory and swap are exhausted, the kernel will kill your user processes, including any McIDAS-X tasks, and log you out. More swap space can be added by modifying your filesystem configuration or by creating a new swap file of the desired size. See the Swap Space chapter of the Red Hat Enterprise Linux Storage Administration Guide for more information.

Increasing the Stack Size

Some Linux operating systems have a hard stack size limit that is too small for decoding large grids. This results in McIDAS-X commands like GRDDISP giving an Error: Unix system stack size on server machine insufficient for grib decoding message when trying to display those grids. If a user reports this occurring while accessing a local dataset, we recommend setting the hard limit to a value of 20480 or larger by completing the instructions in the paragraph below. If a user reports the error while accessing a remote dataset, we recommend contacting the server administrator and asking that they increase the hard limit on the server machine.

To view the current setting, run command ulimit -H -s from the Unix prompt. If the hard limit is less than 20480, have the system administrator increase it to that value or more by editing the /etc/security/limits.conf file and adding a line like the one below. For a larger hard stack value, change the "20480" value to the desired value or "unlimited".

* hard stack 20480

macOS Workstations

Read the sections below and complete the tasks as instructed.

Installing the Required Packages

In order to build and install McIDAS-X on macOS, you must install the compilers and system utilities in a specific order. Complete the steps below in order to do so.

1. Acquire and install the Xcode Developer Tools from the App Store or https://developer.apple.com.

2. Acquire and install the Command Line Tools from https://developer.apple.com/download/more/ (developer login required).

3. If the HPC compilers are installed on the machine, using the uninstall-hpc.sh script downloaded from this link, uninstall the HPC compilers.

   Change to the directory containing the gcc and gfortran installer tar files, and then type:

   sudo /path/to/uninstall-hpc.sh [gcc-version-installer.tar.gz] [gfortran-version-installer.tar.gz]

where the HPC filenames represent the gcc and gfortran tar.gz files as downloaded and installed from hpc.sourceforge.net (e.g. gcc-9.2-bin.tar.gz and gfortran-9.2-bin.tar.gz).  If the tar.gz files no longer exist on the machine, download them again from http://hpc.sourceforge.net.  The tar.gz files are read by the unistall-hpc.sh script, but there is no need to reinstall them.

4. Install the Homebrew package manager, following the instructions at https://brew.sh 

   If installing on a Mac with Apple silicon (e.g. M1 or M2), you must install the Intel version of Homebrew tools using Rosetta, and then force installation of the Intel architecture version of Homebrew by prefixing the install command with arch -x86_64.

   Type:   /usr/sbin/softwareupdate --install-rosetta

   Type:   arch -x86_64 [installation command from https://brew.sh]

5. Install the required tools with the following commands.

   Type:   brew install coreutils

   Type:   brew install gcc

   Type:   brew install gfortran

6. Link the gcc-12 binary to /usr/local/bin/gcc

   Type:   cd /usr/local/bin && sudo ln -s gcc-12 gcc

7. Acquire and install the Latest Release of CMake (3.14+ is required) from https://cmake.org/download/. Then generate the links to the binaries in /usr/local/bin/ by running the command below.

   Type:   sudo "/Applications/CMake.app/Contents/bin/cmake-gui" --install

8. Acquire and install the supported version of XQuartz listed for your version of macOS in the table on the previous page. You can acquire XQuartz from http://xquartz.macosforge.org/.

Allocating Sufficient Shared Memory

McIDAS-X uses shared memory for frame allocation. On some operating systems, the shared memory facility is activated by default and has a sufficient maximum segment size for most McIDAS-X sessions. However, this is not always true for macOS systems. Therefore, on macOS systems the system administrator must complete the steps below to check and, if necessary, force the kernel to activate the shared memory system at boot time and set the maximum segment size to 64 GB or greater the first time McIDAS-X is installed on a macOS workstation (there is no penalty for specifying a larger maximum for shared memory than is physically available).

1. As user root, create the launchctl startup script /Library/LaunchDaemons/edu.wisc.ssec.sharedMemory.plist with the content below.

   <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>edu.wisc.ssec.sharedMemory</string> <key>ProgramArguments</key> <array> <string>/usr/sbin/sysctl</string> <string>-w</string> <string>kern.sysv.shmmax=68719476736</string> <string>kern.sysv.shmall=16777216</string> <string>kern.sysv.shmmin=1</string> <string>kern.sysv.shmmni=131072</string> <string>kern.sysv.shmseg=131072</string> </array> <key>RunAtLoad</key> <true/> </dict> </plist>

   Note the relationship between shmall and shmmax. shmall is the number of pages (4k in size) and shmmax=shmall*4096. The example values above allow shared memory to be addressed in 64 GB segments.

2. As user root, load the new .plist file created in step 1.

   Type:   launchctl load /Library/LaunchDaemons/edu.wisc.ssec.sharedMemory.plist

3. Reboot the workstation.

NOTE: If you previously allocated shared memory on your OS X/macOS workstation using the procedure documented in version 2017.1 or earlier of this manual, you must remove those changes so they don't conflict with the new settings above. (The previous procedure was valid for OS X versions 10.11 and earlier that were supported on those versions of McIDAS-X. Starting with version 2017.2, McIDAS-X is supported on OS X/macOS versions 10.11 and later, which use the new procedure in steps 1 through 3 above for allocating shared memory.)

To remove the previous settings so they don't conflict with the new ones, first check whether file /etc/sysctl.conf exists. If it does, and if it contains five kern.sysv.shm* lines like those in the example below, remove them from the file.

kern.sysv.shmmax=536870912 kern.sysv.shmmin=1 kern.sysv.shmmni=128 kern.sysv.shmseg=32 kern.sysv.shmall=131072

[Search Manual] [Table of Contents] [Go to Previous] [Go to Next]