Using Python with VisAD
November, 2001
Updated: December, 2002
Introduction
The VisAD Library is a powerful and comprehensive software development
package for working with scientific data. While substantial applications
can be built by professional software developers,
scientists and others may find it easier to write VisAD applications in
Python.
The Jython (Java-based implementation of Python) interface
to VisAD is being developed to provide this friendly environment.
This tutorial will continue to grow and expand as we add more Jython
functions to provide interfaces for users. Check back often...
What is Jython?
Jython is an
implementation of the high-level, dynamic, object-oriented language Python seamlessly integrated with the
Java platform. The predecessor
to Jython, JPython, is certified as 100% Pure Java. Jython
is freely available for both commercial and non-commercial use and is
distributed with source code. Jython is complementary to Java and is
especially suited for the following tasks:
-
Embedded scripting - Java programmers can add the Jython libraries
to their system to allow end users to write simple or complicated scripts
that add functionality to the application.
-
Interactive experimentation - Jython provides an interactive
interpreter that can be used to interact with Java packages or with
running Java applications.
This allows programmers to experiment and debug any Java system using
Jython.
-
Rapid application development - Python programs are typically 2-10X
shorter than the equivalent Java program. This translates directly to increased
programmer productivity. The seamless interaction between Python and Java
allows developers to freely mix the two languages both during development
and in shipping products.
There are numerous
alternative
languages implemented for the Java VM. The following features help
to separate Jython from the rest:
-
Dynamic compilation to Java bytecodes - leads to highest possible
performance without sacrificing interactivity.
-
Ability to extend existing Java classes in Jython - allows effective
use of abstract classes.
-
Optional static compilation - allows creation of applets, servlets,
beans, ...
-
Bean Properties - make use of Java packages much easier.
-
Python Language
- combines remarkable power with very clear syntax. It also supports a
full object-oriented programming model which makes it a natural fit for
Java's OO design.
How do I get started?
- If you're running on Windows, you can get a
installer
file (packaged using Zero G's InstallAnywhere) which contains:
- Sun's Java Runtime 1.3.1_03
- Java3D for OpenGL in Windows
- VisAD jar file (library)
- Jython (jar file plus all the library utilities)
- several batch files for starting Jython and VisAD (links to which
will be put into the Start->Programs->VisAD menu item).
Download this file and run it. You will be prompted through the
installation, and given some options about what directory to put
the files into, and where to put the icons for the short-cuts
(for launching the interactive command prompts, the VisAD-Jython
Editor, and the VisAD Spreadsheet).
After the install, click on the "Jython Command Window" short-cut
icon first; this will allow the Jython environment to set up correctly
- you will see several messages about analyzing packages before
you get the familiar ">>>" prompt. The analysis only happens
once (or when one of the jar files changes).
Note that if you use this method to install the Java Runtime
Environment, no system configuration changes will be made (to your
PATH, or Registry). In the following descriptions, if you need to run
the java command, you will have to give an explicit PATH
something like: c:\myinstalldir\jre\bin\java -mx256m .....
Or, you can add this directory to your PATH.
- If you are running Linux on an Intel machine, there is also a quick-install
gzipped-tar file available. Download this file
and then gunzip/untar it -- it will create a jython directory an
put all the files there (including the Java runtime, Java3D, visad.jar, jython.jar,
and several startup scripts for you).
- Otherwise, if you are not using one of those two OS/environments, then
you'll need to install at least Java and Java3D first - see the
VisAD Homepage for details.
For other Unix systems, you can just use your installed Java and Java3D
along with the tar.gz file above - which contains VisAD and Jython.
You may need to edit the startup scripts (although we've tried to code them to avoid
that).
If you cannot use the tar.gz file above, then you'll have to also get VisAD
(the visad.jar file) and Jython.
To get Jython, go to the
Jython project homepage at http://jython.sourceforge.net and
get a copy of the latest release (currently 2.1). Follow the
instructions for installation. The important part of this is that you
end up with the jython.jar file in your CLASSPATH,
It is also important that the jython install directory is
writable by all users. A
jython start-up script will also be installed; it looks something like
this (in Windows) - note that this is for JDK1.2 and above, and forces
the jython.jar into the CLASSPATH in the script:
@echo off
java -mx256m "-Dpython.home=d:\java\jython" -cp "d:\java\jython\jython.jar;%CLASSPATH%" org.python.util.jython %1 %2 %3 %4 %5
The first time you run the script to start the interpreter, you'll get
some messages about analyzing the jar files that are in the
CLASSPATH. This will include the visad.jar if you are
running VisAD from the jar file. If all goes well, you should get the
customary Jython prompt:
>>>
If you're new to Jython (Python), you might want to review the Python tutorial and
other parts of the documentation that are available on-line. For now,
if you simply type a numeric expression and press Enter, Jython
will display the answer:
>>>2 + 3
5
>>>
In many of the examples in this tutorial, you'll see references
to two scripts (subs.py and graph.py) that I am putting a
collection of utilities related to VisAD. Although these are
part of the normal installation, they also change periodically,
and you can always download the latest copies of these, but by
clicking on these links:
The Jython Development Environment
Once you install Jython, you will find a file in its home
directory named registry -- this is where Jython keeps its
environment information. (Note: this is not the "Windows
Registry".) The file is editable and contains numerous examples
and tips (imbedded as comments). Details can be found
here.
Why am I telling you this?
Well, if you are going to create your own Python scripts and library
routines, or you want to use the ones that come with VisAD
without having to always run from the directory containing them,
you will find it much more convenient to set the
python.path value to include your directories.
You may also set up an environment variable that points to
their location. For example:
export PYTHONPATH=./:/usr/local/jython/Lib:/home/me/src/visad/python/
(or the equivalent in Windows).
The visad.python.JPythonEditor
Curtis Rueden has put together an excellent programming editor that you
can use to help simplify your script building and experimenting. To run
the editor, use the following command:
java -cp visad.jar visad.python.JPythonFrame
This editor uses more-or-less standard hotkeys and other GUI-ized editing
functions, like the arrow keys for navigation. It also imports some things
for you every time you start it. With the editor, you can bring in an existing
script (or start a new one), modify it, and run it. If you want to make
iterative changes, this is a very convenient environment. (Tip: if
your scripts are going to use up lots of memory, you might want to add
a -mx128m option on the command line -- or whatever maximum
memory heap size you'd like.)
Here's a screen-shot of the JPythonEditor in action:
If you don't use the editor (see the next section), you'll likely
have to include this line in every module you write:
from visad.python.JPythonMethods import *
Believe me, this is an easy one to mis-type! What does this do? It causes
Jython to look at the JPythonMethods class file and import all the static
methods defined there-in. We have put many VisAD short-cuts there, and
are continuing to define higher-level methods to provide for often used
combinations of VisAD methods. Fortunately, the JPythonEditor
automatically supplies the
from visad.python.JPythonMethods import * for you!
And if you use VisAD classes and methods directly, you'll have to
include an import of the class(es) or the entire package:
from visad import Real, FlatField, etc.
...or...
from visad import *
From the command line
Jython comes with an interactive, command-line interpreter. Keep in mind
that, unlike the JPythonEditor, it does not allow you to build a
script; rather, it allows you to enter Jython commands that are executed
immediately.
Here are two tips if this is going to be your primary way of using
Jython:
- At present in Jython, there is no "initialization" script available
that is automatically started. You can emulate this, however, by doing
this:
-
create a start-up script and put your import statements in it
-
make a shell script (batch file) that starts the Jython interpreter using
the -i option (which leaves the interpreter in interactive mode after
running the command-line-named Jython script)
For example: Put these two lines in a file named .jythonrc.py:
from visad.python.JPythonMethods import *
from visad import *
Then make a shell script (batch file) that looks like:
@echo off
java -mx256m "-Dpython.home=d:\java\jython" -cp "d:\java\jython\jython.jar;%CLASSPATH%" org.python.util.jython -i .jythonrc.py
- If you are at the >>> prompt and want to run a
script from a .py file you've created, use the Jython
execfile() command:
execfile("myscript.py")
Your first display
Once you've gotten this far, and imported all the methods
from JPythonMethods, then you're ready for your first "ahhhhhhhhh".
We tried to keep this simple...it's just a "one-liner":
plot( field ( [0, 7, 10, 7, 0, -7, -10, -7, 0] ) )
(all the extra spaces are optional - but note the "nesting" of
parens around the list, so that Jython will make a single list
and pass that to the field() function)
After a few seconds, a window will pop up with a curve that looks like
a poorly-sampled sine wave:
(If you're doing this from the command line interpreter, be sure you first did:
from visad.python.JPythonMethods import *)
Now, what's happening? First off, the "from
visad.python.JPythonMethods..." line causes the Jython interpreter to
look at the class file "visad.python.JPythonMethods.class" and gather
a list of all the method names contained therein. You may then pick a
method from this list by simply using its name. plot() and
field() are both methods (functions) in this file.
Second, we form a Jython list with the notation:
[ 0, 7, 10, 7, 0, -7, -10, -7, 0 ]
which will be treated just like an array of numbers. We pass this
list to the field() function (in JPythonMethods) which
creates a VisAD Data object as a simple 1-D sampling. (Note:
in Jython, the notation [...] denotes a "list" which is a
mutable sequence; the notation (...) denotes a "tuple" which is
an immutable sequence.)
Then the plot() method does all the VisAD stuff needed to create
the plot from this. You'll notice below the image window, there are a few
buttons:
- Mappings - allows you to set the mappings of the domain and range
components to visuals (axis, colors, etc)
- Controls - brings up the VisAD control widget so you can set values
- Clear - clears the plot and frees up resources
- Close - closes the window
Lots of times, though, you have data in a file that you want to
display. In the following section, check out the page on reading data
from a file, and you'll see an example just about as easy...that looks
something like this:
a = load("example1.txt")
plot(a)
Making Quick Graphs of your data
When you have simple data structures, and just want to make a quick
plot/graph, we have provided the graph.py script which
contains several methods for quickly producing common graphs, like a
bar-style histogram:
To use these quick graphs, you must first
import graph in your Jython script.
This section gives you the details.
Connecting to VisAD
You note from the trivial example in the last section that there are no
explicit references in the code to anything that tells you that
VisAD is underneath. It's all there, of course, but in this simple case,
it is quite hidden away. The trick is in defining the right kind
of methods to consolidate VisAD functions that are often used. You certainly
can go "under the hood" if you want. For example, the variable "a" in the
above example is really a VisAD "FlatField" object. You can use any FlatField
method you want.
Ugo Taddei has created an excellend tutorial on VisAD programming, which
you can view
at this link. Also, Frank Gibbons has translated Ugo's examples into Jython,
which you can find here.
In this section, we'll talk about the model of data that VisAD uses,
and then introduce the built-in functions that operate on these data. Each
sub-section will contain the details and examples, so just follow the links.
How VisAD models data
Objects in Jython may very easily be used to represent VisAD Data objects,
and VisAD Data objects can literally represent any numerical data. This
allows for some very convenient notations to be used. For example, if you
have two image files and want to show their difference, the following Jython
code does the trick:
image1 = load("AREA0001")
image2 = load("AREA0002")
imagediff = image2 - image1
plot(imagediff)
Since the meaning of image2 - image1 is to "subtract one object
from another", we treat this as meaning subtract the data values of one
object from the other. The beauty of this is that the notation is simple;
furthermore, as you'll learn, if these image files also have navigation
information, any re-sampling that must be done in order to perform the
subtraction will take place automatically.
This section gives an overview of the VisAD
Data Model as it relates to Jython scripting. A more thorough treatment
is provided at at this link
Handling missing data values
VisAD represents missing data by the Java/IEEE value NaN (Not a Number). To
test for this value in Jython, you need to do something like:
if Double(x).isNan():
where 'x' is the value you are testing, and you have previously done an import
java.lang.Double.
JPythonMethods
Earlier, I mentioned this Java class file, JPythonMethods. It is
a helper class that contains many static methods that can be used
by Jython to work with VisAD data. Here is a summary of the methods therein,
with links to pages explaining and giving examples for many of them:
- Data File Reading
This section shows you two basic ways to read data from files. The
load()
function reads several different types of data files into VisAD Data objects.
The second method uses Jython's File I/O to read the data, and then
must make VisAD Data objects.
- Data plotting function
This section shows how to use the plot() function to quickly
inspect your data. Later sections will show you how to build your own,
specialized displays.
- Math library functions (that operate on VisAD Data
objects)
In this section, we name all the math and trig functions, like cos()
and abs(), that operate on VisAD Data objects. As discussed in the last
section on the Data modem, operators in Jython apply to the entire object.
- Field construction functions When you have raw,
binary numeric values and need to construct VisAD Data objects, this section
shows you how. A general-purpose method called field() can help
you create one and two-dimensional objects.
- Histogram functions One common rearrangement
of data is into a "histogram", where samples are collected together into
"buckets".
- Matrix and FFT functions A collection of matrix
manipulation and FFT functions are available; this section enumerates these.
If you want to use these functions, you must install
JAMA
- Data Type functions In this section,
we describe several methods for extracting portions of the metadata
from the data objects - for example, the sample set of the
domain, or the units.
- Methods to rearrange and extract data
This section describes a few methods to rearrange your data objects
and to extract portions of them. This can be very important when you
have data with multiple range components...for example,
multi-spectral images that have more than one channel imbedded in the
image file.
Several small examples
In this section, we present a collection of examples that illustrate ways
to use Jython and VisAD together. The beginning of the section has an index,
and we plan to add to this...
An example built in stages
Here the development of an application that
presents two panels to the user - one has an image with a movable horizontal
line; the other shows the the brightness values at points along that line.
This example will be developed in stages, to give the opportunity to discuss
trade-offs and ideas.
Making VisAD Collaborations
In this section, we make some small modifications to the example
from the previous section to illustrate how to make your
application collaborative. Just click
here to see how easy it is...and to see what is perhaps
the smallest, useful VisAD application ever written!!
Making and sharing your own neat stuff
We'll put your contributions in this section.
To encourage you to
share your .py scripts with others, I've
put a seedling here. This script is actually just a set of helper
methods that are used by the example in
the previous section.
Not sure what I want here, except it would be nice to have a two-way street
for folks to share their stuff. Maybe a new visad-jython email list or
something.