KJ135
/
ISCE_INSAR
mirror of http://172.16.0.12:5000/WBJY/ISCE_INSAR.git
4
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1 from piyushrpt/readme 

Updating README to markdown
LT1AB
piyushrpt 2019-02-10 10:26:00 -08:00 committed by GitHub
parent 94cb05e968 edcd3ef5ca
commit 27fa4ac199
2 changed files with 811 additions and 968 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

811
README.md Normal file
View File

@ -0,0 +1,811 @@
# ISCE2
This is the Interferometric synthetic aperture radar Scientific Computing
Environment (ISCE). Its initial development was funded by NASA's Earth Science
Technology Office (ESTO) under the Advanced Information Systems Technology
(AIST) 2008 and is currently being funded under the NASA-ISRO SAR (NISAR)
project.
THIS IS RESEARCH CODE PROVIDED TO YOU "AS IS" WITH NO WARRANTIES OF CORRECTNESS.
USE AT YOUR OWN RISK.
This software is open source under the terms of the the Apache License. Its export
classification is 'EAR99 NLR', which entails some restrictions and responsibilities.
Please read the accompanying LICENSE.txt and LICENSE-2.0 files.
ISCE is a framework designed for the purpose of processing Interferometric
Synthetic Aperture Radar (InSAR) data. The framework aspects of it have been
designed as a general software development framework. It may have additional
utility in a general sense for building other types of software packages. In
its InSAR aspect ISCE supports data from many space-borne satellites and one
air-borne platform. We continue to increase the number of sensors supported.
At this time the sensors that are supported are the following: ALOS, ALOS2,
COSMO_SKYMED, ENVISAT, ERS, KOMPSAT5, RADARSAT1, RADARSAT2, RISAT1, Sentinel1,
TERRASARX, and UAVSAR.
## Contents
1. [Software Dependencies](#software-dependencies)
- [Installing dependencies with Anaconda](#with-anaconda)
- [Installing dependencies with Macports](#with-macports)
- [Note On 'python3' Exectuable Convention](#python3-convention)
- [License required for dependencies to enable some workflows in ISCE](#license-required-for-dependencies-to-enable-some-workflows-in-isce)
2. [Building ISCE](#building-isce)
- [Configuration control: SCONS\_CONFIG\_DIR and SConfigISCE](#configuration-control)
- [Install ISCE](#install-isce)
- [Setup Your Environment](#setup-your-environment)
3. [Running ISCE](#running-isce)
- [Running ISCE from the command line](#running-isce-from-the-command-line)
- [Running ISCE in the Python interpreter](#running-isce-in-the-python-interpreter)
- [Running ISCE with steps](#running-isce-with-steps)
- [Notes on Digital Elevation Models (DEMs)](#notes-on-digital-elevation-models)
4. [Input Files](#input-files)
5. [Component Configurability](#component-configurability)
- [Component Names: Family and Instance](#component-names-family-and-instance)
- [Component Configuration Files: Locations, Names, Priorities](#component-configuration-files-locations-names-priorities)
- [Component Configuration Help](#component-configuration-help)
6. [User community Forums](#user-community-forums)
------
## 1. Software Dependencies
### Basic:
* gcc >= 4.8+ (with C++11 support)
* fftw >= 3.2.2 (with single precision support)
* Python >= 3.5 (3.6 preferred)
* scons >= 2.0.1
* curl - for automatic DEM downloads
* GDAL and its Python bindings >= 2.2
### Optional:
#### For a few sensor types:
* hdf5 >= 1.8.5 and h5py >= 1.3.1 - for COSMO-SkyMed, Kompsat5, and 'Generic' sensor
#### For mdx (image visualization tool) options:
* Motif libraries and include files
* ImageMagick - for mdx production of kml file (advanced feature)
* grace - for mdx production of color table and line plots (advanced feature)
#### For the "unwrap 2 stage" option:
RelaxIV and Pulp are required. Information on getting these packages if
you want to try the unwrap 2 stage option:
* RelaxIV (a minimum cost flow relaxation algorithm coded in C++ by
Antonio Frangioni and Claudio Gentile at the University of Pisa,
based on the Fortran code developed by by Dimitri Bertsekas while
at MIT) available by request at http://www.di.unipi.it/~frangio.
So that ISCE will compile it properly, the RelaxIV files should
be placed in the directory: 'contrib/UnwrapComp/src/RelaxIV'.
* PULP: Use easy\_install or pip to install it or else clone it from,
https://github.com/coin-or/pulp. Make sure the path to the installed
pulp.py is on your PYTHONPATH environment variable (it should be the case
if you use easy\_install or pip).
#### For splitSpectrum and GPU modules:
* cython3 - must have an executable named cython3 (use a symbolic link)
* cuda - for GPUtopozero and GPUgeo2rdr
* opencv - for split spectrum
### With Anaconda
The conda requirements file is shown below:
```bash
cython
gdal
git
h5py
libgdal
pytest
numpy
fftw
scipy
basemap
scons
opencv
```
With the above contents in a textfile named "requirements.txt"
```bash
> conda install --yes --file requirements.txt
```
Ensure that you create a link in the anaconda bin directory for cython3.
### With Macports
The following ports (assuming gcc7 and python36) are needed on OSX
```bash
gcc7
openmotif
python36
fftw-3 +gcc7
fftw-3-single +gcc7
xorg-libXt +flat_namespace
git
hdf5 +gcc7
h5utils
netcdf +gcc7
netcdf-cxx
netcdf-fortran
postgresql95
postgresql95-server
proj
cairo
scons
opencv +python36
ImageMagick
gdal +expat +geos +hdf5 +netcdf +postgresql95 +sqlite3
py36-numpy +gcc7 +openblas
py36-scipy +gcc7 +openblas
py36-matplotlib +cairo +tkinter
py36-matplotlib-basemap
py36-h5py
```
### Python3 Convention
We follow the convention of most package managers in using the executable
'python3' for Python3.x and 'python' for Python2.x. This makes it easy to turn
Python code into executable commands that know which version of Python they
should invoke by naming the appropriate version at the top of the executable
file (as in #!/usr/bin/env python3 or #!/usr/bin/env python). Unfortunately,
not all package managers (such as macports) follow this convention. Therefore,
if you use one of a package manager that does not create the 'python3'
executable automatically, then you should place a soft link on your path to
have the command 'python3' on your path. Then you will be able to execute an
ISCE application such as 'stripmapApp.py as "> stripmapApp.py" rather than as
"> /path-to-Python3/python stripmapApp.py".
### License required for dependencies to enable some workflows in ISCE
Some of the applications, or workflows (such as stripmapApp.py and isceApp.py),
in ISCE that may be familiar to users will not work with this open source version
of ISCE without obtaining licensed components. WinSAR users who have downloaded
ISCE from the UNAVCO website (https://winsar.unavco.org/software/isce) have signed
the licence agreement and will be given access to those licensed components. Others
wanting to use those specific workflows and components may be able to sign the
agreement through UNAVCO if they become members there. Further instructions will
be available for a possible other procedure for obtaining a license directly from
the supplier of those components.
ISCE provides workflows that do not require the licensed components that
may be used effectively and that will be supported going forward by the ISCE team.
Users that need to work with newly processed data along with older processed data
may require those licensed components as a convenience unless they also reprocess
the older data with the same workflows available in this open source release.
-------
## Building ISCE
### Configuration control
Scons requires that configuration information be present in a directory
specified by the environment variable SCONS\_CONFIG\_DIR. First, create a
build configuration file, called SConfigISCE and place it in your chosen
SCONS\_CONFIG\_DIR. The SConfigISCE file should contain the following
information, note that the #-symbol denotes a comment and does not need
to be present in the SConfigISCE file:
NOTE: Locations vary from system to system, so make sure to use the appropriate location.
The one listed here are just for illustrative purpose.
```bash
# The directory in which ISCE will be built
PRJ_SCONS_BUILD = $ISCE_BUILD_ROOT/isce
# The directory into which ISCE will be installed
PRJ_SCONS_INSTALL = $ISCE_INSTALL_ROOT/isce
# The location of libraries, such as libstdc++, libfftw3 (for most system
# it's /usr/lib and/or /usr/local/lib/ and/or /opt/local/lib)
LIBPATH = $YOUR_LIB_LOCATION_HOME/lib64 $YOUR_LIB_LOCATION_HOME/lib
# The location of Python.h. If you have multiple installations of python
# make sure that it points to the right one
CPPPATH = $YOUR_PYTHON_INSTALLATION_LOCATION/include/python3.xm $YOUR_PYTHON_INSTALLATION_LOCATION/lib/python3.x/site-packages/numpy/core/include
# The location of the fftw3.h (most likely something like /usr/include or
# /usr/local/include /opt/local/include
FORTRANPATH = $YOUR_FFTW3_INSTALLATION_LOCATION/include
# The location of your Fortran compiler. If not specified it will use the system one
FORTRAN = $YOUR_COMPILER_LOCATION/bin/gfortran
# The location of your C compiler. If not specified it will use the system one
CC = $YOUR_COMPILER_LOCATION/bin/gcc
# The location of your C++ compiler. If not specified it will use the system one
CXX = $YOUR_COMPILER_LOCATION/bin/g++
#libraries needed for mdx display utility
MOTIFLIBPATH = /opt/local/lib # path to libXm.dylib
X11LIBPATH = /opt/local/lib # path to libXt.dylib
MOTIFINCPATH = /opt/local/include # path to location of the Xm
# directory with various include files (.h)
X11INCPATH = /opt/local/include # path to location of the X11 directory
# with various include files
#Explicitly enable cuda if needed
ENABLE_CUDA = True
CUDA_TOOLKIT_PATH = $YOUR_CUDA_INSTALLATION #/usr/local/cuda
```
In the above listing of the SConfigISCE file, ISCE\_BUILD\_ROOT and
ISCE\_INSTALL\_ROOT may be actual environment variables that you create or else
you can replace them with the actual paths you choose to use for the build files
and the install files. Also, in the following the capitalization of 'isce' as
lower case does matter. This is the case-sensitive package name that Python
code uses for importing isce.
### Install ISCE
cd isce
scons install
For a verbose install run:
scons -Q install
The scons command also allows you to explicitly specify the name of the
SConfigISCE file, which could be used to specify an alternative file for
(say SConfigISCE\_NEW) which must still be located in the same
SCONS\_CONFIG\_DIR, run
```
scons install --setupfile=SConfigISCE_NEW
```
This will build the necessary components and install them into the location
specified in the configuration file as PRJ\_SCONS\_INSTALL.
#### Note about compiling ISCE after an unsuccessful build.
When building ISCE, scons will check the list of header files and libraries that
ISCE requires. Scons will cache the results of this dependency checking. So,
if you try to build ISCE and scons tells you that you are missing headers or
libraries, then you should remove the cached files before trying to build ISCE
again after installing the missing headers and libraries. The cached files are
config.log, .sconfig.dblite, and the files in directory .sconf_temp. You should
run the following command while in the top directory of the ISCE source (the
directory containing the SConstruct file):
```bash
> rm -rf config.log .sconfig.dblite .sconf_temp
```
and then try "scons install" again.
### Setup Your Environment
Once everything is installed, you will need to set the following environment
variables to run the programs included in ISCE ($ISCE_INSTALL_ROOT may be an
environment variable you created [above](#configuration-control) or else replace it with the actual
path to where you installed ISCE):
```bash
export PYTHONPATH=$ISCE\_INSTALL\_ROOT:$PYTHONPATH
```
and to put the executable commands in the ISCE applications directory on your
PATH for convenience,
```bash
export ISCE_HOME=$ISCE_INSTALL_ROOT/isce
export PATH=$ISCE_HOME/applications:$PATH
```
An optional environment variable is $ISCEDB. This variable points to a
directory in which you may place xml files containing global preferences. More
information on this directory and the files that you might place there is
given below in Section on [Input Files](#input-files). For now you can ignore this environment variable.
To test your installation and your environment, do the following:
```bash
> python3
>>> import isce
>>> isce.version.release_version
```
-----
## Running ISCE
### Running ISCE from the command line
Copy the example xml files located in the example directory in the ISCE source
tree to a working directory and modify them to point to your own data. Run
them using the command:
```bash
> $ISCE_HOME/applications/stripmapApp.py isceInputFile.xml
```
or (with $ISCE\_HOME/applications on your PATH) simply,
```bash
> stripmapApp.py isceInputFile.xml
```
The name of the input file on the command line is arbitrary. ISCE also looks
for appropriately named input files in the local directory
You can also ask ISCE for help from the command line:
```bash
> stripmapApp.py --help
```
This will tell you the basic command and the options for the input file.
Example input files are also given in the 'examples/input\_files' directory.
As explained in the [Component Configurability](#component-configurability) section below, it is also possible
to run stripmapApp.py without giving an input file on the command line. ISCE will
automatically find configuration files for applications and components if they
are named appropriately.
### Running ISCE in the Python interpreter
It is also possible to run ISCE from within the Python interpreter. If you have
an input file named insarInputs.xml you can do the following:
```bash
%> python3
>>> import isce
>>> from stripmapApp import Insar
>>> a = Insar(name="stripmapApp", cmdline="insarInputs.xml")
>>> a.configure()
>>> a.run()
```
(As explained in the [Component Configurability](#component-configurability) section below, if the file
insarInputs.xml were named stripmapApp.xml or insar.xml, then the 'cmdline' input
on the line creating 'a' would not be necessary. The file 'stripmapApp.xml' would
be loaded automatically because when 'a' is created above it is given the name
'stripmapApp'. A file named 'insar.xml' would also be loaded automatically if it
exists because the code defining stripmapApp.py gives all instances of it the
'family' name 'insar'. See the Component Configurability section below for
details.)
### Running ISCE with steps
An other way to run ISCE is the following:
```bash
stripmapApp.py insar.xml --steps
```
This will run stripmapApp.py from beginning to end as is done without the
\-\-steps option, but with the added feature that the workflow state is
stored in files after each step in the processing using Python's pickle
module. This method of running stripmapApp.py is only a little slower
and it uses extra disc space to store the pickle files, but it
provides some advantage for debugging and for stopping and starting a
workflow at any predetermined point in the flow.
The full options for running stripmapApp.py with steps is the following:
```bash
stripmapApp.py insar.xml [--steps] [--start=<s>] [--end=<s>] [--dostep=<s>]
```
where "\<s\>" is the name of a step. To see the full ordered list of steps
the user can issue the following command:
```bash
stripmapApp.py insar.xml --steps --help
```
The \-\-steps option was explained above.
The \-\-start and \-\-end option can be used together to process a range of steps.
The \-\-dostep option is used to process a single step.
For the \-\-start and \-\-dostep options to work, of course, requires that the
steps preceding the starting step have been run previously because the
state of the work flow at the beginning of the first step to be run must
be stored from a previous run.
An example for using steps might be to execute the end-to-end workflow
with \-\-steps to store the state of the workflow after every step as in,
```bash
stripmapApp.py insar.xml --steps
```
Then use \-\-steps to rerun some of the steps (perhaps you made a code
modification for one of the steps and want to test it without starting
from the beginning) as in
```bash
stripmapApp.py insar.xml --start=<step-name1> --end=<step-name2>
```
or to rerun a single step as in
```bash
stripmapApp.py insar.xml --dostep=<step-name>
```
Running stripmapApp.py with \-\-steps also enables one to enter the Python
interpreter after a run and load the state of the workflow at any stage
and introspect the objects in the flow and play with them as follows,
for example:
```bash
%> python3
>>> import isce
>>> f = open("PICKLE/formslc")
>>> import pickle
>>> a = pickle.load(f)
>>> o = f.getMasterOrbit()
>>> t, x, p, off = o._unpackOrbit()
>>> print(t)
>>> print(x)
```
Someone with familiarity of the inner workings of ISCE can exploit
this mode of interacting with the pickle object to discover much about
the workflow states and also to edit the state to see its effect
on a subsequent run with \-\-dostep or \-\-start.
### Notes on Digital Elevation Models
- ISCE will automatically download SRTM Digital Elevation Models when you run an
application that requires a DEM. In order for this to work follow the next 2
instructions:
1. You will need to have a user name and password from urs.earthdata.nasa.gov and
you need to include LPDAAC applications to your account.
a. If you don't already have an earthdata username and password,
you can set them at https://urs.earthdata.nasa.gov/
b. If you already have an earthdata account, please ensure that
you add LPDAAC applications to your account:
- Login to earthdata here: https://urs.earthdata.nasa.gov/home
- Click on my applications on the profile
- Click on “Add More Applications”
- Search for “LP DAAC”
- Select “LP DAAC Data Pool” and “LP DAAC OpenDAP” and approve.
2. create a file named .netrc with the following 3 lines:
```bash
machine urs.earthdata.nasa.gov
login your_earthdata_login_name
password your_earthdata_password
```
3. set permissions to prevent others from viewing your credentials:
```bash
> chmod go-rwx .netrc
```
- When you run applications that require a dem, such as stripmapApp.py, if a dem
component is provided but the dem is referenced to the EGM96 geo reference (which
is the case for SRTM DEMs) it will be converted to have the WGS84 ellipsoid as its
reference. A new dem file with suffix wgs84 will be created.
- If no dem component is specified as an input a EGM96 will be automatically
downloaded (provided you followed the preceding instructions to register at
earthdata) and then it will be converted into WGS84.
- If you define an environment variable named DEMDB to contain the path to a
directory, then ISCE applications will download the DEM (and water body mask files
into the directory indicated by DEMDB. Also ISCE applications will look for the
DEMs in the DEMDB directory and the local processing directory before downloading
a new DEM. This will prevent ISCE from downloading multiple copies of a DEM if
you work with data in different subdirectories that cover similar geographic
locations.
## Input Files
Input files are structured 'xml' documents. This section will briefly
introduce their structure using a special case appropriate for processing ALOS
data. Examples for the other sensor types can be found in the directory
'examples/input\_files'.
The basic (ALOS) input file looks like this (indentation is optional):
### stripmapApp.xml (Option 1)
```xml
<stripmapApp>
<component name="stripmapApp">
<property name="sensor name">ALOS</property>
<component name="Master">
<property name="IMAGEFILE">
/a/b/c/20070215/IMG-HH-ALPSRP056480670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20070215/LED-ALPSRP056480670-H1.0__A
</property>
<property name="OUTPUT">20070215</property>
</component>
<component name="Slave">
<property name="IMAGEFILE">
/a/b/c/20061231/IMG-HH-ALPSRP049770670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20061231/LED-ALPSRP049770670-H1.0__A
</property>
<property name="OUTPUT">20061231</property>
</component>
</component>
</stripmapApp>
```
The data are enclosed between an opening tag and a closing tag. The \<stripmapApp\>
tag is closed by the \<\/stripmapApp\> tag for example. This outer tag is necessary
but its name has no significance. You can give it any name you like. The
other tags, however, need to have the names shown above. There are 'property',
and 'component' tags shown in this example.
The component tags have names that match a Component name in the ISCE code.
The component tag named 'stripmapApp' refers to the configuration information for
the Application (which is a Component) named "stripmapApp". Components contain
properties and other components that are configurable. The property tags
give the values of a single variable in the ISCE code. One of the properties
defined in stripmapApp.py is the "sensor name" property. In the above example
it is given the value ALOS. In order to run stripmapApp.py two images need to
be specified. These are defined as components named 'Master' and 'Slave'.
These components have properties named 'IMAGEFILE', 'LEADERFILE', and 'OUTPUT'
with the values given in the above example.
NOTE: the capitalization of the property and component names are not of any
importance. You could enter 'imagefile' instead of 'IMAGEFILE', for example,
and it would work correctly. Also extra spaces in names that include spaces,
such as "sensor name" do not matter.
There is a lot of flexibility provided by ISCE when constructing these input
files through the use of "catalog" tags and "constant" tags.
A "catalog" tag can be used to indicate that the contents that would normally
be found between an opening ad closing "component" tag are defined in another
xml file. For example, the stripmapApp.xml file shown above could have been split
between three files as follows:
### stripmapApp.xml (Option 2)
```xml
<stripmapApp>
<component name="insar">
<property name="Sensor name">ALOS</property>
<component name="master">
<catalog>20070215.xml</catalog>
</component>
<component name="slave">
<catalog>20061231.xml</catalog>
</component>
</component>
</stripmapApp>
```
#### 20070215.xml
```xml
<component name="Master">
<property name="IMAGEFILE">
/a/b/c/20070215/IMG-HH-ALPSRP056480670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20070215/LED-ALPSRP056480670-H1.0__A
</property>
<property name="OUTPUT">20070215 </property>
</component>
```
#### 20061231.xml
```xml
<component name="Slave">
<property name="IMAGEFILE">
/a/b/c/20061231/IMG-HH-ALPSRP049770670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20061231/LED-ALPSRP049770670-H1.0__A
</property>
<property name="OUTPUT">20061231</property>
</component>
```
-----
## Component Configurability
In the examples for running stripmapApp.py ([Here](#running-isce-from-the-command-line) and [Here](#running-isce-in-the-python-interpreter) above) the input
data were entered by giving the name of an 'xml' file on the command line. The
ISCE framework parses that 'xml' file to assign values to the configurable
variables in the isce Application stripmapApp.py. The Application executes
several steps in its workflow. Each of those steps are handled by a Component
that is also configurable from input data. Each component may be configured
independently from user input using appropriately named and placed xml files.
This section will explain how to name these xml files and where to place them.
### Component Names: Family and Instance
Each configurable component has two "names" associated with it. These names
are used in locating possible configuration xml files for those components. The
first name associated with a configurable component is its "family" name. For
stripmapApp.py, the family name is "insar". Inside the stripmapApp.py file an
Application is created from a base class named Insar. That base class defines
the family name "insar" that is given to every instance created from it. The
particular instance that is created in the file stripmapApp.py is given the
'instance name' 'stripmapApp'. If you look in the file near the bottom you will
see the line,
```python
insar = Insar(name="stripmapApp")
```
This line creates an instance of the class Insar (that is given the family name
'insar' elsewhere in the file) and gives it the instance name "stripmapApp".
Other applications could be created that could make several different instances
of the Insar. Each instance would have the family name "insar" and would be
given a unique instance name. This is possible for every component. In the
above example xml files instances name "Master" and "Slave" of a family named
"alos" are created.
### Component Configuration Files: Locations, Names, Priorities
The ISCE framework looks for xml configuration files when configuring every
Component in its flow in 3 different places with different priorities. The
configuration sequence loads configuration parameters found in these xml files
in the sequence lowest to highest priority overwriting any parameters defined
as it moves up the priority sequence. This layered approach allows a couple
of advantages. It allows the user to define common parameters for all instances
in one file while defining specific instance parameters in files named for those
specific instances. It also allows global preferences to be set in a special
directory that will apply unless the user overrides them with a higher priority
xml file.
The priority sequence has two layers. The first layer is location of the xml
file and the second is the name of the file. Within each of the 3 location
priorities indicated below, the filename priority goes from 'family name' to
'instance name'. That is, within a given location priority level, a file
named after the 'family name' is loaded first and then a file with the
'instance name' is loaded next and overwrites any property values read from the
'family name' file.
The priority sequence for location is as follows:
(1) The highest priority location is on the command line. On the command line
the filename can be anything you choose. Configuration parameters can also be
entered directly on the command line as in the following example:
```bash
> stripmapApp.py insar.master.output=master_c.raw
```
This example indicates that the variable named 'output' of the Component
named 'master' belonging to the Component (or Application) named 'insar'
will be given the name "master\_c.raw".
The priority sequence on the command line goes from lowest priority on the left
to highest priority on the right. So, if we use the command line,
```bash
> stripmapApp.py myInputFile.xml insar.master.output=master_c.raw
```
where the myInputFile.xml file also gives a value for the insar master output
file as master\_d.raw, then the one defined on the right will win, i.e.,
master\_c.raw.
(2) The next priority location is the local directory in which stripmapApp.py is
executed. Any xml file placed in this directory named according to either the
family name or the instance name for any configurable component in ISCE will be
read while configuring the component.
(3) If you define an environment variable named ISCEDB, you can place xml files
with family names or instance names that will be read when configuring
Configurable Components. These files placed in the ISCEDB directory have the
lowest priority when configuring properties of the Components. The files placed
in the ISCEDB directory can be used to define global settings that will apply
unless the xml files in the local directory or the command line override those
preferences.
### Component Configuration Structure
However, the component tag has to have the family name of the Component/
Application. In the above examples you see
that the outermost component tag has the name "insar", which is the family name
of the class Insar of which stripmapApp is an instance.
### Component Configuration Help
At this time there is limited information about component configurability
through the command
```bash
> stripmapApp.py --help
```
Future deliveries will improve this situation. In the meantime we describe
here how to discover from the code which Components and parameters are
configurable. One note of caution is that it is possible for a parameter
to appear to be configurable from user input when the particular flow will
not allow this degree of freedom. Experience and evolving documentation will
be of use in determining these cases.
How to find out whether a component is configurable, what its configurable
parameters are, what "name" to use in the xml file, and what name to give to
the xml file.
Let's take as an example, Ampcor.py, which is in components/mroipac/ampcor.
Open it in an editor and search for the string "class Ampcor". It is on
line 263. You will see that it inherits from Component. This is the minimum
requirement for it to be a configurable component.
Now look above that line and you will see several variable names being set
equal to a call to Component.Parameter. These declarations define these
variables as configurable parameters. They are entered in the "parameter\_list"
starting on line 268. That is the method by which these Parameters are made
configurable parameters of the Component Nstage.
Each of the parameters defines the "public\_name", which is the "name" that you
would enter in the xml file. For instance if you want to set the gross offset
in range, which is defined starting on line 88 in the variable
ACROSS\_GROSS\_OFFSET, then you would use an xml tag like the following (assuming
you have determined that the gross offset in range is about 150 pixels):
```xml
<property name="ACROSS_GROSS_OFFSET">150</property>
```
Now, to determine what to call the xml file and what "name" to use in the
component tag. A configurable component has a "family" name and an instance
"name". It is registered as having these names by calling the
Component.\_\_init\_\_ constructor, which is done on line 806. On that line you
will see that the call to \_\_init\_\_ passes 'family=self.class.family' and
'name=name' to the Component constructor (super class of Ampcor). The family
name is given as "nstage" on line 265. The instance name is passed as the
value of the 'name=name' and was passed to it from whatever program created it.
Nstage is created in components/isceobj/StripmapProc/runRefineSlaveTiming.py where
it is given the name 'master_offset1' on line 35. If you are setting a parameter that
should be the same for all uses of Ampcor, then you can use the
family name 'ampcor' for the name of the xml file as 'ampcor.xml'. It is more
likely that you will want to use the instance name 'master\_offset1.xml'
Use the family name 'ampcor' for the component tag 'name'.
Example for SLC matching use of Ampcor:
Filename: master\_offset1.xml:
```xml
<dummy>
<component name="ampcor">
<property name="ACROSS_GROSS_OFFSET">150</property>
</component>
</dummy>
```
## User community forums
Sign up to participate in the ISCE user communities at the following website:
[http://earthdef.caltech.edu/account/register](http://earthdef.caltech.edu/account/register)
After you sign up read helpful information and participate in discussion with
the user/developer community at the following website:
[http://earthdef.caltech.edu/projects/isce_forum/boards](http://earthdef.caltech.edu/projects/isce_forum/boards)

968
README.txt
View File

@ -1,968 +0,0 @@
================================================================================
ISCE2
================================================================================
This is the Interferometric synthetic aperture radar Scientific Computing
Environment (ISCE). Its initial development was funded by NASA's Earth Science
Technology Office (ESTO) under the Advanced Information Systems Technology
(AIST) 2008 and is currently being funded under the NASA-ISRO SAR (NISAR)
project.
THIS IS RESEARCH CODE PROVIDED TO YOU "AS IS" WITH NO WARRANTIES OF CORRECTNESS.
USE AT YOUR OWN RISK.
This software is open source under the terms of the the Apache License. Its export
classification is 'EAR99 NLR', which entails some restrictions and responsibilities.
Please read the accompanying LICENSE.txt and LICENSE-2.0 files.
ISCE is a framework designed for the purpose of processing Interferometric
Synthetic Aperture Radar (InSAR) data. The framework aspects of it have been
designed as a general software development framework. It may have additional
utility in a general sense for building other types of software packages. In
its InSAR aspect ISCE supports data from many space-borne satellites and one
air-borne platform. We continue to increase the number of sensors supported.
At this time the sensors that are supported are the following: ALOS, ALOS2,
COSMO_SKYMED, ENVISAT, ERS, KOMPSAT5, RADARSAT1, RADARSAT2, RISAT1, Sentinel1,
TERRASARX, and UAVSAR.
================================================================================
Contents
================================================================================
1. Software Dependencies
1.1 Installing software dependencies with standard package managers
1.2 Installing Virtual Machine Images with Dependencies Pre-Installed
1.3 Installing dependencies with provided setup script
1.4 Hints for installing dependencies by hand.
1.5 Note On 'python3' Exectuable Convention
1.6 License required for dependencies to enable some workflows in ISCE
2. Building ISCE
2.1 Configuration control: SCONS_CONFIG_DIR and SConfigISCE
2.2 Install ISCE
2.3 Setup Your Environment
3. Running ISCE
3.1 Running ISCE from the command line
3.2 Running ISCE in the Python interpreter
3.3 Running ISCE with steps
3.4 Notes on Digital Elevation Models (DEMs)
4. Input Files
5. Component Configurability
5.1 Component Names: Family and Instance
5.2 Component Configuration Files: Locations, Names, Priorities
5.3 Component Configuration Help
6. User community Forums
================================================================================
1. Software Dependencies
================================================================================
Basic:
------
gcc >= 4.7
fftw 3.2.2
Python >= 3.2 (3.3 preferred)
scons >= 2.0.1
curl - for automatic DEM downloads
gdal, gdal.py >= 2.0
For a few sensor types:
-----------------------
hdf5 >= 1.8.5 and h5py >= 1.3.1 - for COSMO-SkyMed, Kompsat5, and 'Generic' sensor
spiceypy - for RadarSAT1
For mdx (image visualization tool) options:
-------------------------------------------
Motif libraries and include files
ImageMagick - for mdx production of kml file (advanced feature)
grace - for mdx production of color table and line plots (advanced feature)
For the "unwrap 2 stage" option:
--------------------------------
RelaxIV and Pulp are required. Information on getting these packages if
you want to try the unwrap 2 stage option:
* RelaxIV (a minimum cost flow relaxation algorithm coded in C++ by
Antonio Frangioni and Claudio Gentile at the University of Pisa,
based on the Fortran code developed by by Dimitri Bertsekas while
at MIT) available by request at http://www.di.unipi.it/~frangio.
So that ISCE will compile it properly, the RelaxIV files should
be placed in the directory: 'contrib/UnwrapComp/src/RelaxIV'.
* PULP: Use easy_install or pip to install it or else clone it from,
https://github.com/coin-or/pulp. Make sure the path to the installed
pulp.py is on your PYTHONPATH environment variable (it should be the case
if you use easy_install or pip).
Optional for splitSpectrum, GPUtopozero, and GPUgeo2rdr:
--------------------------------------------------------
cython3 - must have an executable named cython3 (use a symbolic link)
cuda - for GPUtopozero and GPUgeo2rdr
--------------------------------------------------------------------------------
1.1 Installing software dependencies with standard package managers
--------------------------------------------------------------------------------
The easiest way to install most of these is with package managers such as
'apt-get' on Linux systems or 'macports' on MacOsX. To use these, however,
will require that you have superuser permission on your computer. The
following URL gives additional information on installing prerequisites for
ISCE:
https://winsar.unavco.org/portal/wiki/Manual%20installation%20using%20repository%20managers/
If it is not possible for you to install the software yourself and you
can't convince the System Administrator on your computer to install the
dependencies, then we provide virtual machine images (VMs) with the
dependencies pre-installed (see Section 1.2).
As a last resort we provide an *experimental* setup script named install.sh
described in Section 1.3 that will allow you to do a "user" installation of the
basic dependencies, setup your environment variables, and install ISCE for you.
For the truly adventurous who want to install dependencies by hand we provide
some hints in Section 1.4.
When you have installed the dependencies you can skip the other sections about
installing the dependencies and read Section 1.5 about the 'python3' convention
and then Section 2 on building ISCE and configuring your environment.
--------------------------------------------------------------------------------
1.2 Installing Virtual Machine Images with Dependencies Pre-Installed
--------------------------------------------------------------------------------
If you don't have superuser privileges on your machine and your system is not
up to date with the software dependencies required to use ISCE, then you can
download Virtual Machine Images (VMs) at the following URL:
Full link: http://earthdef.caltech.edu/boards/4/topics/305
Simple link: http://tinyurl.com/iscevm
Instructions on how to install the Virtual Machines are given there.
--------------------------------------------------------------------------------
1.3 Installing dependencies with provided setup script
--------------------------------------------------------------------------------
This distribution includes an *experimental* script that is designed to
download, build, and install all relevant packages needed for ISCE (except for
h5py, which presently must be built by hand but is only needed for Cosmo-Skymed,
spiceypy, only needed for RadarSAT1, and gdal python bindings). This script is
meant as a last resort for those adventurous persons who may not have root
privileges on their machine to install software with standard package managers
or a virutal machine (VM) image (see Section 1.1 or 1.2).
The script is in the setup directory, and is called install.sh. To run it, you
should cd to the setup directory, then issue the command
> install.sh -h
to see instructions on how to run the script. The minimal command option is
simply,
> install.sh -p <INSTALL_PATH>
where <INSTALL_PATH> is a path where you have permission to create files. This
will check whether the dependencies exist on your default paths and then install
those that do not on the specified path.
The path should be in a local directory away from the system areas to avoid
conflicts and so that administrator privileges are not needed. The config.py
file contains a list of urls where the packages are currently downloaded from
Commenting out a particular package will prevent installation of that package.
If the specified server for a particular package in this file is not available,
then you can simply browse the web for a different server for this package and
replace it in the config.py file. Below under the "Building ISCE" section,
there are instructions on how to point to these packages for building ISCE.
Once all these packages are built, you must setup your PATH and LD_LIBRARY_PATH
variables in the unix shell to ensure that these packages are used for compiling
and linking rather than the default system packages.
--------------------------------------------------------------------------------
1.4 Hints for installing dependencies by hand.
--------------------------------------------------------------------------------
If you would prefer to install all these packages by hand, follow this procedure:
Compile the following
for Radarsat2, Sentinel1A and Tandem-X gdal with python bindings >= 1.9
Building gcc/gfortran
---------------------
Building gcc from source code can be a difficult undertaking. Refer to the
detailed directions at http://gcc.gnu.org/install/ for further help.
Building fftw-3.2.2
-------------------
Get fftw-3.2.2 from: http://www.fftw.org/fftw-3.2.2.tar.gz
Untar the file fftw-3.2.2.tar.gz using
tar -zxvf fftw-3.2.2.tar.gz
cd fftw-3.2.2
then run ./configure --enable-single --enable-shared --prefix=<directory>
where <directory> is the full path to an installation location where you have
write access. Then run,
make
make install
Building Python
---------------
Get the Python source code from http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tgz
Untar the file Python-3.3.5.tgz using
tar -zxvf Python-3.3.5.tgz
cd Python-3.3.5
Then run
./configure --prefix=<directory>
where <directory> is the full path to an installation location where you
have write access. Then run,
make
make install
Building scons
At this time scons only works with Python2. The scons developers have
announced that they are working on a Python3 version of scons. In the
meantime, you should have Python2.6 or Python2.7 available to you by
default if your computer is new enough. Otherwise repeat the above
steps with 3.3.5 replaced with 2.7.8.
Get scons http://prdownloads.sourceforge.net/scons/scons-2.0.1.tar.gz
Untar the file scons-2.0.1.tar.gz using
tar -zxvf scons-2.0.1.tar.gz
cd scons-2.0.1.tar.gz
Then, using the version of Python you just built, run
python2.6 setup.py build
python2.6 setup.py install
Building hdf5 [Only necessary for COSMO-SkyMed support]
-------------
Get the source code from:
http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.6.tar.gz
Building h5py [Only necessary for COSMO-SkyMed support]
Get the h5py source code from:
http://code.google.com/p/h5py/downloads/detail?name=h5py-1.3.1.tar.gz
Building gdal-bindings [Only necessary for Radarsat2, Tandem-X and Sentinel 1A]
----------------
On most linux distributions, gdal can installed along with its python bindings
using standard repository management tools.
If you don't have gdal, you can find instructions on building GDAL here
http://trac.osgeo.org/gdal/wiki/BuildHints
Remember to use configure with --with-python option to automatically build
python bindings.
Else, if you already have gdal installed on your system but without python
buindings, use easy_install corresponding to your python 3 version. You may
need to setup LD_LIBRARY_PATH correctly to include path to libgdal.so
easy_install GDAL
Building SpiceyPy [Only necessary for Radarsat1]
----------------
JPL's CSPICE library (http://naif.jpl.nasa.gov/naif/toolkit_C.html) is needed
for this. Follow instructions at https://github.com/Apollo117/SpiceyPy to
install SpiceyPy, after installing CSPICE.
Once all these packages are built, you must setup your PATH and LD_LIBRARY_PATH
variables in the unix shell to ensure that these packages are used for compiling
and linking rather than the default system packages.
To use the Spice software you will need to download the data files indicated in
the component/isceobj/Orbit/db/kernels.list file. You should download those
files into that directory (or else make soft links in that directory to where
you download them) so that ISCE can find them in the place it expects.
--------------------------------------------------------------------------------
1.5 Note On 'python3' Exectuable Convention
--------------------------------------------------------------------------------
We follow the convention of most package managers in using the executable
'python3' for Python3.x and 'python' for Python2.x. This makes it easy to turn
Python code into executable commands that know which version of Python they
should invoke by naming the appropriate version at the top of the executable
file (as in #!/usr/bin/env python3 or #!/usr/bin/env python). Unfortunately,
not all package managers (such as macports) follow this convention. Therefore,
if you use one of a package manager that does not create the 'python3'
executable automatically, then you should place a soft link on your path to
have the command 'python3' on your path. Then you will be able to execute an
ISCE application such as 'insarApp.py as "> insarApp.py" rather than as
"> /path-to-Python3/python insarApp.py".
--------------------------------------------------------------------------------
1.6 License required for dependencies to enable some workflows in ISCE
--------------------------------------------------------------------------------
Some of the applications, or workflows (such as insarApp.py and isceApp.py),
in ISCE that may be familiar to users will not work with this open source version
of ISCE without obtaining licensed components. WinSAR users who have downloaded
ISCE from the UNAVCO website (https://winsar.unavco.org/software/isce) have signed
the licence agreement and will be given access to those licensed components. Others
wanting to use those specific workflows and components may be able to sign the
agreement through UNAVCO if they become members there. Further instructions will
be available for a possible other procedure for obtaining a license directly from
the supplier of those components.
ISCE provides workflows that do not require the licensed components that
may be used effectively and that will be supported going forward by the ISCE team.
Users that need to work with newly processed data along with older processed data
may require those licensed components as a convenience unless they also reprocess
the older data with the same workflows available in this open source release.
================================================================================
2. Building ISCE
================================================================================
--------------------------------------------------------------------------------
2.1 Configuration control: SCONS_CONFIG_DIR and SConfigISCE
--------------------------------------------------------------------------------
Scons requires that configuration information be present in a directory
specified by the environment variable SCONS_CONFIG_DIR. First, create a
build configuration file, called SConfigISCE and place it in your chosen
SCONS_CONFIG_DIR. The SConfigISCE file should contain the following
information, note that the #-symbol denotes a comment and does not need
to be present in the SConfigISCE file:
NOTE: Locations vary from system to system, so make sure to use the appropriate location.
The one listed here are just for illustrative purpose.
# The directory in which ISCE will be built
PRJ_SCONS_BUILD = $ISCE_BUILD_ROOT/isce
# The directory into which ISCE will be installed
PRJ_SCONS_INSTALL = $ISCE_INSTALL_ROOT/isce
# The location of libraries, such as libstdc++, libfftw3 (for most system
# it's /usr/lib and/or /usr/local/lib/ and/or /opt/local/lib)
LIBPATH = $YOUR_LIB_LOCATION_HOME/lib64 $YOUR_LIB_LOCATION_HOME/lib
# The location of Python.h. If you have multiple installations of python
# make sure that it points to the right one
CPPPATH = $YOUR_PYTHON_INSTALLATION_LOCATION/include/python3.xm $YOUR_PYTHON_INSTALLATION_LOCATION/lib/python3.x/site-packages/numpy/core/include
# The location of the fftw3.h (most likely something like /usr/include or
# /usr/local/include /opt/local/include
FORTRANPATH = $YOUR_FFTW3_INSTALLATION_LOCATION/include
# The location of your Fortran compiler. If not specified it will use the system one
FORTRAN = $YOUR_COMPILER_LOCATION/bin/gfortran
# The location of your C compiler. If not specified it will use the system one
CC = $YOUR_COMPILER_LOCATION/bin/gcc
# The location of your C++ compiler. If not specified it will use the system one
CXX = $YOUR_COMPILER_LOCATION/bin/g++
#libraries needed for mdx display utility
MOTIFLIBPATH = /opt/local/lib # path to libXm.dylib
X11LIBPATH = /opt/local/lib # path to libXt.dylib
MOTIFINCPATH = /opt/local/include # path to location of the Xm
# directory with various include files (.h)
X11INCPATH = /opt/local/include # path to location of the X11 directory
# with various include files
#Explicitly enable cuda if needed
ENABLE_CUDA = True
CUDA_TOOLKIT_PATH = $YOUR_CUDA_INSTALLATION #/usr/local/cuda
In the above listing of the SConfigISCE file, $ISCE_BUILD_ROOT and
$ISCE_INSTALL_ROOT may be actual environment variables that you create or else
you can replace them with the actual paths you choose to use for the build files
and the install files. Also, in the following the capitalization of 'isce' as
lower case does matter. This is the case-sensitive package name that Python
code uses for importing isce.
--------------------------------------------------------------------------------
2.2 Install ISCE
--------------------------------------------------------------------------------
cd isce
scons install
For a verbose install run:
scons -Q install
The scons command also allows you to explicitly specify the name of the
SConfigISCE file, which could be used to specify an alternative file for
(say SConfigISCE_NEW) which must still be located in the same
SCONS_CONFIG_DIR, run
scons install --setupfile=SConfigISCE_NEW
This will build the necessary components and install them into the location
specified in the configuration file as PRJ_SCONS_INSTALL.
--------------------------------------------------------------------------------
2.2.a Note about compiling ISCE after an unsuccessful build.
--------------------------------------------------------------------------------
When building ISCE, scons will check the list of header files and libraries that
ISCE requires. Scons will cache the results of this dependency checking. So,
if you try to build ISCE and scons tells you that you are missing headers or
libraries, then you should remove the cached files before trying to build ISCE
again after installing the missing headers and libraries. The cached files are
config.log, .sconfig.dblite, and the files in directory .sconf_temp. You should
run the following command while in the top directory of the ISCE source (the
directory containing the SConstruct file):
> rm -rf config.log .sconfig.dblite .sconf_temp
and then try "scons install" again.
--------------------------------------------------------------------------------
2.3 Setup Your Environment
--------------------------------------------------------------------------------
Once everything is installed, you will need to set the following environment
variables to run the programs included in ISCE ($ISCE_INSTALL_ROOT may be an
environment variable you created in 2.1 above or else replace it with the actual
path to where you installed ISCE):
export PYTHONPATH=$ISCE_INSTALL_ROOT:$PYTHONPATH
and to put the executable commands in the ISCE applications directory on your
PATH for convenience,
export ISCE_HOME=$ISCE_INSTALL_ROOT/isce
export PATH=$ISCE_HOME/applications:$PATH
An optional environment variable is $ISCEDB. This variable points to a
directory in which you may place xml files containing global preferences. More
information on this directory and the files that you might place there is
given below in Section 4. For now you can ignore this environment variable.
To test your installation and your environment, do the following:
> python3
>>> import isce
>>> isce.version.release_version
================================================================================
3. Running ISCE
================================================================================
---------------------------------------------------------------------------------
3.1 Running ISCE from the command line
---------------------------------------------------------------------------------
Copy the example xml files located in the example directory in the ISCE source
tree to a working directory and modify them to point to your own data. Run
them using the command:
> $ISCE_HOME/applications/insarApp.py isceInputFile.xml
or (with $ISCE_HOME/applications on your PATH) simply,
> insarApp.py isceInputFile.xml
The name of the input file on the command line is arbitrary. ISCE also looks
for appropriately named input files in the local directory
You can also ask ISCE for help from the command line:
> insarApp.py --help
This will tell you the basic command and the options for the input file.
Example input files are also given in the 'examples/input_files' directory.
As explained in the Component Configurability section below, it is also possible
to run insarApp.py without giving an input file on the command line. ISCE will
automatically find configuration files for applications and components if they
are named appropriately.
---------------------------------------------------------------------------------
3.2 Running ISCE in the Python interpreter
---------------------------------------------------------------------------------
It is also possible to run ISCE from within the Python interpreter. If you have
an input file named insarInputs.xml you can do the following:
%> python3
>>> import isce
>>> from insarApp import Insar
>>> a = Insar(name="insarApp", cmdline="insarInputs.xml")
>>> a.configure()
>>> a.run()
(As explained in the Component Configurability section below, if the file
insarInputs.xml were named insarApp.xml or insar.xml, then the 'cmdline' input
on the line creating 'a' would not be necessary. The file 'insarApp.xml' would
be loaded automatically because when 'a' is created above it is given the name
'insarApp'. A file named 'insar.xml' would also be loaded automatically if it
exists because the code defining insarApp.py gives all instances of it the
'family' name 'insar'. See the Component Configurability section below for
details.)
---------------------------------------------------------------------------------
3.3 Running ISCE with steps
---------------------------------------------------------------------------------
An other way to run ISCE is the following:
insarApp.py insar.xml --steps
This will run insarApp.py from beginning to end as is done without the
--steps option, but with the added feature that the workflow state is
stored in files after each step in the processing using Python's pickle
module. This method of running insarApp.py is only a little slower
and it uses extra disc space to store the pickle files, but it
provides some advantage for debugging and for stopping and starting a
workflow at any predetermined point in the flow.
The full options for running insarApp.py with steps is the following:
insarApp.py insar.xml [--steps] [--start=<s>] [--end=<s>] [--dostep=<s>]
where <s> is the name of a step. To see the full ordered list of steps
the user can issue the following command:
insarApp.py insar.xml --steps --help
The --steps option was explained above.
The --start and --end option can be used together to process a range of steps.
The --dostep option is used to process a single step.
For the --start and --dostep options to work, of course, requires that the
steps preceding the starting step have been run previously because the
state of the work flow at the beginning of the first step to be run must
be stored from a previous run.
An example for using steps might be to execute the end-to-end workflow
with --steps to store the state of the workflow after every step as in,
insarApp.py insar.xml --steps
Then use --steps to rerun some of the steps (perhaps you made a code
modification for one of the steps and want to test it without starting
from the beginning) as in
insarApp.py insar.xml --start=<step-name1> --end=<step-name2>
or to rerun a single step as in
insarApp.py insar.xml --dostep=<step-name>
Running insarApp.py with --steps also enables one to enter the Python
interpreter after a run and load the state of the workflow at any stage
and introspect the objects in the flow and play with them as follows,
for example:
%> python3
>>> import isce
>>> f = open("PICKLE/formslc")
>>> import pickle
>>> a = pickle.load(f)
>>> o = f.getMasterOrbit()
>>> t, x, p, off = o._unpackOrbit()
>>> print t
>>> print x
>>>
Someone with familiarity of the inner workings of ISCE can exploit
this mode of interacting with the pickle object to discover much about
the workflow states and also to edit the state to see its effect
on a subsequent run with --dostep or --start.
---------------------------------------------------------------------------------
3.4 Notes on Digital Elevation Models (DEMs)
---------------------------------------------------------------------------------
- ISCE will automatically download SRTM Digital Elevation Models when you run an
application that requires a DEM. In order for this to work follow the next 2
instructions:
1. You will need to have a user name and password from urs.earthdata.nasa.gov and
you need to include LPDAAC applications to your account.
a. If you don't already have an earthdata username and password,
you can set them at https://urs.earthdata.nasa.gov/
b. If you already have an earthdata account, please ensure that
you add LPDAAC applications to your account:
- Login to earthdata here: https://urs.earthdata.nasa.gov/home
- Click on my applications on the profile
- Click on “Add More Applications”
- Search for “LP DAAC”
- Select “LP DAAC Data Pool” and “LP DAAC OpenDAP” and approve.
2. create a file named .netrc with the following 3 lines:
machine urs.earthdata.nasa.gov
login your_earthdata_login_name
password your_earthdata_password
3. set permissions to prevent others from viewing your credentials:
> chmod go-rwx .netrc
- When you run applications that require a dem, such as stripmapApp.py, if a dem
component is provided but the dem is referenced to the EGM96 geo reference (which
is the case for SRTM DEMs) it will be converted to have the WGS84 ellipsoid as its
reference. A new dem file with suffix wgs84 will be created.
- If no dem component is specified as an input a EGM96 will be automatically
downloaded (provided you followed the preceding instructions to register at
earthdata) and then it will be converted into WGS84.
- If you define an environment variable named DEMDB to contain the path to a
directory, then ISCE applications will download the DEM (and water body mask files
into the directory indicated by $DEMDB. Also ISCE applications will look for the
DEMs in the $DEMDB directory and the local processing directory before downloading
a new DEM. This will prevent ISCE from downloading multiple copies of a DEM if
you work with data in different subdirectories that cover similar geographic
locations.
================================================================================
4. Input Files
================================================================================
Input files are structured 'xml' documents. This section will briefly
introduce their structure using a special case appropriate for processing ALOS
data. Examples for the other sensor types can be found in the directory
'examples/input_files'.
The basic (ALOS) input file looks like this (indentation is optional):
insarApp.xml:
-------------
<insarApp>
<component name="insarApp">
<property name="sensor name">ALOS</property>
<component name="Master">
<property name="IMAGEFILE">
/a/b/c/20070215/IMG-HH-ALPSRP056480670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20070215/LED-ALPSRP056480670-H1.0__A
</property>
<property name="OUTPUT">20070215.raw </property>
</component>
<component name="Slave">
<property name="IMAGEFILE">
/a/b/c/20061231/IMG-HH-ALPSRP049770670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20061231/LED-ALPSRP049770670-H1.0__A
</property>
<property name="OUTPUT">20061231.raw </property>
</component>
</component>
</insarApp>
The data are enclosed between an opening tag and a closing tag. The <insarApp>
tag is closed by the </insarApp> tag for example. This outer tag is necessary
but its name has no significance. You can give it any name you like. The
other tags, however, need to have the names shown above. There are 'property',
and 'component' tags shown in this example.
The component tags have names that match a Component name in the ISCE code.
The component tag named 'insarApp' refers to the configuration information for
the Application (which is a Component) named "insarApp". Components contain
properties and other components that are configurable. The property tags
give the values of a single variable in the ISCE code. One of the properties
defined in insarApp.py is the "sensor name" property. In the above example
it is given the value ALOS. In order to run insarApp.py two images need to
be specified. These are defined as components named 'Master' and 'Slave'.
These components have properties named 'IMAGEFILE', 'LEADERFILE', and 'OUTPUT'
with the values given in the above example.
NOTE: the capitalization of the property and component names are not of any
importance. You could enter 'imagefile' instead of 'IMAGEFILE', for example,
and it would work correctly. Also extra spaces in names that include spaces,
such as "sensor name" do not matter.
There is a lot of flexibility provided by ISCE when constructing these input
files through the use of "catalog" tags and "constant" tags.
A "catalog" tag can be used to indicate that the contents that would normally
be found between an opening ad closing "component" tag are defined in another
xml file. For example, the insarApp.xml file shown above could have been split
between three files as follows:
insarApp.xml
------------
<insarApp>
<component name="insar">
<property name="Sensor name">ALOS</property>
<component name="master">
<catalog>20070215.xml</catalog>
</component>
<component name="slave">
<catalog>20061231.xml</catalog>
</component>
</component>
</insarApp>
20070215.xml
------------
<component name="Master">
<property name="IMAGEFILE">
/a/b/c/20070215/IMG-HH-ALPSRP056480670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20070215/LED-ALPSRP056480670-H1.0__A
</property>
<property name="OUTPUT">20070215.raw </property>
</component>
20061231.xml
------------
<component name="Slave">
<property name="IMAGEFILE">
/a/b/c/20061231/IMG-HH-ALPSRP049770670-H1.0__A
</property>
<property name="LEADERFILE">
/a/b/c/20061231/LED-ALPSRP049770670-H1.0__A
</property>
<property name="OUTPUT">20061231.raw</property>
</component>
A "constant" tag can be used to define a constant for convenience inside
an xml file. For example, the dates '20070215' and '20061231' are used
multiple times in the above files. Also, the base path '/a/b/c/' is used
multiple times. A constant defined in a constant tag is used in constructing
values by sandwiching it between two '$' symbols. For example, if a constant
named "date1" is defined then to use it we would enter '$date1'. The following
example insarApp.xml file should make this clear:
insarApp.xml
------------
<insarApp>
<constant name="dir">/a/b/c </constant>
<constant name="date1">20070215</constant>
<constant name="date2">20061231</constant>
<constant name="dir1">$dir$/$date1$</constant>
<constant name="dir2">$dir$/$date2$</constant>
<component name="insarApp">
<property name="sensor name">ALOS</property>
<component name="Master">
<property name="IMAGEFILE">
$dir1$/IMG-HH-ALPSRP056480670-H1.0__A
</property>
<property name="LEADERFILE">
$dir1$/LED-ALPSRP056480670-H1.0__A
</property>
<property name="OUTPUT">$date1$.raw </property>
</component>
<component name="Slave">
<property name="IMAGEFILE">
$dir2$/IMG-HH-ALPSRP049770670-H1.0__A
</property>
<property name="LEADERFILE">
$dir2$/LED-ALPSRP049770670-H1.0__A
</property>
<property name="OUTPUT">$date2$.raw </property>
</component>
</component>
</insarApp>
Note: as of the time of this release constants do not work with catalog files.
This will be fixed in a future release.
================================================================================
5. Component Configurability
================================================================================
In the examples for running insarApp.py (Section 3.1 and 3.3 above) the input
data were entered by giving the name of an 'xml' file on the command line. The
ISCE framework parses that 'xml' file to assign values to the configurable
variables in the isce Application insarApp.py. The Application executes
several steps in its workflow. Each of those steps are handled by a Component
that is also configurable from input data. Each component may be configured
independently from user input using appropriately named and placed xml files.
This section will explain how to name these xml files and where to place them.
----------------------------------------------------------------------------
5.1 Component Names: Family and Instance
----------------------------------------------------------------------------
Each configurable component has two "names" associated with it. These names
are used in locating possible configuration xml files for those components. The
first name associated with a configurable component is its "family" name. For
insarApp.py, the family name is "insar". Inside the insarApp.py file an
Application is created from a base class named Insar. That base class defines
the family name "insar" that is given to every instance created from it. The
particular instance that is created in the file insarApp.py is given the
'instance name' 'insarApp'. If you look in the file near the bottom you will
see the line,
insar = Insar(name="insarApp")
This line creates an instance of the class Insar (that is given the family name
'insar' elsewhere in the file) and gives it the instance name "insarApp".
Other applications could be created that could make several different instances
of the Insar. Each instance would have the family name "insar" and would be
given a unique instance name. This is possible for every component. In the
above example xml files instances name "Master" and "Slave" of a family named
"alos" are created.
----------------------------------------------------------------------------
5.2 Component Configuration Files: Locations, Names, Priorities
----------------------------------------------------------------------------
The ISCE framework looks for xml configuration files when configuring every
Component in its flow in 3 different places with different priorities. The
configuration sequence loads configuration parameters found in these xml files
in the sequence lowest to highest priority overwriting any parameters defined
as it moves up the priority sequence. This layered approach allows a couple
of advantages. It allows the user to define common parameters for all instances
in one file while defining specific instance parameters in files named for those
specific instances. It also allows global preferences to be set in a special
directory that will apply unless the user overrides them with a higher priority
xml file.
The priority sequence has two layers. The first layer is location of the xml
file and the second is the name of the file. Within each of the 3 location
priorities indicated below, the filename priority goes from 'family name' to
'instance name'. That is, within a given location priority level, a file
named after the 'family name' is loaded first and then a file with the
'instance name' is loaded next and overwrites any property values read from the
'family name' file.
The priority sequence for location is as follows:
(1) The highest priority location is on the command line. On the command line
the filename can be anything you choose. Configuration parameters can also be
entered directly on the command line as in the following example:
> insarApp.py insar.master.output=master_c.raw
This example indicates that the variable named 'output' of the Component
named 'master' belonging to the Component (or Application) named 'insar'
will be given the name "master_c.raw".
The priority sequence on the command line goes from lowest priority on the left
to highest priority on the right. So, if we use the command line,
> insarApp.py myInputFile.xml insar.master.output=master_c.raw
where the myInputFile.xml file also gives a value for the insar master output
file as master_d.raw, then the one defined on the right will win, i.e.,
master_c.raw.
(2) The next priority location is the local directory in which insarApp.py is
executed. Any xml file placed in this directory named according to either the
family name or the instance name for any configurable component in ISCE will be
read while configuring the component.
(3) If you define an environment variable named $ISCEDB, you can place xml files
with family names or instance names that will be read when configuring
Configurable Components. These files placed in the $ISCEDB directory have the
lowest priority when configuring properties of the Components. The files placed
in the ISCEDB directory can be used to define global settings that will apply
unless the xml files in the local directory or the command line override those
preferences.
----------------------------------------------------------------------------
5.2 Component Configuration Structure
----------------------------------------------------------------------------
However, the component tag has to have the family name of the Component/
Application. In the above examples you see
that the outermost component tag has the name "insar", which is the family name
of the class Insar of which insarApp is an instance.
----------------------------------------------------------------------------
5.3 Component Configuration Help
----------------------------------------------------------------------------
At this time there is limited information about component configurability
through the command
> insarApp.py --help
Future deliveries will improve this situation. In the meantime we describe
here how to discover from the code which Components and parameters are
configurable. One note of caution is that it is possible for a parameter
to appear to be configurable from user input when the particular flow will
not allow this degree of freedom. Experience and evolving documentation will
be of use in determining these cases.
How to find out whether a component is configurable, what its configurable
parameters are, what "name" to use in the xml file, and what name to give to
the xml file.
Let's take as an example, Nstage.py, which is in components/mroipac/ampcor.
Open it in an editor and search for the string "class Nstage". It is on
line 243. You will see that it inherits from Component. This is the minimum
requirement for it to be a configurable component.
Now look above that line and you will see several variable names being set
equal to a call to Component.Parameter. These declarations define these
variables as configurable parameters. They are entered in the "parameter_list"
starting on line 248. That is the method by which these Parameters are made
configurable parameters of the Component Nstage.
Each of the parameters defines the "public_name", which is the "name" that you
would enter in the xml file. For instance if you want to set the gross offset
in range, which is defined starting on line 130 in the variable
ACROSS_GROSS_OFFSET, then you would use an xml tag like the following (assuming
you have determined that the gross offset in range is about 150 pixels):
<property name="ACROSS_GROSS_OFFSET">150</property>
Now, to determine what to call the xml file and what "name" to use in the
component tag. A configurable component has a "family" name and an instance
"name". It is registered as having these names by calling the
Component.__init__ constructor, which is done on line 672. On that line you
will see that the call to __init__ passes 'family=self.__class__.family' and
'name=name' to the Component constructor (super class of Nstage). The family
name is given as "nstage" on line 245. The instance name is passed as the
value of the 'name=name' and was passed to it from whatever program created it.
Nstage is created in components/isceobj/InsarProc/runOffsetprf_nstage.py where
it is given the name 'insarapp_slcs_nstage' on line 107 and also in
components/isceobj/InsarProc/runRgoffset_nstage.py where it is given the name
'insarapp_intsim_nstage' on line 58. If you are setting a parameter that
should be the same for both of these uses of Nstage, then you can use the
family name 'nstage' for the name of the xml file as 'nstage.xml'. It is more
likely that you will want to use the instance names, 'insarapp_slcs_nstage.xml'
and 'insarapp_intsim_nstage.xml'. Use the family name 'nstage' for the
component tag 'name'.
Example for SLC matching use of Nstage:
Filename: insarapp_slcs_nstage.xml:
<dummy>
<component name="nstage">
<property name="ACROSS_GROSS_OFFSET">150</property>
</component>
</dummy>
----------------------------------------------------------------------------
6. Join the ISCE user Forums for additional help and information from the
ISCE user and developer community
----------------------------------------------------------------------------
Sign up to participate in the ISCE user communities at the following website:
http://earthdef.caltech.edu/account/register
After you sign up read helpful information and participate in discussion with
the user/developer community at the following website:
http://earthdef.caltech.edu/projects/isce_forum/boards
For bug reports and technical usage issues use the folowing website:
http://earthdef.caltech.edu/projects/isce_forum/issues
==============================================================================
END OF FILE
==============================================================================