PyCuAmpcor/examples/cuDenseOffsets:
+ add prepareGeometry() to generate the cooresponding geometry file
+ update file path in xml file to support dir change
+ fix str.decode bug with try/except statement
+ printout used time
+ check --redo before initiate PyCuAmpcor to avoid the long GPU memory error message
PyCuAmpcor/README.md: fix typos for ROIPAC
* stackStripMap.py: change numbering style to match with topsStack
* Stack.py: separate parallel processors in topo.py from runfiles
* stackSentinel.py: add a new argument for parallel processors in topo.py
* Stack.py: remove extra num_process for unpackStackReferenceSLC
* stackSentinel.py: tie numProcess4topo to numProcessif the first not specified
* stackSentinel.py:using geopandas to calculate intersection between bbox and SAFE file coverage region
* stackSentinel.py:using geopandas and shapely package to calculate whether there are intersections between bbox and sentinel SAFE file. And remove import geopandas and shapely part into generate_geopolygon function
* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage
* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage
* stackSentinel.py:using shapely package to find whether bbox intersects with SAFE file
* stackSentinel.py:remove matplotlib DEBUG message#L14-L18
* added a README.md for installation/user guide/procedures
* modified the cuDenseOffsets.py
* expose more options from the CUDA/C++ program
* add an option for varying gross offset input
* clarify the parameter definitions
* removed old SlcImage implementation and cublas dependence
* modified cuSincOversampler
* to be consistent with cpu version
* fix an issue when the extraction of the search window is not around the center
* added a debug mode to output intermediate results
* enable cuda error checking for both Debug/Release build types
* corrected the code to extract raw images when the correlation surface peak is close to edges
* Move utf-8 decoding step inside cython extension
The cython setters take python strings, but the getters return
python bytes, so this makes the types match up. I went with regular
python strings for the interface since they are more common at the
python level, so the encoding/decoding is now an implementation detail
of the cython extension.
Contributed by lijun99, rtburns-jpl, vbrancat, mzzhong
* added option to pass segment number through unpackFrame_UAVSAR
* changed type of segment number
* changed segment type in prepareUAVSAR_coregstack
* Revert "changed type of segment number"
This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.
* Revert "changed segment type in prepareUAVSAR_coregstack"
This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.
* changed type of segment number in unpackFrame_UAVSAR
* Added code to stripmapStack/topo.py to explicitly create the `geom_reference/hgt.rdr.vrt` file to work with newer GDAL
* ImageLib: fix str syntax
* stackSentl: rename slc workflow merge for consistency
rename the mergeSLC step in "slc" workflow, to be consistent with the other workflows.
* topsStack: check inconsistent num of bursts in secondary images
+ extractCommonValidRegion.py: add dropSecondarysWithDifferentNumberOfBursts() to discard those acquisitions during the common region calculation
+ mergeBursts.py: skip secondarys with inconsistent num of bursts by raise exception and exit
+ s1a_isce_utils.adjustValidSampleLine_V2(): remove a redundant line
Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
* stackSentl.createParser(): align break line
+ align linebreak of help msg
+ use choices=[] for auto argument checking
+ use (default: %(default)s) to show defult value
* stackSent: bugfix for new paris if ref date is close to recent
This commit fixs the missing pair between the reference date and the new acquisitions, if the reference date is the most recent "num_conn" acquisition.
* Stack: check existing orbit with glob instead of os.path.exist(dir)
* topsStack: add --num-proc for topo step
stackSentinel: add --num-proc to pass the number of parallel processes to topo.py, so that mp.Pool() can be independent of OMP_NUM_THREADS
Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
* added option to pass segment number through unpackFrame_UAVSAR
* changed type of segment number
* changed segment type in prepareUAVSAR_coregstack
* Revert "changed type of segment number"
This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.
* Revert "changed segment type in prepareUAVSAR_coregstack"
This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.
* changed type of segment number in unpackFrame_UAVSAR
* Silence annoying FindFFTW warnings
* Fix linker errors at import-time
Python extensions are weakly linked, so linker errors can only be
caught at runtime. This commit fixes any such broken extensions,
and adds automated import testing to ctests to prevent future
regressions.
It also adds a helper function `isce2_add_staticlib` which creates a
position-independent static library suitable for usage in multiple
python extensions. This matches the SCons build scripts more closely
and allows for future extensions to be declared in a more terse and
hygienic fashion.
* Update utilLib scripts
* Update combinedLib scripts
* Update offoutliers lib scripts
* Add asa_im_decode cdll
* Update alos lib scripts
* Show ctest output upon failure
* Update DataAccessor scripts
* Remove TARGET_FILE_BASE_NAME
* Remove incomplete stdproc modules (require stanford code)
Co-authored-by: Ryan Burns <rtburns-jpl@users.noreply.github.com>
+ update os.system() call of gdal2isce_xml.py with the python function call, in stripmapStack/topo.py and applications/downsampleDEM.py
+ break the long lines in downsampleDEM.py for easy reading
topsStack/FilterAndCoherence.py:
+ add .full suffix to the complex coherence filename only if the multilook number is 1
+ for multilook number as 1, do not run looks sub-module
topsStack/Stack.filter_coherence(): add .full suffix only when azLooks * rgLooks == 1
topsStack/mergeBursts.py: add python3 at the top of script
Otherwise, if 'stack' folder exists but is empty, the script will return ERROR as below:
```
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack already exists.
Replacing master with existing stack.
updating the valid overlap region of:
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack
******************
swath: 1
Traceback (most recent call last):
File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 109, in <module>
main()
File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 95, in main
topMaster = ut.loadProduct(os.path.join(inps.master , 'IW{0}.xml'.format(swath)))
File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/s1a_isce_utils.py", line 26, in loadProduct
obj = pm.loadProduct(xmlname)
File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
self.load(filename)
File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/Configurable.py", line 1407, in load
tmpProp, tmpFact, tmpMisc = FP.parse(filename)
File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
root = ET.parse(filename)
File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
tree.parse(source, parser)
File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 587, in parse
source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/net/kraken/nobak/zyunjun/TonopahSenAT64/stack/IW1.xml'
```
contrib/PyCuAmpcor/cuDenseOffsets.py:
+ expose --raw-over-samp-factor option to change the raw data oversampling factor with default of 2
+ expose --corr-win-size option, which by default, is calculated automatically based on the search window size and raw oversampling factor
+ change the namespace key for the oversampling factor of correlation surface from 'oversample' to 'corr_oversample' to avoid ambiguity, the cmd option name is still the same, thus no different in the user side.
+ use vrt file for master / slave slc data loading folllowing https://github.com/isce-framework/isce2/pull/78
+ re-organize createParse() for readability
+ add example to argparse help
stripmapStack/MaskAndFilter.py:
+ fix a bug introduced in #https://github.com/isce-framework/isce2/pull/112
+ add example usage to help message
+ mask pixels with SNR == 0, for plotting only without touching data files
+ show input parameter in the figure title
+ show percentage after SNR thresholding
+ add interpolation for matplotlib while plotting
topsStack/grossOffsets: move obsolete basemap import from the top to the related commented code block
related with https://github.com/isce-framework/isce2/pull/97
+ move cpx cohe calculation code from main() to estCpxCoherence()
+ move computeCoherence import from the top to the inside of estCpxCoherence(), to avoid import error/warning sometimes.
+ fix typo in the default value of --cc option from 'fine.cori.full' to 'fine.cor.full'
+ FilterAndCoherence: skip filtering if input filter strength <= 0 and use original phase for coherence estimation
+ Stack: do not add filt_ prefix to output file if input filter strength <= 0
+ topo: merge runMultilookGdal() into runMultilook() with new argument "method" to control to use isce Looks module (averaging) or gdal_translate (extraction; default, same as before).
+ topo: call gdal2isce_xml.py so that xml file for multilooked data are always generate. This fix the warning message from gdal_translate: "Warning 1: Geotransform matrix has non rotational terms"
+ topo: skip multilooking if the data file has vrt/xml file missing, which sometimes happens to incLocal and shadowMask (don't know the cause yet; I used GPU version).
+ crossmul/resampleSlc: re-organize module import at the top of scripts
@falkamelung noted a race condition in #106 due to non-atomic directory
checking followed by os.makedirs. This pattern exists in many places
in the codebase so I went ahead and replaced the rest of them.
The new exist_ok usage should be terser and more idiomatic.
* Fix burst loading in estimateAzimuthMisreg.py
Within the for loop that starts at line 82, looping through range(minBurst, maxBurst) misses the burst with the maximum burst number. In cases where only one burst in a swath is used, the script fails at line 90 when .getWidth() is called on an empty xml. This issue can be fixed by adding the line maxBurst = maxBurst + 1
* Fix burst loading in estimateRangeMisreg.py
Similar to estimateAzimuthMisreg.py, calling range(minBurst,maxBurst) at line 185 will lead to an empty list and the some bursts not being loaded unless line 181 is changed from maxBurst = maxBurst - 1 to maxBurst = maxBurst +1
+ plot the evolution of offset after each step to better check the changes by masking, filling and filtering operations.
+ plot both azimuth and range offset in one figure and move the plotting to the end after file writing
+ update file path in xml file so that script can work even if the files are moved.
+ convert SNR nan value to zero before used as the mask, to avoid warning message
+ add the following options to customize the plot
- add -v and --v-snr option to change the display range for offset and SNR
- add --figsize and --save option
+ suppress the DEBUG message from matplotlib
+ remove obsolete getShape() and resampleOffset()
+ update IML module import
+ adjust indentation
+ dem.py: replace filename in xml file with abspath after geoid to ellipsoid correction
+ update README.md files in contrib/stack/*Stack, as fixImageXml.py is not needed anymore
+ fixImageXml: use native argparse mutually exclusive group for --full/--base options to replace checking with own code
ISCE_HOME was only used to get the location of the default logging config.
Lots of scripts were using boilerplate to set up this config, so I added
an `isce.logging` helper module which is the same as builtin python logging
but already has the configuration defaults set up for isce.
ISCE_HOME setup is retained in the toplevel `__init__.py`
but can now be removed without affecting functionality.
+ convert README.txt to README.md
+ stackStripMap.py: group iono options in createParser()
- group all iono options in createParser() for more easily navigation in --help.
- fix typos in the descriptions for the script and --dem option.
Stack.py: add function call to create water mask file in radar coordinates in the step one "run_1_master_*"
createWaterMask.py: fill pixels without DEM data with value of -1, same as water body
add -d / --dem_file option to input DEM file as an alternative for --bbox option.
add dem2bbox() to grab bbox from existing DEM file
adjust download_waterMask() that: if DEM file is input, put downloaded water body mask file to the same directory as the DEM file.
adjust the default water mask file in radar coordinates that:
1. in the same directory as the input lat/lon files
2. named as "waterMask.rdr" instead of "swbdLat_*.wbd.rdr"
add example docstring and update parser description.
This commit adds experimental CMake build support for ISCE2.
The only changes involved should be adding CMakeLists.txt files.
All current source files, headers, and other functionality, including
the existing SCons build system, should be unaffected.
Some functionality is still a work-in-progress.
These should all be marked as TODO in the CMakeLists.txt files.
`grep TODO **.txt`
Please read CMakeLists.txt and .cmake/*.cmake for more info.
Stack:
+ add alks and rlks options in config.topo()
topo: if number of looks is larger than 1, generate multilooked geometry files
+ generate `geom_master` folder in the root directory, same level as `Igrams`
+ import mroipac.looks.Looks module to multilook the full resolution geometry files from `./merged/geom_master`
+ copy over the full resolution xml/vrt files from `./merged/geome` to `./geom_master` as `*.full.xml/vrt` files, to support the number of looks extraction for mintpy
+ bug fixed in extractInfo(). This was not a problem before because the default values are 1 and the custom values from argument was not inputed in the config file.
Stack:
+ add alks and rlks while writing config file for unwrap
unwrap:
+ calculate the az/rg spacing/resolution and the equivalent number of independent looks in extractInfoFromPickle()
+ uncomment corrlooks setting in runUnwrap()
Zhang Yunjun
Ryan Burns
Bryan Marfito
CunrenLiang
Ryan Burns
Lijun Zhu
Sara Mirzaee
Minyan Zhong
Russell Grew
Ryan Burns
Lijun Zhu
ranneylxr
CunrenLiang
Eric Fielding
Zhang Yunjun
piyushrpt
Gerald Manipon
Heresh Fattahi
Paul A Rosen
Cheryl
shitong01
sssangha
Sara Mirzaee
Forrest Williams
Falk Amelung
Sara Mirzaee
Yujie Zheng
Eric J. Fielding
stoormgeo
Microwave Remote Sensing Laboratory
giangi
Mohammad Mohseni Aref
Eric Jameson Fielding