Commit Graph

173 Commits (500b9e2de2b3575dddb93627749e0d885041762f)

Author SHA1 Message Date
Sara Mirzaee 500b9e2de2 dloadOrbits.py: fix url change 2021-03-09 08:24:02 -06:00
Ryan Burns d0ec8c1794
Merge pull request #175 from CunrenLiang/main
new alosStack, PRF change of ALOS-1, update of TOPS ionospheric correction, new ionospheric filter
2021-03-04 10:52:45 -08:00
Bryan Marfito 903243c1c6
Update CMakeLists.txt 2021-02-28 19:29:13 +08:00
CunrenLiang 89973d7631 Merge remote-tracking branch 'upstream/main' into main 2021-02-26 11:39:02 +08:00
Ryan Burns b60e87d063
Merge pull request #236 from lijun99/pycuampcor
Updates to PyCuAmpcor
2021-02-19 10:27:38 -08:00
Lijun Zhu 7c40da5a74 PyCuAmpcor: offer an option to add gross offset to output offset image 2021-02-16 13:02:04 -08:00
Sara Mirzaee 68c7261b5d
Stack minor modification (#232)
* stackStripMap.py: change numbering style to match with topsStack

* Stack.py: separate parallel processors in topo.py from runfiles

* stackSentinel.py: add a new argument for parallel processors in topo.py

* Stack.py: remove extra num_process for unpackStackReferenceSLC

* stackSentinel.py:  tie numProcess4topo to numProcessif the first not specified
2021-02-15 11:24:56 -08:00
Lijun Zhu 563568561d change runDenseOffset(s) in Alos2Proc/TopsProc in accordance with updated PyCuAmpcor 2021-02-13 12:56:58 -08:00
Lijun Zhu 86a35acd5e merge mzzhong/grossOffsets into pycuampcor: scripts to set varying offsets withe velocity model 2021-02-13 12:55:04 -08:00
CunrenLiang 1d44e16b96 Merge remote-tracking branch 'upstream/main' into main 2021-02-09 15:44:29 -08:00
Lijun Zhu 4ddee2fdaf PyCuAmpcor: fix the help string in cuDenseOffsets.py 2021-02-06 12:02:34 -08:00
Lijun Zhu 5419353a69 PyCuAmpcor: fix the correlation surface variance estimation 2021-02-06 12:02:34 -08:00
Minyan Zhong d5e5067acd update 2021-02-06 12:02:34 -08:00
Minyan Zhong 19ee850211 fix-variance-estimation 2021-02-06 12:02:34 -08:00
Russell Grew e8da2d947a
Bash script edits to make stackSentinel.py faster in places (#225)
* plumb in num-process for time consuming steps

* refine num-process speed enhancements

* refine num-process speed enhancements

* dedicated function for writing batch processes

* refine num-process speed enhancements

* fix indents

Co-authored-by: Russell Grew <russell.grew@douglaspartners.com.au>
2021-02-04 13:58:36 -08:00
CunrenLiang 7b0856fc27 Merge remote-tracking branch 'upstream/main' into main 2021-02-03 13:24:22 -08:00
Bryan Marfito 690fe62c98
Changes to Cmake to include UnwrapComp folder
I've updated the CMakeLists.txt to include the folder UnwrapComp during installation.
2021-02-03 10:21:42 +08:00
Ryan Burns 3673da71cc Bump autoRIFT 1.0.7 -> 1.1.0 2021-02-02 16:29:29 -08:00
Lijun Zhu aa4d6abf99
add some includes for c header files (#221)
* add some includes for c header files

* Trim whitespace

Co-authored-by: Ryan Burns <rtburns-jpl@users.noreply.github.com>
2021-01-13 15:13:59 -08:00
Ryan Burns ad1815ef9c Modernize gdal imports
GDAL has dropped support for the older toplevel `import gdal`
in version 3.2, now requiring `from osgeo import gdal`.
2021-01-06 13:24:26 -08:00
Ryan Burns 86b126f564
Merge pull request #208 from lijun99/pycuampcor
PyCuAmpcor updates:
2021-01-06 13:20:44 -08:00
ranneylxr 48559c6e3e
using geopandas and shapely package to calculate whether there are intersections between bbox and sentinel SAFE file. (#216)
* stackSentinel.py:using geopandas to calculate intersection between bbox and SAFE file coverage region

* stackSentinel.py:using geopandas and shapely package to calculate whether there are intersections between bbox and sentinel SAFE file. And remove import geopandas and shapely part into generate_geopolygon function

* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage

* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage

* stackSentinel.py:using shapely package to find whether bbox intersects with SAFE file

* stackSentinel.py:remove matplotlib DEBUG message#L14-L18
2021-01-06 13:19:55 -08:00
Lijun Zhu 7e9c86910a PyCuAmcpor: set default gpu id to 0 in cuDenseOffsets.py; the auto seek with a negative value is now disabled 2020-12-15 13:33:34 -08:00
CunrenLiang fb96c54581 Update ion_subband.py
correct variable name
2020-12-06 12:07:48 -08:00
Lijun Zhu 6d9f088975 PyCuAmpcor: cuDenseOffsets.py towork with input varying gross offsets 2020-11-30 10:30:23 -08:00
Minyan Zhong c399d3fa03 clean up and update grossOffsets 2020-11-30 00:49:28 -08:00
CunrenLiang ab8b0f4755 Update create_cmds.py
add comments about identifying bad pairs
2020-11-26 23:40:29 -08:00
Lijun Zhu 94caa1ea5c PyCuAmpcor: more code cleanup
* replace tabs with spaces to align the code

  * remove extra spaces at the end of lines

  * add more docstrings
2020-11-25 12:55:38 -08:00
Lijun Zhu 38646456d3 PyCuAmpcor: code cleanup, add docstrings 2020-11-17 23:22:37 -08:00
CunrenLiang 5634f5fa32 Update ion_check.py 2020-11-17 10:59:25 -08:00
CunrenLiang 4e47909428 Update ion_filt.py
minor update
2020-11-15 01:00:27 -08:00
Lijun Zhu a393282b69 PyCuAmpcor updates:
* added a README.md for installation/user guide/procedures

  * modified the cuDenseOffsets.py
     * expose more options from the CUDA/C++ program
     * add an option for varying gross offset input
     * clarify the parameter definitions

  * removed old SlcImage implementation and cublas dependence

  * modified cuSincOversampler
    * to be consistent with cpu version
    * fix an issue when the extraction of the search window is not around the center

  * added a debug mode to output intermediate results

  * enable cuda error checking for both Debug/Release build types

  * corrected the code to extract raw images when the correlation surface peak is close to edges

  * Move utf-8 decoding step inside cython extension

    The cython setters take python strings, but the getters return
    python bytes, so this makes the types match up. I went with regular
    python strings for the interface since they are more common at the
    python level, so the encoding/decoding is now an implementation detail
    of the cython extension.

Contributed by lijun99, rtburns-jpl, vbrancat, mzzhong
2020-11-12 15:02:44 -08:00
CunrenLiang 78a5ddead6 use for loops for dates/pairs processing 2020-11-09 12:43:12 -08:00
Cunren Liang 6cecbe9e11 add executable permission 2020-11-07 16:06:02 -08:00
CunrenLiang 5e4d1604dc put temporary files in a better place 2020-11-07 16:06:02 -08:00
CunrenLiang e9bd7edeb3 new alosStack application 2020-11-07 16:06:02 -08:00
Eric Fielding c601204942
Added code to stripmapStack/topo.py to work with newer GDAL (#196)
* added option to pass segment number through unpackFrame_UAVSAR

* changed type of segment number

* changed segment type in prepareUAVSAR_coregstack

* Revert "changed type of segment number"

This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.

* Revert "changed segment type in prepareUAVSAR_coregstack"

This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.

* changed type of segment number in unpackFrame_UAVSAR

* Added code to stripmapStack/topo.py to explicitly create the `geom_reference/hgt.rdr.vrt` file to work with newer GDAL
2020-10-24 09:36:08 -07:00
Zhang Yunjun 55b9d2c037
topsStack: ignore coreg_secondary images with inconsistent num of bursts (#195)
* ImageLib: fix str syntax

* stackSentl: rename slc workflow merge for consistency

rename the mergeSLC step in "slc" workflow, to be consistent with the other workflows.

* topsStack: check inconsistent num of bursts in secondary images

+ extractCommonValidRegion.py: add dropSecondarysWithDifferentNumberOfBursts() to discard those acquisitions during the common region calculation

+ mergeBursts.py: skip secondarys with inconsistent num of bursts by raise exception and exit

+ s1a_isce_utils.adjustValidSampleLine_V2(): remove a redundant line

Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
2020-10-24 09:31:48 -07:00
Zhang Yunjun 9bfa2dd6db
topsStack: add --num-proc for step topo (#193)
* stackSentl.createParser(): align break line

+ align linebreak of help msg
+ use choices=[] for auto argument checking
+ use (default: %(default)s) to show defult value

* stackSent: bugfix for new paris if ref date is close to recent

This commit fixs the missing pair between the reference date and the new acquisitions, if the reference date is the most recent "num_conn" acquisition.

* Stack: check existing orbit with glob instead of os.path.exist(dir)

* topsStack: add --num-proc for topo step

stackSentinel: add --num-proc to pass the number of parallel processes to topo.py, so that mp.Pool() can be independent of OMP_NUM_THREADS

Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
2020-10-09 07:11:48 -07:00
Eric Fielding 6aa2d065c3
added option to pass segment number through unpackFrame_UAVSAR (#171)
* added option to pass segment number through unpackFrame_UAVSAR

* changed type of segment number

* changed segment type in prepareUAVSAR_coregstack

* Revert "changed type of segment number"

This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.

* Revert "changed segment type in prepareUAVSAR_coregstack"

This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.

* changed type of segment number in unpackFrame_UAVSAR
2020-10-04 16:30:18 -07:00
piyushrpt 5370690b0a Pulling in PR #153 2020-10-04 11:10:24 -07:00
Ryan Burns ba2cb412ea
Bump autoRIFT 1.0.6 -> 1.0.7 (#191)
Co-authored-by: Ryan Burns <rtburns-jpl@users.noreply.github.com>
2020-09-14 12:52:11 -07:00
Ryan Burns 337dc9c158
Fix python extension linking (#168)
* Silence annoying FindFFTW warnings

* Fix linker errors at import-time

Python extensions are weakly linked, so linker errors can only be
caught at runtime. This commit fixes any such broken extensions,
and adds automated import testing to ctests to prevent future
regressions.

It also adds a helper function `isce2_add_staticlib` which creates a
position-independent static library suitable for usage in multiple
python extensions. This matches the SCons build scripts more closely
and allows for future extensions to be declared in a more terse and
hygienic fashion.

* Update utilLib scripts

* Update combinedLib scripts

* Update offoutliers lib scripts

* Add asa_im_decode cdll

* Update alos lib scripts

* Show ctest output upon failure

* Update DataAccessor scripts

* Remove TARGET_FILE_BASE_NAME

* Remove incomplete stdproc modules (require stanford code)

Co-authored-by: Ryan Burns <rtburns-jpl@users.noreply.github.com>
2020-09-14 10:57:10 -07:00
Zhang Yunjun f14981341d update system call of gdal2isce_xml with python interface
+ update os.system() call of gdal2isce_xml.py with the python function call, in stripmapStack/topo.py and applications/downsampleDEM.py

+ break the long lines in downsampleDEM.py for easy reading
2020-09-12 17:05:33 -07:00
Zhang Yunjun f06d22e02c topsStack/topo: use OMP_NUM_THREADS if available
for the parallel processing with multiprocessing.Pool()
2020-09-12 17:05:33 -07:00
yunjunz 6a02f79277 FilterAndCoh: use fine.cor(.full) for multilooked (full-reso) cpx cor file
topsStack/FilterAndCoherence.py:
+ add .full suffix to the complex coherence filename only if the multilook number is 1
+ for multilook number as 1, do not run looks sub-module

topsStack/Stack.filter_coherence(): add .full suffix only when azLooks * rgLooks == 1

topsStack/mergeBursts.py: add python3 at the top of script
2020-09-12 17:05:33 -07:00
Zhang Yunjun 89839bc453 extractCommonValidRegion: check stack/*.xml file existance
Otherwise, if 'stack' folder exists but is empty, the script will return ERROR as below:

```
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack  already exists.
Replacing master with existing stack.
updating the valid overlap region of:
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack
******************
swath:  1
Traceback (most recent call last):
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 109, in <module>
    main()
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 95, in main
    topMaster = ut.loadProduct(os.path.join(inps.master , 'IW{0}.xml'.format(swath)))
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/s1a_isce_utils.py", line 26, in loadProduct
    obj = pm.loadProduct(xmlname)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
    self.load(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/Configurable.py", line 1407, in load
    tmpProp, tmpFact, tmpMisc = FP.parse(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 587, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/net/kraken/nobak/zyunjun/TonopahSenAT64/stack/IW1.xml'
```
2020-09-12 17:05:33 -07:00
Ryan Burns b1c8a599ca Add cmake scripts for autoRIFT 2020-07-21 18:04:16 -07:00
Ryan Burns 48b6bfbccb Bake in autoRIFT 2020-07-21 14:52:28 -07:00
Gerald Manipon f09213eb33
various updates from ARIA-related projects (#149)
* add Dockerfile and SConfigISCE to build cuda-enabled modules

* add Dockerfile and SConfigISCE to build cuda-enabled modules

* set correct name

* update for centos7 and cuda-dev image

* restore

* disable GPUampcor

* update

* save

* sync up Dockerfile updates

* fix base image

* change docker image names

* create gpu vs. cpu specific tags

* update with nodata WBD Stitcher

* fix segmentation fault when running sciflo workflows

- remove dependencies that downgrade the `geos` conda package

* sync fix for segmentation fault to cuda builds

* ping ffi to version 1.12.2

Latest version of ffi, 1.13.0, breaks installation of fpm:
```
$ sudo gem install --no-ri --no-rdoc fpm
Fetching: cabin-0.9.0.gem (100%)
Successfully installed cabin-0.9.0
Fetching: backports-3.17.2.gem (100%)
Successfully installed backports-3.17.2
Fetching: arr-pm-0.0.10.gem (100%)
Successfully installed arr-pm-0.0.10
Fetching: clamp-1.0.1.gem (100%)
Successfully installed clamp-1.0.1
Fetching: ffi-1.13.0.gem (100%)
ERROR:  Error installing fpm:
        ffi requires Ruby version >= 2.3.
```
TODO: remove pin when ffi's backwards compatiblity is restored.

* comment out unused import

* add build circleci job to PR tests

* pin ffi in cuda build

* revert logging hackery

* fix "target_include_directories called with non-compilable target type" errors

* Keep source directory for topsStack

Co-authored-by: dustinlo <dustin.k.lo@jpl.nasa.gov>
Co-authored-by: shitong01 <stchin@ntu.edu.sg>
Co-authored-by: Ryan Burns <47790121+rtburns-jpl@users.noreply.github.com>
2020-07-16 16:08:57 -07:00