Commit Graph

111 Commits (c8af76be104bd51f8a8e613e80e17d77c4b4ac30)

Author SHA1 Message Date
Bryan Marfito c8af76be10
Update fetchOrbit.py 2021-03-14 21:11:21 +08:00
Bryan Marfito a25f80537a
Proposed changes to the fetchOrbit.py
After running the currently modified fetchOrbits.py, I've found out that the downloaded precise orbits were based on the date that they were processed instead of the date that they can be used. I've also modified the code to prevent downloading of restituted orbit if the precise orbit is available.

Pls. check if the changes I've made applies the orbit files to the Sentinel-1 SLC images properly.
2021-03-12 23:36:45 +08:00
Sara Mirzaee 1a86bca5c4 fix url change 2021-03-09 13:43:13 -06:00
Sara Mirzaee e40f70593f
Update README.md
qc.sentinel1.eo.esa.int -> aux.sentinel1.eo.esa.int
2021-03-09 09:57:24 -05:00
Sara Mirzaee 500b9e2de2 dloadOrbits.py: fix url change 2021-03-09 08:24:02 -06:00
CunrenLiang 89973d7631 Merge remote-tracking branch 'upstream/main' into main 2021-02-26 11:39:02 +08:00
Ryan Burns b60e87d063
Merge pull request #236 from lijun99/pycuampcor
Updates to PyCuAmpcor
2021-02-19 10:27:38 -08:00
Sara Mirzaee 68c7261b5d
Stack minor modification (#232)
* stackStripMap.py: change numbering style to match with topsStack

* Stack.py: separate parallel processors in topo.py from runfiles

* stackSentinel.py: add a new argument for parallel processors in topo.py

* Stack.py: remove extra num_process for unpackStackReferenceSLC

* stackSentinel.py:  tie numProcess4topo to numProcessif the first not specified
2021-02-15 11:24:56 -08:00
Lijun Zhu 86a35acd5e merge mzzhong/grossOffsets into pycuampcor: scripts to set varying offsets withe velocity model 2021-02-13 12:55:04 -08:00
CunrenLiang 1d44e16b96 Merge remote-tracking branch 'upstream/main' into main 2021-02-09 15:44:29 -08:00
Russell Grew e8da2d947a
Bash script edits to make stackSentinel.py faster in places (#225)
* plumb in num-process for time consuming steps

* refine num-process speed enhancements

* refine num-process speed enhancements

* dedicated function for writing batch processes

* refine num-process speed enhancements

* fix indents

Co-authored-by: Russell Grew <russell.grew@douglaspartners.com.au>
2021-02-04 13:58:36 -08:00
CunrenLiang 7b0856fc27 Merge remote-tracking branch 'upstream/main' into main 2021-02-03 13:24:22 -08:00
Ryan Burns ad1815ef9c Modernize gdal imports
GDAL has dropped support for the older toplevel `import gdal`
in version 3.2, now requiring `from osgeo import gdal`.
2021-01-06 13:24:26 -08:00
ranneylxr 48559c6e3e
using geopandas and shapely package to calculate whether there are intersections between bbox and sentinel SAFE file. (#216)
* stackSentinel.py:using geopandas to calculate intersection between bbox and SAFE file coverage region

* stackSentinel.py:using geopandas and shapely package to calculate whether there are intersections between bbox and sentinel SAFE file. And remove import geopandas and shapely part into generate_geopolygon function

* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage

* stackSentinel.py: using shapely package to check whether bbox intersect with SAFE file coverage

* stackSentinel.py:using shapely package to find whether bbox intersects with SAFE file

* stackSentinel.py:remove matplotlib DEBUG message#L14-L18
2021-01-06 13:19:55 -08:00
CunrenLiang fb96c54581 Update ion_subband.py
correct variable name
2020-12-06 12:07:48 -08:00
Minyan Zhong c399d3fa03 clean up and update grossOffsets 2020-11-30 00:49:28 -08:00
CunrenLiang ab8b0f4755 Update create_cmds.py
add comments about identifying bad pairs
2020-11-26 23:40:29 -08:00
CunrenLiang 5634f5fa32 Update ion_check.py 2020-11-17 10:59:25 -08:00
CunrenLiang 4e47909428 Update ion_filt.py
minor update
2020-11-15 01:00:27 -08:00
CunrenLiang 78a5ddead6 use for loops for dates/pairs processing 2020-11-09 12:43:12 -08:00
Cunren Liang 6cecbe9e11 add executable permission 2020-11-07 16:06:02 -08:00
CunrenLiang 5e4d1604dc put temporary files in a better place 2020-11-07 16:06:02 -08:00
CunrenLiang e9bd7edeb3 new alosStack application 2020-11-07 16:06:02 -08:00
Eric Fielding c601204942
Added code to stripmapStack/topo.py to work with newer GDAL (#196)
* added option to pass segment number through unpackFrame_UAVSAR

* changed type of segment number

* changed segment type in prepareUAVSAR_coregstack

* Revert "changed type of segment number"

This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.

* Revert "changed segment type in prepareUAVSAR_coregstack"

This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.

* changed type of segment number in unpackFrame_UAVSAR

* Added code to stripmapStack/topo.py to explicitly create the `geom_reference/hgt.rdr.vrt` file to work with newer GDAL
2020-10-24 09:36:08 -07:00
Zhang Yunjun 55b9d2c037
topsStack: ignore coreg_secondary images with inconsistent num of bursts (#195)
* ImageLib: fix str syntax

* stackSentl: rename slc workflow merge for consistency

rename the mergeSLC step in "slc" workflow, to be consistent with the other workflows.

* topsStack: check inconsistent num of bursts in secondary images

+ extractCommonValidRegion.py: add dropSecondarysWithDifferentNumberOfBursts() to discard those acquisitions during the common region calculation

+ mergeBursts.py: skip secondarys with inconsistent num of bursts by raise exception and exit

+ s1a_isce_utils.adjustValidSampleLine_V2(): remove a redundant line

Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
2020-10-24 09:31:48 -07:00
Zhang Yunjun 9bfa2dd6db
topsStack: add --num-proc for step topo (#193)
* stackSentl.createParser(): align break line

+ align linebreak of help msg
+ use choices=[] for auto argument checking
+ use (default: %(default)s) to show defult value

* stackSent: bugfix for new paris if ref date is close to recent

This commit fixs the missing pair between the reference date and the new acquisitions, if the reference date is the most recent "num_conn" acquisition.

* Stack: check existing orbit with glob instead of os.path.exist(dir)

* topsStack: add --num-proc for topo step

stackSentinel: add --num-proc to pass the number of parallel processes to topo.py, so that mp.Pool() can be independent of OMP_NUM_THREADS

Co-authored-by: Zhang Yunjun <yunjunz@users.noreply.github.com>
2020-10-09 07:11:48 -07:00
Eric Fielding 6aa2d065c3
added option to pass segment number through unpackFrame_UAVSAR (#171)
* added option to pass segment number through unpackFrame_UAVSAR

* changed type of segment number

* changed segment type in prepareUAVSAR_coregstack

* Revert "changed type of segment number"

This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.

* Revert "changed segment type in prepareUAVSAR_coregstack"

This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.

* changed type of segment number in unpackFrame_UAVSAR
2020-10-04 16:30:18 -07:00
piyushrpt 5370690b0a Pulling in PR #153 2020-10-04 11:10:24 -07:00
Zhang Yunjun f14981341d update system call of gdal2isce_xml with python interface
+ update os.system() call of gdal2isce_xml.py with the python function call, in stripmapStack/topo.py and applications/downsampleDEM.py

+ break the long lines in downsampleDEM.py for easy reading
2020-09-12 17:05:33 -07:00
Zhang Yunjun f06d22e02c topsStack/topo: use OMP_NUM_THREADS if available
for the parallel processing with multiprocessing.Pool()
2020-09-12 17:05:33 -07:00
yunjunz 6a02f79277 FilterAndCoh: use fine.cor(.full) for multilooked (full-reso) cpx cor file
topsStack/FilterAndCoherence.py:
+ add .full suffix to the complex coherence filename only if the multilook number is 1
+ for multilook number as 1, do not run looks sub-module

topsStack/Stack.filter_coherence(): add .full suffix only when azLooks * rgLooks == 1

topsStack/mergeBursts.py: add python3 at the top of script
2020-09-12 17:05:33 -07:00
Zhang Yunjun 89839bc453 extractCommonValidRegion: check stack/*.xml file existance
Otherwise, if 'stack' folder exists but is empty, the script will return ERROR as below:

```
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack  already exists.
Replacing master with existing stack.
updating the valid overlap region of:
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack
******************
swath:  1
Traceback (most recent call last):
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 109, in <module>
    main()
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 95, in main
    topMaster = ut.loadProduct(os.path.join(inps.master , 'IW{0}.xml'.format(swath)))
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/s1a_isce_utils.py", line 26, in loadProduct
    obj = pm.loadProduct(xmlname)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
    self.load(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/Configurable.py", line 1407, in load
    tmpProp, tmpFact, tmpMisc = FP.parse(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 587, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/net/kraken/nobak/zyunjun/TonopahSenAT64/stack/IW1.xml'
```
2020-09-12 17:05:33 -07:00
Gerald Manipon f09213eb33
various updates from ARIA-related projects (#149)
* add Dockerfile and SConfigISCE to build cuda-enabled modules

* add Dockerfile and SConfigISCE to build cuda-enabled modules

* set correct name

* update for centos7 and cuda-dev image

* restore

* disable GPUampcor

* update

* save

* sync up Dockerfile updates

* fix base image

* change docker image names

* create gpu vs. cpu specific tags

* update with nodata WBD Stitcher

* fix segmentation fault when running sciflo workflows

- remove dependencies that downgrade the `geos` conda package

* sync fix for segmentation fault to cuda builds

* ping ffi to version 1.12.2

Latest version of ffi, 1.13.0, breaks installation of fpm:
```
$ sudo gem install --no-ri --no-rdoc fpm
Fetching: cabin-0.9.0.gem (100%)
Successfully installed cabin-0.9.0
Fetching: backports-3.17.2.gem (100%)
Successfully installed backports-3.17.2
Fetching: arr-pm-0.0.10.gem (100%)
Successfully installed arr-pm-0.0.10
Fetching: clamp-1.0.1.gem (100%)
Successfully installed clamp-1.0.1
Fetching: ffi-1.13.0.gem (100%)
ERROR:  Error installing fpm:
        ffi requires Ruby version >= 2.3.
```
TODO: remove pin when ffi's backwards compatiblity is restored.

* comment out unused import

* add build circleci job to PR tests

* pin ffi in cuda build

* revert logging hackery

* fix "target_include_directories called with non-compilable target type" errors

* Keep source directory for topsStack

Co-authored-by: dustinlo <dustin.k.lo@jpl.nasa.gov>
Co-authored-by: shitong01 <stchin@ntu.edu.sg>
Co-authored-by: Ryan Burns <47790121+rtburns-jpl@users.noreply.github.com>
2020-07-16 16:08:57 -07:00
Heresh Fattahi eb3eff61a6
Merge branch 'master' into BLM 2020-07-02 15:03:10 -07:00
Paul A Rosen e03042f4e5 diversity: replaced oppressive language with neutral definitions of the images that comprise an interferogram through the entire repository 2020-07-02 12:40:49 -07:00
Zhang Yunjun 90d578fa5d cuDenseOffsets: expose --raw-osf and --corr-win-size option
contrib/PyCuAmpcor/cuDenseOffsets.py:
+ expose --raw-over-samp-factor option to change the raw data oversampling factor with default of 2
+ expose --corr-win-size option, which by default, is calculated automatically based on the search window size and raw oversampling factor
+ change the namespace key for the oversampling factor of correlation surface from 'oversample' to 'corr_oversample' to avoid ambiguity, the cmd option name is still the same, thus no different in the user side.
+ use vrt file for master / slave slc data loading folllowing https://github.com/isce-framework/isce2/pull/78
+ re-organize createParse() for readability
+ add example to argparse help

stripmapStack/MaskAndFilter.py:
+ fix a bug introduced in #https://github.com/isce-framework/isce2/pull/112
+ add example usage to help message
+ mask pixels with SNR == 0, for plotting only without touching data files
+ show input parameter in the figure title
+ show percentage after SNR thresholding
+ add interpolation for matplotlib while plotting

topsStack/grossOffsets: move obsolete basemap import from the top to the related commented code block
2020-06-28 19:24:09 -07:00
Cheryl edea69d4b6 added buffer margin to orbit times in get_orbit 2020-06-11 23:33:37 -07:00
shitong01 0fb49e383e fix syntax 2020-06-11 23:33:37 -07:00
shitong01 bb03d8c652 changed made by @cheryltwj to fix orbit file download edge case 2020-06-11 23:33:37 -07:00
sssangha acf0bbb55c Negative valid samples/lines now set to 0 2020-06-06 20:44:00 -07:00
Zhang Yunjun 35a80f0a5f bugfix in stripmapStack/Stack for config unwIfg if filter is turned off
If filter is turned OFF while unwrapping is still ON, the phase unwrapping output filename should not include "filt_" prefix.
2020-05-17 14:58:39 -07:00
yunjunz af5a8540f9 bugfix in topsStack/Stack while writing config_unwrap for rmFilter option
stackSentinel: more msg in checkCurrentStatus()
2020-05-17 14:58:39 -07:00
Zhang Yunjun 659a7ed6b0 stackSentinel: use 2 digit number in run_files to better sort 10 or more steps
update topsStack/README.md accordingly.

stackSentinel: suppress matplotlib DEBUG msg

stackSentinel: remove the unused imp import
2020-05-17 14:58:39 -07:00
Zhang Yunjun 0f60e9f24c FilterAndCoh: move cpx coh calc into a func
related with https://github.com/isce-framework/isce2/pull/97

+ move cpx cohe calculation code from main() to estCpxCoherence()
+ move computeCoherence import from the top to the inside of estCpxCoherence(), to avoid import error/warning sometimes.
+ fix typo in the default value of --cc option from 'fine.cori.full' to 'fine.cor.full'
2020-05-17 14:58:39 -07:00
Zhang Yunjun 4282bb8e0a prepSlcALOS2: add --polarization to pass to unpackFrame* 2020-05-03 14:12:24 -07:00
Zhang Yunjun 6a78c4e6d6 skip filtering if --filter_strength <= 0
+ FilterAndCoherence: skip filtering if input filter strength <= 0 and use original phase for coherence estimation

+ Stack: do not add filt_ prefix to output file if input filter strength <= 0
2020-05-03 14:12:24 -07:00
Zhang Yunjun d494ae3924 Stack: adjust indentation 2020-05-03 14:12:24 -07:00
Zhang Yunjun 34fb0b66fa Stack: skip PU if input method is "no" 2020-05-03 14:12:24 -07:00
Zhang Yunjun d41ffb0ce9 stackStripMap: suppress matplotlib DEBUG msg 2020-05-03 14:12:24 -07:00
Heresh Fattahi c9dd735a7b
Merge pull request #125 from yunjunz/topo
stripmapStack/topo: call gdal2isce_xml.py inside runMultilook() if xml file missing
2020-05-03 09:22:50 -07:00