Commit Graph

85 Commits (6aa2d065c3a1fca95e10fdbd33bf4e19f06a28b6)

Author SHA1 Message Date
Eric Fielding 6aa2d065c3
added option to pass segment number through unpackFrame_UAVSAR (#171)
* added option to pass segment number through unpackFrame_UAVSAR

* changed type of segment number

* changed segment type in prepareUAVSAR_coregstack

* Revert "changed type of segment number"

This reverts commit a18c6452d4b5c95ce869dab2ee1e3bcf6ff2b1d9.

* Revert "changed segment type in prepareUAVSAR_coregstack"

This reverts commit 6cf3ebadb3a6e119c6084c8e03b8cc68d7eccc3f.

* changed type of segment number in unpackFrame_UAVSAR
2020-10-04 16:30:18 -07:00
piyushrpt 5370690b0a Pulling in PR #153 2020-10-04 11:10:24 -07:00
Zhang Yunjun f14981341d update system call of gdal2isce_xml with python interface
+ update os.system() call of gdal2isce_xml.py with the python function call, in stripmapStack/topo.py and applications/downsampleDEM.py

+ break the long lines in downsampleDEM.py for easy reading
2020-09-12 17:05:33 -07:00
Zhang Yunjun f06d22e02c topsStack/topo: use OMP_NUM_THREADS if available
for the parallel processing with multiprocessing.Pool()
2020-09-12 17:05:33 -07:00
yunjunz 6a02f79277 FilterAndCoh: use fine.cor(.full) for multilooked (full-reso) cpx cor file
topsStack/FilterAndCoherence.py:
+ add .full suffix to the complex coherence filename only if the multilook number is 1
+ for multilook number as 1, do not run looks sub-module

topsStack/Stack.filter_coherence(): add .full suffix only when azLooks * rgLooks == 1

topsStack/mergeBursts.py: add python3 at the top of script
2020-09-12 17:05:33 -07:00
Zhang Yunjun 89839bc453 extractCommonValidRegion: check stack/*.xml file existance
Otherwise, if 'stack' folder exists but is empty, the script will return ERROR as below:

```
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack  already exists.
Replacing master with existing stack.
updating the valid overlap region of:
/net/kraken/nobak/zyunjun/TonopahSenAT64/stack
******************
swath:  1
Traceback (most recent call last):
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 109, in <module>
    main()
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/extractCommonValidRegion.py", line 95, in main
    topMaster = ut.loadProduct(os.path.join(inps.master , 'IW{0}.xml'.format(swath)))
  File "/home/zyunjun/tools/isce2/contrib/stack/topsStack/s1a_isce_utils.py", line 26, in loadProduct
    obj = pm.loadProduct(xmlname)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/ProductManager.py", line 25, in loadProduct
    self.load(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Component/Configurable.py", line 1407, in load
    tmpProp, tmpFact, tmpMisc = FP.parse(filename)
  File "/net/kraken/home1/geomod/apps/rhel7/isce2-github/isce/components/iscesys/Parsers/XmlParser.py", line 41, in parse
    root = ET.parse(filename)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 1197, in parse
    tree.parse(source, parser)
  File "/net/kraken/home1/geomod/apps/anaconda3/lib/python3.7/xml/etree/ElementTree.py", line 587, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/net/kraken/nobak/zyunjun/TonopahSenAT64/stack/IW1.xml'
```
2020-09-12 17:05:33 -07:00
Gerald Manipon f09213eb33
various updates from ARIA-related projects (#149)
* add Dockerfile and SConfigISCE to build cuda-enabled modules

* add Dockerfile and SConfigISCE to build cuda-enabled modules

* set correct name

* update for centos7 and cuda-dev image

* restore

* disable GPUampcor

* update

* save

* sync up Dockerfile updates

* fix base image

* change docker image names

* create gpu vs. cpu specific tags

* update with nodata WBD Stitcher

* fix segmentation fault when running sciflo workflows

- remove dependencies that downgrade the `geos` conda package

* sync fix for segmentation fault to cuda builds

* ping ffi to version 1.12.2

Latest version of ffi, 1.13.0, breaks installation of fpm:
```
$ sudo gem install --no-ri --no-rdoc fpm
Fetching: cabin-0.9.0.gem (100%)
Successfully installed cabin-0.9.0
Fetching: backports-3.17.2.gem (100%)
Successfully installed backports-3.17.2
Fetching: arr-pm-0.0.10.gem (100%)
Successfully installed arr-pm-0.0.10
Fetching: clamp-1.0.1.gem (100%)
Successfully installed clamp-1.0.1
Fetching: ffi-1.13.0.gem (100%)
ERROR:  Error installing fpm:
        ffi requires Ruby version >= 2.3.
```
TODO: remove pin when ffi's backwards compatiblity is restored.

* comment out unused import

* add build circleci job to PR tests

* pin ffi in cuda build

* revert logging hackery

* fix "target_include_directories called with non-compilable target type" errors

* Keep source directory for topsStack

Co-authored-by: dustinlo <dustin.k.lo@jpl.nasa.gov>
Co-authored-by: shitong01 <stchin@ntu.edu.sg>
Co-authored-by: Ryan Burns <47790121+rtburns-jpl@users.noreply.github.com>
2020-07-16 16:08:57 -07:00
Heresh Fattahi eb3eff61a6
Merge branch 'master' into BLM 2020-07-02 15:03:10 -07:00
Paul A Rosen e03042f4e5 diversity: replaced oppressive language with neutral definitions of the images that comprise an interferogram through the entire repository 2020-07-02 12:40:49 -07:00
Zhang Yunjun 90d578fa5d cuDenseOffsets: expose --raw-osf and --corr-win-size option
contrib/PyCuAmpcor/cuDenseOffsets.py:
+ expose --raw-over-samp-factor option to change the raw data oversampling factor with default of 2
+ expose --corr-win-size option, which by default, is calculated automatically based on the search window size and raw oversampling factor
+ change the namespace key for the oversampling factor of correlation surface from 'oversample' to 'corr_oversample' to avoid ambiguity, the cmd option name is still the same, thus no different in the user side.
+ use vrt file for master / slave slc data loading folllowing https://github.com/isce-framework/isce2/pull/78
+ re-organize createParse() for readability
+ add example to argparse help

stripmapStack/MaskAndFilter.py:
+ fix a bug introduced in #https://github.com/isce-framework/isce2/pull/112
+ add example usage to help message
+ mask pixels with SNR == 0, for plotting only without touching data files
+ show input parameter in the figure title
+ show percentage after SNR thresholding
+ add interpolation for matplotlib while plotting

topsStack/grossOffsets: move obsolete basemap import from the top to the related commented code block
2020-06-28 19:24:09 -07:00
Cheryl edea69d4b6 added buffer margin to orbit times in get_orbit 2020-06-11 23:33:37 -07:00
shitong01 0fb49e383e fix syntax 2020-06-11 23:33:37 -07:00
shitong01 bb03d8c652 changed made by @cheryltwj to fix orbit file download edge case 2020-06-11 23:33:37 -07:00
sssangha acf0bbb55c Negative valid samples/lines now set to 0 2020-06-06 20:44:00 -07:00
Zhang Yunjun 35a80f0a5f bugfix in stripmapStack/Stack for config unwIfg if filter is turned off
If filter is turned OFF while unwrapping is still ON, the phase unwrapping output filename should not include "filt_" prefix.
2020-05-17 14:58:39 -07:00
yunjunz af5a8540f9 bugfix in topsStack/Stack while writing config_unwrap for rmFilter option
stackSentinel: more msg in checkCurrentStatus()
2020-05-17 14:58:39 -07:00
Zhang Yunjun 659a7ed6b0 stackSentinel: use 2 digit number in run_files to better sort 10 or more steps
update topsStack/README.md accordingly.

stackSentinel: suppress matplotlib DEBUG msg

stackSentinel: remove the unused imp import
2020-05-17 14:58:39 -07:00
Zhang Yunjun 0f60e9f24c FilterAndCoh: move cpx coh calc into a func
related with https://github.com/isce-framework/isce2/pull/97

+ move cpx cohe calculation code from main() to estCpxCoherence()
+ move computeCoherence import from the top to the inside of estCpxCoherence(), to avoid import error/warning sometimes.
+ fix typo in the default value of --cc option from 'fine.cori.full' to 'fine.cor.full'
2020-05-17 14:58:39 -07:00
Zhang Yunjun 4282bb8e0a prepSlcALOS2: add --polarization to pass to unpackFrame* 2020-05-03 14:12:24 -07:00
Zhang Yunjun 6a78c4e6d6 skip filtering if --filter_strength <= 0
+ FilterAndCoherence: skip filtering if input filter strength <= 0 and use original phase for coherence estimation

+ Stack: do not add filt_ prefix to output file if input filter strength <= 0
2020-05-03 14:12:24 -07:00
Zhang Yunjun d494ae3924 Stack: adjust indentation 2020-05-03 14:12:24 -07:00
Zhang Yunjun 34fb0b66fa Stack: skip PU if input method is "no" 2020-05-03 14:12:24 -07:00
Zhang Yunjun d41ffb0ce9 stackStripMap: suppress matplotlib DEBUG msg 2020-05-03 14:12:24 -07:00
Heresh Fattahi c9dd735a7b
Merge pull request #125 from yunjunz/topo
stripmapStack/topo: call gdal2isce_xml.py inside runMultilook() if xml file missing
2020-05-03 09:22:50 -07:00
Zhang Yunjun 8017039ecb topo: replace gdal_translate cmd call with gda.Translate()
+ add more print out message
2020-05-02 11:54:17 -07:00
Zhang Yunjun 08dfd9de21 topo: call gdal2isce_xml.py inside runMultilook() if xml file missing
+ topo: merge runMultilookGdal() into runMultilook() with new argument "method" to control to use isce Looks module (averaging) or gdal_translate (extraction; default, same as before).

+ topo: call gdal2isce_xml.py so that xml file for multilooked data are always generate. This fix the warning message from gdal_translate: "Warning 1: Geotransform matrix has non rotational terms"

+ topo: skip multilooking if the data file has vrt/xml file missing, which sometimes happens to incLocal and shadowMask (don't know the cause yet; I used GPU version).

+ crossmul/resampleSlc: re-organize module import at the top of scripts
2020-05-02 10:26:29 -07:00
Heresh Fattahi 9e7e090548
Merge pull request #113 from mirzaees/stack_runfiles
Stack runfiles
2020-05-02 10:13:25 -07:00
Sara Mirzaee a24f0e99bf Merge branch 'stack_runfiles' of https://github.com/mirzaees/isce2 into stack_runfiles 2020-05-01 14:53:51 -05:00
Sara Mirzaee 08e3849945 looping over lines instead of reading the whole file 2020-05-01 14:45:48 -05:00
Sara Mirzaee fe097c906a add --rmFilter to stackSentinel.py 2020-05-01 14:44:59 -05:00
Ryan Burns 23f2ffaa1e
Use exist_ok argument to makedirs when appropriate (#112)
@falkamelung noted a race condition in #106 due to non-atomic directory
checking followed by os.makedirs. This pattern exists in many places
in the codebase so I went ahead and replaced the rest of them.
The new exist_ok usage should be terser and more idiomatic.
2020-04-13 12:40:32 -07:00
Sara Mirzaee e1729de5a8
fix bug: some lines were commented by mistake 2020-04-11 11:50:10 -04:00
Lijun Zhu 62a10d4c0d topsStack/cuDenseOffsets: call (included not standalone) PyCuAmpcor properly 2020-04-09 21:01:00 -07:00
Lijun Zhu 0855bf2640 DenseOffsets with PyCuAmpcor: use '.slc.vrt' instead of '.slc' as inputs 2020-04-09 21:01:00 -07:00
Forrest Williams 16b7e10d32
Fix Burst Loading in Estimate Range and Azimuth Misreg (#110)
* Fix burst loading in estimateAzimuthMisreg.py

Within the for loop that starts at line 82, looping through range(minBurst, maxBurst) misses the burst with the maximum burst number. In cases where only one burst in a swath is used, the script fails at line 90 when .getWidth() is called on an empty xml. This issue can be fixed by adding the line maxBurst = maxBurst + 1

* Fix burst loading in estimateRangeMisreg.py

Similar to estimateAzimuthMisreg.py, calling range(minBurst,maxBurst) at line 185 will lead to an empty list and the some bursts not being loaded unless line 181 is changed from maxBurst = maxBurst - 1 to maxBurst = maxBurst +1
2020-04-09 20:57:44 -07:00
Sara Mirzaee dc97d160a4 add fine.unw to remove the effect of filtering 2020-04-09 01:14:26 -05:00
Sara Mirzaee 97b7800fda make a separate run_file for some functions to better allocate time and memory 2020-04-09 01:13:36 -05:00
Falk Amelung 7fb9142f1d
Falkamelung/mergeburst race cond (#106)
* race condition fix: avoids raising of exception when multiple jobs create dir at same time
2020-03-25 19:14:37 -07:00
Zhang Yunjun 17b22417fd topo: add more arguments for runMultilookGdal()
+ add in_ext/out_ext/fbase_list for more flexibility

+ fix bug for .full.xml file copying when input/output are in the same directory
2020-03-23 17:09:04 -07:00
Zhang Yunjun b0cecb64cb createWaterMask: add --fill option
+ add --fill option to be able to change the fill value for the missing water body data, because sometimes it's land sometimes it's water.
2020-03-23 17:07:48 -07:00
Zhang Yunjun 7644bf8c93 MaskAndFilter: more ploting options
+ plot the evolution of offset after each step to better check the changes by masking, filling and filtering operations.

+ plot both azimuth and range offset in one figure and move the plotting to the end after file writing

+ update file path in xml file so that script can work even if the files are moved.

+ convert SNR nan value to zero before used as the mask, to avoid warning message

+ add the following options to customize the plot
   - add -v and --v-snr option to change the display range for offset and SNR
   - add --figsize and --save option

+ suppress the DEBUG message from matplotlib

+ remove obsolete getShape() and resampleOffset()

+ update IML module import

+ adjust indentation
2020-03-23 17:04:08 -07:00
piyushrpt 3f82b2cfe9
Merge pull request #99 from mirzaees/run_master
Run master
2020-03-21 17:11:19 -07:00
Zhang Yunjun 82b46df741 dem.py: use abspath filename in xml after correction
+ dem.py: replace filename in xml file with abspath after geoid to ellipsoid correction

+ update README.md files in contrib/stack/*Stack, as fixImageXml.py is not needed anymore

+ fixImageXml: use native argparse mutually exclusive group for --full/--base options to replace checking with own code
2020-03-19 14:11:16 -07:00
Sara Mirzaee 1329a0b961 parallel processing bursts 2020-03-14 13:55:13 -04:00
Sara Mirzaee 84946b6624 Separate master topo job from run_1_unpack 2020-03-14 12:00:25 -04:00
Heresh Fattahi 4998f720d7
Merge pull request #39 from EJFielding/UAVSAR
added back command line option in prepareUAVSAR
2020-03-13 08:27:52 -07:00
Yujie Zheng e95311da68 Added computation of non-filtered correlation 2020-02-28 13:49:13 -08:00
Ryan Burns 35ca7db85e Fix trackCRs script
* Strip leading zero from LLH list
* Cast slice bounds to ints
* Use comma delimiter for reading CSVs
2020-01-22 10:07:18 -08:00
Eric J. Fielding db51787372 Merge branch 'master' of https://github.com/isce-framework/isce2 into UAVSAR 2020-01-07 14:51:56 -08:00
Zhang Yunjun 62871d9806 topsStack: convert README.txt to README.md
+ contrib/stack/topsStack:
   - convert README.txt to README.md
   - add AUX_CAL file download instruction
   - add note for DEM preparation from stripmapStack/README.md

+ contrib/stack:
   - merge References.txt and INSTALL.txt to README.md
   - remove the duplicated README_stripmap/topsStack.txt files.
2020-01-05 19:02:29 -08:00