import numpy as np import os from glob import glob from pathlib import Path from multiprocessing import Pool import hashlib # md5 import time import shutil import datetime ######################################################## # 函数区 ######################################################## spacetySliceEnvPathExEPath=r"d:\ProgramData\anaconda3\envs\spacetySliceEnv\python.exe" def find_tif_files_pathlib(directory): """ 使用pathlib.Path.rglob递归查找指定目录下所有.tif和.tiff文件 参数: directory (str): 要搜索的根目录路径 返回: list: 包含所有找到的.tif/.tiff文件完整路径的列表 """ path = Path(directory) # 使用rglob递归匹配所有.tif和.tiff文件 tif_files = list(path.rglob('*.tif')) + list(path.rglob('*.tiff')) seemname=set() tiffpaths=[] for tiffpath in tif_files: tiffpath =str(tiffpath) rootname = Path(tiffpath).stem if rootname not in seemname: seemname.add(rootname) tiffpaths.append(tiffpath) # 将Path对象转换为字符串路径 return [str(file) for file in tiffpaths] def find_shp_files_pathlib(directory): """ 使用pathlib.Path.rglob递归查找指定目录下所有.tif和.tiff文件 参数: directory (str): 要搜索的根目录路径 返回: list: 包含所有找到的.tif/.tiff文件完整路径的列表 """ path = Path(directory) # 使用rglob递归匹配所有.tif和.tiff文件 tif_files = list(path.rglob('*.shp')) # 将Path对象转换为字符串路径 return [str(file) for file in tif_files] def preProcessTiFF(tiffpath,preFolderPath): file_path = Path(tiffpath) directory_path = str(file_path.resolve().parent.name) rootname=Path(tiffpath).stem outpngpath=os.path.join(preFolderPath,"I"+directory_path+"_"+rootname+'.png') programpath=r"R:\TYSAR-德清院\A-预处理-未标注\A0-算法版本\AA\SpacetySliceDataTools\tools\SpacetyTIFFDataStretch2PNG_AB.py" # cmdtxt=r"{} {} -i {} -o {} --filemode --SquareRoot".format(spacetySliceEnvPathExEPath,programpath,tiffpath,outpngpath) cmdtxt=r"{} {} -i {} -o {} --slicemode --SquareRoot".format(spacetySliceEnvPathExEPath,programpath,tiffpath,preFolderPath) # 直接切片 if os.system(cmdtxt) ==2: print("sucess:",cmdtxt) return "sucess: {}".format(cmdtxt) else: print("failed:",cmdtxt) return "failed: {}".format(cmdtxt) pass def moveDir(srcfolderPath,targetfolderPath): try: shutil.copytree(srcfolderPath,targetfolderPath,dirs_exist_ok=True) print("sucess: copy ", srcfolderPath,targetfolderPath ) shutil.rmtree(srcfolderPath) print("sucess: rmtree ", srcfolderPath) return "sucess: copy and rmtree from {} to {}".format(srcfolderPath,targetfolderPath),True except Exception as e: print("failed: copy ", srcfolderPath, targetfolderPath) return "failed: copy and rmtree from {} to {}".format(srcfolderPath,targetfolderPath),False def writeoutlog(logPath,cmdtxt): with open(logPath,'a',encoding="utf-8") as f: f.write(cmdtxt) f.write("\n") def processPool(srcFolderPath,preFolderPath,targetfolderPath,logPath=r"R:\TYSAR-德清院\A-预处理-未标注\A0-算法版本\AA\SpacetySliceDataTools\log\process.log"): writeoutlog(logPath, "====================================================================================\n") writeoutlog(logPath, "time: {}".format(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))) writeoutlog(logPath, "srcFolderPath: {}".format(srcFolderPath)) writeoutlog(logPath, "preFolderPath: {}".format(preFolderPath)) writeoutlog(logPath, "process start \n") # 1. 预处理流程 async_results = [] with Pool(processes=6) as pool: movepool=[] # 20250813-不分类 条带模式 # srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250813-不分类\0-原图" tiffPaths = find_tif_files_pathlib(srcFolderPath) # preFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250813-不分类\A-预处理\AB-图像预处理" for tiffpath in tiffPaths: async_results.append( pool.apply_async(preProcessTiFF, (tiffpath,preFolderPath,)) ) for pidx in range(len(async_results)): async_result=async_results[pidx] print("{}/{}".format(pidx+1,len(async_results)),async_result.get()) writeoutlog(logPath,"{}/{} {}".format(pidx+1,len(async_results),async_result.get())) writeoutlog(logPath, "\nprocess finished \n") # moverTip,moveflag=moveDir(preFolderPath, targetfolderPath) # writeoutlog(logPath, moverTip) writeoutlog(logPath, "====================================================================================\n") # if moveflag: # pass # else: # exit(3) ######################################################## # 流程执行区 ######################################################## if __name__ == '__main__': logPath = r"R:\TYSAR-德清院\A-预处理-未标注\A0-算法版本\AA\SpacetySliceDataTools\log\process.log" # 20250813-不分类 条带模式 # srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250813-不分类\0-原图" # preFolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250813-不分类\A-预处理\AB-图像预处理" # targetfolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250813-不分类\A-预处理\AB-图像预处理" # processPool(srcFolderPath, preFolderPath, targetfolderPath,logPath) # # # 20250818 条带模式 # srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250818-不分类\0-原图" # preFolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250818-不分类\A-预处理\AB-图像预处理" # targetfolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250818-不分类\A-预处理\AB-图像预处理" # processPool(srcFolderPath, preFolderPath,targetfolderPath, logPath) # # 20250826-不分类 条带模式 srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\航道\20250930-不分类\0-原图" preFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\航道\20250930-不分类\A-预处理\AB-图像预处理" targetfolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250826-不分类\A-预处理\AB-图像预处理" processPool(srcFolderPath, preFolderPath, targetfolderPath,logPath) # # 20250903-不分类 条带模式 # srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250903-不分类\0-原图" # preFolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250903-不分类\A-预处理\AB-图像预处理" # targetfolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250903-不分类\A-预处理\AB-图像预处理" # processPool(srcFolderPath, preFolderPath,targetfolderPath, logPath) # # 20250910-不分类 条带模式 # srcFolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250910-不分类\0-原图" # preFolderPath = r"D:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250910-不分类\A-预处理\AB-图像预处理" # targetfolderPath = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\港口\20250910-不分类\A-预处理\AB-图像预处理" # processPool(srcFolderPath, preFolderPath, targetfolderPath,logPath) result_flag_txt = r"R:\TYSAR-德清院\TYSAR-条带模式(SM)\航道\20250930-不分类\strechResult_finish.log" with open(result_flag_txt,"w",encoding="utf-8") as f: f.write("strech finied!!!\n") writeoutlog(logPath, "strech finied!!!\n")