Optimizing mapping workflows by combining HTCondor (formerly Condor Architecture) with Intergraph 2013 suite (such as GeoMedia 2013, Erdas Imagine, or Smart 3D) relies on leveraging High-Throughput Computing (HTC). Large-scale geospatial and cartographic datasets require massive amounts of CPU cycles to handle orthorectification, spatial queries, and map tiling. Utilizing Condor allows you to distribute these heavy processing pipelines across a network of idle workstations or server clusters.
The optimization of this specific enterprise mapping stack centers on several core strategies:
1. Structure the Map Pipeline into Directed Acyclic Graphs (DAGs)
Condor utilizes an extension tool called DAGMan (Directed Acyclic Graph Manager) to oversee complex dependencies.
Abstract to Concrete Mapping: Divide the mapping process into smaller, independent sub-tasks (e.g., tile-by-tile raster processing, coordinate transformations, and final image stitching).
Define Relationships: Map out these steps sequentially or concurrently in a .dag file. For instance, Task A (Data Ingestion/Extraction) must complete before Task B (Spatial Join/Buffering) can run.
Throttling: Use DAGMan to limit the number of simultaneous job submissions to avoid overloading your Intergraph 2013 license server or database. 2. Implement Job Clustering for Short Map Operations
Mapping workflows often involve thousands of micro-tasks, such as updating small vector attributes or generating individual map tiles.
Avoid Scheduling Overhead: Sending thousands of 5-second jobs to Condor wastes significant scheduling time.
Horizontal Clustering: Group multiple minor mapping jobs occurring at the same workflow level into a single executable package. This maximizes CPU efficiency while reducing the time data spends waiting in queues. 3. Smart Data Management & Minimizing Locality Drag
Geospatial data files (like shapefiles, GeoTIFFs, or Intergraph PCF files) are traditionally massive. Moving them back and forth over a network ruins performance. 12. Optimizing Workflows for Efficiency and Scalability
Leave a Reply