Designing a Hybrid Cloud-Grid Architecture for High-Performance Computing: A Unified Model for Resource Sharing, Task Distribution, and Scalability in Heterogeneous Environments

Abstract

The modern and rapidly growing number of computational problems in the scientific and industrial applications requires not only powerful systems but also adaptive, scalable and fault tolerant high performance computing architectures. This paper presents a new HCG architecture that integrates the merits of the conventional grid computing and the novel cloud services to overcome critical issues including, resource provisioning, scheduling and allocation, failure tolerant systems, and cost minimization in distributed environments. It features an intelligent middle tier to manage resources, cloud resource shedding mechanisms for scaling, and AI-based scheduling for workload distribution. Several simulation-based performance analysis results highlight the proposed hybrid model’s effectiveness in cutting down general task solving time up to 35.7%, improving the usage of resources by up to 81.3% on an average, bringing down recovery time and operational costs by approximately 37.7% than existing standalone systems. Moreover, it provides better energy efficiency and resource reliability based on a given workload level. The results obtained here indicate that the hybrid cloud-grid framework may be considered as a viable model for future HPC solutions that are scalable, reliable, and economically feasible for tackling the requirements posed by data-intensive or near-real time applications. Future works involve deploying the model in a real environment, incorporating the model in a multi-cloud and edge setting, and incorporating more sophisticated auto management techniques that would augment the performance and flexibility of the model.