Sharif Digital Repository

Communication overhead is one of the major performance bottlenecks in large-scale distributed computing systems, in particular for machine learning applications. Conventionally, compression techniques are used to reduce the load of communication by combining intermediate results of the same computation task as much as possible. Recently, via the development of coded distributed computing (CDC), it has been shown that it is possible to enable coding opportunities across intermediate results of different computation tasks to further reduce the communication load. We propose a new scheme, named compressed coded distributed computing (in short, compressed CDC), which jointly exploits the above... 

MapReduce and its open source implementation, Hadoop, are the prevailing platforms for big data processing. MapReduce is a simple programming model for performing large computational problems in large-scale distributed systems. This model consists of two major phases: Map and Reduce. Between these two main phases, partitioner part is embedded which distributes produced keys by Map tasks among Reduce tasks. When the amount of keys and their associated values, which are called intermediate data, is huge, this part has significant impact on execution time of Reduce tasks, and consequently, completion time of jobs. In this paper, we present a network and resource aware key partitioner to... 

A widely dispersed asset, electric grid has enormous impacts on people’s lives. Recently occurred various high-impact low-probability (HILP) threats have raised the concerns about the classical reliability-oriented view. Therefore, withstanding unexpected and less frequent severe situations still remain a significant challenge. As a critical infrastructure (CI), power systems are more and more expected to be resilient to HILPs. So, utilities put their endeavours to harden enough their networks against HILPs. In this regard, there is a great need to a methodology able to optimise the hardening programme investments. This scheme could potentially save a large amount of money, as well as... 

We consider the problem of massive matrix multiplication, which underlies many data analytic applications, in a large-scale distributed system comprising a group of worker nodes. We target the stragglers' delay performance bottleneck, which is due to the unpredictable latency in waiting for slowest nodes (or stragglers) to finish their tasks. We propose a novel coding strategy, named entangled polynomial code, for designing the intermediate computations at the worker nodes in order to minimize the recovery threshold (i.e., the number of workers that we need to wait for in order to compute the final output). We demonstrate the optimality of entangled polynomial code in several cases, and show... 

In deregulated power systems, distribution network operators (DNO) are responsible for maintaining the proper operation and efficiency of distribution networks. This is achieved traditionally through specific investments in network components. The event of distributed generation (DG) has introduced new challenges to these distribution networks. The role of DG units must be correctly assessed to optimise the overall operating and investment cost for the whole system. However, the distributed generation owners (DGOs) have different objective functions which might be contrary to the objectives of DNO. This study presents a long-term dynamic multi-objective model for planning of distribution... 

This paper proposes a hybrid heuristic optimization method based on genetic algorithm and immune systems to maximize the benefits of Distribution Network Operators (DNOs) accrued due to sizing and placement of Distributed Generation (DG) units in distribution networks. The effects of DG units in reducing the reinforcement costs and active power losses of distribution network have been investigated. In the presented method, the integration of DG units in distribution network is done considering both technical and economical aspects. The strength of the proposed method is evaluated by applying it on a small and a realistic large scale distribution network and the results are compared with... 

A widely dispersed asset, electric grid has enormous impacts on people’s lives. Recently occurred various high-impact low-probability (HILP) threats have raised the concerns about the classical reliability-oriented view. Therefore, withstanding unexpected and less frequent severe situations still remain a significant challenge. As a critical infrastructure (CI), power systems are more and more expected to be resilient to HILPs. So, utilities put their endeavours to harden enough their networks against HILPs. In this regard, there is a great need to a methodology able to optimise the hardening programme investments. This scheme could potentially save a large amount of money, as well as... 

Motivated by applications of sensor networks, there has been growing interest in monitoring large scale distributed systems. In these applications, we usually wish to monitor a global system condition defined as a function of local network elements parameters. In this paper, we study Reactive Monitoring in sensor networks, which has the benefit of operating in a decentralized manner. Our primary concern in adopting such a monitoring paradigm is reducing the communication cost which is the dominant factor of energy drain in wireless sensor networks. In this study, we address the reactive aggregate monitoring problem by casting the underlying threshold assignment as an optimization problem....