Loading...
Search for: markov-decision-processes
0.012 seconds
Total 23 records

    Optimal relaying in a slotted aloha wireless network with energy harvesting nodes

    , Article IEEE Journal on Selected Areas in Communications ; Volume 33, Issue 8 , 2015 , Pages 1680-1692 ; 07338716 (ISSN) Moradian, M ; Ashtiani, F ; Sharif University of Technology
    Institute of Electrical and Electronics Engineers Inc  2015
    Abstract
    In this paper, we derive optimal policies for cooperation of a wireless node in relaying the packets of a source node, in a random access environment. In our scenario, the source node sends its packets by its harvested energy and the relay node exploits its harvested energy in relaying the source packets not detected successfully at the destination. The relaying policies determine whether the relay node accepts or rejects the unsuccessfully transmitted source packets and how the relay node prioritizes the accepted source packets to its own packets. The optimization goal is to minimize the average transmission delay of source packets with and without a constraint on the average transmission... 

    Minimizing expected discounted cost in a queueing loss model with discriminating arrivals

    , Article European Journal of Operational Research ; Volume 282, Issue 2 , 2020 , Pages 593-601 Haji, B ; Ross, S ; Sharif University of Technology
    Elsevier B.V  2020
    Abstract
    We consider a queuing loss system with heterogeneous skill based servers and Poisson arrivals. We first assume that each arrival has a vector (X1,…,Xn) of independent binary random variables with Xi=1 if server i is eligible to serve that arrival. The service time at server i is exponential with rate μi. Arrivals finding no servers that are both idle and eligible to serve them are lost. Assuming the system incurs a cost of one unit for each lost customer, our goal is to find the optimal policy for assigning arrivals to idle and eligible servers so as to minimize the expected discounted cost of the system. Later, we generalize our model by considering k server pools where each pool i is... 

    Using Partially-Observable Markov Decision Process for Dialogue Management in Spoken Dialogue Systems

    , M.Sc. Thesis Sharif University of Technology Rahbar Noudehi, Siavash (Author) ; Sameti, Hossein (Supervisor)
    Abstract
    The use of Spoken Dialogue Systems is growing everyday and these systems will substitute current Iterative Voice Response systems in near future. A Spoken Dialogue System consists of Speech Recognition, Language Understanding, Dialogue Management, Speech Generation and Text to Speech Modules. Among these modules the only one that is specific part of Dialogue Systems is Dialogue Management. The responsibility of this part is to determine system behavior to maximize specific variables such as user goal finding accuracy and speed of finding the goal. There were different approaches to dialogue management in recent years the use of Partially-Observable Markov Decision Processes was very popular... 

    Dynamic pricing in a production system with multiple demand classes

    , Article Applied Mathematical Modelling ; Volume 39, Issue 8 , April , 2015 , Pages 2332-2344 ; 0307904X (ISSN) Ahmadi, M ; Shavandi, H ; Sharif University of Technology
    Elsevier Inc  2015
    Abstract
    This paper considers dynamic pricing in a production system with a single product which is demanded by several customer classes. We seek the structure of the optimal policy assuming m available prices and n demand classes that differ based on the lost sales cost they impose on the system. The assumption of different available prices leads to dynamic pricing structure and the assumption of several demand classes leads to rationing which is proposed in the literature of revenue management. We found that an optimal policy structure exists for this combined problem. The optimal policy has a threshold form which lower thresholds are related to the rationing decision and upper thresholds are... 

    Divided POMDP method for complex menu problems in spoken dialogue systems

    , Article 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings, 12 December 2010 through 15 December 2010 ; 2010 , Pages 484-489 ; 9781424479030 (ISBN) Habibi, M ; Rahbar, S ; Sameti, H ; The Institute of Electrical and Electronics Engineers (IEEE); IEEE Signal Processing Society ; Sharif University of Technology
    2010
    Abstract
    In this paper, a problem in spoken dialogue systems namely the menu problem, is introduced and solved by a POMDP model. To overcome the large size of the menu problem, a new method for achieving an optimal policy called divided POMDP method is introduced. Conditions for the problem to be solved by the proposed method are specified and the problem properties resulting in the given conditions are presented. The proposed method is evaluated using a typical menu problem with different menu sizes and it is shown that this method is superior to the conventional methods such as FRTDP for the problems it is capable to solve. Moreover, it converges faster in getting to an optimal policy  

    QoS-aware joint policies in cognitive radio networks

    , Article IWCMC 2011 - 7th International Wireless Communications and Mobile Computing Conference, 4 July 2011 through 8 July 2011 ; July , 2011 , Pages 2220-2225 ; 9781424495399 (ISBN) Salehkaleybar, S ; Majd, S. A ; Pakravan, M. R ; Sharif University of Technology
    2011
    Abstract
    One of the most challenging problems in Opportunistic Spectrum Access (OSA) is to design channel sensing-based protocol in multi secondary users (SUs) network. Quality of Service (QoS) requirements for SUs have significant implications on this protocol design. In this paper, we propose a new method to find joint policies for SUs which not only tries to guarantee QoS requirements but also maximize network throughput. We use Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to formulate interactions between SUs. Meanwhile, a tractable approach for Dec-POMDP is utilized to extract sub-optimum joint policies for large horizons. Among these policies, the QoS-aware joint... 

    Minimizing uplink delay in delay-sensitive 5G CRAN platforms

    , Article 2nd IEEE 5G World Forum, 5GWF 2019, 30 September 2019 through 2 October 2019 ; 2019 , Pages 154-160 ; 9781728136271 (ISBN) Ataie, A ; Kanaanian, B ; Khalaj, B. H ; Sharif University of Technology
    Institute of Electrical and Electronics Engineers Inc  2019
    Abstract
    In this paper, we consider the problem of minimizing the uplink delays of users in a 5G cellular network. Such cellular network is based on a Cloud Radio Access Network (CRAN) architecture with limited fronthaul capacity, where our goal is to minimize delays of all users through an optimal resource allocation. Earlier works minimize average delay of each user assuming same transmit power for all users. Combining Pareto optimization and Markov Decision Process (MDP), we show that every desired balance in the trade-off among infinite-horizon average-reward delays, is achievable by minimizing a properly weighted sum delays. In addition, we solve the problem in two realistic scenarios;... 

    Appropriate MAC Sub-layer Algorithms for Body Area Networks and Related Analysis

    , M.Sc. Thesis Sharif University of Technology Omidvar, Hamed (Author) ; Nasiri-Kenari, Masoumeh (Supervisor) ; Vosougi Vahdat, Bijan (Co-Advisor)
    Abstract
    WBAN , as one of the most important wireless networks in the near future, has received significant attentions in recent years. The most important designing challenge in these networks is the scarcity of energy resources especially in their sensor nodes. Additionally, special characteristics of these networks have led the other network’s designs, especially the general sensor network protocols to be inefficient for the WBAN. Among others, the existences of recurring patterns in each sensor channel fading and the high correlation among different sensor channels are some of the most important characteristics of WBANs. Considering these characteristics, in this thesis, the MAC sub-layer of... 

    Opportunistic RF Energy Harvesting in Cognitive Radio Networks

    , M.Sc. Thesis Sharif University of Technology Miri, Zoheir (Author) ; Nasiri-kenari, Masoumeh (Supervisor) ; Ashtiani, Farid (Supervisor)
    Abstract
    Considering energy efficiency ways a key role in designing future wireless networks( 5G mobile networks). Moreover Spectrum efficiency is another critical issues in designing wireless networks. Cognitive radios can improve the spectrum efficiency. On the other hand, radio frequency (RF) energy harvesting has emerged as a promising technique to supply energy for wireless networks and thereby increase their energy efficiency. In this thesis, we propose a new technique for the RF-powered CRNs.To this end,We consider a cognitive radio network comprised of a primary user and a secondary user. The primary user, uses a typical frequency band for transmit data in a time slot basis and both the... 

    Using Probabilistic Models and Data Mining Techniques in Online Computer Games

    , M.Sc. Thesis Sharif University of Technology Khosravinia, Sina (Author) ; Haji, Babak (Supervisor)
    Abstract
    Computer video games have had a huge growth in recent years, and Dota 2 is one of the most popular video games and e-sports right now. In this research, Dota 2 drafting phase is studied by using stochastic processes and probabilistic models. This phase is modeled as a POMDP (Partially Observable Markov Decision Process) for the first time. Afterwards, the model is solved using data mining methods and machine learning algorithms. A large database of the game's match data is created, and six different machine learning algorithms are used to predict the match outcome based on the drafting stage. The prediction accuracy and power of these algorithms is compared, and finally, a program is... 

    Markov Decision Process with Timeconsuming Transition

    , M.Sc. Thesis Sharif University of Technology Qarehdaghi, Hassan (Author) ; Alishahi, Kasra (Supervisor)
    Abstract
    Mankind according to his authority (or delusion of authority) always finds himself in a situation which need decision-¬making. Usually, he seeks to make the best possible decision. The basis for measuring the goodness of choices is different in different occasions. This measure could be level of enjoyment, economic profit, probability of reaching a goal, etc. These decisions have consequences such that the situations before and after the decisions are not the same. Most challenging decision¬-making situations are those which the decision¬maker has not the complete authority over the situation and the results of decisions are influenced by out of control factors. A significant part of... 

    Bayesian approach to updating markov-based models for predicting pavement performance

    , Article Transportation Research Record ; Issue 2366 , 2013 , Pages 34-42 ; 03611981 (ISSN) Tabatabaee, N ; Ziyadi, M ; Sharif University of Technology
    2013
    Abstract
    The Markov decision process is one of the most common probabilistic prediction models used in infrastructure management. When existing data are insufficient, expert knowledge is commonly used to derive a Markovian transition probability matrix. Eventually, every pavement management system will progress to a level at which inspection measurements from the network will be organized into a database to be used for performance prediction. The best way to use this body of data to improve the initially developed transition probability matrix is to combine prior expert knowledge with new observations. This paper proposes a method for periodically updating Markovian transition probabilities as new... 

    On tradeoff between collision and cooperation in a random access wireless network with energy harvesting nodes

    , Article IEEE Transactions on Vehicular Technology ; Volume 67, Issue 3 , 2018 , Pages 2501-2513 ; 00189545 (ISSN) Moradian, M ; Ashtiani, F ; Sharif University of Technology
    Institute of Electrical and Electronics Engineers Inc  2018
    Abstract
    In this paper, we investigate an energy harvesting cooperative network in which the cooperation is done in a random access environment. Although the cooperation provides an extra harvested energy supply for transmissions of the source, it causes probable collisions for the transmitted source packets. Thus, this kind of cooperation can improve or degrade the QoS of the source packets. We find the optimal policy in such a scenario to maximize the source throughput and derive the necessary and sufficient condition for no-cooperation policy to be throughput-optimal. Then, we prove that the maximum throughput is obtained by considering no-cooperation or full-cooperation policy depending on the... 

    Faster Algorithms for Quantitative Analysis of MCs and MDPs with Small Treewidth

    , Article 18th International Symposium on Automated Technology for Verification and Analysis, ATVA 2020, 19 October 2020 through 23 October 2020 ; Volume 12302 LNCS , 2020 , Pages 253-270 Asadi, A ; Chatterjee, K ; Kafshdar Goharshady, A ; Mohammadi, K ; Pavlogiannis, A ; Sharif University of Technology
    Springer Science and Business Media Deutschland GmbH  2020
    Abstract
    Discrete-time Markov Chains (MCs) and Markov Decision Processes (MDPs) are two standard formalisms in system analysis. Their main associated quantitative objectives are hitting probabilities, discounted sum, and mean payoff. Although there are many techniques for computing these objectives in general MCs/MDPs, they have not been thoroughly studied in terms of parameterized algorithms, particularly when treewidth is used as the parameter. This is in sharp contrast to qualitative objectives for MCs, MDPs and graph games, for which treewidth-based algorithms yield significant complexity improvements. In this work, we show that treewidth can also be used to obtain faster algorithms for the... 

    Power allocation of sensor transmission for remote estimation over an unknown gilbert-elliott channel

    , Article 18th European Control Conference, ECC 2020, 12 May 2020 through 15 May 2020 ; 2020 , Pages 1461-1467 Farjam, T ; Fardno, F ; Charalambous, T ; Sharif University of Technology
    Institute of Electrical and Electronics Engineers Inc  2020
    Abstract
    In this paper, we consider the problem of scheduling the power of a sensor when transmitting over an unknown Gilbert-Elliott (GE) channel for remote state estimation. The sensor supports two power modes, namely low power and high power, which are to be selected for transmission over the channel in order to minimize a cost on the error covariance, while satisfying the energy constraints. The remote estimator provides error-free acknowledgement/negative-acknowledgement (ACK/NACK) messages to the sensor only when low power is utilized. We first consider the Partially Observable Markov Decision Process (POMDP) problem for the case of known GE channels and derive conditions for optimality of a... 

    On Coordination of Smart Grid and Cooperative Cloud Providers

    , Article IEEE Systems Journal ; Volume 15, Issue 1 , 2021 , Pages 672-683 ; 19328184 (ISSN) Mohebbi Moghaddam, M ; Manshaei, M. H ; Naderi Soorki, M ; Saad, W ; Goudarzi, M ; Niyato, D ; Sharif University of Technology
    Institute of Electrical and Electronics Engineers Inc  2021
    Abstract
    Cooperative cloud providers in the form of cloud federations can potentially reduce their energy costs by exploiting electricity price fluctuations across different locations. In this environment, on the one hand, the electricity price has a significant influence on the federations formed, and, thus, on the profit earned by the cloud providers, and on the other hand, the cloud cooperation has an inevitable impact on the performance of the smart grid. In this regard, the interaction between independent cloud providers and the smart grid is modeled as a two-stage Stackelberg game interleaved with a coalitional game in this article. In this game, in the first stage the smart grid, as a leader... 

    Optimal control of parallel queues with impatient customers

    , Article Performance Evaluation ; Volume 60, Issue 1-4 , 2005 , Pages 327-343 ; 01665316 (ISSN) Movaghar, A ; Sharif University of Technology
    2005
    Abstract
    We consider a queueing system with a number of identical exponential servers. Each server has its own queue with unlimited capacity. The service discipline in each queue is first-come-first-served (FCFS). Customers arrive according to a state-dependent Poisson process with an arrival rate which is a non-increasing function of the number of customers in the system. Upon arrival, a customer must join a server's queue according to a stationary state-dependent policy, where the state is taken to be the number of customers in servers' queues. No jockeying among queues is allowed. Each arriving customer is limited to a generally distributed patience time after which it must depart the system and... 

    Energy Efficient Policies for TDMA based Wireless Body Area Network (WBAN)

    , M.Sc. Thesis Sharif University of Technology Karimzadeh Farshbafan, Mohammad (Author) ; Nasiri-Kenari, Masoumeh (Supervisor) ; Ashtiani, Farid (Supervisor)
    Abstract
    Recent advancements in wireless communication along with the urgent need for new care systems for the patients, has attracted the attention of many communication researchers to Wireless Body Area Networks (WBANs), which has led to IEEE 802.15.6 and IEEE 802.15.4j standards, accordingly. The main goal of such networks, is to send the vital data of the associate, through the over/in-body-implanted sensors, to one central node. One of the key features of these networks is the limited energy sources of the sensors, transmit the data. Meanwhile, on-time error-free transmission of the data is also of great importance. Therefore, designing a proper algorithm, satisfying the aforementioned... 

    Change Point Detection in Molecular Carrier Based Nano Networks

    , M.Sc. Thesis Sharif University of Technology Ghoroghchian, Nafiseh (Author) ; Nasiri Kenari, Masoumeh (Supervisor) ; Aminzadeh Gohari, Amin (Co-Advisor)
    Abstract
    Molecular communication (MC) is an emerging communication paradigm, whereas molecules are used as information carriers to establish communication among elements in nano-meter to meter scales. In this thesis, we investigate the problem of detecting and monitoring changes (abnormality) based on molecular communication, using quickest change point detection scheme. We assume the distributions and parameters of the system are known. To this end, we consider a network of multiple sensors, each sensing its surrounding and employing On-Off-keying modulation for data transmission toward a fusion center (FC). An abnormality initiates randomly in time and location, and further propagates in the... 

    Optimal rate and delay performance in non-cooperative opportunistic spectrum access

    , Article Proceedings of the International Symposium on Wireless Communication Systems, 28 August 2012 through 31 August 2012 ; August , 2012 , Pages 56-60 ; 21540217 (ISSN) ; 9781467307604 (ISBN) Perez, J ; Khodaian, M ; Sharif University of Technology
    2012
    Abstract
    We study transmission rate control and performance delay in cognitive radio (CR) links from a cross-layer perspective. We assume a hierarchical CR network where the secondary users (SU) access the spectrum band in an opportunistic and noncooperative way. The SU goal is to transmit a fixed-size file (fixed amount of data packets) during the sojourn time of the primary users (PU's) idle state. We assume that the SU's support frames retransmission through an automatic repeat request (ARQ) mechanism. By formulating the problem as a Markov decision process, we demonstrate that there is always an optimal stationary rate adaptation policy, and we propose a simple algorithm to obtain it. We derive...