Sharif Digital Repository

Conjugate gradient methods are appealing for large scale nonlinear optimization problems, because they avoid the storage of matrices. Recently, a new conjugacy condition proposed by Dai and Liao, considers an inexact line search scheme that reduces to the old one if the line search is exact. Based on this condition, a new conjugate gradient method was proposed that has fast convergence. Later, Yabe and Takano, based on new conjugacy condition and modified secant condition, proposed another conjugate gradient method. This method takes both the available gradient and function value information and achieves a high-order accuracy in approximating the second-order curvature of the objective... 

We discuss conjugate gradient methods for which both the gradient and func-tion values are considered in computing the conjugate gradient parameter. We pro-pose new conjugate gradient methods as members of Dai-Liao’s family of conjugate gradient methods and Andrei’s family of hybrid conjugate gradient methods. For computing the conjugate gradient parameter in our methods, three modified secant equations proposed by Zhang, Deng and Chen, Li and Fukushima, and Yuan are used. It is shown that under proper conditions, three of the proposed methods are globally convergent for uniformly convex functions and two other methods are glob-ally convergent for general functions. It is also shown that... 

Considering the special importance of network flow problems in human life, as well as the complexity of solving these problems in very large scales, there are numerous methods to solve them and the interior point methods are the most important approaches among them. In a number of methods, a preconditioned conjugate gradient solver has been applied for the solution of the Karush-Kuhn-Tucker (KKT) system, in each interior point iteration; therefore, the selection of an appropriate preconditioner is a special issue. In spite of presenting different preconditioners in recent years, discussion and implementation of a particular class of triangulated graph-based preconditioners is our main... 

Conjugate gradient methods are useful for large scale nonlinear optimization problem, because they avoid the storage of any matrices. In this thesis, we have investigated an accelerated hybrid conjugate gradient algorithm, recently proposed in the literature. The combining parameter is calculated so that the corresponding direction to the conjugate gradient algorithm, while satisfies the modified secant condition, is a Newton direction. It is shown that for uniformly convex functions and for general nonlinear functions the algorithm with strong Wolfe line search is globally convergent. The algorithm uses an accelerated approach for the reduction of the objective function values by modifying... 

Design, analysis and practical implementation of the filter trust-region algorithms are investigated. First, we introduce two filter trust-region algorithms for solving the unconstrained optimization problem. These algorithms belong to two different class of optimization algorithms: (1) The monotone class, and (2) The non-monotone class. We prove the global convergence of the sequence of the iterates generated by the new algorithms to the first and second order critical points. Then, we propose a filter trust-region algorithm for solving bound constrained optimization problems and show that the algorithm converges to a first order critical point. Moreover, we address some well known... 

An interior point method for nonlinear programming problem recently proposed by Waltz, Morales, Nocedal and Orban is described and implemented [2].The steps are computed by line search based on the primal-dual equations, or trust region based on the conjugate gradient iteration. Steps computed by line search are tried first, but if they are determinded to be ineffective, a trust region iteration that guarantees progress toward a stationary point is used. In order to reduce the calculations, here we propose some modifications. The algorithms are implemented and the programs are tested on a variety of problems. Numerical results based on Dolan-More’ confirm the effectiveness of the algorithms  

The main goal of this work is solving system of linear equations Ax = b, where A is a n_n square matrix, b is a n_1 vector and x is the vector of unknowns. When n is large, using direct methods is not economical. Thus, the system is solved by iterative methods. At first, projection method onto subspace K _ Rn with dimension m _ n is described, and then this subspace K is equalized with the krylov subspace. Then,some samples of projection methods onto the krylov subspace, such as FOM, GMRES and CG (Conjugate Gradient), are considered. The preconditioning of the linear system is explained, that is, instead of solving system Ax = b, the system PAx = Pb (P nonsingular), is solved, such that the... 

Time-dependent origin-destination demand is a key input in dynamic traffic assignment in advanced traffic management systems, and the result of dynamic traffic assignment is dependent on the accuracy of this information. One method to achieve time dependent demand matrices is using a primary demand matrix and volume traffic counts in some links of network. In this thesis a bi-level model is used to correct the demand matrix and the extended gradient method is suggested to solve the problem. The extended gradient is an iterative method that in each iteration, corrects the demand matrix in a way that the estimated traffic flow be close to the observed traffic flow. Execution of this method in... 

We explain a filter-trust-region algorithm for solving nonlinear optimization problems with simple bounds recently proposed by Sainvitu and Toint. The algorithm is shown to be globally convergent to at least one first-order critical point. We implement the algorithm and test the program on various problems. The results show the effectiveness of the algorithm  

Numerical Simulation of two-dimensional steady compressible fluid flow on unstructured grids was accomplished using a fast implicit algorithm. To solve the copmlete two-dimensional Navier-Stokes equations, implicit time stepping was used which results in a large sparse linear system in each iteration. To solve the linear system, the biconjugate gradient method which belongs to Krylov subspace methods family, with an ILU(0) preconditioner was used. For accelerating the solution in large problems, parallel processing was used for linear system to be solved faster. Two upwind methods, namely Roe’s and AUSM+ methods were used for spatial descritizaion of inviscid fluxes with a MUSCL algorithm... 

We describe two modified secant equations proposed by Yuan, Li and Fukushima. First, we study the approach proposed by Andrei. Then, we explain two hybrid conjugate gradient methods for unconstrained optimization problems. The methods are hybridizations of Hestenes-Stiefel and Dai-Yuan conjugate gradient methods. It is shown that one of the algorithms is globally convergent for uniformly convex functions and the other is globally convergent for general functions. Two approaches for computing the initial value of the steplength proposed by Babaie, Fatemi, and Mahdavi-Amiri and Andrei are used for accelerating the performance of the line search. We implement the algorithms and compare the... 

Worst-case complexity guarantees for non-convex optimization algorithms is a topic that have received increasing attention. Here , we review trust-region Newton methods recently proposed in the literature . After a slight modification of the main model , two methods are proposed : one of them is based on the exact solution of the sub-problem , and the other is based on the inexact solution of the sub-problem , such as ``trust-region Newton-conjugate gradient " method with the complexity bounds corresponding to the best known bounds for this class of algorithms . We implement the proposed algorithms and test the programs in the Python software environment  

Nowadays, solving large-scale unconstrained optimization problems has wide applications in data science and machine learning. Therefore, the development and analysis of efficient algorithms for solving unconstrained optimization problems is of great interest. Line search and trust region are two general frameworks for guaranteeing the convergence of algorithms for solving unconstrained optimization problems. Conjugate gradient (CG) methods and the conjugate residual (CR) balance by Hestenes and Stiefel, have been presented for solving linear systems with symmetric and positive definite coefficient matrices. The basic feature of CR, that is, residual minimization, is important and can be used... 

Line search-based methods are known as a category of the most efficient iterative algo- rithms for solving unconstrained optimization problems. Among them, the conjugate gradient method is of particular importance in solving large-scale contemporary world problems due to its simplicity of structure, low memory requirement and strong convergence characteristics. In spite of the desirable numerical behavior of the conjugate gradient method, this method generally lacks the descent property even for uniformly convex objective functions. To overcome this defect, some effective modifications have been presented in the literature. Amidst, the three-term extension attracted the attention of many...