This work presents a modification of deep neural networks for time series analysis. We used kernel layer(s), as a novel approach, at the beginning of the common deep neural networks. These kernels learn based on dynamic time warping (DTW). In each kernel, DTW is calculated between the kernel value and a part of input time series or a part of last layer output (if the kernel is not in the first layer). DTW also gives an alignment path for the input series. This alignment path is used to defining a loss function with the goal of getting better alignment (lower DTW distance) between the kernel and the other input. Besides getting better accuracy on the examined datasets, the other achievement...