---
title: Using Deep Convolutional Neural Networks (DCNNs) for Time Series Forecasting Using Tensorflow - Part 2
description: Learn about the data processing codes required to use DCNNs for time series forecasting.
---

[Avenue Code Snippets](https://blog.avenuecode.com)

# [Using Deep Convolutional Neural Networks (DCNNs) for Time Series Forecasting Using Tensorflow - Part 2](https://blog.avenuecode.com/using-deep-convolutional-neural-networks-dcnns-for-time-series-forecasting-using-tensorflow-part-2)

 Written by [Hossein Javedani Sadaei](https://blog.avenuecode.com/author/hossein-javedani-sadaei) | 5/16/18 8:00 PM

[In our last post](https://blog.avenuecode.com/using-deep-convolutional-neural-networks-dcnns-for-time-series-forecasting-using-tensorflow-part-1?hsLang=en-us), we discussed the importance of developing a strong forecasting engine to predict future energy consumption based on data from past energy consumption. We then discussed the suitability of using a Deep Convolutional Neural Network (DCNN) to form accurate predictions and examined how to use this model by converting a series of load data into images.In today’s blog, we explain the codes we’re using for data processing.

## Data Processing

Before utilizing our DCNN, we need to convert a time series into gray-scale images by developing a class with some methods. We’ll begin this process by importing the required libraries as follows:

    ## import required libraries

    from sklearn.model\_selection import train\_test\_split

    import math

    import pandas as pd

    import numpy as np

    import tensorflow as tf

    import random

In this application, we will use the libraries listed above to facilitate coding. We’ll use Pandas to help process the data and NumPy to convert the data into a format that we can later input into Tensorflow. To facilitate programming, we will create a class (“MakeImage”) that has some methods.

    ### building a class to convert time series to images

    # defining variables of the class

    class MakeImage:

        def \_\_init\_\_(self):

            self.\_df = None

            self.\_size\_width = None

           self.\_index\_of\_columns\_2\_be\_batched = None

Our “size\_width” variable is used to specify image width. “index\_of\_columns\_2\_be\_batched”   indicates which columns of our Pandas dataframe (“df”) we will use for further operations like creating batches of data.

    # defining setters and getters

        @property     

        def df(self):

              return self.\_df   

        @property

       def size\_width(self):

               return self.\_size\_width

    @property

    def index\_of\_columns\_2\_be\_batched(self):

            return self.\_index\_of\_columns\_2\_be\_batched

     @df.setter

Our data must be saved in the first column of an excel file in a zero index sheet. In this class we have named this specific data column “load\_data.” You may rename this as follows:

    def df(self, value):

         self.\_df = pd.read\_excel(value, 0, names = \["load\_data"\])

    @size\_width.setter

    def size\_width(self, value):

         self.\_size\_width = value

     @index\_of\_columns\_2\_be\_batched.setter

     def index\_of\_columns\_2\_be\_batched(self, value):

        self.\_index\_of\_columns\_2\_be\_batched = value

The following function can be used to append another column (“label”) to our defined “df” (Pandas dataframe). As discussed in [Part 1 of this blog series](https://blog.avenuecode.com/using-deep-convolutional-neural-networks-dcnns-for-time-series-forecasting-using-tensorflow-part-1?hsLang=en-us), these labels indicate whether a given instance of energy consumption has increased (1) or decreased (0) compared to the last instance:

    def add\_label(self):

        df = self.df

        df.ix\[0,'label'\] = 0.0

        df.ix\[1:,'label'\] = \[1.0 if df.ix\[i,'load\_data'\] > df.ix\[i-1,'load\_data'\] else 0.0  for i in range(1,len(df.index))\]

Our next method creates chunked data from features and labels with fixed sizes that will later be used to make images. The output of this method is two NumPy arrays, namely “nparray\_features” and “nparray\_label” for chunked features and labels, respectively.

         def chunk\_features\_and\_label(self, col\_name, col\_label):

               df = self.df

               image\_width = self.size\_width

               nparray\_features = np.zeros(shape = (len(df.index) - image\_width, image\_width))

               nparray\_label = np.zeros(shape = (len(df.index) - image\_width, 1))

               for i in range(len(df.index) - image\_width):

                        nparray\_features\[i, :\] = df.ix\[i: i + image\_width - 1, col\_name\].as\_matrix().reshape(image\_width)

                        nparray\_label\[i, :\] = df.ix\[i + image\_width, col\_label\].reshape(-1,1)

             return nparray\_features, nparray\_label

The following function receives a feature set NumPy and readies it for image conversion:

        def features\_2\_images(self, x):

    size\_width = self.size\_width

    x\_image = np.zeros(shape = (x.shape\[0\], x.shape\[1\]\*x.shape\[1\]))

    print(x.shape\[0\])

    for i in range(x.shape\[0\]):

    x\_temp = x\[i, :\].copy()

   x\_copy = np.copy(x\[i,:\])

   x\_int = np.zeros(shape = (x\_copy.shape), dtype = int)

   x\_copy2 = np.copy(x\[i,:\])

   x\_copy.sort()

    for j in range(np.size(x\_copy)):

        for k in range(np.size(x\_copy)):

               if x\_copy2\[j\] == x\_copy\[k\]:

               x\[i, j\] = int(k)

    n\_values = np.max(x\[i,:\].astype(int)) + 1

    squared = np.eye(n\_values)\[x\[i,:\].astype(int)\]

    flatten = squared.flatten()

    x\_image\[i, :\] = flatten

return x\_image

To convert labels into an appropriate format for our proposed methodology, we need to turn them into NumPy arrays with the same dimension as the number of classes (in our case, 2). The following function uses a one hot encoding technique to convert them:

        def convert\_2\_onehot\_encoding(self, vector\_of\_one\_dim):

                 nparray\_lab\_onehot = np.zeros(shape = (vector\_of\_one\_dim.shape\[0\], 2))

                 for i in range(vector\_of\_one\_dim.shape\[0\]):

             n\_values = 2

            nparray\_lab\_onehot\[i, :\] = np.eye(n\_values)\[vector\_of\_one\_dim\[i,:\].astype(int)\]           

      return nparray\_lab\_onehot

In our next method, we simply receive “batch\_size” as an input argument and calculate the number of batches based on input.

def number\_of\_batches(self, x, batch\_size):

return   int(x.shape\[0\]/batch\_size)

Using the input images and their corresponding labels, the following function will randomly select a batch of combinations in each call, e.g. “x\_batch” and “y\_batch” with “batch\_size.”

         def next\_batch(self, x, y, batch\_size):

      index\_for\_batch = self.index\_of\_columns\_2\_be\_batched

      if len(index\_for\_batch) >= batch\_size:

               selected\_index = random.sample(index\_for\_batch, batch\_size)

else:

            selected\_index = random.sample(index\_for\_batch, len(index\_for\_batch))    

       x\_batch = \[x\[i\] for i in selected\_index\]

       y\_batch = \[y\[i\] for i in selected\_index\]

      self.index\_of\_columns\_2\_be\_batched = \[i for i in index\_for\_batch if i not in selected\_index\]

      return x\_batch, y\_batch

The function above concludes our set-up process. Now, we must create a class and an instance of it. Then, by calling the methods of this object, the required input data for our proposed DCNN will be prepared as follows:

         ### create a class and initialize an instance of it

        makeImage = MakeImage()

        ## Setter instance variables

        #  put the path of your time series data in your system as shown below

        path\_2\_excel\_file = 'path to data/load\_data.xlsx'

        makeImage.df = path\_2\_excel\_file  # a string that shows path of data

        # set the image size (we determined for this application that 32 is a good choice)

       makeImage.size\_width = 32

        makeImage.columns\_2\_pad = \['load\_data', 'label'\]

        makeImage.add\_label()

       # process load data to be proper for the model

       X\_not\_ready\_yet, Y\_not\_ready\_yet = makeImage.chunk\_features\_and\_label('load\_data', 'label')

       X\_ready = makeImage.features\_2\_images(X\_not\_ready\_yet)

      Y\_ready = makeImage.convert\_2\_onehot\_encoding(Y\_not\_ready\_yet)

      # select 70% of data for training the model and 30% for testing

      X\_train, X\_test, y\_train, y\_test = train\_test\_split(X\_ready, Y\_ready, test\_size = 0.3) 

      # the batch size is selected to be 100; it is possible to adjust it based on the problem’s requirements

      batch\_size = 100

      # we need to calculate number of batches

      number\_of\_batches = makeImage.number\_of\_batches(X\_train, batch\_size)

## Conclusion

In this blog, we described how to develop some codes to facilitate the data processing component of our proposed methodology. These codes are purposefully developed so that minor changes can make them reusable for solving most time series problems. In our final blog, we’ll discuss our proposed DCNN topology and establish a computation graph, using Tensorflow to process the output data from today’s codes.

[View full post](https://blog.avenuecode.com/using-deep-convolutional-neural-networks-dcnns-for-time-series-forecasting-using-tensorflow-part-2)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Hossein Javedani Sadaei"
  },
  "dateModified" : "2018-05-16T20:00:02.169Z",
  "datePublished" : "2018-05-16T20:00:00Z",
  "headline" : "Using Deep Convolutional Neural Networks (DCNNs) for Time Series Forecasting Using Tensorflow - Part 2",
  "image" : {
    "@type" : "ImageObject",
    "height" : 3000,
    "url" : "https://info.avenuecode.com/hubfs/daniel-hjalmarsson-597537-unsplash.jpg",
    "width" : 4000
  },
  "mainEntityOfPage" : "https://blog.avenuecode.com/using-deep-convolutional-neural-networks-dcnns-for-time-series-forecasting-using-tensorflow-part-2",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60,
      "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
      "width" : 60
    },
    "name" : "Avenue Code Snippets"
  }
}
```