Category: Machine Learning

Time Series – Data Processing and Visualization Time Series is a sequence of observations indexed in equi-spaced time intervals. Hence, the order and continuity should be maintained in any time series. The dataset we will be using is a multi-variate time series having hourly data for approximately one year, for air quality in a significantly polluted Italian city. The dataset can be downloaded from the link given below − . It is necessary to make sure that − The time series is equally spaced, and There are no redundant values or gaps in it. In case the time series is not continuous, we can upsample or downsample it. Showing df.head() In [122]: import pandas In [123]: df = pandas.read_csv(“AirQualityUCI.csv”, sep = “;”, decimal = “,”) df = df.iloc[ : , 0:14] In [124]: len(df) Out[124]: 9471 In [125]: df.head() Out[125]: For preprocessing the time series, we make sure there are no NaN(NULL) values in the dataset; if there are, we can replace them with either 0 or average or preceding or succeeding values. Replacing is a preferred choice over dropping so that the continuity of the time series is maintained. However, in our dataset the last few values seem to be NULL and hence dropping will not affect the continuity. Dropping NaN(Not-a-Number) In [126]: df.isna().sum() Out[126]: Date 114 Time 114 CO(GT) 114 PT08.S1(CO) 114 NMHC(GT) 114 C6H6(GT) 114 PT08.S2(NMHC) 114 NOx(GT) 114 PT08.S3(NOx) 114 NO2(GT) 114 PT08.S4(NO2) 114 PT08.S5(O3) 114 T 114 RH 114 dtype: int64 In [127]: df = df[df[”Date”].notnull()] In [128]: df.isna().sum() Out[128]: Date 0 Time 0 CO(GT) 0 PT08.S1(CO) 0 NMHC(GT) 0 C6H6(GT) 0 PT08.S2(NMHC) 0 NOx(GT) 0 PT08.S3(NOx) 0 NO2(GT) 0 PT08.S4(NO2) 0 PT08.S5(O3) 0 T 0 RH 0 dtype: int64 Time Series are usually plotted as line graphs against time. For that we will now combine the date and time column and convert it into a datetime object from strings. This can be accomplished using the datetime library. Converting to datetime object In [129]: df[”DateTime”] = (df.Date) + ” ” + (df.Time) print (type(df.DateTime[0])) <class ”str”> In [130]: import datetime df.DateTime = df.DateTime.apply(lambda x: datetime.datetime.strptime(x, ”%d/%m/%Y %H.%M.%S”)) print (type(df.DateTime[0])) <class ”pandas._libs.tslibs.timestamps.Timestamp”> Let us see how some variables like temperature changes with change in time. Showing plots In [131]: df.index = df.DateTime In [132]: import matplotlib.pyplot as plt plt.plot(df[”T”]) Out[132]: [<matplotlib.lines.Line2D at 0x1eaad67f780>] In [208]: plt.plot(df[”C6H6(GT)”]) Out[208]: [<matplotlib.lines.Line2D at 0x1eaaeedff28>] Box-plots are another useful kind of graphs that allow you to condense a lot of information about a dataset into a single graph. It shows the mean, 25% and 75% quartile and outliers of one or multiple variables. In the case when number of outliers is few and is very distant from the mean, we can eliminate the outliers by setting them to mean value or 75% quartile value. Showing Boxplots In [134]: plt.boxplot(df[[”T”,”C6H6(GT)”]].values) Out[134]: {”whiskers”: [<matplotlib.lines.Line2D at 0x1eaac16de80>, <matplotlib.lines.Line2D at 0x1eaac16d908>, <matplotlib.lines.Line2D at 0x1eaac177a58>, <matplotlib.lines.Line2D at 0x1eaac177cf8>], ”caps”: [<matplotlib.lines.Line2D at 0x1eaac16d2b0>, <matplotlib.lines.Line2D at 0x1eaac16d588>, <matplotlib.lines.Line2D at 0x1eaac1a69e8>, <matplotlib.lines.Line2D at 0x1eaac1a64a8>], ”boxes”: [<matplotlib.lines.Line2D at 0x1eaac16dc50>, <matplotlib.lines.Line2D at 0x1eaac1779b0>], ”medians”: [<matplotlib.lines.Line2D at 0x1eaac16d4a8>, <matplotlib.lines.Line2D at 0x1eaac1a6c50>], ”fliers”: [<matplotlib.lines.Line2D at 0x1eaac177dd8>, <matplotlib.lines.Line2D at 0x1eaac1a6c18>],”means”: [] }

Time Series – Variations of ARIMA In the previous chapter, we have now seen how ARIMA model works, and its limitations that it cannot handle seasonal data or multivariate time series and hence, new models were introduced to include these features. A glimpse of these new models is given here − Vector Auto-Regression (VAR) It is a generalized version of auto regression model for multivariate stationary time series. It is characterized by ‘p’ parameter. Vector Moving Average (VMA) It is a generalized version of moving average model for multivariate stationary time series. It is characterized by ‘q’ parameter. Vector Auto Regression Moving Average (VARMA) It is the combination of VAR and VMA and a generalized version of ARMA model for multivariate stationary time series. It is characterized by ‘p’ and ‘q’ parameters. Much like, ARMA is capable of acting like an AR model by setting ‘q’ parameter as 0 and as a MA model by setting ‘p’ parameter as 0, VARMA is also capable of acting like an VAR model by setting ‘q’ parameter as 0 and as a VMA model by setting ‘p’ parameter as 0. In [209]: df_multi = df[[”T”, ”C6H6(GT)”]] split = len(df) – int(0.2*len(df)) train_multi, test_multi = df_multi[0:split], df_multi[split:] In [211]: from statsmodels.tsa.statespace.varmax import VARMAX model = VARMAX(train_multi, order = (2,1)) model_fit = model.fit() c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelstsastatespacevarmax.py:152: EstimationWarning: Estimation of VARMA(p,q) models is not generically robust, due especially to identification issues. EstimationWarning) c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelstsabasetsa_model.py:171: ValueWarning: No frequency information was provided, so inferred frequency H will be used. % freq, ValueWarning) c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelsbasemodel.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals “Check mle_retvals”, ConvergenceWarning) In [213]: predictions_multi = model_fit.forecast( steps=len(test_multi)) c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelstsabasetsa_model.py:320: FutureWarning: Creating a DatetimeIndex by passing range endpoints is deprecated. Use `pandas.date_range` instead. freq = base_index.freq) c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelstsastatespacevarmax.py:152: EstimationWarning: Estimation of VARMA(p,q) models is not generically robust, due especially to identification issues. EstimationWarning) In [231]: plt.plot(train_multi[”T”]) plt.plot(test_multi[”T”]) plt.plot(predictions_multi.iloc[:,0:1], ”–”) plt.show() plt.plot(train_multi[”C6H6(GT)”]) plt.plot(test_multi[”C6H6(GT)”]) plt.plot(predictions_multi.iloc[:,1:2], ”–”) plt.show() The above code shows how VARMA model can be used to model multivariate time series, although this model may not be best suited on our data. VARMA with Exogenous Variables (VARMAX) It is an extension of VARMA model where extra variables called covariates are used to model the primary variable we are interested it. Seasonal Auto Regressive Integrated Moving Average (SARIMA) This is the extension of ARIMA model to deal with seasonal data. It divides the data into seasonal and non-seasonal components and models them in a similar fashion. It is characterized by 7 parameters, for non-seasonal part (p,d,q) parameters same as for ARIMA model and for seasonal part (P,D,Q,m) parameters where ‘m’ is the number of seasonal periods and P,D,Q are similar to parameters of ARIMA model. These parameters can be calibrated using grid search or genetic algorithm. SARIMA with Exogenous Variables (SARIMAX) This is the extension of SARIMA model to include exogenous variables which help us to model the variable we are interested in. It may be useful to do a co-relation analysis on variables before putting them as exogenous variables. In [251]: from scipy.stats.stats import pearsonr x = train_multi[”T”].values y = train_multi[”C6H6(GT)”].values corr , p = pearsonr(x,y) print (”Corelation Coefficient =”, corr,”nP-Value =”,p) Corelation Coefficient = 0.9701173437269858 P-Value = 0.0 Pearson’s Correlation shows a linear relation between 2 variables, to interpret the results, we first look at the p-value, if it is less that 0.05 then the value of coefficient is significant, else the value of coefficient is not significant. For significant p-value, a positive value of correlation coefficient indicates positive correlation, and a negative value indicates a negative correlation. Hence, for our data, ‘temperature’ and ‘C6H6’ seem to have a highly positive correlation. Therefore, we will In [297]: from statsmodels.tsa.statespace.sarimax import SARIMAX model = SARIMAX(x, exog = y, order = (2, 0, 2), seasonal_order = (2, 0, 1, 1), enforce_stationarity=False, enforce_invertibility = False) model_fit = model.fit(disp = False) c:usersnavekshaappdatalocalprogramspythonpython37libsite-packagesstatsmodelsbasemodel.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals “Check mle_retvals”, ConvergenceWarning) In [298]: y_ = test_multi[”C6H6(GT)”].values predicted = model_fit.predict(exog=y_) test_multi_ = pandas.DataFrame(test) test_multi_[”predictions”] = predicted[0:1871] In [299]: plt.plot(train_multi[”T”]) plt.plot(test_multi_[”T”]) plt.plot(test_multi_.predictions, ”–”) Out[299]: [<matplotlib.lines.Line2D at 0x1eab0191c18>] The predictions here seem to take larger variations now as opposed to univariate ARIMA modelling. Needless to say, SARIMAX can be used as an ARX, MAX, ARMAX or ARIMAX model by setting only the corresponding parameters to non-zero values. Fractional Auto Regressive Integrated Moving Average (FARIMA) At times, it may happen that our series is not stationary, yet differencing with ‘d’ parameter taking the value 1 may over-difference it. So, we need to difference the time series using a fractional value. In the world of data science there is no one superior model, the model that works on your data depends greatly on your dataset. Knowledge of various models allows us to choose one that work on our data and experimenting with that model to achieve the best results. And results should be seen as plot as well as error metrics, at times a small error may also be bad, hence, plotting and visualizing the results is essential. In the next chapter, we will be looking at another statistical model, exponential smoothing.

Time Series – Introduction A time series is a sequence of observations over a certain period. A univariate time series consists of the values taken by a single variable at periodic time instances over a period, and a multivariate time series consists of the values taken by multiple variables at the same periodic time instances over a period. The simplest example of a time series that all of us come across on a day to day basis is the change in temperature throughout the day or week or month or year. The analysis of temporal data is capable of giving us useful insights on how a variable changes over time, or how it depends on the change in the values of other variable(s). This relationship of a variable on its previous values and/or other variables can be analyzed for time series forecasting and has numerous applications in artificial intelligence.