TCN
ADLStream.models.TCN
Temporal Convolutional Network.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
input_shape |
tuple |
Shape of the input data |
required |
output_size |
int |
Number of neurons of the last layer. |
required |
loss |
tf.keras.Loss |
Loss to be use for training. |
required |
optimizer |
tf.keras.Optimizer |
Optimizer that implements theraining algorithm. |
required |
nb_filters |
int |
Number of convolutional filtersto use in the TCN blocks. Defaults to 64. |
64 |
kernel_size |
int |
The size of the convolutional kernel. Defaults to 2. |
2 |
nb_stacks |
int |
The number of stacks of residual blocks to use. Defaults to 1. |
1 |
dilations |
list |
The list of the dilations. Defaults to [1, 2, 4, 8, 16]. |
[1, 2, 4, 8, 16] |
tcn_dropout |
float between 0 and 1 |
Fraction of the input units to drop. Defaults to 0.0. |
0.0 |
return_sequences |
bool |
Whether to return the last output in the output sequence, or the full sequence. Defaults to True. |
True |
activation |
tf activation function |
The activation used in the residual blocks o = Activation(x + F(x)). Defaults to "linear". |
'linear' |
out_activation |
tf activation function |
Activation of the output layer. Defaults to "linear". |
'linear' |
padding |
str |
The padding to use in the convolutional layers, can be 'causal' or 'same'. Defaults to "causal". |
'causal' |
use_skip_connections |
bool |
If we want to add skip connections from input to each residual block. Defaults to True. |
True |
use_batch_norm |
bool |
Whether to use batch normalization in the residual layers or not. Defaults to False. |
False |
dense_layers |
list |
List with the number of hidden neurons for each layer of the dense block before the output. Defaults to []. |
[] |
dense_dropout |
float between 0 and 1 |
Fraction of the dense units to drop. Defaults to 0.0. |
0.0 |
dense_activation |
tf activation function |
Activation function of the dense layers after the convolutional block. Defaults to "linear". |
'linear' |
Returns:
Type | Description |
---|---|
tf.keras.Model |
TCN model |
Source code in ADLStream/models/tcn.py
def TCN(
input_shape,
output_size,
loss,
optimizer,
nb_filters=64,
kernel_size=2,
nb_stacks=1,
dilations=[1, 2, 4, 8, 16],
tcn_dropout=0.0,
return_sequences=True,
activation="linear",
out_activation="linear",
padding="causal",
use_skip_connections=True,
use_batch_norm=False,
dense_layers=[],
dense_dropout=0.0,
dense_activation="linear",
):
"""Temporal Convolutional Network.
Args:
input_shape (tuple): Shape of the input data
output_size (int): Number of neurons of the last layer.
loss (tf.keras.Loss): Loss to be use for training.
optimizer (tf.keras.Optimizer): Optimizer that implements theraining algorithm.
nb_filters (int, optional): Number of convolutional filtersto use in the TCN blocks.
Defaults to 64.
kernel_size (int, optional): The size of the convolutional kernel.
Defaults to 2.
nb_stacks (int, optional): The number of stacks of residual blocks to use.
Defaults to 1.
dilations (list, optional): The list of the dilations.
Defaults to [1, 2, 4, 8, 16].
tcn_dropout (float between 0 and 1, optional): Fraction of the input units to drop.
Defaults to 0.0.
return_sequences (bool, optional):
Whether to return the last output in the output sequence, or the full sequence.
Defaults to True.
activation (tf activation function, optional):
The activation used in the residual blocks o = Activation(x + F(x)).
Defaults to "linear".
out_activation (tf activation function, optional): Activation of the output layer.
Defaults to "linear".
padding (str, optional): The padding to use in the convolutional layers,
can be 'causal' or 'same'.
Defaults to "causal".
use_skip_connections (bool, optional):
If we want to add skip connections from input to each residual block.
Defaults to True.
use_batch_norm (bool, optional):
Whether to use batch normalization in the residual layers or not.
Defaults to False.
dense_layers (list, optional): List with the number of hidden neurons for each
layer of the dense block before the output.
Defaults to [].
dense_dropout (float between 0 and 1, optional): Fraction of the dense units to drop.
Defaults to 0.0.
dense_activation (tf activation function, optional): Activation function of the dense
layers after the convolutional block.
Defaults to "linear".
Returns:
tf.keras.Model: TCN model
"""
input_shape = input_shape[-len(input_shape) + 1 :]
inputs = tf.keras.layers.Input(shape=input_shape)
x = inputs
if len(input_shape) < 2:
x = tf.keras.layers.Reshape((inputs.shape[1], 1))(x)
x = keras_tcn.TCN(
nb_filters=nb_filters,
kernel_size=kernel_size,
nb_stacks=nb_stacks,
dilations=dilations,
use_skip_connections=use_skip_connections,
dropout_rate=tcn_dropout,
activation=activation,
use_batch_norm=use_batch_norm,
padding=padding,
)(x)
# Dense block
if return_sequences:
x = tf.keras.layers.Flatten()(x)
for hidden_units in dense_layers:
x = tf.keras.layers.Dense(hidden_units, activation=dense_activation)(x)
if dense_dropout > 0:
tf.keras.layers.Dropout(dense_dropout)(x)
x = tf.keras.layers.Dense(output_size, activation=out_activation)(x)
model = tf.keras.Model(inputs=inputs, outputs=x)
model.compile(optimizer=optimizer, loss=loss)
return model