Autograd¶
Python API¶
-
class
singa.autograd.
Abs
(name=None)¶ y = abs(x), is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(a)¶ Return abs(a), where a is CTensor.
-
-
class
singa.autograd.
Acos
¶ Calculates the arccosine (inverse of cosine) of the given input tensor, element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Acosh
¶ Calculates the hyperbolic arccosine of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Add
¶ Performs element-wise binary addition.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
a tuple for (dx0, dx1), dx0 is data for dL / da, dx1 is data for dL / db.
-
forward
(a, b)¶ Return a+b, where a and b are CTensor.
-
-
class
singa.autograd.
AddBias
(axis=0)¶ Add Bias to each row / column of the Tensor, depending on the axis arg.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss.
- Returns
a tuple for (db, dx), db is data for dL / db, dx is data for dL / dx.
-
forward
(x, b)¶ - Parameters
x (CTensor) – matrix.
b (CTensor) – bias to be added.
- Returns
the result Tensor
-
-
class
singa.autograd.
And
¶ Returns the tensor resulted from performing the and logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Raises
AssertionError – no backward function for this operator
-
forward
(a, b)¶ Return np.logical_and(a,b), where a and b are CTensor.
-
-
class
singa.autograd.
Asin
¶ Calculates the arcsine (inverse of sine) of the given input tensor, element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Asinh
¶ Calculates the hyperbolic arcsine of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Atan
¶ Calculates the arctangent (inverse of tangent) of the given input tensor, element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Atanh
¶ Calculates the hyperbolic arctangent of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
AvgPool1d
(kernel_size, stride=None, padding=0, odd_padding=0, 0, 0, 0)¶ Generate a Avg Pooling 1d operator
-
class
singa.autograd.
AvgPool2d
(kernel_size, stride=None, padding=0, odd_padding=0, 0, 0, 0)¶
-
class
singa.autograd.
BatchNorm2d
(num_features, momentum=0.9)¶ Generate a BatchNorm 2d operator
-
class
singa.autograd.
Cast
(to)¶ The operator casts the elements of a given input tensor to a data type specified by the ‘to’ argument and returns an output tensor of the same size in the converted type.
-
backward
(dy)¶ backward of Cast Args:f
dy (CTensor), gradient tensor.
- Raises
AssertionError – no backward function for this operator
-
forward
(x)¶ forward of Cast :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Ceil
¶ Ceil takes one input data (Tensor) and produces one output data (Tensor) where the ceil is, y = ceil(x), is applied to the tensor elementwise.
-
backward
(dy)¶ backward of Ceil :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Ceil :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Clip
(min, max)¶ Clip operator limits the given input within an interval. The interval is specified by the inputs ‘min’ and ‘max’.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – input tensor
- Returns
a new CTensor with np.clip(x,min,max)
-
-
class
singa.autograd.
Concat
(axis=0)¶ Concatenate a list of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss
- Returns
data for the dL / dxs, L is the loss,
- Return type
dxs (a tuple of CTensor)
-
forward
(*xs)¶ - Parameters
xs (a list of CTensor) – List of tensors for concatenation
- Returns
a CTensor for the result
-
-
class
singa.autograd.
ConstantOfShape
(value=0.0)¶ Init a ConstantOfShape, generate a tensor with given value and shape.
-
backward
(dy)¶ backward of ConstantOfShape :param dy: gradient tensor. :type dy: CTensor
- Raises
AssertionError – no backward function for this operator
-
forward
(x)¶ forward of ConstantOfShape :param x: CTensor, 1D tensor. The shape of the expected output tensor.
All values must be >= 0.
- Returns
- the output CTensor. If attribute ‘value’ is specified, the value
and datatype of the output tensor is taken from ‘value’. If attribute ‘value’ is not specified, the value in the output defaults to 0, and the datatype defaults to float32.
-
-
class
singa.autograd.
Conv2d
(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, group=1, bias=True, pad_mode='NOTSET', **kwargs)¶ Generate a Conv 2d operator
-
class
singa.autograd.
Cos
¶ Calculates the cosine of the given input tensor, element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Cosh
¶ Calculates the hyperbolic cosine of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
CrossEntropy
¶ -
backward
(dy=1.0)¶ - Parameters
dy (float or CTensor) – scalar, accumulate gradient from outside of current network, usually equal to 1.0
- Returns
- data for the dL /dx, L is the loss, x is the output
of current network. note that this is true for dy = 1.0
- Return type
dx (CTensor)
-
forward
(x, t)¶ - Parameters
x (CTensor) – 1d or 2d tensor, the prediction data(output) of current network.
t (CTensor) – 1d or 2d tensor, the target data for training.
- Returns
scalar.
- Return type
loss (CTensor)
-
-
class
singa.autograd.
Div
¶ Performs element-wise binary division (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
- a CTensor tuple for (da, db), da is data for dL / da, db is data
for dL / db.
-
forward
(a, b)¶ Return np.div(a,b), where a and b are CTensor.
-
-
class
singa.autograd.
Dropout
(ratio=0.5)¶ Init a Dropout, which scales the masked input data by the following equation: output = scale * data * mask, scale = 1. / (1. - ratio).
-
backward
(dy)¶ backward of Dropout :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Dropout :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Dummy
(tensor, name=None)¶ Dummy operation whice serves as a placehoder for autograd :param name: set it for debug :type name: string
-
grad_name
(idx)¶ - Parameters
idx – index of the output among all outputs
- Returns
the name of the gradient of the output tensor
-
output_name
(idx)¶ - Parameters
idx – index of the output among all outputs
- Returns
the name of the output tensor
-
-
class
singa.autograd.
Elu
(alpha=1.0)¶ f(x) = alpha * (exp(x) - 1.) for x < 0, f(x) = x for x >= 0., is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – matrix
- Returns
a CTensor for the result
-
-
class
singa.autograd.
Equal
¶ Returns the tensor resulted from performing the equal logical operation elementwise on the input tensors x and y.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss
- Raises
AssertionError – no backward function for this operator
-
forward
(x, y)¶ Return a=b, where a and b are CTensor.
-
-
class
singa.autograd.
Exp
(name=None)¶ y = exp(x), is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(a)¶ Return exp(a), where a is Tensor.
-
-
class
singa.autograd.
Flatten
(axis=1)¶ Flattens the input tensor into a 2D matrix. If input tensor has shape (d_0, d_1, … d_n) then the output will have shape (d_0 X d_1 … d_(axis-1), d_axis X d_(axis+1) … X dn).
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss
- Returns
data for the dL / dx, L is the loss,
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – the input tensor
- Returns
the result CTensor
-
-
class
singa.autograd.
Gather
(axis, indices)¶ Init a Gather, Given data tensor of rank r >= 1, and indices tensor of rank q, gather entries of the axis dimension of data (by default outer-most one as axis=0) indexed by indices, and concatenates them in an output tensor of rank q + (r - 1).
-
backward
(dy)¶ backward of Gather :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Gather :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Gemm
(alpha=1.0, beta=1.0, transA=0, transB=0)¶ Init a General Matrix multiplication(Gemm) operator. Compute Y = alpha * A’ * B’ + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A’ = transpose(A) if transA else A B’ = transpose(B) if transB else B
-
backward
(dy)¶ backward propogation of Gemm :param dy: The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero. :type dy: CTensor
- Returns
CTensor, the gradient over A CTensor, the gradient over B CTensor(optional), the gradient over C
-
forward
(A, B, C=None)¶ forward propogation of Gemm :param A: The shape of A should be (M, K) if transA is 0, or
(K, M) if transA is non-zero.
- Parameters
B (CTensor) – The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (CTensor) – (optional), Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).
- Returns
tensor, the output
-
-
class
singa.autograd.
GlobalAveragePool
(data_format='channels_first')¶ Init a GlobalAveragePool operator
-
backward
(dy)¶ backward propogation of GlobalAveragePool :param dy: the gradient tensor from upper operations :type dy: CTensor
- Returns
CTensor, the gradient over input
-
forward
(x)¶ forward propogation of GlobalAveragePool :param x: the input tensor :type x: CTensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Greater
¶ Returns the tensor resulted from performing the greater logical operation elementwise on the input tensors A and B.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss.
- Raises
AssertionError – no backward function for this operator.
-
forward
(x, y)¶ Return a>b, where a and b are CTensor.
-
-
class
singa.autograd.
HardSigmoid
(alpha=0.2, gamma=0.5)¶ y = max(0, min(1, alpha * x + beta)), is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – matrix
- Returns
a CTensor for the result
-
-
class
singa.autograd.
Identity
¶ Init a identity operator
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy.
- Returns
dL / dx.
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – input tensor.
- Returns
the same CTensor x.
-
-
class
singa.autograd.
LSTM
(input_size, hidden_size, nonlinearity='tanh', num_layers=1, bias=True, batch_first=False, dropout=0, bidirectional=False)¶ Generate a LSTM operator
-
class
singa.autograd.
LeakyRelu
(a)¶ f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Less
¶ Returns the tensor resulted from performing the less logical operation elementwise on the input CTensors x and y.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss.
- Raises
AssertionError – no backward function for this operator.
-
forward
(x, y)¶ Return a<b, where a and b are CTensor.
-
-
class
singa.autograd.
Linear
(in_features, out_features, bias=True)¶ Generate a Linear operator
-
class
singa.autograd.
Log
¶ y = log(x), is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return log(x), where x is CTensor.
-
-
class
singa.autograd.
Matmul
¶ Init matrix multiplication operator.
-
backward
(dy)¶ - Parameters
dy (CTensor) – data for the dL / dy, L is the loss.
- Returns
a tuple for (dx, dw).
-
forward
(x, w)¶ Return np.matmul(x,w), where x and w are CTensor.
-
-
class
singa.autograd.
Max
¶ Element-wise max of each of the input tensors (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
a tuple for (*dx), dx is data for dL / dx.
-
forward
(*x)¶ - Parameters
*x (a list of CTensor) – List of tensors for max.
- Returns
CTensor, the output
-
-
class
singa.autograd.
MaxPool1d
(kernel_size, stride=None, padding=0, odd_padding=0, 0, 0, 0)¶ Generate a Max Pooling 1d operator
-
class
singa.autograd.
MaxPool2d
(kernel_size, stride=None, padding=0, odd_padding=0, 0, 0, 0)¶ Generate a Max Pooling 2d operator
-
class
singa.autograd.
Mean
¶ Element-wise mean of each of the input CTensors.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy.
- Returns
a list of dx (CTensor).
-
forward
(*l)¶ - Parameters
l (a list of CTensor) – a list of CTensor for element-wise mean.
- Returns
a new CTensor.
-
-
class
singa.autograd.
MeanSquareError
¶ -
backward
(dy=1.0)¶ Backward propagation. :param dys: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
forward
(x, t)¶ Forward propagation. :param xs: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
-
class
singa.autograd.
Min
¶ Element-wise min of each of the input tensors (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
a tuple for (*dx), dx is data for dL / dx.
-
forward
(*x)¶ - Parameters
*x (a list of CTensor) – List of tensors for max.
- Returns
CTensor, the output
-
-
class
singa.autograd.
Mul
¶ Performs element-wise binary multiplication (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
- a tuple for (da, db), da is data for dL / da, db is data
for dL / db.
-
forward
(a, b)¶ Return np.multiply(a,b), where a and b are CTensor.
-
-
class
singa.autograd.
Negative
¶ y = -x, is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return -x, where x is CTensor.
-
-
class
singa.autograd.
NonZero
¶ Init a NonZero, Constructs a tensor by tiling a given tensor. This is the same as function tile in Numpy: https://docs.scipy.org/doc/numpy/reference/generated/numpy.tile.html
-
backward
(dy)¶ backward of NonZero :param dy: gradient tensor. :type dy: CTensor
- Raises
AssertionError – no backward function for this operator
-
forward
(x)¶ forward of NonZero :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Not
¶ Returns the negation of the input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Raises
AssertionError – no backward function for this operator
-
forward
(x)¶ Return np.logical_not(x), where x is CTensor.
-
-
class
singa.autograd.
OneHot
(axis, depth, values)¶ Produces a one-hot tensor based on inputs.
-
backward
(dy)¶ backward of OneHot :param dy: gradient tensor. :type dy: CTensor
- Raises
AssertionError – no backward function for this operator
-
forward
(indices)¶ forward of OneHot, we borrow this function from onnx :param indices: Scalar specifying the number of classes in
one-hot tensor. The values in the ‘indices’ input tensor are expected to be in the range [-depth, depth-1].
- Returns
the output CTensor.
-
-
class
singa.autograd.
Operation
(name=None)¶ An operation includes the forward and backward function of tensor calculation. Steps to add a specific operation Xxxx: 1. create a subclass of Operation, name it as Xxxx 2. override the forward() and backward(); The arguments of forward()
and backward() should only include CTensor;
-
backward
(*dys)¶ Backward propagation. :param dys: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
forward
(*xs)¶ Forward propagation. :param xs: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
grad_name
(idx)¶ - Parameters
idx – index of the output among all outputs
- Returns
the name of the gradient of the output tensor
-
output_name
(idx)¶ - Parameters
idx – index of the output among all outputs
- Returns
the name of the output tensor
-
-
class
singa.autograd.
Or
¶ Returns the tensor resulted from performing the or logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Raises
AssertionError – no backward function for this operator
-
forward
(a, b)¶ Return np.logical_or(a,b), where a and b are CTensor.
-
-
class
singa.autograd.
PRelu
¶ PRelu applies the function f(x) = slope * x for x < 0, f(x) = x for x >= 0 to the data tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x, slope)¶ - Parameters
x (CTensor) – matrix.
- Returns
the result CTensor
-
-
class
singa.autograd.
Pooling2d
(kernel_size, stride=None, padding=0, is_max=True, pad_mode='NOTSET')¶ Generate a Pooling 2d operator
-
class
singa.autograd.
Pow
¶ f(x) = a^b, is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
- a tuple for (da, db), da is data for dL / da, db is data
for dL / db.
-
forward
(a, b)¶ Return a^b, where a and b are CTensor.
-
-
class
singa.autograd.
RNN
(input_size, hidden_size, num_layers=1, nonlinearity='tanh', bias=True, batch_first=False, dropout=0, bidirectional=False)¶ Generate a RNN operator
-
class
singa.autograd.
ReLU
¶ Relu means rectified linear function, i.e, y = max(0, x) is applied to the CTensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy.
- Returns
dL / dx = dy if x >= 0; otherwise 0.
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – input tensor.
- Returns
a new CTensor whose element y = x if x >= 0; otherwise 0.
-
-
class
singa.autograd.
Reciprocal
¶ y = 1/x, is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return 1/x, where x is CTensor.
-
-
class
singa.autograd.
ReduceMean
(axes=None, keepdims=1)¶ Init a ReduceMean, computes the mean of the input tensor’s element along the provided axes.
-
backward
(dy)¶ backward of ReduceMean :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of ReduceMean :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
ReduceSum
(axes=None, keepdims=1)¶ Init a ReduceSum, computes the sum of the input tensor’s element along the provided axes.
-
backward
(dy)¶ backward of ReduceSum :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of ReduceSum :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Reshape
(shape)¶ Reshape the input tensor similar to np.reshape.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – matrix.
- Returns
the result CTensor
-
-
class
singa.autograd.
SeLU
(alpha=1.67326, gamma=1.0507)¶ y = gamma * (alpha * e^x - alpha) for x <= 0, y = gamma * x for x > 0 is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – matrix
- Returns
a CTensor for the result
-
-
class
singa.autograd.
SeparableConv2d
(in_channels, out_channels, kernel_size, stride=1, padding=0, bias=False)¶ Generate a Conv 2d operator
-
class
singa.autograd.
Shape
¶ Takes a tensor as input and outputs a tensor containing the shape of the input tensor.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Sigmoid
¶ y = 1 / (1 + exp(-x)), is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Sign
¶ Calculate the sign of the given input tensor element-wise. If input > 0, output 1. if input < 0, output -1. if input == 0, output 0.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(a)¶ - Parameters
a (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Sin
¶ Calculates the sine of the given input tensor, element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Sinh
¶ Calculates the hyperbolic sine of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Slice
(starts, ends, axes=None, steps=None)¶ Init a Slice, Produces a slice of the input tensor along multiple axes. Similar to numpy: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
-
backward
(dy)¶ backward of Slice :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Slice :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
SoftMax
(axis=1)¶ Apply SoftMax for each row of the Tensor or each column of the Tensor according to the parameter axis.
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(x)¶ - Parameters
x (CTensor) – the input 1d or 2d tensor
- Returns
the result CTensor
-
-
class
singa.autograd.
SoftMaxCrossEntropy
(t)¶ -
backward
(dy=1.0)¶ Backward propagation. :param dys: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
forward
(x)¶ Forward propagation. :param xs: input args consisting of only CTensors.
- Returns
CTensor instance(s)
-
-
class
singa.autograd.
SoftPlus
¶ y = ln(exp(x) + 1) is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return ln(exp(x) + 1), where x is CTensor.
-
-
class
singa.autograd.
SoftSign
¶ Calculates the softsign (x/(1+|x|)) of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return (x/(1+|x|)), where x is CTensor.
-
-
class
singa.autograd.
Split
(axis, parts, num_output=None)¶ Init a Split, Split a tensor into a list of tensors, along the specified ‘axis’.
-
backward
(*dys)¶ backward of Split :param dys: list of CTensor, gradient tensor.
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Split :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Sqrt
¶ y = x^0.5, is applied to the tensor elementwise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ Return x^0.5, where x is CTensor.
-
-
class
singa.autograd.
Squeeze
(axis=[])¶ Remove single-dimensional entries from the shape of a tensor. Takes a parameter axes with a list of axes to squeeze. If axes is not provided, all the single dimensions will be removed from the shape. If an axis is selected with shape entry not equal to one, an error is raised.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Sub
¶ Performs element-wise binary subtraction (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
- a tuple for (da, db), da is data for dL / da, db is data
for dL / db.
-
forward
(a, b)¶ Return a-b, where x is CTensor.
-
-
class
singa.autograd.
Sum
¶ Element-wise sum of each of the input tensors
-
backward
(dy)¶ - Parameters
dy (CTensor) – dL / dy
- Returns
dL / dx
- Return type
dx (CTensor)
-
forward
(*l)¶ - Parameters
l (a list of CTensor) – element-wise sum operator
- Returns
a CTensor for the result
-
-
class
singa.autograd.
Tan
¶ Insert single-dimensional entries to the shape of an input tensor (data).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Tanh
¶ Calculates the hyperbolic tangent of the given input tensor element-wise.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Tile
(repeats)¶ Init a Tile, Constructs a tensor by tiling a given tensor. This is the same as function tile in Numpy: https://docs.scipy.org/doc/numpy/reference/generated/numpy.tile.html
-
backward
(dy)¶ backward of Tile :param dy: gradient tensor. :type dy: CTensor
- Returns
the gradient tensor over input tensor.
-
forward
(x)¶ forward of Tile :param x: input tensor. :type x: CTensor
- Returns
the output CTensor.
-
-
class
singa.autograd.
Transpose
(perm)¶ Transpose the input tensor similar to numpy.transpose.
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Unsqueeze
(axis)¶ Insert single-dimensional entries to the shape of an input tensor (data).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Returns
CTensor, the gradient over input
-
forward
(x)¶ - Parameters
x (CTensor) – Input tensor
- Returns
CTensor, the output
-
-
class
singa.autograd.
Xor
¶ Performing the xor logical operation elementwise on the input tensors A and B (with Numpy-style broadcasting support).
-
backward
(dy)¶ - Parameters
dy (CTensor) – the gradient tensor from upper operations
- Raises
AssertionError – no backward function for this operator
-
forward
(a, b)¶ Return np.logical_xor(a,b), where a and b are CTensor.
-
-
singa.autograd.
abs
(a)¶ Return abs(a), where a is Tensor.
-
singa.autograd.
acos
(x)¶ Calculates the arccosine (inverse of cosine) of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
acosh
(x)¶ Calculates the hyperbolic arccosine of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
add
(a, b)¶ Return a+b, where a and b are Tensor.
-
singa.autograd.
add_bias
(x, b, axis=0)¶ Add Bias to each row / column of the Tensor, depending on the axis arg. :param x: matrix. :type x: Tensor :param b: bias to be added. :type b: Tensor :param axis: 0 or 1, default is 0. :type axis: int
- Returns
the result Tensor
-
singa.autograd.
asin
(x)¶ Calculates the arcsine (inverse of sine) of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
asinh
(x)¶ Calculates the hyperbolic arcsine of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
atan
(x)¶ Calculates the arctangent (inverse of tangent) of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
atanh
(x)¶ Calculates the hyperbolic arctangent of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
axis_helper
(y_shape, x_shape)¶ check which axes the x has been broadcasted :param y_shape: the shape of result :param x_shape: the shape of x
- Returns
a tuple refering the axes
-
singa.autograd.
back_broadcast
(y_shape, x_shape, x)¶ for a brodcasted tensor, restore its shape of x from y_shape to x_shape :param y_shape: the shape of result :param x_shape: the shape of x :param x: the input
- Returns
a tensor
-
singa.autograd.
backward
(y, dy=None)¶ Run the backward propagation starting at y. :param y: a Tensor instance, usually the loss :param dy: a number or a Tensor instance, for the gradient of the
objective/loss w.r.t y, usually None, i.e., 1.0
- Returns
- yeild the parameter (tensor with stores_grad true) and the
gradient tensors.
-
singa.autograd.
batchnorm_2d
(handle, x, scale, bias, running_mean, running_var)¶ Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. :param handle: BatchNormHandle for cpu and CudnnBatchNormHandle
for gpu
-
singa.autograd.
cast
(x, to)¶ The operator casts the elements of a given input tensor to a data type specified by the ‘to’ argument and returns an output tensor of the same size in the converted type. :param x: input tensor. :type x: Tensor :param to: data type, float32 = 0; int = 2. :type to: int
- Returns
the output Tensor.
-
singa.autograd.
cat
(xs, axis=0)¶ Concatenate a list of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on. :param xs: List of tensors for concatenation :type xs: a list of Tensor :param axis: Which axis to concat on. A negative value means
counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(inputs).
- Returns
a Tensor for the result
-
singa.autograd.
ceil
(x)¶ Ceil takes one input data (Tensor) and produces one output data (Tensor) where the ceil is, y = ceil(x), is applied to the tensor elementwise. :param x: input tensor. :type x: Tensor
- Returns
the output Tensor.
-
singa.autograd.
clip
(x, min=None, max=None)¶ Clip operator limits the given input within an interval. The interval is specified by the inputs ‘min’ and ‘max’. :param x: input tensor :type x: Tensor :param min: Minimum value, under which element is replaced by min. :type min: float :param max: Maximum value, above which element is replaced by max. :type max: float
- Returns
a new Tensor with np.clip(x,min,max).
-
singa.autograd.
constant_of_shape
(x, value=0)¶ Init a ConstantOfShape, generate a tensor with given value and shape. :param x: Tensor, 1D tensor. The shape of the expected output tensor.
All values must be >= 0.
- Parameters
value (float) – (Optional) The value of the output elements. Should be a one-element value. If not specified, it defaults to 0 and datatype float32
- Returns
- the output Tensor. If attribute ‘value’ is specified, the value
and datatype of the output tensor is taken from ‘value’. If attribute ‘value’ is not specified, the value in the output defaults to 0, and the datatype defaults to float32.
-
singa.autograd.
conv2d
(handle, x, W, b=None, odd_padding=0, 0, 0, 0)¶ Conv 2d operator :param handle: ConvHandle for cpu or CudnnConvHandle for gpu :type handle: object :param x: input :type x: Tensor :param W: weight :type W: Tensor :param b: bias :type b: Tensor :param odd_padding: , the odd paddding is the value
that cannot be handled by the tuple padding (w, h) mode so we need to firstly handle the input, then use the nomal padding method.
-
singa.autograd.
cos
(x)¶ Calculates the cosine of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
cosh
(x)¶ Calculates the hyperbolic cosine of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
ctensor2numpy
(x)¶ To be used in SoftMax Operation. Convert a singa_tensor to numpy_tensor.
-
singa.autograd.
div
(a, b)¶ Return np.div(a,b), where a and b are Tensor.
-
singa.autograd.
dropout
(x, ratio=0.5)¶ Init a Dropout, which scales the masked input data by the following equation: output = scale * data * mask, scale = 1. / (1. - ratio). :param x: input tensor. :type x: Tensor :param ratio: the ratio of random dropout, with value in [0, 1). :type ratio: float
- Returns
the output Tensor.
-
singa.autograd.
elu
(x, alpha=1)¶ f(x) = alpha * (exp(x) - 1.) for x < 0, f(x) = x for x >= 0., is applied to the tensor elementwise. :param x: matrix :type x: Tensor :param alpha: Coefficient of ELU, default is 1.0 :type alpha: float
- Returns
a Tensor for the result
-
singa.autograd.
equal
(x, y)¶ Return a=b, where a and b are Tensor.
-
singa.autograd.
exp
(a)¶ Return exp(a), where a is Tensor.
-
singa.autograd.
flatten
(x, axis=1)¶ Flattens the input tensor into a 2D matrix. If input tensor has shape (d_0, d_1, … d_n) then the output will have shape (d_0 X d_1 … d_(axis-1), d_axis X d_(axis+1) … X dn). :param x: the input tensor :type x: Tensor :param axis: Indicate up to which input dimensions (exclusive)
should be flattened to the outer dimension of the output. The value for axis must be in the range [-r, r], where r is the rank of the input tensor. Negative value means counting dimensions from the back. When axis = 0, the shape of the output tensor is (1, (d_0 X d_1 … d_n), where the shape of the input tensor is (d_0, d_1, … d_n).
- Returns
the result Tensor
-
singa.autograd.
gather
(x, axis, indices)¶ Init a Gather, Given data tensor of rank r >= 1, and indices tensor of rank q, gather entries of the axis dimension of data (by default outer-most one as axis=0) indexed by indices, and concatenates them in an output tensor of rank q + (r - 1). :param x: input tensor. :type x: Tensor :param axis: which axis to slice on. A negative value means counting
dimensions from the back. Accepted range is [-rank, rank-1] where r = rank(input).
- Parameters
indices (list of int) – entries of the axis dimension of data.
- Returns
the output Tensor.
-
singa.autograd.
gemm
(A, B, C=None, alpha=1.0, beta=1.0, transA=0, transB=0)¶ Init a General Matrix multiplication(Gemm) operator. Compute Y = alpha * A’ * B’ + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A’ = transpose(A) if transA else A B’ = transpose(B) if transB else B :param A: The shape of A should be (M, K) if transA is 0, or
(K, M) if transA is non-zero.
- Parameters
B (Tensor) – The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
C (Tensor) – (optional), Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).
alpha (float) – Scalar multiplier for the product of input tensors A * B.
beta (float) – Scalar multiplier for input tensor C.
ransA (int) – Whether A should be transposed
transB (int) – Whether B should be transposed
- Returns
Tensor, the output
-
singa.autograd.
globalaveragepool
(x, data_format='channels_first')¶ GlobalAveragePool operator :param x: the input tensor :type x: Tensor :param data_format: A string, we support two formats:
channels_last and channels_first, default is channels_first. channels_first means the format of input is (N x C x H x W) channels_last means the format of input is (N x H x W x C)
- Returns
Tensor, the output
-
singa.autograd.
gradients
(y, dy=None)¶ Compute the gradients of the output w.r.t the parameters
- Parameters
y – the output tensor, e.g., the loss
dy – gradient of the target w.r.t y; None indicates the gradient is 1.0; it can be used to rescale the loss.
- Returns
- a dictionary storing the gradient tensors of all tensors
whose stores_grad is true (e.g. parameter tensors)
-
singa.autograd.
greater
(x, y)¶ Return a>b, where a and b are Tensor.
-
singa.autograd.
hardsigmoid
(x, alpha=0.2, gamma=0.5)¶ y = max(0, min(1, alpha * x + beta)), is applied to the tensor elementwise. :param x: matrix :type x: Tensor :param alpha: Value of alpha. :type alpha: float :param gamma: Value of beta. :type gamma: float
- Returns
a Tensor for the result
-
singa.autograd.
identity
(x)¶ Init a identity operator. :param x: input tensor. :type x: Tensor
- Returns
the same Tensor with x.
-
singa.autograd.
infer_dependency
(op)¶ Infer the dependency of all operations with the given op as the last operation. Operation A is depending on B if A uses the output(s) of B.
- Parameters
op – an Operation instance, e.g. the loss operation.
- Returns
a Counter instance with the operation as the key, and the number of operations that are depending on it as the value; and a Counter instance with the id of the output tensor as the key, and the number of operations that are depending on it as the value.
-
singa.autograd.
leakyrelu
(x, a=0.01)¶ f(x) = alpha * x for x < 0, f(x) = x for x >= 0 is applied to the tensor elementwise. :param x: Input tensor :type x: Tensor :param a: Coefficient of leakage, default to 0.01. :type a: float
- Returns
Tensor, the output
-
singa.autograd.
less
(x, y)¶ Return a<b, where a and b are CTensor.
-
singa.autograd.
log
(x)¶ Return log(x), where x is Tensor.
-
singa.autograd.
matmul
(x, w)¶ Return np.matmul(x,w), where x and w are Tensor.
-
singa.autograd.
max
(*l)¶ Element-wise max of each of the input tensors (with Numpy-style broadcasting support). :param *x: List of tensors for max. :type *x: a list of Tensor
- Returns
CTensor, the output
-
singa.autograd.
mean
(*l)¶ Element-wise mean of each of the input tensors. :param l: element-wise mean operator. :type l: a list of Tensor
- Returns
a new Tensor.
-
singa.autograd.
min
(*l)¶ Element-wise min of each of the input tensors (with Numpy-style broadcasting support). :param *x: List of tensors for max. :type *x: a list of Tensor
- Returns
Tensor, the output
-
singa.autograd.
mul
(x, y)¶ Return np.multiply(x,y), where a and b are Tensor.
-
singa.autograd.
negative
(x)¶ Return -x, where x is Tensor.
-
singa.autograd.
nonzero
(x)¶ Init a NonZero, Constructs a tensor by tiling a given tensor. This is the same as function tile in Numpy: https://docs.scipy.org/doc/numpy/reference/generated/numpy.tile.html :param x: input tensor. :type x: Tensor
- Returns
the output Tensor.
-
singa.autograd.
onehot
(axis, indices, depth, values)¶ Produces a one-hot tensor based on inputs. :param axis: Axis along which one-hot representation in added.
Default: axis=-1. axis=-1 means that the additional dimension will be inserted as the innermost/last dimension in the output tensor.
- Parameters
indices (Tensor) – Scalar specifying the number of classes in one-hot tensor. The values in the ‘indices’ input tensor are expected to be in the range [-depth, depth-1].
depth (int) – Scalar specifying the number of classes in one-hot tensor. This is also the size of the one-hot dimension (specified by ‘axis’ attribute) added on in the output tensor. The values in the ‘indices’ input tensor are expected to be in the range [-depth, depth-1].
values (float) – Rank 1 tensor containing exactly two elements, in the format [off_value, on_value], where ‘on_value’ is the value used for filling locations specified in ‘indices’ input tensor,
- Returns
the output Tensor.
-
singa.autograd.
pooling_2d
(handle, x, odd_padding=0, 0, 0, 0)¶ Pooling 2d operator :param handle: PoolingHandle for cpu or CudnnPoolingHandle for
gpu
- Parameters
x (Tensor) – input
odd_padding (tuple of four int) – the odd paddding is the value that cannot be handled by the tuple padding (w, h) mode so it needs to firstly handle the input, then use the normal padding method.
- Returns
the result Tensor
-
singa.autograd.
pow
(a, b)¶ Return a^b, where a and b are Tensor.
-
singa.autograd.
prelu
(x, slope)¶ PRelu applies the function f(x) = slope * x for x < 0, f(x) = x for x >= 0 to the data tensor elementwise. :param x: matrix. :type x: Tensor
- Returns
the result Tensor
-
singa.autograd.
reciprocal
(x)¶ Return 1/x, where x is Tensor.
-
singa.autograd.
reduce_mean
(x, axes=None, keepdims=1)¶ Init a ReduceMean, computes the mean of the input tensor’s element along the provided axes. :param x: input tensor. :type x: Tensor :param axes: A list of integers, along which to reduce.
Accepted range is [-r, r-1] where r = rank(data). The default is None, which reduces over all the dimensions of the input tensor.
- Parameters
keepdims (int) – Keep the reduced dimension or not, default 1 mean keep reduced dimension.
- Returns
the output Tensor.
-
singa.autograd.
reduce_sum
(x, axes=None, keepdims=1)¶ Init a ReduceSum, computes the sum of the input tensor’s element along the provided axes. :param x: input tensor. :type x: Tensor :param axes: A list of integers, along which to reduce.
Accepted range is [-r, r-1] where r = rank(data). The default is None, which reduces over all the dimensions of the input tensor.
- Parameters
keepdims (int) – Keep the reduced dimension or not, default 1 mean keep reduced dimension.
- Returns
the output Tensor.
-
singa.autograd.
relu
(x)¶ Relu means rectified linear function, i.e, y = max(0, x) is applied to the CTensors elementwise. :param x: input tensor. :type x: Tensor
- Returns
a new Tensor whose element y = x if x >= 0; otherwise 0.
-
singa.autograd.
reshape
(x, shape)¶ Reshape the input tensor similar to mp.reshape. :param x: matrix. :type x: Tensor :param shape: Specified shape for output. At most one
dimension of the new shape can be -1. In this case, the value is inferred from the size of the tensor and the remaining dimensions. A dimension could also be 0, in which case the actual dimension value is unchanged (i.e. taken from the input tensor).
- Returns
the result Tensor
-
singa.autograd.
selu
(x, alpha=1.67326, gamma=1.0507)¶ y = gamma * (alpha * e^x - alpha) for x <= 0, y = gamma * x for x > 0 is applied to the tensor elementwise. :param x: matrix :type x: Tensor :param alpha: Coefficient of SELU default to 1.67326 :type alpha: float :param gamma: Coefficient of SELU default to 1.0507 :type gamma: float
- Returns
a Tensor for the result
-
singa.autograd.
shape
(x)¶ Takes a tensor as input and outputs a tensor containing the shape of the input tensor. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
sigmoid
(x)¶ y = 1 / (1 + exp(-x)), is applied to the tensor elementwise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
sign
(a)¶ Calculate the sign of the given input tensor element-wise. If input > 0, output 1. if input < 0, output -1. if input == 0, output 0. :param a: Input tensor :type a: Tensor
- Returns
Tensor, the output
-
singa.autograd.
sin
(x)¶ Calculates the sine of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
sinh
(x)¶ Calculates the hyperbolic sine of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
slice
(x, starts, ends, axes=None, steps=None)¶ Init a Slice, Produces a slice of the input tensor along multiple axes. Similar to numpy: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html :param x: input tensor. :type x: Tensor :param starts: starting indices of corresponding axis :type starts: list of int :param ends: ending indices of corresponding axis :type ends: list of int :param axes: axes that starts and ends apply to.
Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).
- Parameters
steps (list of int) – slice step of corresponding axis in axes. Negative value means slicing backward. ‘steps’ cannot be 0. Defaults to 1.
- Returns
the output Tensor.
-
singa.autograd.
softmax
(x, axis=1)¶ Apply SoftMax for each row of the Tensor or each column of the Tensor according to the parameter axis. :param x: the input 1d or 2d tensor :type x: Tensor :param axis: axis of softmax, default to 1 :type axis: int
- Returns
the result Tensor
-
singa.autograd.
softplus
(x)¶ Return ln(exp(x) + 1), where x is Tensor.
-
singa.autograd.
softsign
(x)¶ Return (x/(1+|x|)), where x is Tensor.
-
singa.autograd.
split
(x, axis, parts, num_output=None)¶ Init a Split, Split a tensor into a list of tensors, along the specified ‘axis’. :param x: input tensor. :type x: Tensor :param axis: which axis to split on. A negative value means
counting dimensions from the back. Accepted range is [-rank, rank-1] where r = rank(input).
- Parameters
parts (list of int) – length of each output, which can be specified using argument ‘parts’. Otherwise, the tensor is parts to equal sized parts.
num_output (bool) – once parts is none, the tensor is split to equal sized parts for each output.
- Returns
the output Tensor.
-
singa.autograd.
sqrt
(x)¶ Return x^0.5, where x is Tensor.
-
singa.autograd.
squeeze
(x, axis=[])¶ Remove single-dimensional entries from the shape of a tensor. Takes a parameter axes with a list of axes to squeeze. If axes is not provided, all the single dimensions will be removed from the shape. If an axis is selected with shape entry not equal to one, an error is raised. :param x: Input tensor :type x: Tensor :param axis: List of integers indicating the dimensions
to squeeze. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).
- Returns
Tensor, the output
-
singa.autograd.
sub
(a, b)¶ Return a-b, where a and b are Tensor.
-
singa.autograd.
sum
(*l)¶ Element-wise sum of each of the input tensors :param l: element-wise sum operator :type l: a list of Tensor
- Returns
a Tensor for the result
-
singa.autograd.
tan
(x)¶ Calculates the tangent of the given input tensor, element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
tanh
(x)¶ Calculates the hyperbolic tangent of the given input tensor element-wise. :param x: Input tensor :type x: Tensor
- Returns
Tensor, the output
-
singa.autograd.
tile
(x, repeats)¶ Init a Tile, Constructs a tensor by tiling a given tensor. This is the same as function tile in Numpy: https://docs.scipy.org/doc/numpy/reference/generated/numpy.tile.html :param x: input tensor. :type x: Tensor :param repeats: 1D int matrix of the same length as input’s
dimension number, includes numbers of repeated copies along input’s dimensions.
- Returns
the output Tensor.
-
singa.autograd.
transpose
(x, shape)¶ Transpose the input tensor similar to numpy.transpose. :param x: Input tensor :type x: Tensor :param perm: A list of integers. By default, reverse the
dimensions, otherwise permute the axes according to the values given.
- Returns
Tensor, the output
-
singa.autograd.
unsqueeze
(x, axis=- 1)¶ Insert single-dimensional entries to the shape of an input tensor (data). :param x: Input tensor :type x: Tensor :param axis: the dimensions to be inserted. :type axis: list of int
- Returns
Tensor, the output