Image by Editor | Midjourney & Canva
Deep Learning is widely used in many areas of Artificial Intelligence research and has contributed to technological advancements. For example, text generation, facial recognition, and voice synthesis applications are based on deep learning research.
One of the most used Deep Learning packages is PyTorch. It is an open-source package created by Meta AI in 2016 and has since been used by many.There are a lot of PyTorch advantages, including:
Flexible model architecture
Native Support for CUDA (Can use GPU)
Python-based
Providing lower-level controls, which are useful for research and many use cases
Active development by the developer and community
Let’s explore PyTorch with this article to help you get started.
Preparation
You should visit their installation webpage and select the one that suits your environment’s requirements. The below code is the installation example.
pip3 install torch torchvision torchaudio –index-url https://download.pytorch.org/whl/cpu
With the PyTorch ready, let’s get into the central part.
PyTorch Tensor
Tensor is the building block in PyTorch. It is similar to the NumPy array but has access to a GPU. We can try to create a PyTorch Tensor using the following code:
a = torch.tensor([2, 4, 5])
print(a)
Output>>
tensor([2, 4, 5])
Like the NumPy array Tensor, it allows matrix operations.
e = torch.tensor([[1, 2, 3],
[4, 5, 6]])
f = torch.tensor([7, 8, 9])
print(e * f)
Output>>
tensor([[ 7, 16, 27],
[28, 40, 54]])
It’s also possible to perform the matrix multiplication.
g = torch.randn(2, 3)
h = torch.randn(3, 2)
print( g @ h)
Output>>
tensor([[-0.8357, 0.0583],
[-2.7121, 2.1980]])
We can access the Tensor information by using the code below.
x = torch.rand(3,4)
print(“Shape:”, x.shape)
print(“Data type:”, x.dtype)
print(“Device:”, x.device)
Output>>
Shape: torch.Size([3, 4])
Data type: torch.float32
Device: cpu
Neural Network Training with PyTorch
By defining the NN using the nn.Module class, we can develop a simple model. Let’s try it out with the code below.
import torch
class SimpleNet(nn.Module):
def __init__(self, input, hidden, output):
super(SimpleNet, self).__init__()
self.fc1 = torch.nn.Linear(input, hidden)
self.fc2 = torch.nn.Linear(hidden, output)
def forward(self, x):
x = torch.nn.functional.relu(self.fc1(x))
x = self.fc2(x)
return x
inp = 10
hid = 10
outp = 2
model = SimpleNet(inp, hid, out)
print(model)
Output>>
SimpleNet(
(fc1): Linear(in_features=10, out_features=10, bias=True)
(fc2): Linear(in_features=10, out_features=2, bias=True)
)
The above code defines a SimpleNet class that inherits from nn.Module, which sets up the layers. We use nn.Linear is for the layers, and relu as the activation function.
We can add more layers or use different layers like Conv2D or CNN. But we would not use that.
Next, we would train the SimpleNet we developed with sample Tensor data.
import torch
inp = torch.randn(100, 10)
tar = torch.randint(0, 2, (100,))
criterion = torch.nn.CrossEntropyLoss()
optimizr = torch.optim.SGD(model.parameters(), lr=0.01)
epochs = 100
batchsize = 10
for epoch in range(numepochs):
model.train()
for i in range(0, inp.size(0), batchsize):
batch_inp = inputs[i:i+batch_size]
batch_tar = targets[i:i+batch_size]
out = model(batch_inp)
loss = criterion(out, batch_tar)
optimizer.zero_grad()
loss.backward()
optimizr.step()
if (epoch + 1) % 10 == 0:
print(f’Epoch [{epoch+1}/{num_epochs}], Loss: {round(loss.item(),4})’)
During the training above, we use random Tensor data and initiate the loss function called CrossEntropyLoss. Also, we initiate the SGD optimizer to manage the model parameters to minimize the loss.
The training process runs multiple times according to the epoch numbers and then performs the optimization process. This is the usual deep-learning process.
We can add several steps to more complex training to improve training, like early stopping, learning rate, and other techniques.
Lastly, we can evaluate the model we have trained with the unseen data. The following code allows us to do that.
from sklearn.metrics import classification_report
model.eval()
test_inputs = torch.randn(20, 10)
test_targets = torch.randint(0, 2, (20,))
with torch.no_grad():
test_outputs = model(test_inputs)
_, predicted = torch.max(test_outputs, 1)
print(classification_report(test_targets, predicted))
What happened above is that we switched the model into the evaluation mode, which turned off dropout and batch normalization updates. Additionally, we disable the gradient computation process to speed up the process.
You can visit the PyTorch documentation to learn further about what you can do.
Conclusion
In this article, we will go through the basics of PyTorch. From tensor creation to tensor operations and developing a simple NN model. The article is an introductory level that every beginner should be able to follow quickly.
Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and data tips via social media and writing media. Cornellius writes on a variety of AI and machine learning topics.