Stories interesting to Practitioners of Machine Learning and A.I.
A lot of chatter about Pytorch for NLP and Deep Learning lately. PyTorch is a Python-centric, open-source deep learning library, coming most recently out of Facebook’s AI research group. It is being given respect due to the fact that FB depends heavily on AI processing at scale.
PyTorch is often compared to Google Brain’s efforts with TensorFlow and makes use of the Tensor construct as well. Many believe that Pytorch is oriented towards practitioners rather than academics, evidenced by a simplified api.
In the arms race for ‘most powerful deep learning stack’, TensorFlow shows up for a host of reasons affecting several important factors depending on your objectives. Now PyTorch is vying for the top spot due in part to speed of execution. Let’s walk through the high points.
Efficiency is Important
Some significant optimizations to Recurrent and Convolutional neural networks were realized after experimentation with Tensors for deep learning. In TensorFlow and PyTorch, the computational graphs employ automatic differentiation. The main difference is that in the case of PyTorch, the ‘autograd’ operator uses dynamic computational graphs to derive the gradients. This adds up as we all know due to the need to optimize up-front between epoch passes across GPUs or machines. With this dynamic control, all of the optimization can happen before spawning the new graphs iteratively.
This beginning tutorial which they say will take you 60 minutes is a good place to start.
Cloud Partners, O/S, GPU & Package Managers
Here are some articles comparing PyTorch against TensorFlow