Introduction
In this article, I am going to review and implement the deep learning paper titled “CBAM: Convolutional Block Attention Module” by Woo et al. [1]. As the name suggests, this is essentially a block we can attach to a CNN-based model to enhance feature quality by performing an attention mechanism. Despite the name attention, it is completely different from the one in the ViT (Vision Transformer) architecture. Keep in mind that CBAM was first released in 2018, while ViT was only introduced in 2020. So, we can simply say that CBAM is the older approach to apply an attention mechanism to image data. Despite being older, we should not worry about its relevance since CBAM is a lot more lightweight as compared to ViT, which makes it suitable to be used as a starting point for deployment on low-power devices.
Better Than SENet
If we were to talk about the history, CBAM was actually proposed as the improvement of SENet (Squeeze-and-Excitation Network), which was introduced a year before CBAM. If you remember the SENet architecture, it essentially works by performing attention across the channel dimension. By doing so, the channels that seem unimportant would be weighted less than that of the more important ones. — I actually got a separate article talking more thoroughly about SENet, which you can access through the link given in reference number [2].
Instead of just assigning weights to each channel, CBAM also gives weights to every single pixel in the spatial dimension of the image. So, we can say that CBAM has two attention mechanisms, which the authors refer to as the CAM (Channel Attention Module) and the SAM (Spatial Attention Module). So, based solely on this theory, CBAM should perform better than SENet.
···
CBAM Architecture
Let’s talk more specifically about the CBAM architecture which I display in Figure 1 below. As I’ve mentioned earlier, CBAM consists of CAM and SAM. These two sub-blocks are responsible for creating attention weights, which will then be applied to the original tensor by multiplication. The output tensor of this block (the one referred to as Refined Features) has the exact same dimension as the input (Input Feature), meaning that we can easily plug CBAM to any backbone model without needing to worry about altering the tensor shapes.
Figure 1. The high-level view of the CBAM architecture [1].
Channel Attention Module (CAM)
Now let’s take a closer look at the channel attention module in Figure 2 below. This component is actually very similar to the SENet block, except that CAM also uses global maxpooling layer in addition to the global average-pooling layer. It is explained in the paper that the two operations capture different kind of information, allowing the tensor produced by CAM to be more informative as compared to that of the SENet block.
Figure 2. The structure of the channel attention module [1].
Remember that the spatial dimension of the tensor collapses to 1×1 when we apply global pooling operation to it. This essentially means that the input tensor, which has the original shape of C×H×W, now becomes C×1×1, allowing us to process it further easily with the MLP in the subsequent step. There are two linear layers in this MLP, where the first one is used to shrink the number of features according to the reduction ratio parameter, whereas the second one works by expanding the feature vector back to the original length (i.e., the number of channels C). These two linear layers in the MLP are together responsible to learn the importance of each channel. Also, keep in mind that this MLP is shared for the tensor produced by the maxpooling and the average-pooling operations, meaning that these two tensors will be processed by the exact same MLP.
As these two tensors have been processed, we then combine them by element-wise summation and pass it through a sigmoid function. Since this function shrinks any number to the range of 0 to 1, we can now perceive the resulting tensor as the channel attention weight. The elements that correspond to the more important channels will have the value close to 1, indicating that these channels will be weighted more than the others. According to the paper, this kind of mechanism helps the model to understand what kind of features to attend.
You can see the formal mathematical expression of the channel attention module in Figure 3 below, where F is an arbitrary intermediate tensor within a network. One thing you need to keep in mind here is that there should be a ReLU activation function placed between the two linear layers (i.e., W₀ and W₁) yet is somehow not written in this equation.
Figure 3. The formal mathematical expression of the channel attention module [1].
Spatial Attention Module (SAM)
The spatial attention module is conceptually similar to the channel attention module. Take a look at the illustration of this sub-block in Figure 4 below.
Figure 4. The structure of the spatial attention module [1].
What essentially differentiates SAM from CAM is the axis where the pooling operation is performed. Previously in CAM the pooling is done across the spatial dimension, allowing each channel to have a single value representing that channel. Meanwhile, here in SAM the pooling is done across the channel dimension for each spatial pixel location. Thus, every pixel now contains a single value that represents all channels at once. By doing so, the input tensor that initially has the shape of C×H×W will collapse to 1×H×W. Remember that since we use maximum and average-pooling operations, we will thus have two tensors of that size. These two tensors are then concatenated, forming a new tensor of shape 2×H×W. This tensor is then processed with a 7×7 convolution layer of a single kernel, which effectively combines the information from the two channels into one. So, at this point the tensor becomes 1×H×W again and is then forwarded to the sigmoid function. Similar to CAM, the tensor produced by this sigmoid acts as the spatial attention weight. By using this weight tensor, we can essentially let the model know where it should pay more attention to. Below is what the formal mathematical definition of the spatial attention module looks like.
Figure 5. The mathematical definition of the spatial attention module [1].
Integrating CBAM to Any Backbone Model
Previously I mentioned that the output shape of CBAM is exactly the same as the input, allowing it to be integrated to any backbone model easily. In fact, the authors also provide an illustration regarding how we can do that, which I show you in Figure 6 below. In this example, they illustrate how to plug CBAM into a ResNet building block. Once we have successfully integrated them like this, we can just stack these blocks as usual. Later in the coding part I will demonstrate how to implement CBAM from scratch and how to plug it into a ResNeXt model.
Figure 6. How to integrate CBAM into any backbone model [1].
···
Experimental Results
The design of the CBAM architecture itself was not chosen arbitrarily. Instead, it was built based on empirical results on their ablation studies, in which they proved that their final model is indeed the most optimal one.
Ablation Study on the Channel Attention Module
The first ablation study they conducted was related to the pooling layers in the CAM. The results of this experimental set are shown in Figure 7. We can see in this table that the errors when we use either of the two poolings are significantly lower than the plain backbone ResNet50 model. This essentially indicates that the information extracted by the maximum-pooling and the average-pooling are both important. Furthermore, when we utilize both pooling mechanisms simultaneously, the top-1 error goes even lower to 22.80%, which I believe this proves that the two tensors contain information that are not only important but also complementary (i.e., completing each other). Theoretically speaking, the features produced by maxpooling and average-pooling should indeed be complementary since the former captures the most prominent pixel value within a channel while the latter extracts the general information of the channel. So, this is essentially the reason why in Figure 2 the authors ended up using both pooling operations.
Figure 7. Ablation study on the use of average and maximum-pooling operations in the CAM [1].
Ablation Study on the Spatial Attention Module
Regarding the spatial attention module, it is explained in the paper that the authors also used varying configurations as displayed in Figure 8 below. You can see here that the configuration in the last row produces the best result, where it utilizes the two pooling operations followed by a convolution layer with 7×7 kernel. In the case of SAM, both the maximum and the average-pooling operations are technically replaceable by a 1×1 convolution (which will combine information across channel dimension with learnable parameters instead of using a “fixed” average and max operations), yet the classification performance appears to be suboptimal.
Figure 8. Ablation study on the configuration of the SAM [1].
Ablation Study on the Placement of CAM and SAM
The last ablation study the authors conducted was related to how the CAM and SAM are arranged within the CBAM. It is shown in Figure 9 below that using sequential method, especially CAM followed by SAM, allows the model to perform best with the top-1 error of only 22.66%. You can see in the next row that they tried to swap the sequence of the two modules, but they found that the error increases instead. Furthermore, the classification performance was getting even worse when they tried to parallelize CAM and SAM, even though this approach is still better than the ResNet50 with SE module only.
Figure 9. Ablation study on how the CAM and SAM are arranged [1].
Comparison with Other Models
In the subsequent experiment the authors compared the performance of a plain model, the model with SE module, and the model with CBAM on different backbones. You can see in Figure 10 below that the model that uses CBAM almost always performs better than the other two as highlighted in green. Moreover, in ResNeXt50, although the model with SE module is better than the same model with CBAM, the error gap is only 0.01%, which I think is negligible.
I also found in this figure that the error of ResNet50 with CBAM is lower than the plain ResNet101 as highlighted in orange. Interestingly, it is seen here that the number of params and the GFLOPs of ResNet50 with CBAM are much smaller. These facts show that CBAM enables a shallower network to outperform the deeper one while significantly conserving computational resources, which is ideal for deployment on low-end devices.
Figure 10. How CBAM performs on different backbone CNN models [1][3].
Attention Heatmap
In addition to the quantitative results explained above, the authors also used Grad-CAM to perform qualitative evaluation. If you’re not yet familiar with Grad-CAM, it is essentially a method we can use to find out the specific area that contributes more to the predicted class. Figure 11 below displays several examples of the attention heatmap produced using Grad-CAM, where the area highlighted in red indicates that it gives more contribution to the predicted class.
Figure 11. Attention heatmap obtained by Grad-CAM [1].
The results are pretty interesting. Let’s now take a look at the Croquet ball class. With the plain ResNet50, it looks like the model focuses on both the ball and the person. As the SE module is applied (i.e., channel-wise attention only), the attention map becomes more refined toward the ball. And then, after we replace the SE module with CBAM, the model achieves an even sharper focus on the target object.
A similar thing can also be observed in the other classes. In Eskimo dog and Snow leopard, for example, we can see that the model only pays attention to the eyes. If I were to say, this is basically not wrong as long as the predicted class is correct. However, if we were to predict something (as a human), it would make more sense to see the entire object whenever possible, right? And so, this is exactly what the attention modules do. You can see that when CBAM is used, the red area in the attention heatmap covers the entire face, indicating the model now take that facial region into consideration to make predictions.
Additionally, it is also seen in the figure that by using CBAM we can make the model more confident when making predictions. Take a look at the School bus image in the above figure. You can see here that the bus is not centered at the middle of the image. This basically causes the plain ResNet50 to have a confidence score of only 0.07 in predicting the bus (which I believe this should have been misclassified). Meanwhile, SE module allows the model to correctly classify it with the confidence of 0.92, and then CBAM improves it even further to 0.98.
···
CBAM Implementation
As we have understood all the theories behind CBAM, let’s now roll our sleeves and get our hands dirty with some code! As I’ve mentioned earlier, here I am going to implement CBAM and try to integrate it into the ResNeXt backbone. Although I am implementing both from scratch, I’ll focus the discussion primarily on the CBAM module. So if you’re not yet familiar with ResNeXt, I do encourage you read my previous article about that model beforehand, which you can access through the link at reference [4].
As usual, the very first thing we need to do is to import the required modules, i.e., the base torch module and its nn submodule.
Next, in Codeblock 2 below I initialize the configurable variables. The reduction ratio R is used to adjust the width of the MLP layer inside the CAM. I set the value for this to 16 as suggested in the paper. Meanwhile, CARDINALITY, NUM_CHANNELS, and NUM_BLOCKS are the ones belong to ResNeXt.
···
CAM Implementation
Let’s start with the CAM first. If you go back to Figure 2, you can see that we have two pooling operations. In Codeblock 3 below, the two layers that correspond to them are initialized at lines #(1) and #(2). Don’t forget to set the output_size parameter to (1,1) since we want each channel to be represented as a single number.
What we do next inside the __init__() method is initializing the MLP that consists of two linear layers. The first linear layer is responsible to reduce the number of features according to the R parameter (#(3)), whereas the second one is used to expand it back to the original number of features (#(5)). Also, don’t forget to place the ReLU activation function in between (#(4)). In fact, the structure of this MLP layer is exactly the same as the one used in SENet. — You can read more about the underlying idea behind this structure in my previous article about that module at reference [2]. — The last thing we do inside the __init__() method is to initialize the sigmoid activation function (#(6)), which is responsible to rescale the tensor such that the values will always be between 0 and 1, suitable to be used as an attention weight.
Now let’s move on to the forward() method where the execution would happen. You can see at line #(7) in the above codeblock that we take a single tensor x as the input. This input tensor will then be saved in the original variable, which is done because we will later multiply it with the resulting channel attention weight tensor (#(17)). The x tensor itself will be processed by maxpooling and average-pooling operations in parallel (#(8–9)). Both x_max and x_avg are forwarded to the same MLP, which is the reason why this MLP is called “shared” (#(12–13)). Then at line #(14), we combine x_max and x_avg through element-wise summation before forwarding the resulting tensor to the sigmoid function (#(15)). There is a little technical thing we do at line #(16), which is used to reintroduce the spatial dimension we previously dropped at lines #(10) and #(11). Finally, as the weight tensor is ready, we can then actually weight the original tensor by multiplying them (#(17)).
At this point we already got our CAM class completed. What we are going to do next is to test it with the following code. Here I initialize a CAM instance that accepts a 512-channel image and pass a dummy tensor of size 512×28×28 through it, simulating an arbitrary intermediate tensor within a network.
If you run the above code, you should get the following output. Notice that previously in Codeblock 3 I wrote lots of print functions, which is the reason why here you can clearly see the detailed flow of the network.
It is necessary to understand that although the MLP looks like it doesn’t change the tensor dimension at all (#(1–2)), you need to know that the feature vector length is internally reduced to 32 by the first linear layer before eventually expanded back to 512 by the second one. Next, it might also be worth noting that the channel attention weight tensor originally has the shape of 512×1×1 (#(3)), indicating that every single channel in the original tensor has a single weighting number associated with it. This attention weight is then applied to the original tensor by using a simple multiplication, which technically speaking, this weight tensor is broadcasted along the spatial dimension of the original tensor (#(4)). At this point our tensor is now ready to be forwarded to the SAM, which we are going to build very soon.
···
SAM Implementation
The implementation of the spatial attention module is displayed in Codeblock 5 below. What we need to initialize inside the __init__() method is only a single 7×7 convolution layer (#(1)) and a sigmoid activation function (#(2))
In the forward() method, the first thing we do is to store the original input into a separate variable (#(3)), which is exactly the same as what we did in the CAM. The pooling mechanism in the SAM is a bit unique since here we want to do that across the channel dimension. This is essentially the reason that I didn’t initialize any pooling layers in the __init__() method since nn.AdaptiveMaxPool2d() and nn.AdaptiveAvgPool2d() operate on spatial dimension, which is irrelevant for this case. Instead, here we use a simple torch.max() and torch.mean() functions to do the maxpooling (#(4)) and average-pooling (#(5)) operations, respectively. Just don’t forget to set the dim parameter to 1 so that they really do the operations across the channel dimension.
Despite taking different values, the tensor shape produced by the two poolings are exactly the same, which is the reason that we can just concatenate them as shown at line #(6). Keep in mind that tensor concatenation does not really combine information as it only stacks the two without blending the numbers. Thus, in the subsequent step we apply the convolution layer we initialized earlier to actually do that (#(7)). This convolution only consists of a single kernel, which implies that the resulting tensor will have a single channel as well. This idea is conceptually different from the one in the CAM, where in that module we combine the information by element-wise summation. Technically speaking, we can essentially use summation for the SAM too as it will produce the exact same tensor dimension. However, I do believe that the authors might also intended to capture the correlation between neighboring pixels instead of independently giving weight to each pixel, which is the reason why they decided to use convolution over summation.
As the two tensors have been combined, the next thing we do is to pass the resulting tensor through the sigmoid activation function to actually obtain the spatial attention weight (#(8)). And finally, we will multiply this weight tensor with the original SAM input as shown at line #(9).
Now let’s run the Codeblock 6 below to test if our spatial attention module works properly.
And below is what the flow of the SAM looks like. We can see here that as the pooling operations are applied to the input tensor, the channel dimension collapses to 1 (#(1–2)). This essentially means that every pixel is now represented as a single number aggregated from all channels in that spatial location. Then at line #(3), the tensor becomes 2×28×28 as we concatenate the two before eventually reducing it again to 1×28×28 using the convolution layer (#(4)). After being processed by the sigmoid function, the resulting spatial attention weight is then multiplied with the original tensor, in which the former is broadcasted along the channel dimension of the latter, allowing the final output tensor to have the exact same shape as the input (#(5)).
···
The Complete CBAM Block
Now as the CAM and SAM are done, we will now put them together in the CBAM class. See the details in Codeblock 7 below. You can see here that this class is very simple as what we need to do is just to initialize the two attention modules and place them sequentially. Keep in mind that we need to pass the num_channels parameter every time we want to initialize a CBAM instance (#(1)) since we will later integrate this module into ResNeXt, in which every single one of its building blocks accepts different number of channels, and so we need to make this CBAM block flexible as well.
Again, just to ensure that this class works properly, let’s pass a dummy tensor through it using the Codeblock 8 below.
···
Implementing CBAM on ResNeXt Building Block
Alright, so at this point our CBAM is ready and in this section I am going to actually demonstrate how we can attach this module to a ResNeXt building block. The code I write in Codeblock 9 onwards are basically the same as the one used when I demonstrated how to integrate SENet on ResNeXt. I do encourage you to read that article [2] and my explanation on the pure ResNeXt backbone [4] because it would be too long if I explain everything here.
The only thing I want to emphasize in Codeblock 9 is that the CBAM module itself is initialized at line #(1) which is then attached to the flow at line #(2).
And now we can test the Block class above by running the Codeblock 10 below. You can see in the following output that the tensor successfully passes through the entire network, including the CBAM block we attached at the end of the main flow (#(1)).
···
The Final CBAM-ized ResNeXt
As the CBAM module has been attached to the main ResNeXt building block, we can just stack these blocks according to the structure given in the ResNeXt paper. The CBAMResNeXt class in Codeblock 11 is literally copy-pasted from my SENet article [2] since the only thing we need to do to attach CBAM module to ResNeXt is modifying the Block class back in Codeblock 9.
And now we can check if the entire network works properly by running the following testing code. Here I initialize a CBAMResNeXt instance and pass a dummy RGB image of size 224×224 through it.
Below is what the resulting output looks like. Here we can see that the model successfully passes the original image through the entire network up until the classification head. This essentially indicates that our CBAM is properly attached, and thus this model is ready to be trained for image classification.
···
Ending
And well I think that’s pretty much everything about CBAM and how to implement it from scratch. You can also find the code used in this article in my GitHub repository [6]. Please let me know if you find any mistakes in the discussion or in the code. Thanks for reading, I hope you learn something new today. See ya in my next article!
···
References
[1] Sanghyun Woo et al. CBAM: Convolutional Block Attention Module. Arxiv. https://arxiv.org/abs/1807.06521 [Accessed November 12, 2025].
[2] Muhammad Ardi Putra. SENet Paper Walkthrough: The Channel-Wise Attention. Towards Data Science. https://towardsdatascience.com/the-channel-wise-attention/ [Accessed November 12, 2025]. Also available at https://medium.com/ai-advances/senet-paper-walkthrough-the-channel-wise-attention-8ac72b9cc252.
[3] Image originally created by author.
[4] Muhammad Ardi Putra. ResNeXt Paper Walkthrough: Taking ResNet to the Next Level. Towards Data Science. https://towardsdatascience.com/taking-resnet-to-the-next-level/ [Accessed November 12, 2025]. Also available at https://medium.com/ai-advances/taking-resnet-to-the-next-level-resnext-77088c245698.
[5] Saining Xie et al. Aggregated Residual Transformations for Deep Neural Networks. Arxiv. https://arxiv.org/abs/1611.05431 [Accessed November 12, 2025].
[6] MuhammadArdiPutra. CBAM. GitHub. https://github.com/MuhammadArdiPutra/medium_articles/blob/main/Deep%20Learning%20From%20Scratch/CBAM.ipynb [Accessed November 12, 2025].
Instagram Engineering
Shopify Engineering