I code with pytorch and I want to use resnet-18 as backbone of Faster R-RCNN. When I print structure of resnet18, this is the output:
>>import torch
>>import torchvision
>>import numpy as np
>>import torchvision.models as models
>>resnet18 = models.resnet18(pretrained=False)
>>print(resnet18)
ResNet(
(conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(layer1): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer2): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer3): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
(fc): Linear(in_features=512, out_features=1000, bias=True)
)
My question is, until which layer it is feature extractor? is AdaptiveAvgPool2d should be part of backbone of Faster R-CNN?
In this toturial, it is shown how to train a Mask R-CNN with an arbitrary backbone, I want to do the same thing with Faster R-CNN and train a Faster R-CNN with resnet-18 but until which layer should be part of feature extractor is confusing for me.
I know how to use resnet+Feature Pyramid Network as backbone, My question is about resent.
If we want to use output of Adaptive Average Pooling we use this code for different Resnets:
# backbone
if backbone_name == 'resnet_18':
resnet_net = torchvision.models.resnet18(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 512
elif backbone_name == 'resnet_34':
resnet_net = torchvision.models.resnet34(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 512
elif backbone_name == 'resnet_50':
resnet_net = torchvision.models.resnet50(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 2048
elif backbone_name == 'resnet_101':
resnet_net = torchvision.models.resnet101(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 2048
elif backbone_name == 'resnet_152':
resnet_net = torchvision.models.resnet152(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 2048
elif backbone_name == 'resnet_50_modified_stride_1':
resnet_net = resnet50(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 2048
elif backbone_name == 'resnext101_32x8d':
resnet_net = torchvision.models.resnext101_32x8d(pretrained=True)
modules = list(resnet_net.children())[:-1]
backbone = nn.Sequential(*modules)
backbone.out_channels = 2048
If we want to use convolution feature map we use this code:
# backbone
if backbone_name == 'resnet_18':
resnet_net = torchvision.models.resnet18(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnet_34':
resnet_net = torchvision.models.resnet34(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnet_50':
resnet_net = torchvision.models.resnet50(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnet_101':
resnet_net = torchvision.models.resnet101(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnet_152':
resnet_net = torchvision.models.resnet152(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnet_50_modified_stride_1':
resnet_net = resnet50(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
elif backbone_name == 'resnext101_32x8d':
resnet_net = torchvision.models.resnext101_32x8d(pretrained=True)
modules = list(resnet_net.children())[:-2]
backbone = nn.Sequential(*modules)
torchvision automatically takes in the feature extraction layers for vgg and mobilenet. .features automatically extracts out the relevant layers that are needed from the backbone model and passes it onto the object detection pipeline. You can read more about this in resnet_fpn_backbone function.
In the object detection link that you shared, you just need to change backbone = torchvision.models.mobilenet_v2(pretrained=True).features tobackbone = resnet_fpn_backbone('resnet50', pretrained_backbone).
Just to give you a brief understanding,resnet_fpn_backbone function utilizes the resnet backbone_name (18, 34, 50 ...) that you provide, instantiate retinanet and extract layers 1 through 4 using forward. This backbonewithFPN will be used in faster RCNN as backbone.
I use something like this with the fresh versions of torch and torchvision.
def get_resnet18_backbone_model(num_classes, pretrained):
from torchvision.models.detection.backbone_utils import resnet_fpn_backbone
print('Using fasterrcnn with res18 backbone...')
backbone = resnet_fpn_backbone('resnet18', pretrained=pretrained, trainable_layers=5)
anchor_generator = AnchorGenerator(
sizes=((16,), (32,), (64,), (128,), (256,)),
aspect_ratios=tuple([(0.25, 0.5, 1.0, 2.0) for _ in range(5)]))
roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0', '1', '2', '3'],
output_size=7, sampling_ratio=2)
# put the pieces together inside a FasterRCNN model
model = FasterRCNN(backbone, num_classes=num_classes,
rpn_anchor_generator=anchor_generator,
box_roi_pool=roi_pooler)
return model
Note that resnet_fpn_backbone() is already setting backbone.out_channels to the correct value.
Related
I am doing an experiment of transfer learning.
I trained 2 CNNs that have exactly the same structure, one for MNIST and one for SVHN.
I obtained the parameters (weights and bias) of the 2 models.
Now, I want to combine (sum, or other operations) these weights. A thing like this:
modelMNIST.parameters()
modelSVHN.parameters()
#now the new model
model3 = MyCNN(1)
model3.parameters = modelMNIST.parameters()+modelSVHN.parameters()
If I do in this way, I obtain this error:
SyntaxError: can't assign to function call
And in this way:
model3.block_1[0].weight = modelMNIST.block_1[0].weight + modelSVHN.block_1[0].weight
I get this error:
TypeError: cannot assign 'torch.cuda.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)
Is there any way to combine weights of different models?
You need to update the .data attribute of the parameter. Parameter is not FloatTensor and hence the error.
Since the two networks are identical you can use the below code to update the weights.
for param1, param2 in zip(modelMNIST.parameters(), modelSVHN.parameters()):
param1.data += param2.data
My solution is this:
class VGG16SUM(nn.Module):
def __init__(self, model1, model2, num_classes):
super(VGG16SUM, self).__init__()
# calculate same padding:
# (w - k + 2*p)/s + 1 = o
# => p = (s(o-1) - w + k)/2
self.block_1 = nn.Sequential(
nn.Conv2d(in_channels=1,
out_channels=64,
kernel_size=(3, 3),
stride=(1, 1),
# (1(32-1)- 32 + 3)/2 = 1
padding=1),
nn.BatchNorm2d(64),
nn.ReLU(),
nn.Conv2d(in_channels=64,
out_channels=64,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(64),
nn.ReLU(),
nn.MaxPool2d(kernel_size=(2, 2),
stride=(2, 2))
)
self.block_2 = nn.Sequential(
nn.Conv2d(in_channels=64,
out_channels=128,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(128),
nn.ReLU(),
nn.Conv2d(in_channels=128,
out_channels=128,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(128),
nn.ReLU(),
nn.MaxPool2d(kernel_size=(2, 2),
stride=(2, 2))
)
self.block_3 = nn.Sequential(
nn.Conv2d(in_channels=128,
out_channels=256,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(256),
nn.ReLU(),
nn.Conv2d(in_channels=256,
out_channels=256,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(256),
nn.ReLU(),
nn.Conv2d(in_channels=256,
out_channels=256,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(256),
nn.ReLU(),
nn.MaxPool2d(kernel_size=(2, 2),
stride=(2, 2))
)
self.block_4 = nn.Sequential(
nn.Conv2d(in_channels=256,
out_channels=512,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(512),
nn.ReLU(),
nn.Conv2d(in_channels=512,
out_channels=512,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(512),
nn.ReLU(),
nn.Conv2d(in_channels=512,
out_channels=512,
kernel_size=(3, 3),
stride=(1, 1),
padding=1),
nn.BatchNorm2d(512),
nn.ReLU(),
nn.MaxPool2d(kernel_size=(2, 2),
stride=(2, 2))
)
self.classifier = nn.Sequential(
nn.Linear(2048, 4096),
nn.ReLU(True),
nn.Dropout(p=0.25),
nn.Linear(4096, 4096),
nn.ReLU(True),
nn.Dropout(p=0.25),
nn.Linear(4096, num_classes),
)
for p_out, p_in1, p_in2 in zip(self.parameters(), model1.parameters(), model2.parameters()):
p_out.data = nn.Parameter(p_in1 +p_in2);
def forward(self, x):
x = self.block_1(x)
x = self.block_2(x)
x = self.block_3(x)
x = self.block_4(x)
# x = self.avgpool(x)
x = x.view(x.size(0), -1)
x = self.classifier(x)
return x
#logits = self.classifier(x)
#probas = F.softmax(logits, dim=1)
# probas = nn.Softmax(logits)
#return probas
# return logits
It works!!!
I can get subsets of a specific size or all of them.
scala> println((1 to 25).toSet.subsets.size)
33554432
scala> println((1 to 25).toSet.subsets(1).size)
25
scala> println((1 to 25).toSet.subsets(10).size)
1
To get just the subsets of size, say 10 or less, I could fall back to a loop and take the union. What would be a functional programming way of doing this?
You can use flatMap as follows:
(1 to 10).flatMap((1 to 25).toSet.subsets(_))
For example, below is a list of all subsets of size up to 3 in a Set of 6 elements:
(1 to 3).flatMap((1 to 6).toSet.subsets(_))
// res1: scala.collection.immutable.IndexedSeq[scala.collection.immutable.Set[Int]] = Vector(
// Set(5), Set(1), Set(6), Set(2), Set(3), Set(4),
// Set(5, 1), Set(5, 6), Set(5, 2), Set(5, 3), Set(5, 4), Set(1, 6),
// Set(1, 2), Set(1, 3), Set(1, 4), Set(6, 2), Set(6, 3), Set(6, 4),
// Set(2, 3), Set(2, 4), Set(3, 4),
// Set(5, 1, 6), Set(5, 1, 2), Set(5, 1, 3), Set(5, 1, 4), Set(5, 6, 2),
// Set(5, 6, 3), Set(5, 6, 4), Set(5, 2, 3), Set(5, 2, 4), Set(5, 3, 4),
// Set(1, 6, 2), Set(1, 6, 3), Set(1, 6, 4), Set(1, 2, 3), Set(1, 2, 4),
// Set(1, 3, 4), Set(6, 2, 3), Set(6, 2, 4), Set(6, 3, 4), Set(2, 3, 4)
// )
I think this is what you're after. (Using smaller numbers for visual verification.)
(2 to 4).flatMap(Set(1 to 5:_*).subsets)
//res0 = Vector(Set(5, 1), Set(5, 2), Set(5, 3), Set(5, 4), Set(1, 2), Set(1, 3), Set(1, 4), Set(2, 3), Set(2, 4), Set(3, 4), Set(5, 1, 2), Set(5, 1, 3), Set(5, 1, 4), Set(5, 2, 3), Set(5, 2, 4), Set(5, 3, 4), Set(1, 2, 3), Set(1, 2, 4), Set(1, 3, 4), Set(2, 3, 4), Set(5, 1, 2, 3), Set(5, 1, 2, 4), Set(5, 1, 3, 4), Set(5, 2, 3, 4), Set(1, 2, 3, 4))
I want to count elements value of dictionary. I try with this code:
def f_items(data, steps=0):
items = defaultdict(int)
for element in data:
if element in data:
items[element] += 1
else:
items[element] = 1
return items.items()
data = [[1, 2, 3, 'E'], [1, 2, 3, 'E'], [5, 2, 7, 112, 'A'] ]
rdd = sc.parallelize(data)
items = rdd.flatMap(lambda data: [y for y in f_items(data)], True)
print (items.collect())
The output of this code is shown below:
[(1, 1), (2, 1), (3, 1), ('E', 1), (1, 1), (2, 1), (3, 1), ('E', 1), (5, 1), (2, 1), (7, 1), (112, 1), ('A', 1)]
But, it should show the result following:
[(1, 2), (2, 3), (3, 3), ('E', 2), (5, 1), (7, 1), (112, 1), ('A', 1)]
How to achieve this?
Your last step should be a reduceByKey function call on the items rdd.
final_items = items.reduceByKey(lambda x,y: x+y)
print (final_items.collect())
You can look into this link to see some examples of reduceByKey in scala, java and python.
I have an RDD which has 2 partition and key value pair data as value:
rdd5.glom().collect()
[[(u'hive', 1), (u'python', 1), (u'spark', 1), (u'hive', 1),
(u'spark', 1), (u'python', 1)], [(u'spark', 1), (u'java', 1),
(u'java', 1), (u'spark', 1)]]
When I perform aggregateByKey
rdd6=rdd5.aggregateByKey((0,0), lambda acc,val: (acc[0]+1,acc[1]+val), lambda acc1,acc2 : (acc1[1]+acc2[1])/acc1[0]+acc2[0])
It is not giving me expected result:
Output:
[(u'python', (2, 2)), (u'spark', 1), (u'java', (2, 2)), (u'hive', (2,
2))]
Expected:
[(u'python', 1), (u'spark', 1), (u'java', 1), (u'hive', 1)]
I can see key present in one partition only not giving me expected output. What changes should I make to achieve that?
Ok so below is the way to do this using reduceByKey and aggregateByKey.
The problem you had with aggregateByKey is that the last function is responsiable for adding two accumulators. It has to return the same structure as all other functions so that when adding another new accumulator (From another partition) it will work again.
It is very similar to combineByKey, see here.
rdd = sc.parallelize([(u'hive', 1), (u'python', 1), (u'spark', 1),\
(u'hive', 1), (u'spark', 1), (u'python', 1), (u'spark', 1), (u'java', 1), (u'java', 1), (u'spark', 1)])
print rdd.aggregateByKey( (0, 0), lambda acc, val: (acc[0] + 1,acc[1] + val),\
lambda acc1, acc2 : (acc1[0] + acc2[0], acc1[1] + acc2[1])).collect()
print rdd.mapValues(lambda x: (1, x)).reduceByKey(lambda x, y: (x[0] + y[0], x[1] + y[1])).collect()
[(u'spark', (4, 4)), (u'java', (2, 2)), (u'hive', (2, 2)), (u'python',
(2, 2))]
[(u'spark', (4, 4)), (u'java', (2, 2)), (u'hive', (2, 2)), (u'python',
(2, 2))]
If you are trying to average the values, you can add another mapValues at the end like so:
print rdd.aggregateByKey( (0, 0),\
lambda acc, val: (acc[0] + 1,acc[1] + val),\
lambda acc1, acc2 : (acc1[0] + acc2[0], acc1[1] + acc2[1]))\
.mapValues(lambda x: x[1] * 1.0 / x[0])\
.collect()
[(u'spark', 1.0), (u'java', 1.0), (u'hive', 1.0), (u'python', 1.0)]
How can I project / transform the following
List(("A", 1.0), ("A", 3.0), ("B", 2.0), ("B", 2.0))
to
List(("A", 4.0),("B", 4.0))
So I aggregate by the string and sum the doubles?
val x = List(("A", 1.0), ("A", 3.0), ("B", 2.0), ("B", 2.0))
val y = x.groupBy(_._1).map { case (a,bs) => a -> bs.map(_._2).sum }
y.toList // List((A,4.0), (B,4.0))