Number of layers of the model - neural-network

I'm training a CNN. For reporting purposes, I want to find out the number of layers my model has.
From what I see in the code below, I have a total of 6 layers, layer1, layer2, conv2_drop, fc1, fc2, fc3. Am I right?
Net(
(layer1): Sequential(
(0): Conv2d(3, 10, kernel_size=(5, 5), stride=(1, 1))
(1): BatchNorm2d(10, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU()
(3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(4): Dropout(p=0.2, inplace=False)
)
(layer2): Sequential(
(0): Conv2d(10, 20, kernel_size=(5, 5), stride=(1, 1))
(1): BatchNorm2d(20, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU()
(3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(4): Dropout(p=0.2, inplace=False)
)
(conv2_drop): Dropout2d(p=0.5, inplace=False)
(fc1): Linear(in_features=500, out_features=250, bias=True)
(fc2): Linear(in_features=250, out_features=50, bias=True)
(fc3): Linear(in_features=50, out_features=10, bias=True)
)
Does my NN have 6 layers, or layer1 and layer2 themselves add 4 more each? That would be 14 layers total?

It is not entirely clear what one would consider a layer (is flattening a layer or just an operation? What if it's implemented as torch.nn.Module?). Neural networks are, in essence, graphs performing operations, layers are a helpful abstraction which helps us reason about them.
In PyTorch, IMO, it would be a single instance of torch.nn.Module, so in this case it would be 14. Even more so, BatchNorm is considered a layer universally and for ease I would consider others (ReLU, MaxPool2d, Dropout) as ones as well.
layer1 and layer2 is more of a block (stack of layers) in this case. Still your fc1, fc2 and fc3 should be IMO (notice I've used it again) coded as another block (wrapped in torch.nn.Seqeuntial probably).

Related

passing data from a dataset into a neural network

I am an absolute beginner in neural networks and I have problems passing data into the network.
This is the model:
model = keras.Sequential([keras.layers.Flatten(input_shape=(300,150,3)),keras.layers.Dense(128,activation='relu'),keras.layers.Dense(10,activation='softmax')])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
The data consists of images with labels as tuples. Its a mutli-label dataset, where each label, e.g. varroa_output, contains 1 if the characterisitic was present in the image and a 0 if it wasn't. This is the train_batch I tried to feed into the neural network:
<BatchDataset element_spec=(TensorSpec(shape=(None, 300, 150, 3), dtype=tf.uint8, name=None), {'cooling_output': TensorSpec(shape=(None,), dtype=tf.float64, name=None), 'pollen_output': TensorSpec(shape=(None,), dtype=tf.float64, name=None), 'varroa_output': TensorSpec(shape=(None,), dtype=tf.float64, name=None), 'wasps_output': TensorSpec(shape=(None,), dtype=tf.float64, name=None)})>
This is how I tried to train the model:
history = model.fit(train_batches,epochs=5,validation_data=validation_batches)
acc = history.history['accuracy']
print(acc)
And the following error occurs:
ValueError: Found unexpected losses or metrics that do not correspond to any Model output: dict_keys(['cooling_output', 'pollen_output', 'varroa_output', 'wasps_output']). Valid mode output names: ['dense_11']. Received struct is: {'cooling_output': <tf.Tensor 'IteratorGetNext:1' shape=(None,) dtype=float64>, 'pollen_output': <tf.Tensor 'IteratorGetNext:2' shape=(None,) dtype=float64>, 'varroa_output': <tf.Tensor 'IteratorGetNext:3' shape=(None,) dtype=float64>, 'wasps_output': <tf.Tensor 'IteratorGetNext:4' shape=(None,) dtype=float64>}.
I am sure there is an obvious explanation but I am an absolute beginner and would appreciate any help.

How to view Neural Network model internal architecture (embeds Module List) and (Layer) details in FastAi?

I am trying to view the internal architecture of my trained neural network in FastAi. My model is trained using FastAi Tabular. How can I view the internal details of the model? I need to see the Embedding Module List for my categorical variables. I found the output from a Blog, but they didn't have the code for it. The output looks something like this:
TabularModel(
(embeds): ModuleList(
(0): Embedding(388505, 600)
(1): Embedding(72, 18)
(2): Embedding(331868, 600)
(3): Embedding(5155, 192)
...
(60): Embedding(3, 3)
(61): Embedding(2, 2)
(62): Embedding(3, 3)
)
(emb_drop): Dropout(p=0.04, inplace=False)
(bn_cont): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(layers): Sequential(
(0): Linear(in_features=2102, out_features=1000, bias=True)
(1): ReLU(inplace=True)
(2): BatchNorm1d(1000, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(3): Dropout(p=0.001, inplace=False)
(4): Linear(in_features=1000, out_features=500, bias=True)
(5): ReLU(inplace=True)
(6): BatchNorm1d(500, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Dropout(p=0.01, inplace=False)
(8): Linear(in_features=500, out_features=1, bias=True)
)
)
It can simply be done using
print(model)

Can't replace classifier on Densenet121 in pytorch

I am trying to do some transfer learning using this github DenseNet121 model (https://github.com/gaetandi/cheXpert.git). I'm running into issues resizing the classification layer from 14 to 2 outputs.
Relevant part of the github code is:
class DenseNet121(nn.Module):
"""Model modified.
The architecture of our model is the same as standard DenseNet121
except the classifier layer which has an additional sigmoid function.
"""
def __init__(self, out_size):
super(DenseNet121, self).__init__()
self.densenet121 = torchvision.models.densenet121(pretrained=True)
num_ftrs = self.densenet121.classifier.in_features
self.densenet121.classifier = nn.Sequential(
nn.Linear(num_ftrs, out_size),
nn.Sigmoid()
)
def forward(self, x):
x = self.densenet121(x)
return x
I load and init with:
# initialize and load the model
model = DenseNet121(nnClassCount).cuda()
model = torch.nn.DataParallel(model).cuda()
modeldict = torch.load("model_ones_3epoch_densenet.tar")
model.load_state_dict(modeldict['state_dict'])
It looks like DenseNet doesn't split layers up into children so model = nn.Sequential(*list(modelRes.children())[:-1]) won't work.
model.classifier = nn.Linear(1024, 2) seems to work on default DenseNets, but with the modified classifier (additional sigmoid function) here it ends up just adding an additional classifier layer without replacing the original.
I've tried
model.classifier = nn.Sequential(
nn.Linear(1024, dset_classes_number),
nn.Sigmoid()
)
But am having the same added instead of replaced classifier issue:
...
)
(classifier): Sequential(
(0): Linear(in_features=1024, out_features=14, bias=True)
(1): Sigmoid()
)
)
)
(classifier): Sequential(
(0): Linear(in_features=1024, out_features=2, bias=True)
(1): Sigmoid()
)
)
If you want to replace the classifier inside densenet121 that is a member of your model you need to assign
model.densenet121.classifier = nn.Sequential(...)
if i understand your problem, the following code will solve
import torchvision.models as models
import torch
from torch import nn
import numpy as np
np.random.seed(0)
torch.manual_seed(0)
densenet121 = models.densenet121(pretrained=True)
for param in densenet121.parameters():
param.requires_grad = False
densenet121.classifier = nn.Sequential(
nn.Linear(1024, 14),
nn.ReLU(),
nn.Dropout(0.4),
nn.Linear(14, 2),
)
densenet121.cuda()

What VIF value limit (like 4,5,6,7,....) should I select for Linear Regression model which has 30 discrete variables and 4 continuous variables?

What should be the VIF value limit (like 4,5,6,7,....) for Linear Regression model which has 30 discrete, 4 continuous input variables and 1 continuous variable?
It's confusing to see that different researcher recommend different VIF values to use.
I have tried it in SPSS and by creating dummy variables for discrete variables. Here is the result
Coefficients
Model Unstandardized Coefficients Standardized Coefficients t Sig. Collinearity Statistics
B Std. Error Beta Tolerance VIF
(Constant) .076 1.262 .060 .952
absences .014 .012 .020 1.170 .243 .776 1.289
G1 .129 .039 .109 3.326 .001 .214 4.665
G2 .857 .036 .773 23.541 .000 .215 4.645
age .027 .050 .010 .548 .584 .649 1.540
school_new -.170 .135 -.025 -1.265 .206 .588 1.702
sex_new .150 .121 .023 1.239 .216 .680 1.471
address_new -.119 .127 -.017 -.937 .349 .712 1.405
famsize_new .038 .118 .005 .320 .749 .830 1.205
pstatus_new .004 .169 .000 .025 .980 .786 1.272
schoolsup_new .197 .178 .019 1.105 .269 .811 1.234
famsup_new -.070 .110 -.011 -.632 .528 .836 1.197
paid_new .147 .222 .011 .659 .510 .865 1.156
activities_new -.009 .108 -.001 -.087 .931 .830 1.204
nursery_new .070 .132 .009 .531 .596 .879 1.137
higher_new -.124 .189 -.012 -.655 .513 .712 1.404
internet_new -.115 .134 -.015 -.858 .391 .755 1.324
romantic_new .022 .112 .003 .200 .842 .832 1.202
M_prim_edu -.046 .556 -.006 -.083 .934 .046 21.942
M_5th_TO_9th -.114 .560 -.016 -.203 .839 .038 26.474
M_secon_edu -.143 .566 -.018 -.253 .801 .045 22.328
M_higher_edu -.309 .583 -.042 -.529 .597 .036 27.719
F_prim_edu -.454 .518 -.062 -.875 .382 .046 21.795
F_5th_TO_9th -.318 .522 -.046 -.608 .543 .041 24.624
F_secon_edu -.300 .532 -.037 -.563 .574 .053 18.873
F_higher_edu -.269 .547 -.033 -.492 .623 .051 19.613
M_health_job -.195 .253 -.025 -.770 .441 .229 4.373
M_other_job .050 .256 .004 .197 .844 .541 1.849
M_services_job -.273 .225 -.041 -1.211 .226 .199 5.016
M_teacher_job -.013 .226 -.002 -.055 .956 .286 3.496
F_health_job .470 .335 .036 1.400 .162 .355 2.814
F_other_job .003 .362 .000 .008 .993 .539 1.854
F_services_job .151 .269 .023 .563 .574 .136 7.336
F_teacher_job .015 .275 .002 .054 .957 .159 6.293
reason_school_repu .239 .194 .031 1.235 .217 .364 2.746
reason_course_pref .176 .202 .023 .873 .383 .347 2.886
reason_other .364 .175 .056 2.074 .039 .320 3.129
guard_mother -.030 .129 -.004 -.234 .815 .699 1.431
guard_other .311 .259 .023 1.204 .229 .612 1.635
tra_time_15_TO_30min .043 .120 .006 .356 .722 .764 1.309
tra_time_30_TO_60min .274 .206 .023 1.327 .185 .745 1.342
tra_time_GT_60min .791 .351 .038 2.254 .025 .816 1.225
study_2_TO_5hrs_time .171 .129 .026 1.325 .186 .584 1.713
study_5_TO_10hrs_time .151 .177 .017 .853 .394 .605 1.654
study_GT_10hrs_time .073 .253 .005 .290 .772 .743 1.347
failure_1_time -.532 .189 -.051 -2.814 .005 .704 1.421
failure_2_time -.691 .362 -.033 -1.906 .057 .766 1.305
failure_3_time -.428 .375 -.019 -1.140 .255 .813 1.230
family_rela_bad -.002 .381 .000 -.004 .997 .391 2.558
family_rela_avg .012 .322 .001 .038 .970 .177 5.642
family_rela_good .011 .303 .002 .037 .971 .106 9.470
family_rela_excel -.101 .308 -.014 -.329 .743 .127 7.885
freetime_low .105 .236 .012 .447 .655 .315 3.172
freetime_avg -.038 .217 -.006 -.174 .862 .217 4.600
freetime_high -.026 .231 -.004 -.111 .911 .228 4.384
freetime_very_high -.153 .266 -.014 -.572 .567 .363 2.753
go_out_low .095 .223 .012 .424 .672 .280 3.576
go_out_avg .135 .218 .019 .619 .536 .236 4.244
go_out_high .186 .232 .024 .801 .423 .264 3.781
go_out_very_high -.132 .246 -.015 -.537 .591 .284 3.521
Dalc_low -.157 .156 -.019 -1.006 .315 .655 1.527
Dalc_avg .274 .250 .021 1.097 .273 .628 1.592
Dalc_high -.877 .352 -.043 -2.488 .013 .763 1.310
Dalc_very_high .102 .407 .005 .250 .802 .571 1.751
Walc_low .031 .144 .004 .213 .831 .656 1.526
Walc_avg -.148 .164 -.018 -.901 .368 .594 1.683
Walc_high .000 .205 .000 .002 .998 .495 2.020
Walc_very_high -.059 .309 -.005 -.190 .849 .393 2.542
health_low -.065 .205 -.006 -.314 .754 .542 1.845
health_avg -.125 .185 -.015 -.677 .499 .459 2.179
health_high -.088 .190 -.010 -.465 .642 .482 2.075
health_very_high -.234 .169 -.035 -1.381 .168 .357 2.801
a. Dependent Variable: G3

Keras does not mach model with classes

I am new to Keras and I am trying to make a Neuronal Network to recognize 38 cases. I created such a model, but it just does not work. There is some problem with last layer I think. I checked summary and it looks like output of last layers is 38 as it should. Can someone help me with making it work?
My code is:
model = Sequential()
model.add(Convolution2D(16, 5, 5, border_mode='valid', input_shape=(168, 192, 3)) )
model.add( Activation('relu') )
model.add( MaxPooling2D(2,2) )
model.add( Convolution2D(16, 5, 5) )
model.add( Activation('relu') )
model.add( MaxPooling2D(2,2) )
model.add( Flatten() )
model.add( Dense(512, activation='relu'))
model.add(Dense(38, activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer=adam(0.001),metrics=['accuracy'])
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'data/train',
target_size=(168, 192),
batch_size=38,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'data/validation',
target_size=(168, 192),
batch_size=38,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=2000,
epochs=10,
validation_data=validation_generator,
validation_steps=800)
and the error looks like:
ValueError: Error when checking target: expected dense_129 to have shape (None, 38) but got array with shape (38, 1)
According to Keras documentation of from_from_directory, the specified directory ('data/train' in your case) should contain one subdirectory per class.
Since the error is saying the model is getting an array of shape (38, 1), this means you do not have 38 folders with data/train. (Note do not confuse that the first 38 dimension is the batch size, which coincidentally you have set it to same as number of classes, but does not have to be).
So you should either reaarange your subfolders into one class per subfolder, or load data manually, and flow from memory.