I want to test the explainability of a multiclass semantic segmentation model, deeplab_v3plus with shap to know which features contribute the most to semantic classification. However I have a ValueError: max_evals=500 is too low when running my file, and I struggle to understand the reason.
import glob
from PIL import Image
import torch
from torchvision import transforms
from torchvision.utils import make_grid
import torchvision.transforms.functional as tf
from deeplab import deeplab_v3plus
import shap
def test(args):
# make a video prez
model = deeplab_v3plus('resnet101', num_classes=args.nclass, output_stride=16, pretrained_backbone=True)
model.load_state_dict(torch.load(args.seg_file,map_location=torch.device('cpu'))) # because no gpu available on sandbox environnement
model = model.to(args.device)
model.eval()
explainer = shap.Explainer(model)
with torch.no_grad():
for i, file in enumerate(args.img_folder):
img = img2tensor(file, args)
pred = model(img)
print(explainer(img))
if __name__ == '__main__':
class Arguments:
def __init__(self):
self.device = torch.device("cuda:1" if torch.cuda.is_available() else "cpu")
self.seg_file = "Model_Woodscape.pth"
self.img_folder = glob.glob("test_img/*.png")
self.mean = [0.485, 0.456, 0.406]
self.std = [0.229, 0.224, 0.225]
self.h, self.w = 483, 640
self.nclass = 10
self.cmap = {
1: [128, 64, 128], # "road",
2: [69, 76, 11], # "lanemarks",
3: [0, 255, 0], # "curb",
4: [220, 20, 60], # "person",
5: [255, 0, 0], # "rider",
6: [0, 0, 142], # "vehicles",
7: [119, 11, 32], # "bicycle",
8: [0, 0, 230], # "motorcycle",
9: [220, 220, 0], # "traffic_sign",
0: [0, 0, 0] # "void"
}
args = Arguments()
test(args)
But it returns:
(dee_env) jovyan#jupyter:~/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+$ python test_shap.py
BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
Traceback (most recent call last):
File "/home/jovyan/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+/test_shap.py", line 85, in <module>
test(args)
File "/home/jovyan/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+/test_shap.py", line 37, in test
print(explainer(img))
File "/home/jovyan/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+/dee_env/lib/python3.9/site-packages/shap/explainers/_permutation.py", line 82, in __call__
return super().__call__(
File "/home/jovyan/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+/dee_env/lib/python3.9/site-packages/shap/explainers/_explainer.py", line 266, in __call__
row_result = self.explain_row(
File "/home/jovyan/use-cases/Scene_understanding/Code_Woodscape/deeplab_v3+/dee_env/lib/python3.9/site-packages/shap/explainers/_permutation.py", line 164, in explain_row
raise ValueError(f"max_evals={max_evals} is too low for the Permutation explainer, it must be at least 2 * num_features + 1 = {2 * len(inds) + 1}!")
ValueError: max_evals=500 is too low for the Permutation explainer, it must be at least 2 * num_features + 1 = 1854721!
In the source code it looks like it's because I don't give enough arguments. I only have three images in my test_img/* folder, is that why?
I have the same issue. A possible solution I found which seems to be working for my case is to replace this line
explainer = shap.Explainer(model)
With this line
explainer = shap.explainers.Permutation(model, max_evals = 1854721)
shap.Explainer by default has algorithm='auto'. From the documentation: shape.Explainer
By default the “auto” options attempts to make the best choice given
the passed model and masker, but this choice can always be overriden
by passing the name of a specific algorithm.
Since 'permutation' has been selected you can directly use shap.explainers.Permutation and set max_evals to the value suggested in the error message above.
Given the high number of your use case, this might take a really long time. I would suggest to use an easier model just for testing the above solution.
When I have this data frame
library(ggplot)
library(gganimate)
data <- tribble(
~year, ~num,
1950, 56,
1951, 59,
1952, 64,
1953, 76,
1954, 69,
1955, 74,
1956, 78,
1957, 98,
1958, 85,
1959, 88,
1960, 91,
1961, 87,
1962, 99,
1963, 104
)
and want to make an animated line plot with gganimate:
ggplot(data, aes(year, num))+geom_point()+geom_line()+transition_reveal(year, num)
I get a diagram, in which points and lines are drawn in the wrong sequence.
What is the reason for this and how can I correct it?
In
transition_reveal()
the first argument (id) regards the group aesthetic (which you don't have). I found that just using id = 1 for a single time series works.
The second argument (along) should be your x aesthetic (in your case the year).
Try:
ggplot(data, aes(year, num))+
geom_point()+
geom_line()+
transition_reveal(1, year)
I'm trying to train some data with CNN.
x_train.shape
(67197, 99, 81, 1)
y_train.shape
(67197, 12)
and trying to use Keras's Resnet method
import keras
import keras_resnet.models
input_shape = (98,81,1)
nclass = 12
x = keras.layers.Input(input_shape)
model = keras_resnet.models.ResNet50(x,classes=nclass)
model.compile("adam","categorical_crossentropy",["accuracy"])
model.fit(x_train,y_train,
batch_size = 300,
nb_epoch=5,
validation_data = (x_test,y_test),
shuffle = True,
)
but I got some shape error.
ValueError: Error when checking input: expected input_3 to have shape (None, 98, 81, 1) but got array with shape (67197, 99, 81, 1)
Its just a typo, your training data has shape (samples, 99, 81, 1), so the input shape should be (99, 81, 1), not (98, 81, 1). Its just off by one.
I want to plot some lines with the date on the x axis, but all examples I could find use the American style like 12-31-2012. But I want 31.12.2012, but it doesn't seem to work by just changing the date formatter from
dateFormatter = dates.DateFormatter('%Y-%m-%d')
to
dateFormatter = dates.DateFormatter('%d.%m.%y')
My date list works like this: I want to define a "firstDay" manually, and then generate X succeeding days. That works as I can see when I print that result list.
But when I want to plot that list (converted by num2date) I have totally different dates.
E.g. I set my firstDay to 734517.0 which is January the 15th in 2012. Then I print my dates on the axis and I get as a first date 01.01.87 ??
Here is my full code:
import numpy as np
import matplotlib.pyplot as plot
import matplotlib.ticker as mticker
from matplotlib import dates
import datetime
fig = plot.figure(1)
DAU = ( 2, 20, 25, 60, 190, 210, 18, 196, 212, 200, 160, 150, 185, 175, 316, 320, 294, 260, 180, 145, 135, 97, 84, 80, 60, 45, 37, 20, 20, 24, 39, 73, 99)
WAU = ( 50, 160, 412, 403, 308, 379, 345, 299, 258, 367, 319, 381, 461, 412, 470, 470, 468, 380, 290, 268, 300, 312, 360, 350, 316, 307, 289, 321, 360, 378, 344, 340, 346)
MAU = (760, 620, 487, 751, 612, 601, 546, 409, 457, 518, 534, 576, 599, 637, 670, 686, 574, 568, 816, 578, 615, 520, 499, 503, 529, 571, 461, 614, 685, 702, 687, 649, 489)
firstDay = 734517.0 #15. Januar 2012
#create an array with len(DAU) entries from given starting day...
dayArray = []
for i in xrange(len(DAU)):
dayArray.append(firstDay + i)
#...and fill them with the converted dates
dayLabels = [dates.num2date(dayArray[j]) for j in xrange(len(DAU))]
for k in xrange(len(DAU)):
print dayLabels[k]
spacing = np.arange(len(DAU)) + 1
line1 = plot.plot(spacing, DAU, 'o-', color = '#336699')
line2 = plot.plot(spacing, WAU, 'o-', color = '#993333')
line3 = plot.plot(spacing, MAU, 'o-', color = '#89a54e')
ax = plot.subplot(111)
plot.ylabel('', weight = 'bold')
plot.title('', weight = 'bold')
ticks, labels = plot.xticks(spacing, dayLabels)
plot.setp(labels, rotation = 90, fontsize = 11)
dateFormatter = dates.DateFormatter('%d.%m.%y')
ax.xaxis.set_major_formatter(dateFormatter)
#ax.fmt_xdata = dates.DateFormatter('%Y-%m-%d')
#fig.autofmt_xdate()
yMax = max(np.max(DAU), np.max(WAU), np.max(MAU))
yLimit = 100 - (yMax % 100) + yMax
plot.yticks(np.arange(0, yLimit + 1, 100))
plot.grid(True, axis = 'y')
plot.subplots_adjust(bottom = 0.5)
plot.subplots_adjust(right = 0.82)
legend = plot.legend((line1[0], line2[0], line3[0]),
('DAU',
'WAU',
'MAU'),
'upper left',
bbox_to_anchor = [1, 1],
shadow = True)
frame = legend.get_frame()
frame.set_facecolor('0.80')
for t in legend.get_texts():
t.set_fontsize('small')
plot.show()
It would be fine as well with this date formatter:
ax.fmt_xdata = dates.DateFormatter('%Y-%m-%d')
but that gives me the timestamps as well, e.g. 2012-01-15 00:00:00+00:00 .
If someone could tell me how to cut the time off, it would be really great!!
It seems to me that the easiest way is to use real Datetime objects. This way you can use the datetime.timedelta(days=i) to make your date range. And matplotlib automatically takes spacing into account in case your dates are not regular. It also allows you to use the default date formatting options from matplotlib.
I left some code out to keep it simpler but you should be able to mix this with your script:
![import numpy as np
import matplotlib.pyplot as plot
import matplotlib.ticker as mticker
from matplotlib import dates
import datetime
fig = plot.figure(1)
DAU = ( 2, 20, 25, 60, 190, 210, 18, 196, 212, 200, 160, 150, 185, 175, 316, 320, 294, 260, 180, 145, 135, 97, 84, 80, 60, 45, 37, 20, 20, 24, 39, 73, 99)
WAU = ( 50, 160, 412, 403, 308, 379, 345, 299, 258, 367, 319, 381, 461, 412, 470, 470, 468, 380, 290, 268, 300, 312, 360, 350, 316, 307, 289, 321, 360, 378, 344, 340, 346)
MAU = (760, 620, 487, 751, 612, 601, 546, 409, 457, 518, 534, 576, 599, 637, 670, 686, 574, 568, 816, 578, 615, 520, 499, 503, 529, 571, 461, 614, 685, 702, 687, 649, 489)
firstDay = datetime.datetime(2012,1,15) #15. Januar 2012
dayArray = [firstDay + datetime.timedelta(days=i) for i in xrange(len(DAU))]
ax = plot.subplot(111)
line1 = ax.plot(dayArray, DAU, 'o-', color = '#336699')
line2 = ax.plot(dayArray, WAU, 'o-', color = '#993333')
line3 = ax.plot(dayArray, MAU, 'o-', color = '#89a54e')
ax.xaxis.set_major_formatter(dates.DateFormatter('%d.%m.%Y'))]
The main difference with your script is the way dayArray is created (and used as the x-value in the plotting command) and the last line which sets the format of the x-axis.