Related
I'm trying to train a network for facial recognition, I can't get past this error. I'm really new to programming in Python and even more so with libraries like Tensorflow.
I keep on getting this error related to input shape. Any help would be highly appreciated. Thanks!
labeled_images = data.map(process_file)
print('labeled images: ', labeled_images)
test_data = labeled_images.take(50)
train_data = labeled_images.skip(50)
print('test data: ', test_data)
print('train data: ', train_data)
this is where the problems start
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(192, 192, 3)))
model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(40))
model.summary()
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy, metrics=['accuracy'])
history = model.fit(train_data, epochs=10, validation_data=test_data)
I printed the "variables"
data: <ZipDataset element_spec=(TensorSpec(shape=(), dtype=tf.string, name=None), TensorSpec(shape=(40,), dtype=tf.int64, name=None))>
labeled images: <MapDataset element_spec=(TensorSpec(shape=(192, 192, 3), dtype=tf.float32, name=None), TensorSpec(shape=(40,), dtype=tf.int64, name=None))>
test data: <TakeDataset element_spec=(TensorSpec(shape=(192, 192, 3), dtype=tf.float32, name=None), TensorSpec(shape=(40,), dtype=tf.int64, name=None))>
train data: <SkipDataset element_spec=(TensorSpec(shape=(192, 192, 3), dtype=tf.float32, name=None), TensorSpec(shape=(40,), dtype=tf.int64, name=None))>
this is the error
ValueError: in user code:
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\engine\training.py", line 1021, in train_function *
return step_function(self, iterator)
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\engine\training.py", line 1010, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\engine\training.py", line 1000, in run_step **
outputs = model.train_step(data)
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\engine\training.py", line 859, in train_step
y_pred = self(x, training=True)
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Dion\.conda\envs\KerASS\lib\site-packages\keras\engine\input_spec.py", line 264, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" is '
ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 192, 192, 3), found shape=(192, 192, 3)
I got a series of data as bytes, I tried to encode it
let encoder = JSONEncoder()
encoder.dataEncodingStrategy = .deferredToData
let encodedData = try encoder.encode(data)
let data = encodedData.compactMap { $0 }
my data is looks like that:
[ 91,91,48, 44, 49, 57, 51, ..., 49, 44, 48, 93, 93]
It works, but the outcome is not what I expected (the numbers inside the data.element are different), then I tried to change the encoding strategy.
encoder64.dataEncodingStrategy = .base64
let data = encodedData.compactMap { $0 }
[91, 34, 65, 77, 72, ..., 65, 61, 61, 34, 93]
Then I have a different results, but still not is the what that I expected.
there is another custom encoding strategy, could you please give me some example of this custom strategy that I can test another forms to compare the results.
the example data that I expected is like :
[ 0, 193, 193, 193, 193, 72, 20, 193, ..., 255, 91, 0]
the 0 in the beginning and in the end is so important to me.
Thank you so much
The output is correct.
The JSONEncoder creates a JSON string from the data and compactMap maps each character to its UInt8 value
91 is [
91 is [
48 is 0
44 is ,
49 is 1
57 is 9
51 is 3
...
49 is 1
44 is ,
48 is 0
93 is ]
93 is ]
And consider the different representations: 91hex is 145dec
When I have this data frame
library(ggplot)
library(gganimate)
data <- tribble(
~year, ~num,
1950, 56,
1951, 59,
1952, 64,
1953, 76,
1954, 69,
1955, 74,
1956, 78,
1957, 98,
1958, 85,
1959, 88,
1960, 91,
1961, 87,
1962, 99,
1963, 104
)
and want to make an animated line plot with gganimate:
ggplot(data, aes(year, num))+geom_point()+geom_line()+transition_reveal(year, num)
I get a diagram, in which points and lines are drawn in the wrong sequence.
What is the reason for this and how can I correct it?
In
transition_reveal()
the first argument (id) regards the group aesthetic (which you don't have). I found that just using id = 1 for a single time series works.
The second argument (along) should be your x aesthetic (in your case the year).
Try:
ggplot(data, aes(year, num))+
geom_point()+
geom_line()+
transition_reveal(1, year)
Dears, does anyone has an ideia why 'dropout_rate' and 'learning_rate' returne only 0 and does not search for the range I gave when I am doing a RandomizedSearchCV on the hyperparameters?
Here is my code for a ANN using keras/tensoflow:
# Create the model
def create_model(neurons = 1, init_mode = 'uniform', activation='relu', inputDim = 8792, dropout_rate=0.7, learn_rate=0.01, momentum=0, weight_constraint=0): #, learn_rate=0.01, momentum=0):
model = Sequential()
model.add(Dense(neurons, input_dim=inputDim, kernel_initializer=init_mode, activation=activation, kernel_constraint=maxnorm(weight_constraint), kernel_regularizer=regularizers.l2(0.001))) # one inner layer
#model.add(Dense(neurons, input_dim=inputDim, activation=activation)) # second inner layer
model.add(Dropout(dropout_rate))
model.add(Dense(1, activation='sigmoid'))
optimizer = RMSprop(lr=learn_rate)
# compile model
model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
return model
# model
model = KerasClassifier(build_fn=create_model, verbose=0)
# Define K-fold cross validation test harness
kfold = StratifiedKFold(n_splits=3, shuffle=True, random_state=seed)
for train, test in kfold.split(X_train, Y_train):
print("TRAIN:", train, "VALIDATION:", test)
# Define Hyperparameters
# specify parameters and distributions to sample from
from scipy.stats import randint as sp_randint
param_dist = {'neurons': sp_randint(300, 360), #, 175, 180, 185, 190, 195, 200],
'learn_rate': sp_randint (0.001, 0.01),
'batch_size': sp_randint(50, 60),
'epochs': sp_randint(20, 30),
'dropout_rate': sp_randint(0.2, 0.8),
'weight_constraint': sp_randint(3, 8)
}
# run randomized search
n_iter_search = 100
print("[INFO] Starting training digits")
print("[INFO] Tuning hyper-parameters for accuracy")
grid = RandomizedSearchCV(estimator=model, param_distributions=param_dist,
n_iter=n_iter_search, n_jobs=10, cv=kfold)
start = time.time()
grid_result = grid.fit(X_train, Y_train)
print("[INFO] GridSearch took {:.2f} seconds".format(time.time() - start))
My answer:
[INFO] GridSearch took 1164.39 seconds
[INFO] GridSearch best score 1.000000 using parameters: {'batch_size': 54, 'dropout_rate': 0, 'epochs': 20, 'learn_rate': 0, 'neurons': 331, 'weight_constraint': 7}
[INFO] Grid scores on development set:
0.614679 (0.034327) with: {'batch_size': 54, 'dropout_rate': 0, 'epochs': 29, 'learn_rate': 0, 'neurons': 354, 'weight_constraint': 6}
0.883792 (0.008650) with: {'batch_size': 53, 'dropout_rate': 0, 'epochs': 27, 'learn_rate': 0, 'neurons': 339, 'weight_constraint': 7}
0.256881 (0.012974) with: {'batch_size': 59, 'dropout_rate': 0, 'epochs': 27, 'learn_rate': 0, 'neurons': 308, 'weight_constraint': 4}
...
Thanks for helping.
0.2 and 0.8 are not integers, so when you use sp_randint(0.2, 0.8), these are converted to integers so its the same as sp_randint(0, 0). You have to use an equivalent function that generates floating point numbers, not integers.
For example, you can use a uniform distribution (uniform from scipy.stats) to generate real numbers.
I'm trying to read a .csv file with Octave (I suppose it's equivalent on Matlab). One of the columns contains hexadecimal values identifying MAC addresses, but I'd like to have it parsed anyway, I don't mind if it's converted to decimal.
Is it possible to do this automatically with functions such as dlmread? Or do I have to create a custom function?
This is how the file looks like:
Timestamp, MAC, LastBsn, PRR, RSSI, ED, SQI, RxGain, PtxCoord, Channel: 26
759, 0x35c8cc, 127, 99, -307, 29, 237, 200, -32
834, 0x32d710, 183, 100, -300, 55, 248, 200, -32
901, 0x35c8cc, 227, 100, -300, 29, 238, 200, -32
979, 0x32d6a0, 22, 95, -336, 10, 171, 200, -32
987, 0x32d710, 27, 96, -328, 54, 249, 200, -32
1054, 0x35c8cc, 71, 92, -357, 30, 239, 200, -32
1133, 0x32d6a0, 122, 95, -336, 11, 188, 200, -32
I can accept any output value for the (truncated) MAC addresses, from sequence numbers (1-6) to decimal conversion of the value (e.g. 0x35c8cc -> 3524812).
My current workaround is to use a text editor to manually replace the MAC addresses with decimal numbers, but an automated solution would be handy.
The functions dlmread and csvread will handle numeric files. You can use textscan (which is also present in Matlab), but since you're using Octave, you're better off using csv2cell (part of Octave's io package). It basically reads a csv file and returns a cell array of strings and doubles:
octave-3.8.1> type test.csv
1,2,3,"some",1c:6f:65:90:6b:13
4,5,6,"text",0d:5a:89:46:5c:70
octave-3.8.1> plg load io; # csv2cell is part of the io package
octave-3.8.1> data = csv2cell ("test.csv")
data =
{
[1,1] = 1
[2,1] = 4
[1,2] = 2
[2,2] = 5
[1,3] = 3
[2,3] = 6
[1,4] = some
[2,4] = text
[1,5] = 1c:6f:65:90:6b:13
[2,5] = 0d:5a:89:46:5c:70
}
octave-3.8.1> class (data{1})
ans = double
octave-3.8.1> class (data{9})
ans = char
>> type mycsv.csv
Timestamp, MAC, LastBsn, PRR, RSSI, ED, SQI, RxGain, PtxCoord, Channel: 26
759, 0x35c8cc, 127, 99, -307, 29, 237, 200, -32
834, 0x32d710, 183, 100, -300, 55, 248, 200, -32
901, 0x35c8cc, 227, 100, -300, 29, 238, 200, -32
979, 0x32d6a0, 22, 95, -336, 10, 171, 200, -32
987, 0x32d710, 27, 96, -328, 54, 249, 200, -32
1054, 0x35c8cc, 71, 92, -357, 30, 239, 200, -32
1133, 0x32d6a0, 122, 95, -336, 11, 188, 200, -32
You can read the file with csv2cell. The values starting with "0x" will be automatically converted from hex to decimal values. See:
>> pkg load io % load io package for csv2cell
>> data = csv2cell ("mycsv.csv");
>> data(2,1)
ans =
{
[1,1] = 759
}
To access the cell values use:
>> data{2,1}
ans = 759
>> data{2,2}
ans = 3524812
>> data{2,5}
ans = -307