i would like to fine tune A BERT model with my own labels, like [COLOR, MATERIAL] and not the normal "NAME", "ORG".
I'm following this Colab: https://colab.research.google.com/drive/14rYdqGAXJhwVzslXT4XIwNFBwkmBWdVV
I prepared train.txt, eval.txt, test.txt like this:
-DOCSTART- -X- -X- O
blue B-COLOR
motorcicle B-CATEGORY
steel B-MATERIAL
etc.
But whene i execute this command
!python run_ner.py --data_dir=data/ --bert_model=bert-base-multilingual-cased --task_name=ner --output_dir=out_ner --max_seq_length=128 --do_train --num_train_epochs 5 --do_eval --warmup_proportion=0.1
i get this error
06/08/2020 13:30:27 - INFO - pytorch_transformers.modeling_utils - loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-pytorch_model.bin from cache at /root/.cache/torch/pytorch_transformers/5b5b80054cd2c95a946a8e0ce0b93f56326dff9fbda6a6c3e02de3c91c918342.7131dcb754361639a7d5526985f880879c9bfd144b65a0bf50590bddb7de9059
06/08/2020 13:30:33 - INFO - pytorch_transformers.modeling_utils - Weights of Ner not initialized from pretrained model: ['classifier.weight', 'classifier.bias']
06/08/2020 13:30:33 - INFO - pytorch_transformers.modeling_utils - Weights from pretrained model not used in Ner: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
File "run_ner.py", line 594, in
main()
File "run_ner.py", line 464, in main
train_examples, label_list, args.max_seq_length, tokenizer)
File "run_ner.py", line 210, in convert_examples_to_features
label_ids.append(label_map[labels[i]])
KeyError: 'B-COLOR'
Did i create wrongly train.txt file?
add these label to the get_labels() metod in the run_ner.py file, it will work
Did you change labels name in this file?
Related
Hello everyone I am working on CIFAR10 dataset using pytorch. I have developed a model which works absolutely fine but the main problem occurrs while runing the following code:
import time
start_time=time.time()
epochs=5
train_losses=[]
test_losses=[]
train_correct=[]
test_correct=[]
for i in range(epochs):
tsn_corr=0
tst_corr=0
for b, (X_train,y_train) in enumerate(train_loader):
b+=1
y_pred=model(X_train)
loss=criterion(y_pred,y_train)
#Tally the number of correct predictions
predicted= torch.max(y_pred.data, 1)[1]
batch_corr=(predicted==y_train).sum()
tsn_corr += batch_corr
#optimize paramters
optimizer.zero_grad()
loss.backward()
optimizer.step()
#print interim results
if b%600 == 0:
print(f"epochs: {i}, batch: {b}, loss: {loss.item():10.8f}")
loss=loss.detach().numpy()
train_losses.append(loss)
train_correct.append(tsn_corr)
#Running the test_batches
with torch.no_grad():
for b, (X_test,y_test) in enumerate(test_loader):
b+=1
y_val=model(X_test)
#TALLY THE NUMBER OF CORRECT PREDICTIONS
predicted=torch.max(y_val.data, 1)[1]
batch_corr= (predicted==y_test).sum()
tst_corr += batch_corr
loss=criterion(y_val,y_test)
loss=loss.detach().numpy()
test_losses.append(loss)
test_correct.append(tst_corr)
the following error occurrs while running the following code:
NotImplementedError Traceback (most recent call last)
<ipython-input-43-48e21e83e9f7> in <module>
15 b+=1
16
---> 17 y_pred=model(X_train)
18 loss=criterion(y_pred,y_train)
19
~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _call_impl(self, *input, **kwargs)
887 result = self._slow_forward(*input, **kwargs)
888 else:
--> 889 result = self.forward(*input, **kwargs)
890 for hook in itertools.chain(
891 _global_forward_hooks.values(),
~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in _forward_unimplemented(self, *input)
199 registered hooks while the latter silently ignores them.
200 """
--> 201 raise NotImplementedError
202
203
NotImplementedError:
Can someone tell me what can I do to fix this code. Apart from this previous all codes work fine and the model which I made using ConvolutionalNeural-Networks also runs successfully meaning that there is no problem with the model. I guess this detail might help. It might be noted that this code works just fine on MNIST dataset. I dont know what is the problem with CIFAR datasets
Your model class needs to implement a forward method. See the PyTorch Example on Subclassing to see an example.
I am using tensorRT to perform inference with CUDA. I'd like to use CuPy to preprocess some images that I'll feed to the tensorRT engine. The preprocessing function, called my_function, works fine as long as tensorRT is not run between different calls of the my_function method (see code below). Specifically, the issue is not strictly related by tensorRT but by the fact that tensorRT inference requires to be wrapped by push and pop operations of the pycuda context.
With respect to the following code, the last execution of my_function will raise the following error:
File "/home/ubuntu/myfile.py", line 188, in _pre_process_cuda
img = ndimage.zoom(img, scaling_factor)
File "/home/ubuntu/.local/lib/python3.6/site-packages/cupyx/scipy/ndimage/interpolation.py", line 482, in zoom
kern(input, zoom, output)
File "cupy/core/_kernel.pyx", line 822, in cupy.core._kernel.ElementwiseKernel.__call__
File "cupy/cuda/function.pyx", line 196, in cupy.cuda.function.Function.linear_launch
File "cupy/cuda/function.pyx", line 164, in cupy.cuda.function._launch
File "cupy_backends/cuda/api/driver.pyx", line 299, in cupy_backends.cuda.api.driver.launchKernel
File "cupy_backends/cuda/api/driver.pyx", line 124, in cupy_backends.cuda.api.driver.check_status
cupy_backends.cuda.api.driver.CUDADriverError: CUDA_ERROR_INVALID_HANDLE: invalid resource handle
Note: in the following code I haven't reported the entire tensorRT inference code. In fact, simply pushing and popping a pycuda context generates the error
Code:
import numpy as np
import cv2
import time
from PIL import Image
import requests
from io import BytesIO
from matplotlib import pyplot as plt
import cupy as cp
from cupyx.scipy import ndimage
import tensorrt as trt
import pycuda.driver as cuda
import pycuda.autoinit
def my_function(numpy_frame):
dtype = 'float32'
img = cp.array(numpy_frame, dtype='float32')
# print(img)
img = ndimage.zoom(img, (0.5, 0.5, 3))
img = (cp.array(2, dtype=dtype) / cp.array(255, dtype=dtype)) * img - cp.array(1, dtype=dtype)
img = img.transpose((2, 0, 1))
img = img.ravel()
return img
# load image
url = "https://www.pexels.com/photo/109919/download/?search_query=&tracking_id=411xe21veam"
response = requests.get(url)
img = Image.open(BytesIO(response.content))
img = np.array(img)
# initialize tensorrt
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt_runtime = trt.Runtime(TRT_LOGGER)
cfx = cuda.Device(0).make_context()
my_function(img) # ok
my_function(img) # ok
# ----- TENSORRT ---------
cfx.push()
# .... tensorrt inference....
cfx.pop()
# ----- TENSORRT ---------
my_function(img) # <---- error
I even tried to do it other ways, but unfortunately with the same result:
cfx.push()
my_function(img) # ok
cfx.pop()
cfx.push()
my_function(img) # error
cfx.pop()
#admin: if you can think of a better name for this question feel free to edit it :)
There were multiple contexts open. For instance, it seems that all of the following open a context:
import pycuda.autoinit
cfx.cuda.Device(0).make_context()
cfx.push()
So if you run the three command above, then simply running one cfx.pop() won't be enough. You will need to run cfx.pop() three times to pop all the contexts.
Im trying to train my own dataset on SegNet (with caffe), I prepared the dataset same as segnet tutorial. when I try to run the train, it shows me this error:
I0915 08:33:50.851986 49060 net.cpp:482] Collecting Learning Rate and Weight Decay.
I0915 08:33:50.852017 49060 net.cpp:247] Network initialization done.
I0915 08:33:50.852030 49060 net.cpp:248] Memory required for data: 1064448016
I0915 08:33:50.852730 49060 solver.cpp:42] Solver scaffolding done.
I0915 08:33:50.853065 49060 solver.cpp:250] Solving VGG_ILSVRC_16_layer
I0915 08:33:50.853080 49060 solver.cpp:251] Learning Rate Policy: step
F0915 08:33:51.324506 49060 math_functions.cu:123] Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0) CUBLAS_STATUS_MAPPING_ERROR
*** Check failure stack trace: ***
# 0x7fa27a0d3daa (unknown)
# 0x7fa27a0d3ce4 (unknown)
# 0x7fa27a0d36e6 (unknown)
# 0x7fa27a0d6687 (unknown)
# 0x7fa27a56946e caffe::caffe_gpu_asum<>()
# 0x7fa27a54b264 caffe::SoftmaxWithLossLayer<>::Forward_gpu()
# 0x7fa27a440b29 caffe::Net<>::ForwardFromTo()
# 0x7fa27a440f57 caffe::Net<>::ForwardPrefilled()
# 0x7fa27a436745 caffe::Solver<>::Step()
# 0x7fa27a43707f caffe::Solver<>::Solve()
# 0x406676 train()
# 0x404bb1 main
# 0x7fa2795e5f45 (unknown)
# 0x40515d (unknown)
# (nil) (unknown)
my dataset is .jpg (train) .png (labels gray-scale images) and .txt file as in the tutorial. what can be the problem? thanks for helping
The ground-truth images should be 1 channel 0-255 images without alpha layer, so the NN will recognise the difference between the classes.
img = Image.open(filename).convert('L') # Not 'LA' (A - alpha)
Thanks to isn4, Here is the resolution:
Turns out you have to change the range of the pixel values as well as the actual number of pixel values. Segnet gets confused if you have 256 possible pixel values (0-255) and don't have class weightings for each of them. So I changed all of my PNG label images from 255 and 0 as the pixel possibilities to 1 and 0 as the pixel possibilities.
Here's my python script for doing so:
import os
import cv2
import numpy as np
img = cv2.imread('/usr/local/project/old_png_labels/label.png, 0)
a_img = np.array(img, np.double)
normalized = cv2.normalize(img, a_img, 1.0, 0.0, cv2.NORM_MINMAX)
cv2.imwrite('/usr/local//project/png_labels/label.png, normalized)
I am trying to find the indices 2d max pooling in lasagne
network = batch_norm(Conv2DLayer(
network, num_filters=filter_size, filter_size=(kernel, kernel),pad=pad,
nonlinearity=lasagne.nonlinearities.rectify,
W=lasagne.init.GlorotUniform(),name="conv"), name="BN")
pool_in = lasagne.layers.get_output(network)
network = MaxPool2DLayer(network, pool_size=(pool_size, pool_size),stride=2,name="pool")
pool_out = lasagne.layers.get_output(network)
ind1 = T.grad(T.sum(pool_out), wrt=pool_in)
When I try to build the model it raises a error
DisconnectedInputError: grad method was asked to compute the gradient with respect to a variable that is not part of the computational graph of the cost, or is used only by a non-differentiable operator: Elemwise{mul,no_inplace}.0
Backtrace when the node is created:
File "//anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2871, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "//anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_ast_nodes
if self.run_code(code, result):
File "//anaconda/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 3035, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-28-0b136cc660e2>", line 1, in <module>
network = build_model()
File "<ipython-input-27-20acc3fe0d98>", line 8, in build_model
pool_in = lasagne.layers.get_output(network)
File "//anaconda/lib/python2.7/site-packages/lasagne/layers/helper.py", line 191, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "//anaconda/lib/python2.7/site-packages/lasagne/layers/special.py", line 52, in get_output_for
return self.nonlinearity(input)
File "//anaconda/lib/python2.7/site-packages/lasagne/nonlinearities.py", line 157, in rectify
return theano.tensor.nnet.relu(x)
What is the right way of coding functions on lasagne layers intermediate outputs.
I had a similar problem a while ago, check out my solution for 2d and 3d max pooling indices:
Theano max_pool_3d
(Its based on the same Google-groups post, i guess)
I'm a beginner in Java and a geomatics student.
I would like to run the gdal unit from QGIS in the DOS command. I managed to open the executable in the command. But the command shows several types of sub folder.
C:\Program Files\QGISWien\bin>"C:\Program Files\QGISWien\bin\gdalwarp.exe"
Usage: gdalwarp [--help-general] [--formats]
[-s_srs srs_def] [-t_srs srs_def] [-to "NAME=VALUE"]
[-order n | -tps | -rpc | -geoloc] [-et err_threshold]
[-refine_gcps tolerance [minimum_gcps]]
[-te xmin ymin xmax ymax] [-tr xres yres] [-tap] [-ts width height]
[-wo "NAME=VALUE"] [-ot Byte/Int16/...] [-wt Byte/Int16]
[-srcnodata "value [value...]"] [-dstnodata "value [value...]"] -dstalpha
[-r resampling_method] [-wm memory_in_mb] [-multi] [-q]
[-cutline datasource] [-cl layer] [-cwhere expression]
[-csql statement] [-cblend dist_in_pixels] [-crop_to_cutline]
[-of format] [-co "NAME=VALUE"]* [-overwrite]
[-nomd] [-cvmd meta_conflict_value]
srcfile* dstfile
And I would like for example to open the [--help-general]
Any Ideas, how can I do that?
Thanks
hey with my knowledge if you have to open a file there is some basic tags you should follow
Opens an existing file and allows you to specify an editor.
"File.OpenFile filename [/e:editorname]"