Annotating a corpus using Syntaxnet - corpus

I am trying to annotate a corpus using Syntaxnet. I added the following lines in the end of the /models/syntaxnet/syntaxnet/models/parsey_mcparseface/context.pbtxt file:
input {
name: 'input_file'
record_format: 'english-text'
Part {
file_pattern: '/home/melvyn/text.txt'
}
}
output {
name: 'output_file'
record_format: 'english-text'
Part {
file_pattern: '/home/melvyn/text-tagged.txt'
}
}
When i run the command:
./demo.sh --input=input_file --output=output_file
I am getting:
./demo.sh: line 31: bazel-bin/syntaxnet/parser_eval: No such file or directory
./demo.sh: line 43: bazel-bin/syntaxnet/parser_eval: No such file or directory
./demo.sh: line 55: bazel-bin/syntaxnet/conll2tree: No such file or directory
According to the answer given ## here ## I changed my demo.sh file and now I get some errors which say:
[libprotobuf ERROR external/tf/google/protobuf/src/google/protobuf/text_format.cc:291] Error parsing text-format syntaxnet.TaskSpec: 200:8: Message type "syntaxnet.TaskOutput" has no field named "Part".
E external/tf/tensorflow/core/framework/op_segment.cc:53] Create kernel failed: Invalid argument: Could not parse task context at syntaxnet/models/parsey_mcparseface/context.pbtxt
E external/tf/tensorflow/core/common_runtime/executor.cc:333] Executor failed to create kernel. Invalid argument: Could not parse task context at syntaxnet/models/parsey_mcparseface/context.pbtxt
[[Node: DocumentSource = DocumentSourcebatch_size=32, corpus_name="stdin-conll", task_context="syntaxnet/models/parsey_mcparseface/context.pbtxt", _device="/job:localhost/replica:0/task:0/cpu:0"]]
What could be a possible solution?

Though it's not certain but I think you are not running the shell script from the root directory. Please try running it as per the instructions mentioned here
I hope it helps.

Related

Adjust wait_time in locust using custom arguments via CLI

I want to adjust wait_time parameter by passing it via CLI.
I have tried the following way:
custom_wait_time = None
# Add custom argument to locust
#events.init_command_line_parser.add_listener
def init_parser(parser):
parser.add_argument("--locust-wait-time",
type=int,
include_in_web_ui=True,
default=None,
help="Wait time per each request of a user.")
#events.init.add_listener
def _(environment, **kwargs):
global custom_wait_time
custom_wait_time = int(environment.parsed_options.locust_wait_time)
print(custom_wait_time) # First print
class MyUser(HttpUser):
global custom_wait_time
print(custom_wait_time) # Second print
wait_time = constant(custom_wait_time)
Assume that custom_wait_time=10 when I pass it via CLI, the First print gives me custom_wait_time=10 while the Second print gives me custom_wait_time=None instead of 10, so the wait_time = constant(custom_wait_time) will break and give me the error below:
Traceback (most recent call last):
File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/locust/user/users.py", line 176, in run_user
user.run()
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/locust/user/users.py", line 144, in run
self._taskset_instance.run()
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/locust/user/task.py", line 367, in run
self.wait()
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/locust/user/task.py", line 445, in wait
self._sleep(self.wait_time())
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/locust/user/task.py", line 451, in _sleep
gevent.sleep(seconds)
File "/Users/opt/anaconda3/envs/ai/lib/python3.7/site-packages/gevent/hub.py", line 157, in sleep
if seconds <= 0:
TypeError: '<=' not supported between instances of 'NoneType' and 'int'
Any help would be appreciated.
The problem is that the code will run in the wrong order - MyUser is defined before any of the init-methods are called.
If you instead do MyUser.wait_time = constant(custom_wait_time) inside your init handler (and dont set it at all in the class) it should work.
That way you dont need any globals either :)
I've just do the same work.
#events.init_command_line_parser.add_listener
def _(parser):
parser.add_argument("--waitTime", type=float, env_var="LOCUST_WAIT_TIME", default=1.0, help="wait time between each task of an user")
# Set `include_in_web_ui` to False if you want to hide from the web UI
#parser.add_argument("--my-ui-invisible-argument", include_in_web_ui=False, default="I am invisible")
and in the test class, just use it value like this
class GetInfoUser(HttpUser):
def wait_time(self):
return self.environment.parsed_options.waitTime

Adding neovim to home-manager config leads to permission problem

When adding neovim to my home-manager config (nix on MacOs), like so
programs.neovim = {
enable = true;
package = pkgs.neovim;
viAlias = true;
extraPackages = [
pkgs.rnix-lsp
];
};
I get the following error
these 5 derivations will be built:
/nix/store/7z24wy5ccgjhkw8zc39926z5r0pqfd59-neovim-0.8.0.drv
/nix/store/18kwwzrpsqaa6whyz3shlmidz0lp5sx5-home-manager-fonts.drv
/nix/store/zq40m1z7bl7azgay2gd39c769j5ypszd-home-manager-path.drv
/nix/store/37hvbw396bp6j89grhyqqxb772vbj6j4-activation-script.drv
/nix/store/dghqgkn78cpzq2fwclbg1rkjybwg7dr4-home-manager-generation.drv
building '/nix/store/7z24wy5ccgjhkw8zc39926z5r0pqfd59-neovim-0.8.0.drv'...
/nix/store/lig4bdiwsr9lfb8x8plpnasg95vrw98q-hook/nix-support/setup-hook: line 115: /nix/store/42yqjb325cpgs1gycv3sk9dki1y0qli0-neovim-0.8.0/bin/nvim-python3: Permission denied
error: builder for '/nix/store/7z24wy5ccgjhkw8zc39926z5r0pqfd59-neovim-0.8.0.drv' failed with exit code 1;
last 1 log lines:
> /nix/store/lig4bdiwsr9lfb8x8plpnasg95vrw98q-hook/nix-support/setup-hook: line 115: /nix/store/42yqjb325cpgs1gycv3sk9dki1y0qli0-neovim-0.8.0/bin/nvim-python3: Permission denied
For full logs, run 'nix log /nix/store/7z24wy5ccgjhkw8zc39926z5r0pqfd59-neovim-0.8.0.drv'.
error: 1 dependencies of derivation '/nix/store/18kwwzrpsqaa6whyz3shlmidz0lp5sx5-home-manager-fonts.drv' failed to build
error: 1 dependencies of derivation '/nix/store/zq40m1z7bl7azgay2gd39c769j5ypszd-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dghqgkn78cpzq2fwclbg1rkjybwg7dr4-home-manager-generation.drv' failed to build
Any ideas what might cause this?
package takes as input an unwrapped version of neovim (without the python/nodeā€¦ wrapper since home-manager does the wrapping itself) as you can see here. Since this value is anyway set by default you can just remove it:
programs.neovim = {
enable = true;
viAlias = true;
extraPackages = [
pkgs.rnix-lsp
];
};

Yocto variable not defined but set with _ operator

I'm struggling with something I'm not sure to address correctly.
In a Yocto environment (for STM32MP1 by the way) I have to configure a new target.
Hence I added to meta-st/meta-st-stm32mp/conf/machine/include/st-machine-extlinux-config-stm32mp.inc this section, that looks like the other already available:
EXTLINUX_BOOTDEVICE_EMMC = "mmc1"
EXTLINUX_BOOTDEVICE_SDCARD = "mmc0"
EXTLINUX_ROOT_EMMC = "${#bb.utils.contains('ST_VENDORFS','1','root=/dev/mmcblk1p4','root=/dev/mmcblk1p3',d)}"
EXTLINUX_ROOT_NAND = "ubi.mtd=UBI rootfstype=ubifs root=ubi0:rootfs"
# Define available targets to use
UBOOT_EXTLINUX_CONFIGURED_TARGETS += "mp151a_sdcard"
UBOOT_EXTLINUX_CONFIGURED_TARGETS += "mp151a_emmc"
# Define bootprefix for each target
UBOOT_EXTLINUX_BOOTPREFIXES_mp151a_sdcard = "${EXTLINUX_BOOTDEVICE_SDCARD}_stm32mp151a_"
UBOOT_EXTLINUX_BOOTPREFIXES_mp151a_emcc = "${EXTLINUX_BOOTDEVICE_EMCC}_stm32mp151a_"
# Define labels for each target
UBOOT_EXTLINUX_LABELS_mp151a_sdcard = "stm32mp151a-sdcard"
UBOOT_EXTLINUX_LABELS_mp151a_emcc = "stm32mp151a-emcc"
# Define default boot config for each target
UBOOT_EXTLINUX_DEFAULT_LABEL_mp151a_sdcard ?= "stm32mp151a-sdcard"
UBOOT_EXTLINUX_DEFAULT_LABEL_mp151a_emcc ?= "stm32mp151a-emcc"
# Define FDT overrides for all labels
UBOOT_EXTLINUX_FDT_stm32mp151a-sdcard = "/stm32mp151a.dtb"
UBOOT_EXTLINUX_FDT_stm32mp151a-emcc = "/stm32mp151a.dtb"
# Define ROOT overrides for all labels
UBOOT_EXTLINUX_ROOT_stm32mp151a-sdcard = "${EXTLINUX_ROOT_SDCARD}"
UBOOT_EXTLINUX_ROOT_stm32mp151a-emcc = "${EXTLINUX_ROOT_EMCC}"
But when I bitbake <image> (that includes the file above) I get this output:
DEBUG: Executing python function update_extlinuxconf_targets
NOTE: UBOOT_EXTLINUX_CONFIGURED_TARGETS: mp157a-dk1_sdcard mp157a-dk1_sdcard-optee mp157c-dk2_sdcard mp157c-dk2_sdcard-optee mp157c-ed1_emmc mp157c-ed1_emmc-optee mp157c-ed1_sdcard mp157c-ed1_sdcard-optee mp157c-ev1_emmc mp157c-ev1_emmc-optee mp157c-ev1_nand mp157c-ev1_nor-sdcard mp157c-ev1_nor-emmc mp157c-ev1_sdcard mp157c-ev1_sdcard-optee mp151a_sdcard mp151a_emmc
NOTE: UBOOT_EXTLINUX_CONFIG_FLAGS: emmc sdcard
NOTE: *** Loop for config_label: emmc
NOTE: *** Loop for devicetree: stm32mp151a
NOTE: >>> New target label: mp151a_emmc
NOTE: >>> Append mp151a_emmc to UBOOT_EXTLINUX_TARGETS
NOTE: *** Loop for config_label: sdcard
NOTE: *** Loop for devicetree: stm32mp151a
NOTE: >>> New target label: mp151a_sdcard
NOTE: >>> Append mp151a_sdcard to UBOOT_EXTLINUX_TARGETS
NOTE: >>> UBOOT_EXTLINUX_TARGETS (updated): mp151a_emmc mp151a_sdcard
DEBUG: Python function update_extlinuxconf_targets finished
DEBUG: Executing python function do_create_multiextlinux_config
ERROR: UBOOT_EXTLINUX_ROOT not defined
DEBUG: Python function do_create_multiextlinux_config finished
ERROR: Function failed: do_create_multiextlinux_config
As you can see, the file is actually processed because it added the targets I've defined.
But it doesn't find the UBOOT_EXTLINUX_ROOT even if it's "set" with the _ operator:
UBOOT_EXTLINUX_ROOT_stm32mp151a-sdcard = "${EXTLINUX_ROOT_SDCARD}"
UBOOT_EXTLINUX_ROOT_stm32mp151a-emcc = "${EXTLINUX_ROOT_EMCC}"
I also tried to set the main variable to something like:
UBOOT_EXTLINUX_ROOT = ""
or
UBOOT_EXTLINUX_ROOT = "root=/dev/mmcblk1p4"
to see if it was the problem but it doesn't change nothing.
Is this something related to Yocto itself (I mean, something wrong in my syntax) or it's very specific to the SDK (meta-st) ?
The error above should be raised by this file:
root = localdata.getVar('UBOOT_EXTLINUX_ROOT')
if not root:
bb.fatal('UBOOT_EXTLINUX_ROOT not defined')
UPDATE
I checked the (huge) output of bitbake -e and among other targets I see:
# $UBOOT_EXTLINUX_ROOT [41 operations]
[...]
# "${EXTLINUX_ROOT_NOREMMC}"
# override[stm32mp157c-ev1-m4-examples-sdcard]:set /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/meta-st/meta-st-stm32mp/conf/machine/include/st-machine-extlinux-config-stm32mp.inc:274
# "${EXTLINUX_ROOT_SDCARD}"
# override[stm32mp157c-ev1-m4-examples-sdcard-optee]:set /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/meta-st/meta-st-stm32mp/conf/machine/include/st-machine-extlinux-config-stm32mp.inc:275
# "${EXTLINUX_ROOT_SDCARD_OPTEE}"
# override[stm32mp151a-sdcard]:set /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/meta-st/meta-st-stm32mp/conf/machine/include/st-machine-extlinux-config-stm32mp.inc:296
# "${EXTLINUX_ROOT_SDCARD}"
# override[stm32mp151a-emcc]:set /local/STM32MP15-Ecosystem-v1.1.0/Distribution-Package/openstlinux-4.19-thud-mp1-19-10-09/layers/meta-st/meta-st-stm32mp/conf/machine/include/st-machine-extlinux-config-stm32mp.inc:297
[...]
# pre-expansion value:
# ""
UBOOT_EXTLINUX_ROOT=""
# $UBOOT_EXTLINUX_ROOT_cubemx-nor-sdcard
UBOOT_EXTLINUX_ROOT_cubemx-nor-sdcard="root=/dev/mmcblk0p3"
# $UBOOT_EXTLINUX_ROOT_cubemx-sdcard
UBOOT_EXTLINUX_ROOT_cubemx-sdcard="root=/dev/mmcblk0p6"
# $UBOOT_EXTLINUX_ROOT_stm32mp151a-emcc
UBOOT_EXTLINUX_ROOT_stm32mp151a-emcc="\${EXTLINUX_ROOT_EMCC}"
# $UBOOT_EXTLINUX_ROOT_stm32mp151a-sdcard
UBOOT_EXTLINUX_ROOT_stm32mp151a-sdcard="root=/dev/mmcblk0p6"
So far, if I understand correctly, the override values are correctly assigned (but not the ${EXTLINUX_ROOT_EMCC} - I don't understand where the \ comes from) but the main variable is still empty.
Adding UBOOT_EXTLINUX_ROOT = "root=/dev/mmcblk1p4" at the beginning of the above file, seems to do the trick (even if before I wrote the opposite, perhaps I forgot to clear the cache?) but I don't think it's the right way to do it.
You should specify the wanted name of the machine as a target to build, i.e.:
MACHINE=stm32mp151a-sdcard bitbake <image>
This way, the UBOOT_EXTLINUX_ROOT gets the non-empty value "root=/dev/mmcblk0p6" (from the UBOOT_EXTLINUX_ROOT_stm32mp151a-sdcard variant of the variable).

Errors in codeigniter-restserver library

I want to use restful in my ci 3.03 application:
I found this tutplus tutorial
I downloaded codeigniter-restserver-master.zip file and copied Format.php and REST_Controller.php(#version 3.0.0) files into /application/libraries/REST directory
I created control application/controllers/api/Users.php :
require_once("application/libraries/REST/REST_Controller.php");
require_once("application/libraries/REST/Format.php");
class Users extends REST_Controller
{
//protected $rest_format = 'json';
function users_get()
{
//$users = $this->user_model->get_all();
$filter_username= $this->get('filter_username');
$filter_user_group= $this->get('filter_user_group');
$filter_active= $this->get('filter_active');
$sort= $this->get('sort');
$sort_direction= $this->get('sort_direction');
//, $filter_user_group, $filter_active, $sort, $sort_direction
$users_list = $this->muser->getUsersList(false, ''/*, $filter_username, $filter_user_group, $filter_active, $sort, $sort_direction, ''*/);
echo '<pre>'.count($users_list).'::$users_lists::'.print_r($users_list,true).'</pre>';
if($users_list)
{
$this->response($users, 200);
}
else
{
$this->response(NULL, 404);
}
}
AND RUNNING URL http://local-ci3.com/api/users I got many errors:
A PHP Error was encountered
Severity: Notice
Message: Undefined property: Users::$format
Filename: REST/REST_Controller.php
Line Number: 734
Backtrace:
File: /mnt/diskD_Work/wwwroot/ci3/application/libraries/REST/REST_Controller.php
Line: 734
Function: _error_handler
File: /mnt/diskD_Work/wwwroot/ci3/application/libraries/REST/REST_Controller.php
Line: 649
Function: response
File: /mnt/diskD_Work/wwwroot/ci3/index.php
Line: 292
Function: require_once
A PHP Error was encountered
Severity: Notice
Message: Undefined property: Users::$format
Filename: REST/REST_Controller.php
Line Number: 752
Backtrace:
File: /mnt/diskD_Work/wwwroot/ci3/application/libraries/REST/REST_Controller.php
Line: 752
Function: _error_handler
File: /mnt/diskD_Work/wwwroot/ci3/application/libraries/REST/REST_Controller.php
Line: 649
Function: response
File: /mnt/diskD_Work/wwwroot/ci3/index.php
Line: 292
Function: require_once
Actually I wanted to get some workable library to help me with REST api creation. I think that is preferable way istead of making from zero.
But is this library not workable or does it needs for some fixing? Sorry, what I missed is if this library only for ci 2?
I made search on this forum and found such hint :
I have the same problem when I load both Format.php and
Rest_Controller.php into a controller. After have a quick glance at
Format.php, it appears to be a standalone format conversion helper.
Try to just load Rest_Controller.php and see if your problem goes
away.
I commented line
//require_once("application/libraries/REST/Format.php");
in my controller, but I still get errors like :
Message: Undefined property: Users::$format.
I tried to review code of this library and see that invalid block when data are converted to json format, line 731-757 :
elseif ($data !== NULL)
{
// If the format method exists, call and return the output in that format
if (method_exists($this->format, 'to_' . $this->response->format))
{
// Set the format header
$this->output->set_content_type($this->_supported_formats[$this->response->format], strtolower($this->config->item('charset')));
$output = $this->format->factory($data)->{'to_' . $this->response->format}();
// An array must be parsed as a string, so as not to cause an array to string error
// Json is the most appropriate form for such a datatype
if ($this->response->format === 'array')
{
$output = $this->format->factory($output)->{'to_json'}();
}
}
else
{
// If an array or object, then parse as a json, so as to be a 'string'
if (is_array($data) || is_object($data))
{
$data = $this->format->factory($data)->{'to_json'}();
}
// Format is not supported, so output the raw data as a string
$output = $data;
}
}
If I tried to commented this block, but get error
Message: Array to string conversion
Looks like data are not converted in this case...
Is is possible to fix these errors?
Or can you, please, to tell me advice some codeigniter 3 REST api workable library with similar interface like library above?
Thanks!
I use that lib, work just fine. My suggestion is follow the more relevant installation instruction on github .
you also wrong place the lib file :
Tutorial say :
require(APPPATH'.libraries/REST_Controller.php');
You try :
require_once("application/libraries/REST/REST_Controller.php");
require_once("application/libraries/REST/Format.php");
No need to include the format because on line 407 the lib will load it. And also good to know on line 404 it will load the configuration (application/config/rest.php) it will be your default configuration, and also you can change it to suit your need.
Please let me know if you still got error using my answer :)

flickr.auth.getFrob by tcl (REST)

I'm trying to get response from Flickr by using Flickr API but I have no idea and no examples in TCL for that.
I wrote the following code:
#!/usr/bin/tclsh
package require rest
set flickr(auth.getFrob) {
url http://api.flickr.com/services/rest/
req_args { api_key: }
}
rest::create_interface flickr
puts [flickr::auth::getFrob -api_key ea4a4134e2821898e5e31713d2ad74fd ]
When I execute it I get this error:
invalid command name "flickr::auth::getFrob"
while executing
"flickr::auth::getFrob -api_key ea4a4134e2821898e5e31713d2ad74fd "
invoked from within
"puts [flickr::auth::getFrob -api_key ea4a4134e2821898e5e31713d2ad74fd ]"
(file "./flickr.tcl" line 17)
=====================================
I've updated the last line of the code as proposed by Johannes to:
puts [flickr::auth.getFrob -api_key ea4a4134e2821898e5e31713d2ad74fd ]
but still got the strange response:
rsp {stat fail} {{err {code 112 msg {Method "unknown" not found}} {}}}
when supposed something like:
<frob>746563215463214621</frob>
as described in Flickr API help:
auth.getFrob
It looks like the command name is
::flickr::auth.getFrob