CWL: Creating input yaml - issue with ruamel comments.py - workflow

I am using cwltool version 3.0.20201203173111 that came with pip install cwl-runner. This might be issue with ruamel or with my workflow but here is run command and traceback:
cwltool --make-template --debug tumor_only_exome.cwl > hiv_input.yml
traceback:
Traceback (most recent call last):
File "/home/coyote/anaconda3/envs/cwl/lib/python3.6/site-packages/cwltool/main.py", line 991, in main
make_template(tool)
File "/home/coyote/anaconda3/envs/cwl/lib/python3.6/site-packages/cwltool/main.py", line 744, in make_template
generate_input_template(tool),
File "/home/coyote/anaconda3/envs/cwl/lib/python3.6/site-packages/cwltool/main.py", line 301, in generate_input_template
template.insert(0, name, value, comment)
File "/home/coyote/anaconda3/envs/cwl/lib/python3.6/site-packages/ruamel/yaml/comments.py", line 722, in insert
self.yaml_add_eol_comment(comment, key=key)
File "/home/coyote/anaconda3/envs/cwl/lib/python3.6/site-packages/ruamel/yaml/comments.py", line 295, in yaml_add_eol_comment
if comment[0] != '#':
IndexError: string index out of range

Related

SQLFluff Lint not working for multilevel subdirectory sql file

we are trying to use SQLFluff in our project to avoid sql parser errors before deployment.
in our case we have subdirectories which contains sql files.
During Development we are running command sqlfluff lint command in root directory, we found that lint command is working for one level subdirectory sql path like below,
sqlfluff lint demo/complexquery.sql --dialect snowflake
But when we try for 2 level subdirectory sql file path, lint command is not working and giving error as below. could you please let me know whether iam missing syntax.
sqlfluff lint SQLScript/demo/complexquery.sql --dialect snowflake
Traceback (most recent call last):
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 395, in loads
value, vtype = decoder.load_value(multilinestr)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 849, in load_value
raise ValueError("Found tokens after a closed " +
ValueError: Found tokens after a closed string. Invalid TOML.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\ar\AppData\Roaming\Python\Python310\Scripts\sqlfluff.exe_main.py", line 7, in
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 549, in lint
config = get_config(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 361, in get_config
return FluffConfig.from_root(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 613, in from_root
c = loader.load_config_up_to_path(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in load_config_up_to_path
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 338, in load_config_at_path
configs = self.load_config_file(p, fname, configs=configs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 305, in load_config_file
elems = self._get_config_elems_from_toml(file_path)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 191, in _get_config_elems_from_toml
config = toml.load(fpath)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 134, in load
return loads(ffile.read(), _dict, decoder)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 397, in loads
raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Found tokens after a closed string. Invalid TOML. (line 54 column 1 char 4192)

Error while executing basic code in Locust

from locust import Locust, TaskSet
def login(l):
print("I am logged In")
def logout(m):
print("I am logged Out")
class UserBehaviour(TaskSet):
task=[login,logout]
class User(Locust):
task_set = UserBehaviour
Error Message---
(venv) C:\pythnprojects\LearnLocustProject\venv\locust_test>locust -f firstlocust.py
[2020-03-11 00:38:57,259] DELLXPS/INFO/locust.main: Starting web monitor at *:8089
[2020-03-11 00:38:57,259] DELLXPS/INFO/locust.main: Starting Locust 0.11.0
[2020-03-11 00:39:05,581] DELLXPS/INFO/locust.runners: Hatching and swarming 1 clients at the rate 1 clients/s...
[2020-03-11 00:39:05,585] DELLXPS/ERROR/stderr: Traceback (most recent call last):
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 358, in run
self.schedule_task(self.get_next_task())
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 419, in get_next_task
return random.choice(self.tasks)
File "C:\DOWNLOADS\lib\random.py", line 290, in choice
raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence
[2020-03-11 00:39:06,582] DELLXPS/INFO/locust.runners: All locusts hatched: User: 1
[2020-03-11 00:39:06,591] DELLXPS/ERROR/stderr: Traceback (most recent call last):
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 358, in run
self.schedule_task(self.get_next_task())
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 419, in get_next_task
return random.choice(self.tasks)
File "C:\DOWNLOADS\lib\random.py", line 290, in choice
raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence
[2020-03-11 00:39:07,597] DELLXPS/ERROR/stderr: Traceback (most recent call last):
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 358, in run
self.schedule_task(self.get_next_task())
File "c:\pythnprojects\learnlocustproject\venv\lib\site-packages\locust\core.py", line 419, in get_next_task
return random.choice(self.tasks)
File "C:\DOWNLOADS\lib\random.py", line 290, in choice
raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence
It looks like you've misspelled tasks (it currently seems to say task).

Failing to install pandas due to an error

I am Failing to install pandas in cmd using pip-install pandas due to the error below that pops up when installing. I am using python version 3.7
Traceback (most recent call last):
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\urllib3\response.py", line 360, in _error_catcher
yield
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\urllib3\response.py", line 442, in read
data = self._fp.read(amt)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\cachecontrol\filewrapper.py", line 62, in read
data = self.__fp.read(amt)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\http\client.py", line 447, in read
n = self.readinto(b)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\http\client.py", line 491, in readinto
n = self.fp.readinto(b)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\socket.py", line 589, in readinto
return self._sock.recv_into(b)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\ssl.py", line 1052, in recv_into
return self.read(nbytes, buffer)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\ssl.py", line 911, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\cli\base_command.py", line 179, in main
status = self.run(options, args)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\commands\install.py", line 315, in run
resolver.resolve(requirement_set)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\resolve.py", line 131, in resolve
self._resolve_one(requirement_set, req)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\resolve.py", line 294, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\resolve.py", line 242, in _get_abstract_dist_for
self.require_hashes
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\operations\prepare.py", line 334, in prepare_linked_requirement
progress_bar=self.progress_bar
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 878, in unpack_url
progress_bar=progress_bar
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 702, in unpack_http_url
progress_bar)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 946, in _download_http_url
_download_url(resp, link, content_file, hashes, progress_bar)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 639, in _download_url
hashes.check_against_chunks(downloaded_chunks)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\utils\hashes.py", line 62, in check_against_chunks
for chunk in chunks:
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 607, in written_chunks
for chunk in chunks:
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\utils\ui.py", line 159, in iter
for x in it:
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_internal\download.py", line 596, in resp_read
decode_content=False):
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\urllib3\response.py", line 494, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\urllib3\response.py", line 459, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "c:\users\chiya\appdata\local\programs\python\python37-32\lib\site-packages\pip\_vendor\urllib3\response.py", line 365, in _error_catcher
raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
I tried updating pip with the command pip install -U pip setuptools
but it showed that pip is up to date.
I tried again pip install fbchat which downloaded successfully but after trying to install pandas with the same "pip install pandas" the error popped up again.**

KeyError: 'found' on Elastic2DocManager when syncing data from MongoDB

From MongoDB to Elastic Search(5.6.5), I sync the database with Mongo-Connector using Elastic2DocManager:
mongo-connector -m localhost:27017 -t localhost:9200 -d elastic2_doc_manager
After seeing some update on docs.deleted of mongodb_meta on Elastic Search:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open mongodb_meta 3wd6OjTT6tD3f6ZGezZw 5 1 1337173 8372 192.9mb 192.9mb
mongo-connector stops working with below error:
2018-07-11 07:16:41,977 [WARNING] elasticsearch:97 - POST http://localhost:9200/_bulk [status:N/A request:10.003s]
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "c:\programdata\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
httplib_response = conn.getresponse()
File "c:\programdata\anaconda3\lib\http\client.py", line 1331, in getresponse
response.begin()
File "c:\programdata\anaconda3\lib\http\client.py", line 297, in begin
version, status, reason = self._read_status()
File "c:\programdata\anaconda3\lib\http\client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "c:\programdata\anaconda3\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
...
Exception in thread Thread-1:
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 150, in run
self._docman.send_buffered_operations()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 482, in send_buffered_operations
action_buffer = self.BulkBuffer.get_buffer()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 696, in get_buffer
self.update_sources()
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\util.py", line 35, in wrapped
return f(*args, **kwargs)
File "c:\programdata\anaconda3\lib\site-packages\mongo_connector\doc_managers\elastic2_doc_manager.py", line 628, in update_sources
if ES_doc['found']:
KeyError: 'found'
What is the reason for this error?
Make sure the versions are compatible.
The required python packages are installed, this is my requirements.txt look like.
astroid==1.6.5
autopep8==1.3.5
certifi==2018.8.24
elasticsearch==6.3.1
elasticsearch-dsl==6.2.1
isort==4.3.4
lazy-object-proxy==1.3.1
mccabe==0.6.1
pycodestyle==2.4.0
pylint==1.9.2
pymongo==3.7.1
rope==0.11.0
six==1.11.0
urllib3==1.23
wrapt==1.10.11

django pipeline Error: The file build/fonts/glyphicons-halflings-regular.eot' could not be found with <pipeline.storage.PipelineCachedStorage object

I am using django-pipeline, On running "sudo python manage.py collectstatic"
Getting this error:
Traceback (most recent call last):
File "manage.py", line 10, in <module> execute_from_command_line(sys.argv)
File "/Users/office/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line utility.execute()
File "/Users/office/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/office/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv self.execute(*args, **cmd_options)
File "/Users/office/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute output = self.handle(*args, **options)
File "/Users/office/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 168, in handle collected = self.collect()
File "/Users/office/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect raise processed ValueError: The file 'bower_components/eonasdan-bootstrap-datetimepicker/build/fonts/glyphicons-halflings-regular.eot' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x10d274e10>.
Any idea how to fix this ?
Using django-pipeline-forgiving(https://pypi.python.org/pypi/django-pipeline-forgiving) solved the issue.
pip install django-pipeline-forgiving
Set in your settings.py: STATICFILES_STORAGE = 'django_pipeline_forgiving.storages.PipelineForgivingStorage'