Error converting Ipython Notebook - ipython

I have two ipynb files in two folders. Once converts no problem, the other wont convert. Am I doing something wrong? I am converting to html
File 1:
cd C:\Users\rcreedon\Dropbox\GIZSupervisor\DATA\Production_Data\STP_Data\Data_Sets\Wave1\1004
ipython nbconvert 1004ProdData.ipynb --to html
A total success
File 2:
cd C:\Users\rcreedon\Dropbox\GIZSupervisor\DATA\Production_Data\STP_Data\Data_Sets\Wave1\1006
ipython nbconvert 1006prodData.ipynb --to html
A Horrid failure.
Error message looks like this
C:\Users\rcreedon\Dropbox\GIZSupervisor\DATA\Production_Data\STP_Data\Data_Sets
Wave1\1006>ipython nbconvert 1006ProdData.ipynb --to html
[NbConvertApp] Using existing profile dir: u'C:\Users\rcreedon\.ipython\pro
ile_default'
[NbConvertApp] Converting notebook 1006ProdData.ipynb to html
[NbConvertApp] Support files will be in 1006ProdData_files\
[NbConvertApp] Loaded template html_full.tpl
[NbConvertApp] ERROR | Error while converting '1006ProdData.ipynb'
Traceback (most recent call last):
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\nbconvertapp.py", line
00, in convert_notebooks
output, resources = exporter.from_filename(notebook_filename, resources=res
urces)
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters\exporter.py",
line 289, in from_filename
return self.from_notebook_node(nbformat.read(f, 'json'), resources=resource
,**kw)
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters\exporter.py",
line 260, in from_notebook_node
output = self.template.render(nb=nb_copy, resources=resources)
File "C:\Anaconda\lib\site-packages\jinja2\environment.py", line 969, in rend
r
return self.environment.handle_exception(exc_info, True)
File "C:\Anaconda\lib\site-packages\jinja2\environment.py", line 742, in hand
e_exception
reraise(exc_type, exc_value, tb)
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
tml_full.tpl", line 1, in top-level template code
{%- extends 'html_basic.tpl' -%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
tml_basic.tpl", line 1, in top-level template code
{%- extends 'display_priority.tpl' -%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
keleton\display_priority.tpl", line 1, in top-level template code
{%- extends 'null.tpl' -%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
keleton\null.tpl", line 26, in top-level template code
{%- block body -%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
tml_full.tpl", line 62, in block "body"
{{ super() }}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
keleton\null.tpl", line 29, in block "body"
{%- block any_cell scoped -%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
keleton\null.tpl", line 76, in block "any_cell"
{%- block headingcell scoped-%}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\exporters..\templates\
tml_basic.tpl", line 64, in block "headingcell"
{{ ("#" * cell.level + cell.source) | replace('\n', ' ') | strip_math_space
| markdown2html | strip_files_prefix | add_anchor }}
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\filters\markdown.py", l
ne 55, in markdown2html
return pandoc(source, 'markdown', 'html', extra_args=['--mathjax'])
File "C:\Anaconda\lib\site-packages\IPython\nbconvert\utils\pandoc.py", line
4, in pandoc
"http://johnmacfarlane.net/pandoc/installing.html"
PandocMissing: The command 'pandoc -f markdown -t html --mathjax' returned an e
ror: [Error 2] The system cannot find the file specified.
Please check that pandoc is installed:
http://johnmacfarlane.net/pandoc/installing.html

not sure if this is still active or not, but a very similar problem was solved for me by adding the cabal binary path (ex $HOME/.cabal/bin) to bash $PATH.

Related

SQLFluff Lint not working for multilevel subdirectory sql file

we are trying to use SQLFluff in our project to avoid sql parser errors before deployment.
in our case we have subdirectories which contains sql files.
During Development we are running command sqlfluff lint command in root directory, we found that lint command is working for one level subdirectory sql path like below,
sqlfluff lint demo/complexquery.sql --dialect snowflake
But when we try for 2 level subdirectory sql file path, lint command is not working and giving error as below. could you please let me know whether iam missing syntax.
sqlfluff lint SQLScript/demo/complexquery.sql --dialect snowflake
Traceback (most recent call last):
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 395, in loads
value, vtype = decoder.load_value(multilinestr)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 849, in load_value
raise ValueError("Found tokens after a closed " +
ValueError: Found tokens after a closed string. Invalid TOML.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\ar\AppData\Roaming\Python\Python310\Scripts\sqlfluff.exe_main.py", line 7, in
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 549, in lint
config = get_config(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 361, in get_config
return FluffConfig.from_root(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 613, in from_root
c = loader.load_config_up_to_path(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in load_config_up_to_path
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 338, in load_config_at_path
configs = self.load_config_file(p, fname, configs=configs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 305, in load_config_file
elems = self._get_config_elems_from_toml(file_path)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 191, in _get_config_elems_from_toml
config = toml.load(fpath)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 134, in load
return loads(ffile.read(), _dict, decoder)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 397, in loads
raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Found tokens after a closed string. Invalid TOML. (line 54 column 1 char 4192)

How can I initialize db ini on Ckan?

I can't initialize my db on Ckan. I try to install Ckan 2.8 on ubuntu 16.04 and postgres 9.6 and I followed the instruction on the oficial page.
All is good but when I try to initialize my db, I can't.
This is the error.
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
sys.exit(run())
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 363, in command
model.repo.init_db()
File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 188, in init_db
self.upgrade_db()
File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 266, in upgrade_db
mig.upgrade(self.metadata.bind, self.migrate_repository, version=version)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
return _migrate(url, repository, version, upgrade=True, err=err, **opts)
File "<decorator-gen-16>", line 2, in _migrate
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
return f(*a, **kw)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
schema.runchange(ver, change, changeset.step)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
change.run(self.engine, step)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
script_func(engine)
File "/usr/lib/ckan/default/src/ckan/ckan/migration/versions/016_uuids_everywhere.py", line 69, in upgrade
drop_sequencies(migrate_engine)
File "/usr/lib/ckan/default/src/ckan/ckan/migration/versions/016_uuids_everywhere.py", line 186, in drop_sequencies
migrate_engine.execute('drop sequence %s_id_seq;' % sequence)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2064, in execute
return connection.execute(statement, *multiparams, **params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 939, in execute
return self._execute_text(object, multiparams, params)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1097, in _execute_text
statement, parameters
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) no existe la secuencia «package_extra_revision_id_seq»
[SQL: 'drop sequence package_extra_revision_id_seq;']
I try to run:
paster db init -c /etc/ckan/default/production.ini
sudo ckan db init
I checked my db while the script was running and script created the sequence "package_extra_revision_id_seq" correctly. But when the script finishes the sequence "package_extra_revision_id_seq" no longer exists.
"CKAN 2.8 installing from source" paster db init is broken and not reported, so it won't work in anyway. Solution is to install CKAN 2.7 instead as dependencies are different and working.
https://docs.ckan.org/en/2.7/
Once installed, paster command for init database works

Basic Grako example gives IndexError

I'd like to get started with Grako (3.6.6) and as a first experience with parsers I wanted to generate an HTML table from a custom syntax. The following basic test
import grako
grammar = """table = { row }+ ;
row = (cell1:cell "|" cell2:cell) "\n";
cell = /[a-z]+/ ;
"""
model = grako.genmodel("model", grammar)
ast = model.parse(
"""a | b
c | d
""", "table")
print(ast)
results in an error
File "test.py", line 13, in <module>
""", "table")
File "grako\grammars.py", line 790, in grako.grammars.Grammar.parse (grako\grammars.c:27773)
File "grako\grammars.py", line 97, in grako.grammars.GrakoContext.parse (grako\grammars.c:4391)
File "grako\contexts.py", line 180, in grako.contexts.ParseContext.parse (grako\contexts.c:4313)
File "grako\grammars.py", line 594, in grako.grammars.Rule.parse (grako\grammars.c:22253)
File "grako\grammars.py", line 597, in grako.grammars.Rule._parse_rhs (grako\grammars.c:22435)
File "grako\contexts.py", line 399, in grako.contexts.ParseContext._call (grako\contexts.c:10088)
File "grako\contexts.py", line 433, in grako.contexts.ParseContext._invoke_rule (grako\contexts.c:11135)
File "grako\grammars.py", line 435, in grako.grammars.PositiveClosure.parse (grako\grammars.c:17285)
File "grako\contexts.py", line 695, in grako.contexts.ParseContext._positive_closure (grako\contexts.c:19286)
File "grako\contexts.py", line 696, in grako.contexts.ParseContext._positive_closure (grako\contexts.c:19240)
File "grako\grammars.py", line 435, in grako.grammars.PositiveClosure.parse.lambda10 (grako\grammars.c:17195)
File "grako\grammars.py", line 547, in grako.grammars.RuleRef.parse (grako\grammars.c:20774)
File "grako\grammars.py", line 594, in grako.grammars.Rule.parse (grako\grammars.c:22253)
File "grako\grammars.py", line 597, in grako.grammars.Rule._parse_rhs (grako\grammars.c:22435)
File "grako\contexts.py", line 399, in grako.contexts.ParseContext._call (grako\contexts.c:10088)
File "grako\contexts.py", line 433, in grako.contexts.ParseContext._invoke_rule (grako\contexts.c:11135)
File "grako\grammars.py", line 326, in grako.grammars.Sequence.parse (grako\grammars.c:11582)
File "grako\grammars.py", line 268, in grako.grammars.Token.parse (grako\grammars.c:9463)
File "grako\contexts.py", line 543, in grako.contexts.ParseContext._token (grako\contexts.c:13772)
File "grako\buffering.py", line 301, in grako.buffering.Buffer.match (grako\buffering.c:9168)
IndexError: string index out of range
which happens to be partial_match = (token[0].isalpha() and token.isalnum() and self.is_name_char(self.current()) )
Despite me being new to parsers and a little lack of documentation, I'd like to stick to Grako.
Can you help me set up a basic example which outputs the HTML for a table?
Grako is not seeing the "\n" in the grammar correctly because newlines are not allowed in tokens, and the \n is being evaluated in the context of the outer, triple-quote ("""), string. Things work fine if you use /\n/ instead.
Also note that if \n will be part of the language, then you should probably write a ##whitespace clause so the parser doesn't skip over the character:
##whitespace :: /[\t ]+/
This is the correct grammar for your language:
grammar = """
##whitespace :: /[\t ]+/
table = { row }+ ;
row = (cell1:cell "|" cell2:cell) "\\n";
cell = /[a-z]+/ ;
"""
I'm currently patching Grako to detect and report errors like the one in your grammar. The changes are already in the Bitbucket repository. I'll make a release after I finish testing.

get unicode error when print in html flask

Getting unnicode error when print in html Flask.
Here is my code.
{% if(backpaths) %}
{% for n in backpaths:%}
{% print '%s'%n %}
</br>
{% endfor %}
{% endif %}
I tried to use n.decode('utf-8') but it didn't work and got same error
backpaths is set to:
['1\xe6\x9c\x89 --(HYPER)--> quantifier={indefinite|\xe4\xb8\x8d\xe5\xae\x9a\xe6\x8c\x87} --(HYPO)--> \xe6\x9c\x89 ', '2\xe6\x9c\x89 --(HYPER)--> exist|\xe5\xad\x98\xe5\x9c\xa8 --(HYPO)--> \xe6\x9c\x89 ']
here is the traceback
(most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/ganchimeg/FlaskApp/FlaskApp/__init__.py", line 125, in homepage
return render_template('index.html', backpaths=successPaths)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128, in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110, in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/ganchimeg/FlaskApp/FlaskApp/templates/index.html", line 31, in top-level template code
{{ n }}
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 16: ordinal not in range(128)
You don't have Unicode strings, you have byte strings. Python tries to implicitly decode those using the standard ASCII codec. Explicitly decode them:
{% if(backpaths) %}
{% for n in backpaths:%}
{{ n.decode('utf8') }}
</br>
{% endfor %}
{% endif %}
It'll be better if you passed in backpaths into the template ready-decoded.

How can I modify/merge Jinja2 dictionaries?

I have a Jinja2 dictionary and I want a single expression that modifies it - either by changing its content, or merging with another dictionary.
>>> import jinja2
>>> e = jinja2.Environment()
Modify a dict: Fails.
>>> e.from_string("{{ x[4]=5 }}").render({'x':{1:2,2:3}})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "jinja2/environment.py", line 743, in from_string
return cls.from_code(self, self.compile(source), globals, None)
File "jinja2/environment.py", line 469, in compile
self.handle_exception(exc_info, source_hint=source)
File "<unknown>", line 1, in template
jinja2.exceptions.TemplateSyntaxError: expected token
'end of print statement', got '='
Two-stage update: Prints superfluous "None".
>>> e.from_string("{{ x.update({4:5}) }} {{ x }}").render({'x':{1:2,2:3}})
u'None {1: 2, 2: 3, 4: 5}'
>>> e.from_string("{{ dict(x.items()+ {3:4}.items()) }}").render({'x':{1:2,2:3}})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "jinja2/environment.py", line 868, in render
return self.environment.handle_exception(exc_info, True)
File "<template>", line 1, in top-level template code
TypeError: <lambda>() takes exactly 0 arguments (1 given)
Use dict(x,**y): Fails.
>>> e.from_string("{{ dict((3,4), **x) }}").render({'x':{1:2,2:3}})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "jinja2/environment.py", line 868, in render
return self.environment.handle_exception(exc_info, True)
File "<template>", line 1, in top-level template code
TypeError: call() keywords must be strings
So how does one modify the dictionary x in Jinja2 by changing an attribute or merging with another dictionary?
This question is similar to: How can I merge two Python dictionaries as a single expression? -- insofar as Jinja2 and Python are analogous.
I found another solution without any extension.
{% set _dummy = x.update({4:5}) %}
It makes x updated. Don't use _dummy.
Sounds like the Jinja2 "do" statement extension may help. Enabling this extension would allow you to rewrite:
{{ x.update({4:5}) }} {{ x }}
as
{% do x.update({4:5}) %} {{ x }}
Example:
>>> import jinja2
>>> e = jinja2.Environment(extensions=["jinja2.ext.do",])
>>> e.from_string("{% do x.update({4:5}) %} {{ x }}").render({'x':{1:2,2:3}})
u' {1: 2, 2: 3, 4: 5}'
>>>
I added a filter to merge dictionaries, namely:
>>> def add_to_dict(x,y): return dict(x, **y)
>>> e.filters['add_to_dict'] = add_to_dict
>>> e.from_string("{{ x|add_to_dict({4:5}) }}").render({'x':{1:2,2:3}})
u'{1: 2, 2: 3, 4: 5}'