Error handling of PipelineModel.load(PATH) if path doesn't exist Pyspark - pyspark

I want to do an error handling of PipelineModel.load(PATH) i.e want to check if path exists if it doesn't exist, I want to exit the code so was trying try except block but don't exactly know what the error I should be expecting, Below is the snippet for same
try:
pipelineModel = PipelineModel.load(PATH)
except InvalidInputException:
print('Path Does not exist')
sys.exit(0)
Below is the error message
NameError: name 'InvalidInputException' is not defined

try:
pipelineModel = PipelineModel.load(S3LOC)
except Exception as e:
print('Exception Message ',e.args)
print('Model is not deployed')
sys.exit(0)
This snippet worked for me

Related

Test that an exception is handled correctly with pytest

I have a block of try-exception code and I want to test if the block works properly? I wonder if there is a way to test this block with a unit test using pylint? This is the snipent I have:
class data_type(Enum):
SESSIONS = 0
RUNS = 1
def find_filter(data_type)
try:
if data_type.name == 'SESSIONS':
return {}
elif data_type.name == 'RUNS':
#Filter out rerun jobs
filter = {
"filter":{
"attName":"is_rerun",
"operand":"IN",
"#c":".InFilter",
"values":["FALSE"]
}
}
return filter
except AttributeError as erro:
loghandler.critical('Error in creating count query: {}'.format(erro))
I want to test if the data_type is not a valid enum member, the exception catches it before the execution goes to if statement. For example if:
find_filter('')
is called I like to see that the except AttributeError as erro captures the error and the error is logged.
Is there a way to do that?
PS: I know with pytest.raises() I can test if a raised error is really raised? But as far as I understand pytest.raises() does not work for testing the exception block of a try-exception case.
Any suggestion please?
Thanks.

Can I raise and handle exceptions in the same function

I am raising an exception and trying to handle the exception in snippet. The raising exception part and handling exception part are done in a function. Is it wrong to do so?
import sys
def water_level(lev):
if(lev<10):
raise Exception("Invalid Level!")
print"New level=" # If exception not raised then print new level
lev=lev+10
print lev
try:
if(level<10):
print"Alarming situation has occurred."
except Exception:
sys.stdout.write('\a')
sys.stdout.flush()
else:
os.system('say "Liquid level ok"')
print"Enter the level of tank"
level=input()
water_level(level) #function call
The output is not handling exception. Can someone explain me why?
It is better to just raise the exception in the function and then catch it it when you call the function so your function does not do too much and your error handling is independent. And it makes your code simpler.
Your code never reached your except clause because if the water level is too low it raises an exception and jumps out of the function and if it was okay it just reaches the else clause. The print statement in your try clause is also never reached because it is the same condition than the one that raises your exception and that one jumps out.
Your code should be something like that...
import sys
import os
def water_level(level):
#just raise exception in function here
if level < 10:
raise Exception("Invalid Level!")
level = level + 10
print("New level=") # If exception not raised then print new level
print(level)
#function call
print("Enter the level of tank")
#cast to int
level=int(input())
try:
#call function in here
water_level(level)
#catch risen exceptions
except Exception as e:
sys.stdout.write('\a')
sys.stdout.flush()
#print exception(verification)
print(e)
print("Alarming situation has occurred.")
else:
os.system('say "Liquid level ok"')
Note that i corrected some other flaws
import os was missing
you should cast your input() to a number so you can do number comparisions and additions
try to avoid to catch the least specific Exception Exception because you will catch every other Exception too.(thats why i added the print(e)) -> think about custom exceptions

ScalaTest display original Failure stacktrace

How can display the original stacktrace for a Try Failure as part of the test output?
When I do the following:
result.success.value should equal blah
And the result is a Failure I get
The Try on which success was invoked was not a Success.
Or, if I do this first:
result should be a 'success
It's a little more informative, because I can see the exception:
Failure(java.lang.IllegalArgumentException: Cannot format given Object as a Date) was not a success
But the stack trace shows where it failed in the test, not the original stack trace of the Failure.
If some error-causing code is wrapped into Try, it means that exception is handled somewhere inside calculation, and it's cause and message will not be printed somewhere until explicitly requested. In order to see original cause, you can access failure object directly (docs), or handle failure manually:
val (ok, message) = result match {
case Success(v) => (true, "")
case Failure(ex) =>
(false, ex.getMessage + "\n" + ex.getStackTrace.mkString("\n"))
}
assert(ok, message)
This link also looks helpful.
If you want original stack trace to be printed - use result.get, but check not for 'success but inner type/value:
result.get should equal blah

Python3 Catch errors when from self.connect(('badhost',6667))

Looks like asyncio is the module to use. I'll leave this question up anyway, because it doesn't look like there is a way to catch specific errors with asynchat.
class mysocket(asynchat.async_chat):
terminator = b'\n'
def __init__(self,sock=None):
asynchat.async_chat.__init__(self,sock)
self.create_socket()
# Try always succeeds with self.connect
try:
self.connect(('badhost',6667))
print('command always successful')
except:
print('This never gets printed')
How do I catch errors from the self.connect() method that causes an uncaught exception.
error: uncaptured python exception, closing channel <main.mysocket badhost:6667 at 0x7f0a03e66a58> (:[Errno 111] Connection refused [/usr/lib/python3.4/asyncore.py|read|83] [/usr/lib/python3.4/asyncore.py|handle_read_event|439] [/usr/lib/python3.4/asyncore.py|handle_connect_event|447])
All that is left to try is overwrite the handle_connect_event() method and put asyncore.handle_connect_event(self). I would like to get a professional answer to this dilemma.
Try to override default handle_error method:
def handle_error(self):
t, v, tb = sys.exc_info()
if t == socket.error:
# do smth with it
print("Connect error")

txpostgres : Deferred instance has no attribute 'addCallBack'

I want to use an asynchronous connection to a postgres database for inserting a realtime incomming data. I am using Twisted for the TCP communication and i am giving txpostgres a shot for the interaction with the database.I am stacked with a weird message when i try to add a callback for my asynchronous insert.Here my code:
try:
conn = txpostgres.ConnectionPool(50,params)
d = conn.start()
def save_conn(c):
self.list_cnx.append(c)
print str(c),"Connect OK"
def print_err(m):
print m
d.addCallbacks(lambda _: save_conn(conn),lambda __: print_err('Connect NO'))
except Exception as e:
print "Cannot connect to database!!"
I am adding the refrence of the connction pool in a list for future query.
def insert_data(self,dic):
try:
insArRq="""INSERT INTO test_pool(date_msg, msg) VALUES ('%s','%s')"""%(dic['date'],dic['msg'])
for c in self.list_cnx:
def insert_finich():
print "insert finich"
def insert_error():
print "insert error"
d = c.runOperation(insArRq) # return a deferred as mentioned in the documentation
print d # for debug
d.addCallBack(insert_finich) # error mesage
except Exception as ee:
print "Insert error : ",ee
When i try to add a callback for the deferred returned by the runOperation this error appear:
<Deferred at 0x8d9782c waiting on Deferred at 0x8d9786c>
Insert error : Deferred instance has no attribute 'addCallBack'
and sometimes:
<Deferred at 0x8d97a0c>
Insert error : Deferred instance has no attribute 'addCallBack'
Help me please, i'am new to defrred concepts so i think i'am missing something.Thanks
The method is named addCallback; case is important.