Why would a Spark job fail without throwing run-time exceptions? - scala

Question

I'm going through a problem in which my job fails at a particular stage when it invokes a class.
Here's the line
val stockDataFilteredRDD: RDD[stockPriceInfo] =
lineMapToStockPriceInfoObjectRDD
.map(new stockDataFilter(_).requirementsMet.get)
Here's what I see
15/10/0916:02:28INFOClient:ApplicationreportfromResourceManager:
applicationidentifier:application_1438798768056_0254
appId:254
clientToAMToken:null
appDiagnostics:
appMasterHost:ip-10-0-142-138.ec2.internal
appQueue:root.add_twitter_user
appMasterRpcPort:0
appStartTime:1444421062565
yarnAppState:RUNNING
distributedFinalState:UNDEFINED
appTrackingUrl:http://myIP.ip
15/10/0916:02:29INFOClient:ApplicationreportfromResourceManager:
applicationidentifier:application_1438798768056_0254
appId:254
clientToAMToken:null
appDiagnostics:
appMasterHost:ip-10-0-142-138.ec2.internal
appQueue:root.add_twitter_user
appMasterRpcPort:0
appStartTime:1444421062565
yarnAppState:FINISHED
distributedFinalState:FAILED
appTrackingUrl:http://myIP.ip
appUser:add_twitter_user
The class invoked
class stockDataFilter(val s:stockPriceInfo){
val dateDelim="-"
val timeDelim=":"
val dateAndTime=s.dateTime
val splitDateTime=dateAndTime.split("#")
val dateStamp=splitDateTime(0)
val time=splitDateTime(1)
val splitDate=dateStamp.split(dateDelim)
val splitTime=time.split(timeDelim)
val year=splitDate(0);
val month=splitDate(1);
val day=splitDate(2)
val hour=splitTime(0);
val minute=splitTime(1);
val second=splitTime(2)
val openingBell:LocalTime=newLocalTime(9,30)
val closingBell:LocalTime=newLocalTime(16,0)
val currentTime:LocalTime=newLocalTime(hour.toInt,minute.toInt)
//NonTradingSessions
val newYearsDay:Date=newDate(year.toInt-1900,0,1)
val weekends:List[String]=List("Saturday","Sunday")
val date=new Date(year.toInt-1900,month.toInt-1,day.toInt)
val currentDate=new LocalDate(year.toInt-1990,month.toInt-1,day.toInt)
//NewYorkStockExchangeoperatesfrom9:30a.m.to4:00p.m
def isWithinTradingSession:Boolean={
val isAfterOpen:Boolean=currentTime.isAfter(openingBell)
val isBeforeClose:Boolean=currentTime.isBefore(closingBell)
isAfterOpen&&isBeforeClose
}//return Trueifitiswithingtradingtime
def requirementsMet:Option[stockPriceInfo]=isWithinTradingSessionmatch{
case true=>Some(s)
case false=>None
}
}
I am able to display(store in HDFS) anything before that, but once I add this line, it fails. I've looked at the logs, there are no obvious issues. There was also no compile-time or run-time exception. I've been stuck on this for days out of options. Your help would be appreciated.. Regards
LOGS:
LogType: stderr
LogLength: 6638
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/jars/spark-assembly-1.1.0-cdh5.2.1-hadoop2.5.0-cdh5.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data01/yarn/nm/usercache/add_twitter_user/filecache/62/twitteryahoofinanceanalytics.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15/10/10 11:27:24 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
15/10/10 11:27:25 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:25 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:25 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:25 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:25 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher#IP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://driverPropsFetcher#IP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'driverPropsFetcher' on port port.
15/10/10 11:27:26 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:26 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:26 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor#myIP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkExecutor#myIP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'sparkExecutor' on port 54841.
15/10/10 11:27:26 INFO Remoting: Remoting shut down
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
15/10/10 11:27:26 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://sparkDriver#IP:port/user/CoarseGrainedScheduler
15/10/10 11:27:26 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver
15/10/10 11:27:26 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:26 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:26 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor#myIP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkExecutor#myIP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'sparkExecutor' on port 36011.
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to MapOutputTracker: akka.tcp://sparkDriver#IP:PORT/user/MapOutputTracker
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to BlockManagerMaster: akka.tcp://sparkDriver#IP/user/BlockManagerMaster
15/10/10 11:27:26 INFO storage.DiskBlockManager: Created local directory at /data01/yarn/nm/usercache/add_twitter_user/appcache/application_1438798768056_0263/spark-local-20151010112726-839d
15/10/10 11:27:26 INFO storage.DiskBlockManager: Created local directory at /data02/yarn/nm/usercache/add_twitter_user/appcache/application_1438798768056_0263/spark-local-20151010112726-4371
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'Connection manager for block manager' on port port.
15/10/10 11:27:26 INFO network.ConnectionManager: Bound socket to port port with id = ConnectionManagerId(myIP,port)
15/10/10 11:27:26 INFO storage.MemoryStore: MemoryStore started with capacity 530.3 MB
15/10/10 11:27:26 INFO storage.BlockManagerMaster: Trying to register BlockManager
15/10/10 11:27:26 INFO storage.BlockManagerMaster: Registered BlockManager
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver#IP/user/HeartbeatReceiver
15/10/10 11:27:27 INFO executor.CoarseGrainedExecutorBackend: Driver commanded a shutdown
15/10/10 11:27:27 INFO network.ConnectionManager: Selector thread was interrupted!
15/10/10 11:27:27 INFO network.ConnectionManager: ConnectionManager stopped
15/10/10 11:27:27 INFO storage.MemoryStore: MemoryStore cleared
15/10/10 11:27:27 INFO storage.BlockManager: BlockManager stopped
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:27 INFO Remoting: Remoting shut down
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
15/10/10 11:27:27 INFO Remoting: Remoting shut down
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
LogType: stdout
LogLength: 0
Log Contents: