I'm building a discord bot that will play cards against humanity. The issue is that when I try to load the JSON file, the program doesn't work.
`#bot.command(pass_context=True)
async def loadCards(ctx):
with open('wcards.json') as f:
wtcards = json.load(f)
with open('bcards.json') as f:
bkcards = json.load(f)
if len(wtcards) > 1 and len(bkcards) > 1:
await ctx.send('Cards Loaded')`
I believe that you need to use the read and write parameters when using open().
e.g.
#bot.command(pass_context=True)
async def loadCards(ctx):
with open('wcards.json', 'r') as f:
wtcards = json.load(f)
with open('bcards.json', 'r') as f:
bkcards = json.load(f)
if len(wtcards) > 1 and len(bkcards) > 1:
await ctx.send('Cards Loaded')
Related
I am trying to download pdf file from S3 using the akka-stream-alpakka connector. I have the s3 path and try to download the pdf using a wrapper method over the alpakka s3Client.
def getSource(s3Path: String): Source[ByteString, NotUsed] = {
val (source, _) = s3Client.download(s3Bucket, s3Path)
source
}
From my main code, I call the above method and try to convert it to a file
val file = new File("certificate.pdf")
val res: Future[IOResult] = getSource(data.s3PdfPath)
.runWith(FileIO.toFile(file))
However, instead of it getting converted to a file, I am stuck with a type of IOResult. Can someone please guide as to where I am going wrong regarding this ?
def download(bucket: String, bucketKey: String, filePath: String) = {
val (s3Source: Source[ByteString, _], _) = s3Client.download(bucket, bucketKey)
val result = s3Source.toMat(FileIO.toPath(Paths.get(filePath)))(Keep.right)
.run()
result
}
download(s3Bucket, key, newSigFilepath).onComplete {
}
Inspect the IOResult, and if successful you can use your file:
res.foreach {
case IOResult(bytes, Success(_)) =>
println(s"$bytes bytes written to $file")
... // do whatever you want with your file
case _ =>
println("some error occurred.")
}
I have the following piece of code which is producing 0KB Files in S3 whats wrong with the following piece of code
def extractFilesFromZipStream(zipInputStream: ZipArchiveInputStream,
tgtPath: String, storageType:String): scala.collection.mutable.Map[String, (String, Long)] = {
var filePathMap = scala.collection.mutable.Map[String, (String, Long)]()
Try {
storageWrapper.mkDirs(tgtPath)
Stream.continually(zipInputStream.getNextZipEntry).takeWhile(_ != null).foreach {file =>
val storagePathFilePath = s"$tgtPath/${file.getName}"
storageWrapper.write(zipInputStream, storagePathFilePath)
LOGGER.info(storagePathFilePath)
val lineCount = Source.fromInputStream(storageWrapper.read(storagePathFilePath)).getLines().count(s => s!= null)
There is nothing wrong with the Storage wrapper, it takes an input stream and path and has been working well so far. Can anyone suggest what's wrong with my implementation of using zipArchive Stream.
Link of my screenshot I am a beginner in Scala, trying to read the file but getting the java.io.FileNotFoundException,can someone help.
package standardscala
case class TempData(day :Int,doy :Int, month:Int, year :Int, precip :Double, snow :Double, tave :Double, tmax :Double, tmin :Double )
object TempData {
def main(args: Array[String]): Unit = {
val source = scala.io.Source.fromFile("DATA/MN212.csv")
val lines = source.getLines().drop(1) // to get the lines of files,drop(1) to drop the header
val data= lines.map { line => val p = line.split(",")
TempData(p(0).toInt,p(1).toInt,p(2).toInt,p(4).toInt,p(5).toDouble,p(6).toDouble,p(7).toDouble,p(8).toDouble,p(9).toDouble)
}.toArray
source.close() //Closing the connection
data.take(5) foreach println
}
}
Try to use absolute path, and the problem will disappear.
One option would be to move your csv file into a resources folder and load it as a resource like:
val f = new File(getClass.getClassLoader.getResource("your/csv/file.csv").getPath)
Or you could try loading it from an absolute path!
Please read this post about reading CSV by Alvin Alexander, writer of the Scala Cookbook:
object CSVDemo extends App {
println("Month, Income, Expenses, Profit")
val bufferedSource = io.Source.fromFile("/tmp/finance.csv")
for (line <- bufferedSource.getLines) {
val cols = line.split(",").map(_.trim)
// do whatever you want with the columns here
println(s"${cols(0)}|${cols(1)}|${cols(2)}|${cols(3)}")
}
bufferedSource.close
}
As Silvio Manolo pointed out, you should not use fromFile with absolute path, as you code will require the same file hierarchy to run. In a first draft, this is acceptable so you can move on and test the real job!
I want to read files from given directory then read contents from file and create Map of filename as key and its contetns as value.
I did not got any success but I have tried like this,
def getFileLists(): List[File] = {
val directory = "./input"
// print(new File(directory).listFiles().toList)
return new File(directory).listFiles().toList
}
val contents = getFileLists().map(file => Source.fromFile(file).getLines())
print(contents)
you can change your following line
val contents = getFileLists().map(file => Source.fromFile(file).getLines())
to
val contents = getFileLists().map(file => (file.getName, Source.fromFile(file).getLines()))
which would give you
contents: List[(String, Iterator[String])]
Furthermore you can add .toMap method call as
val contents = getFileLists().map(file => (file.getName, Source.fromFile(file).getLines())).toMap
which would give you
contents: scala.collection.immutable.Map[String,Iterator[String]]
What you are doing is transforming the list of file-names to a list of their contents. You want a Map[File, List[String]] instead. To do that the easiest ist to map to tuples of files and content, and then call toMap on the result:
getFileLists().map(file => file -> Source.fromFile(file).getLines().toList).toMap
toMap works when the input sequence has Tuple2 as element type. file -> contents is such a tuple (File, List[String]).
Or in two steps:
val xs: Seq[(File, List[String])] = getFileLists().map(file =>
file -> Source.fromFile(file).getLines().toList)
val m: Map[File, List[String]] = xs.toMap
You can try this:
getFileLists().map(file => (file.getName, Source.fromFile(file).getLines().toList)).toMap
I am reading data from AWS S3. The following code works fine if the input file is small. It failed when the input file is big. Is there any parameter I can modify to increase the buffer size or anything so it can handle bigger input file as well? Thanks!
val s3Object= s3Client.getObject(new GetObjectRequest("myBucket", "myPath/myFile.csv"));
val myData = Source.fromInputStream(s3Object.getObjectContent()).getLines()
for (line <- myData) {
val data = line.split(",")
myMap.put(data(0), data(1).toDouble)
}
println(" my map : " + myMap.toString())
If you look at the source code you can see that internally it calls Source.createBufferedSource. You can use that to create your own version with a bigger buffer size.
These are the lines of code from scala:
def createBufferedSource(
inputStream: InputStream,
bufferSize: Int = DefaultBufSize,
reset: () => Source = null,
close: () => Unit = null
)(implicit codec: Codec): BufferedSource = {
// workaround for default arguments being unable to refer to other parameters
val resetFn = if (reset == null) () => createBufferedSource(inputStream, bufferSize, reset, close)(codec) else reset
new BufferedSource(inputStream, bufferSize)(codec) withReset resetFn withClose close
}
def fromInputStream(is: InputStream, enc: String): BufferedSource =
fromInputStream(is)(Codec(enc))
def fromInputStream(is: InputStream)(implicit codec: Codec): BufferedSource =
createBufferedSource(is, reset = () => fromInputStream(is)(codec), close = () => is.close())(codec)
Edit: Now that I have thought about your issue a bit more, you can increase the buffer size in this way, but I'm not sure that this will actually fix your issue