entity decode for json that return a list/seq of any val - scala

Question

I am building the back-end of an app using http4s. In the app i receive a json response from an external api (not the one i am working on). The api response is of this pattern below.
json response:
`{
"datatable" : {
"data" : [["AAPl", "MRT", "2020-03-20", 123, 123, 12.4, 233, 3234],
["AAPl", "MRT", "2020-03-20", 123, 123, 12.4, 233, 3234]],
"meta": {
"next_date : null
}
}`
my question is?
can someone show me how to create an entity decoder and entity encoder that would decode the pattern. i can seem to get it to work.
currently i have:
object Stocks {
case class tickerInfo(ticker: String, dim:String, date: String, a: Int, b: Int, c: Float, d: Int, e: Int)
case class data(data: Seq[tickerInfo])
case class meta(next_date : Option[String])
case class table(data: data, meta:meta)
case class stockInfo(datatable:table)
object stockInfo {
implicit def stockInfoEntityDecoder[F[_]:Sync]: EntityDecoder [F, stockInfo] = jsonOf
implicit def stockInfoEntityEncoder[F[_] : Applicative]: EntityEncoder[F, stockInfo] = jsonEncoderOf
}
val decodedJson = C.expect[stockInfo](GET(Uri.uri("www.externalApi.com")
}
But this doesn't work. Please can someone tell me where i am going wrong.
I am getting a run time error *not a compile error). and its an http4s error that says - InvalidMessageBodyFailure.
Thanks

Answer 1

You have a couple of errors in your model, but the main problem is that circe won't be able to automatically decode an array of jsons into a case class.
If you can not modify the source of the data, you would need to create your own custom codec.
object Stocks {
final case class TickerInfo(ticker: String, dim: String, date: String, a: Int, b: Int, c: Float, d: Int, e: Int)
final case class Meta(next_date : Option[String])
final case class Table(data: List[TickerInfo], meta: Meta)
final case class StockInfo(datatable: Table)
object StockInfo {
implicit final val TickerInfoDecoder: Decoder[TickerInfo] = Decoder[List[Json]].emap {
case ticker :: dim :: date :: a :: b :: c :: d :: e :: Nil =>
(
ticker.as[String],
dim.as[String],
date.as[String],
a.as[Int],
b.as[Int],
c.as[Float],
d.as[Int],
e.as[Int]
).mapN(TickerInfo).left.map(_.toString)
case list =>
Left(s"Bad number of fields in: ${list.mkString("[", ", ", "]")}")
}
implicit final val MetaDecoder: Decoder[Meta] = deriveDecoder
implicit final val TableDecoder: Decoder[Table] = deriveDecoder
implicit final val StockInfoDecoder: Decoder[StockInfo] = deriveDecoder
}
}
(you can see it working here, I leave outside the http4s part as it would be tricky to mock, but it shouldn't matter)
The error reporting of that custom encoder could be improved by providing more useful messages for each field. That is left as an exercise for the reader.

Answer 2

You need to:
create a data model, presumably consisting of some case classes and possibly sealed traits. What this data model should look like depends on the external API that you're talking to.
create JSON decoders for this data model. These should go in the case classes' companion objects, because that will allow the compiler to find them when needed without having to import anything.
Use the http4s-circe library in order to integrate these decoders with http4s. https://http4s.org/v0.19/json/
If you did everything correctly, you should then be able to use the Http4s client to retrieve the data, e. g. httpClient.expect[YourModelClass](Uri.uri("http://somewhere.com/something/")).

Answer 3

Welcome! I'd probably start inside and work your way out:
So for elements in the data array, perhaps something like (I'm guessing on the domain):
case class Stock(
name: String,
mart: String,
datePosted: LocalDate,
a: Int,
b: Int,
price: Double,
c: Int,
d: Int
)
Using Circe's automatic derevation should handle this pretty well.
Data seems like an array of arrays of exploded elements? So you may have to write some extraction logic to convert to the inner Object manually if that is the case. Something like this in the Controller may help:
elem match {
case name :: mart :: date :: a :: b :: price :: c :: d :: Nil => Stock(name, mart, date, a, b, price, c, d)
case invalid # _ => log.warn(s"Invalid record: $invalid")
}
Hopefully the above snippet returns something useful like an Either[E, A], etc.
Finally, you'd need objects for the outer JSON. Something simple to capture, such as case class ExternalApiRequest(dataTable: T), where T is a type appropriate for the above case. List[String] in the worst case?
Hope this helped! Let me know if you have any specific errors you were running into