I know if I have JSON with specific known fields, like "date" and "title"
{
"date": "any string",
"title": "any string"
}
Then, I can decode it by defining
myDecodeFunc :: Json -> Either JsonDecodeError ({ date :: String, title :: String })
myDecodeFunc = decodeJson
However, I can't seem to find how to decode the same JSON with arbitrary fields like this.
{
"any field": "any string",
"any field": "any string"
}
I am new to Purescript, so any pointers would be appreciated. Thanks.
A fancy term for a "record"-ish thing where you don't know all possible fields in advance is dictionary.
And to parse JSON as a dictionary, use a dictionary-ish type, such as Foreign.Object for example, which just happens to have a handy instance of FromJSON:
import Foreign.Object (Object)
myDecodeFunc :: Json -> Either JsonDecodeError (Object String)
myDecodeFunc = decodeJson
Related
I have a JSON object that I've transformed that I need to filter down to only a subset of its original keys. I've looked through the docs for the Json object in circe but it doesn't appear to expose any API around filtering the object. Do I have to use a cursor for this? I considered creating a decoder from a case class however my keys have a special character . in them. Here is some more code/data for context.
{
"field.nested.this": "value",
"field.nested.that": "value",
"field.nested.where": "value"
}
What's the best approach to create a new JSON instance that doesn't contain the field.nested.that field?
I'm not sure if this is what you need:
object Circe extends App {
import io.circe._
import io.circe.literal._
import io.circe.syntax._
//I'm using a json literal here.
//If you have a runtime string from an external source
// you would need to parse it with `io.circe.parser.parse` first
val json: Json = json"""
{
"field.nested.this": "value",
"field.nested.that": "value",
"field.nested.where": "value"
}
"""
val maybeJsonFiltered =
json.asObject.map(_.filterKeys(_ != "field.nested.that").asJson)
println(maybeJsonFiltered)
// Some({
// "field.nested.this" : "value",
// "field.nested.where" : "value"
// })
}
Alternatively you could also parse it as a map (json.as[Map[String, String]]) or a custom case class with only the fields you need, and encode them back to json. You will probably need a #JsonKey annotation for all your fields with ..
I have a request like the following
val request =
Request[IO](
method = POST,
uri = Uri.uri("..."),
headers = Headers(
Authorization(BasicCredentials("...", "..."))
)
)
.withEntity(PaymentIntentRequest2(2000, "usd"))
I am looking at the source code and it looks like the withEntity inherits the headers from the nested EntityDecoder so the code above defaults to Content-Type: application/json. Where as if I explicitly pass in UrlForm everything is fine.
Unfortunately the API I am hitting expected the data as x-www-form-urlencoded and given the complexity of the target API with all the different endpoints/requests I would like to find a way to encode the given case class as a form. What is the best way of doing that?
I have tried:
Explicitly specifying the Content-Type but this doesn't work because the inherited type takes priority
Building an implicit generic conversion from Product to UrlForm (extension method for now)
implicit class UrlFormEncode[+B <: Product](val u: B) {
def asUrlForm: UrlForm =
u.productElementNames
.zip(u.productIterator)
.foldLeft(UrlForm()) { (a, b) =>
a.combine(UrlForm(b._1 -> b._2.toString))
}
}
The problem here is UrlForm expects a string in both sides of the mapping. And if I just convert things with .toString it doesn't work because of nested typed for example:
ChargeRequest(Amount(refInt), EUR, source = Some(SourceId("...."))
Results in the following json which is not valid
{
"currency": "EUR",
"amount": "2000",
"source": "Some(SourceId(....))",
"customer": "None"
}
I tried asJson instead of toString but circe can not decide on the proper KeyEncoder
What is the right way of approaching this so the given Product is encoded down the stream ?
I just faced the same issue and this is the way it worked for me.
From https://http4s.org/v0.20/client/
// This import will add the right `apply` to the POST.
import org.http4s.client.dsl.io._
val form = UrlForm(
OAuthAttribute.Code -> code,
OAuthAttribute.RedirectUri -> callbackUri,
OAuthAttribute.GrantType -> "authorization_code"
)
private def buildRequest(tokenUri: Uri, form: UrlForm, header: String): Request[IO] =
POST(
form,
tokenUri,
Header.Raw(CIString("Authorization"), header),
Header.Raw(CIString("Content-Type"), "application/x-www-form-urlencoded"),
Header.Raw(CIString("Accept"), "application/json")
)
And that's it. For some strange reason using .withHeaders didn't work for me, seems like they are overridden or so.
I am trying to parse MongoDB Extended JSON using Circe JSON Parser, works fine in most of the cases except for special datatypes, for eg. in below case class i have priorityOrder which is of long datatype.
case class relinfo(id:String,assetId:String,insureeId:String,queue:String,priorityOrder:Long) extends baseDomain
But when it is converted to MongoDB JSON format, it is converted to special mongo format described below (check priorityOrder field)
{
"_id" : "4abf009d-64b1-496c-b0e8-9061f5e183a0",
"id" : "4abf009d-64b1-496c-b0e8-9061f5e183a0",
"assetId" : "e26d5310-ab0c-4672-9971-4babd3420302",
"insureeId" : "cdee05a1-a09c-4e10-81df-c3f112298cc3",
"queue" : "Low",
"priorityOrder" : {
"$numberLong" : "1930926795621"
}
}
The challenge is during de-serialization process, if i try to take this JSON and convert back to concrete object type using circe parser then it fails to map priorityOrder attribute, is there any way i can write custom encoder/decoder that will treat long data type in a special manner. The custom encoder/decoder will read value from "$numberLong" nested type and convert that value to Long datatype.
I get this exception from circe parser
Left(DecodingFailure(Long, List(El(DownField(priorityOrder),true,false))))
I was able to figure out solution to this problem by creating custom decoder for long datatype. Here is the code for individuals in similar boat
implicit val decodeLong: Decoder[Long] = new Decoder[Long] {
final def apply(c: HCursor): Decoder.Result[Long] =
{
val longval = c.downField("$numberLong").as[String] match
{
case Right(x) => x.toLong
case _ => throw new Exception("Unable to find $numberLong")
}
Right(longval)
}
}
I am building a GraphQL endpoint for an API using Finch, Circe and Sangria. The variables that come through in a GraphQL query are basically an arbitrary JSON object (let's assume there's no nesting). So for example, in my test code as Strings, here are two examples:
val variables = List(
"{\n \"foo\": 123\n}",
"{\n \"foo\": \"bar\"\n}"
)
The Sangria API expects a type for these of Map[String, Any].
I've tried a bunch of ways but have so far been unable to write a Decoder for this in Circe. Any help appreciated.
The Sangria API expects a type for these of Map[String, Any]
This is not true. Variables for an execution in sangria can be of an arbitrary type T, the only requirement that you have an instance of InputUnmarshaller[T] type class for it. All marshalling integration libraries provide an instance of InputUnmarshaller for correspondent JSON AST type.
This means that sangria-circe defines InputUnmarshaller[io.circe.Json] and you can import it with import sangria.marshalling.circe._.
Here is a small and self-contained example of how you can use circe Json as a variables:
import io.circe.Json
import sangria.schema._
import sangria.execution._
import sangria.macros._
import sangria.marshalling.circe._
val query =
graphql"""
query ($$foo: Int!, $$bar: Int!) {
add(a: $$foo, b: $$bar)
}
"""
val QueryType = ObjectType("Query", fields[Unit, Unit](
Field("add", IntType,
arguments = Argument("a", IntType) :: Argument("b", IntType) :: Nil,
resolve = c ⇒ c.arg[Int]("a") + c.arg[Int]("b"))))
val schema = Schema(QueryType)
val vars = Json.obj(
"foo" → Json.fromInt(123),
"bar" → Json.fromInt(456))
val result: Future[Json] =
Executor.execute(schema, query, variables = vars)
As you can see in this example, I used io.circe.Json as variables for an execution. The execution would produce following result JSON:
{
"data": {
"add": 579
}
}
Here's a decoder that works.
type GraphQLVariables = Map[String, Any]
val graphQlVariablesDecoder: Decoder[GraphQLVariables] = Decoder.instance { c =>
val variablesString = c.downField("variables").focus.flatMap(_.asString)
val parsedVariables = variablesString.flatMap { str =>
val variablesJsonObject = io.circe.jawn.parse(str).toOption.flatMap(_.asObject)
variablesJsonObject.map(j => j.toMap.transform { (_, value: Json) =>
val transformedValue: Any = value.fold(
(),
bool => bool,
number => number.toDouble,
str => str,
array => array.map(_.toString),
obj => obj.toMap.transform((s: String, json: Json) => json.toString)
)
transformedValue
})
}
parsedVariables match {
case None => left(DecodingFailure(s"Unable to decode GraphQL variables", c.history))
case Some(variables) => right(variables)
}
}
We basically parse the JSON, turn it into a JsonObject, then transform the values within the object fairly simplistically.
Although the above answers work for the specific case of Sangria, I'm interested in the original question: What's the best approach in Circe (which generally assumes that all types are known up front) for dealing with arbitrary chunks of Json?
It's fairly common when encoding/decoding Json that 95% of the Json is specified, but the last 5% is some type of "additional properties" chunk which can be any Json object.
Solutions I've played with:
Encode/Decode the free-form chunk as Map[String,Any]. This means you'll have to introduce implicit encoders/decoders for Map[String, Any], which can be done, but is dangerous as that implicit can be pulled into places you didn't intend.
Encode/Decode the free-form chunk as Map[String, Json]. This is the easiest approach and is supported out of the box in Circe. But now the Json serialization logic has leaked out into your API (often you'll want to keep the Json stuff completely wrapped, so you can swap in other non-json formats later).
Encode/Decode to a String, where the string is required to be a valid Json chunk. At least you haven't locked your API into a specific Json library, but it doesn't feel very nice to have to ask your users to create Json chunks in this manual way.
Create a custom trait hierarchy to hold the data (e.g sealed trait Free; FreeInt(i: Int) extends Free; FreeMap(m: Map[String, Free] extends Free; ...). Now you you can create specific encoders/decoders for it. But what you've really done is replicate the Json type hierarchy that already exists in Circe.
I'm leaning more toward option 3. Since it's the most flexible, and will introduce the least dependencies in the API. But none of them are entirely satisfying. Any other ideas?
I'm receiving an JSON graph (payload) from the client, so at the web api I get a JsValue. I simply want to take that, decorate it with a couple of fields and store it in Mongo. Something like this:
case class Plan(_id: ObjectId, name: String, payload: JsValue)
{
"_id" = 12345,
"name" : "test model",
"payload" : {a JSON graph}
}
From JsValue to database...
builder += "payload" -> JSON.parse(Json.stringify(model.payload))
From database back to a JsValue...
payload = Json.parse(dbo.as[MongoDBList]("payload").toString))
While it works to go from JsValue -> String -> MongoDBObject, I have two valid typed objects and I have to use an untyped intermediate format to go from one to another.
If you just want to store the graph as a string "payload", you can of course do that.
You may want to consider using ReactiveMongo instead of Casbah, along with Play-ReactiveMongo, which provides direct-to-JSON capability. I have not used Play-ReactiveMongo.