I have written KSQL Client Java app to fetch the topic message details from KSQL Tables. This is the code snippet to read the topic messages from KSQL table. But when i run this program getting below error. Please let me know how to resolve this issue.
KSQLClient Code:
ClientOptions options = ClientOptions.create()
.setHost(KSQLDB_SERVER_HOST)
.setPort(KSQLDB_SERVER_HOST_PORT);
Client client = Client.create(options);
System.out.println("Client object value ---->"+client);
// Send requests with the client by following the other examples
String query = "SELECT * FROM TESTKSQLTBLE EMIT CHANGES;";
Map<String, Object> properties = Collections.singletonMap("auto.offset.reset", "earliest");
client.streamQuery(query, properties)
.thenAccept(streamedQueryResult -> {
System.out.println("Result column names: " + streamedQueryResult.columnNames());
// RowSubscriber subscriber = new RowSubscriber();
//streamedQueryResult.subscribe(subscriber);
}).exceptionally(e -> {
System.out.println("Push query request failed: " + e);
return null;
});
Exception details:
Exception in thread "main" java.util.concurrent.ExecutionException: io.confluent.ksql.api.client.exception.KsqlClientException: Received 404 response from server: HTTP 404 Not Found. Error code: 40400
at java.util.concurrent.CompletableFuture.reportGet(Unknown Source)
at java.util.concurrent.CompletableFuture.geta(Unknown Source)
at my.ksqldb.app.KSQLExampleApp.main(KSQLExampleApp.java:55)
i'm trying to get bid and ask a primeXM broker. but i don't know what i'm doing wrong. could anyone show me an example of an answer to Massquote?
As soon as I get a MassQuote response, I'm sending a MassQuoteAsk, but the error "Required tag missing" occurs.
The logs:
ToApp 8=FIX.4.4 |9=86 |35=V |34=2 |49=XXXX|52=20200826-15:10:02.528 |56=XXXX |262=0 |263=1 |264=0 |146=1 |55=USD/JPY |10=205 |
FromApp 8=FIX.4.4 |9=135 |35=i |34=2 |49=XC80 |52=20200826-15:10:02.769 |56=Q097 |117=1 |296=1 |302=0 |295=1 |299=0 |106=10 |134=3000000 |135=1000000 |188=106.11 |190=106.112 |10=048 |
OnMessage MassQuote 8=FIX.4.4 |9=135 |35=i |34=2 |49=XC80 |52=20200826-15:10:02.769 |56=Q097 |117=1 |296=1 |302=0 |295=1 |299=0 |106=10 |134=3000000 |135=1000000 |188=106.11 |190=106.112 |10=048 |
ToApp 8=FIX.4.4 |9=57 |35=b |34=3 |49=XXXX|52=20200826-15:10:02.814 |56=XXXX |117=1 |10=002 |
ToAdmin 8=FIX.4.4 |9=100 |35=3 |34=4 |49=XXXX|52=20200826-15:10:02.931 |56=XXXX |45=3 |58=Required tag missing |371=117 |372=i |373=1 |10=238 |
Request of MarketData:
private void QueryMarketDataRequest() {
MDReqID mdReqId = new MDReqID(GetFreeID);
SubscriptionRequestType subscriptionRequestType =
new SubscriptionRequestType(SubscriptionRequestType.SNAPSHOT_PLUS_UPDATES);
MarketDepth marketDepth = new MarketDepth(0);
MarketDataRequest.NoRelatedSymGroup symbolGroup = new MarketDataRequest.NoRelatedSymGroup();
symbolGroup.Set(new Symbol("USD/JPY"));
MarketDataRequest message = new MarketDataRequest(mdReqId, subscriptionRequestType, marketDepth);
message.AddGroup(symbolGroup);
SendMessage(message);
}
MassQuote message response:
public void OnMessage(MassQuote message, SessionID sessionId) {
_logger.LogDebug($"OnMessage MassQuote {message}");
MassQuoteAcknowledgement m = new MassQuoteAcknowledgement();
m.QuoteID = message.QuoteID;
SendMessage(m);
}
It sounds like you need to get ahold of PrimeXM's connection specification and update your DataDictionary XML file to match it.
For instance, if their spec indicates that they are not going to send 117 in their MassQuote message, then you need remove 117 from the MassQuote definition in your dictionary.
my .cfg was UseDataDictionary=Y, but sometimes it seems that PrimeXM answers MassQuote (MsgType=i) without QuoteID (117), but in my FIX44.xml 117 is set as required. and this was causing the error.
so I changed .cfg UseDataDictionary=N and it stopped giving the error.
I don't know why PrimeXM sends MassQuote without QuoteID, but I think I solved my problem.
Thanks.
dbug: QuickFix.IApplication[0]
FromApp 8=FIX.4.4╔9=95╔35=i╔34=63╔49=XC80╔52=20200826-16:58:12.444╔56=Q097╔106=10╔190=106.008╔295=1╔296=1╔299=0╔302=0╔10=229╔
dbug: QuickFix.IApplication[0]
OnMessage MassQuote 8=FIX.4.4╔9=95╔35=i╔34=63╔49=XC80╔52=20200826-16:58:12.444╔56=Q097╔106=10╔190=106.008╔295=1╔296=1╔299=0╔302=0╔10=229╔
warn: QuickFix.IApplication[0]
field not found for tag: 117
my goal is to maintain the traceId (of a brave.Span) between a rabbitMq sender and consumer. In order to achieve this I use spring-cloud-sleuth autoconfigured in version 2.1.0.RELEASE
When creating a span and eventually sending a message over RabbitTemplate to the broker and receiving it in the same (test) application I would expect spring-cloud-sleuth to maintain the traceId between sender and consumer.
My observation is that when sending a Message over RabbitMq the traceId is not correctly appended to the headers of the message. There is indeed a traceId/context appended and submitted, but it is a new one, different from that I read when creating the span (see code below).
On the consumer the newly created (but unrelated) traceId then is correctly processed and can be read from the Tracer.
#Test
public void messaging_ShouldPreserveTraceId() {
final Span spanProbe = tracer.newTrace().name("TraceIT").start();
spanProbe.start();
final String traceIdBefore = spanProbe.context().traceIdString();
log.info("TRACE CONTEXT BEFORE: " + spanProbe.context());
log.info("TRACE ID BEFORE: " + traceIdBefore);
log.info("TRACE ID BEFORE Parent: " + spanProbe.context().parentIdString());
// send - the actual rabbitTemplate Call (in the producer) happens in the same (main) thread
try{
producer.sendAsString(ROUTING_KEY, "CONTENT");
}finally {
spanProbe.finish();
}
// consume
Awaitility.await().atMost(TEN_MINUTES).until(() -> {
assertThat(consumer.spans(), hasSize(1));
});
// assert
final Span consumerSpan = consumer.spans().get(0);
final String traceIdAfter = consumerSpan.context().traceIdString();
log.info("TRACE CONTEXT AFTER: " + consumerSpan.context());
log.info("TRACE ID AFTER: " + traceIdAfter);
assertEquals(traceIdAfter, traceIdBefore);
}
The consumer (test class field) is:
static class TraceTestListener implements MessageListener {
private final List<Span> spans = new ArrayList<>();
#Autowired
private Tracing tracing;
#Override
public void onMessage(Message message) {
log.info("---> Received MESSAGE: {}", message);
spans.add(tracing.tracer().currentSpan());
}
public List<Span> spans() {
return spans;
}
}
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(114) - TRACE CONTEXT BEFORE: 23ca5b3b9f068716/23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(115) - TRACE ID BEFORE: 23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(116) - TRACE ID BEFORE Parent: null
[Rabbit-4 ] org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$InternalConsumer.handleDelivery(897) - Storing delivery for consumerTag: 'amq.ctag-HnuJEiuRHAHTYfzypJDW6w' with deliveryTag: '1' in Consumer#6c27e700: tags=[[amq.ctag-HnuJEiuRHAHTYfzypJDW6w]], channel=Cached Rabbit Channel: AMQChannel(amqp://asdasdaa#35.243.142.228:5672/asdasdaa,2), conn: Proxy#33ebe4f0 Shared Rabbit Connection: SimpleConnection#3a88f6fb [delegate=amqp://asdasdaa#35.243.142.228:5672/asdasdaa, localPort= 58539], acknowledgeMode=AUTO local queue size=0
// Please mind how in the received message's headers a different traceId is present
[test_rabbitConsumer1] org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.handle(469) - Received message: (Body:'[B#1989ac6d(byte[7])' MessageProperties [headers={X-B3-SpanId=b335bbaf06a08879, X-B3-Sampled=0, X-B3-TraceId=b335bbaf06a08879}, timestamp=Tue May 21 13:44:57 CEST 2019, contentType=text/plain; charset=utf-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=test-exchange, receivedRoutingKey=test-routing, deliveryTag=1, consumerTag=amq.ctag-HnuJEiuRHAHTYfzypJDW6w, consumerQueue=test-queue])
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.log(180) - Starting scope for span: b335bbaf06a08879/46a25dd87dc63878
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.decorateScope(102) - With parent: 4663306299116113188
[test_rabbitConsumer1] sl.euth.debug.boot.rabbit.trace.TraceIT$TraceTestListener.onMessage(150) - ---> Received MESSAGE: (Body:'[B#1989ac6d(byte[7])' MessageProperties [headers={}, timestamp=Tue May 21 13:44:57 CEST 2019, contentType=text/plain; charset=utf-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=test-exchange, receivedRoutingKey=test-routing, deliveryTag=1, consumerTag=amq.ctag-HnuJEiuRHAHTYfzypJDW6w, consumerQueue=test-queue])
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.log(180) - Closing scope for span: b335bbaf06a08879/46a25dd87dc63878
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(133) - TRACE CONTEXT AFTER: b335bbaf06a08879/46a25dd87dc63878
//💥 Here we would expect the traceId to be 23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(134) - TRACE ID AFTER: b335bbaf06a08879
[main ] sl.euth.debug.commons.lib.test.junit.rules.LoggingRule$1.evaluate(77) - Finished test messaging_ShouldPreserveTraceId(sl.euth.debug.boot.rabbit.trace.TraceIT) in 13102 ms
I try to add a custom claim to an existing Identity user but I get an exception at run-time:
Npgsql.PostgresException: 23502: null value in column "Id" violates not-null constraint
Help!
What I did. I've created a simple web app on windows using the following command line
dotnet new mvc --auth Individual --framework netcoreapp1.1
I made changes found here to make the app use PostgreSQL as the database back-end. The created default webapp works fine. I can register as a new user, login, log out, etc...
Then I modified the Test method of the Home controller (I know the exceptions are ugly):
[Authorize]
public async Task<IActionResult> Test()
{
var user = await GetCurrentUserAsync();
if (user == null) {
_logger.LogWarning("User is null.");
throw new Exception("Not logged in");
}
_logger.LogWarning("User: {0}, {1}", user.Email, user);
var claim = new Claim("TestClaimType", "TestClaimValue");
IdentityResult idRes = IdentityResult.Failed();
if (_userManager.SupportsUserClaim) {
idRes = await _userManager.AddClaimAsync(user, claim); <------- Adding the claim
}
_logger.LogWarning("Return from adding claim");
if (idRes != IdentityResult.Success) {
throw new Exception("Failed to add claim.");
}
return View();
}
After logging in, I trigger the Test method and get the following logging (the PostgresException is near the end):
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/1.1 GET http://localhost:5000/Home/Test
info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationMiddleware[3]
HttpContext.User merged via AutomaticAuthentication from authenticationScheme: Identity.Application.
info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
Authorization was successful for user: mark#mark.com.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
Executing action method AlumniConnect.Controllers.HomeController.Test (AlumniConnect) with arguments ((null)) - ModelState is Valid
info: Microsoft.EntityFrameworkCore.Storage.IRelationalCommandBuilderFactory[1]
Executed DbCommand (1ms) [Parameters=[#__get_Item_0='?'], CommandType='Text', CommandTimeout='30']
SELECT "e"."Id", "e"."AccessFailedCount", "e"."ConcurrencyStamp", "e"."Email", "e"."EmailConfirmed", "e"."LockoutEnabled", "e"."LockoutEnd", "e"."NormalizedEmail", "e"."NormalizedUserName", "e"."PasswordHash", "e"."PhoneNumber", "e"."PhoneNumberConfirmed", "e"."SecurityStamp", "e"."TwoFactorEnabled", "e"."UserName"
FROM "AspNetUsers" AS "e"
WHERE "e"."Id" = #__get_Item_0
LIMIT 1
warn: AlumniConnect.Controllers.HomeController[0]
User: mark#mark.com, mark#mark.com
warn: AlumniConnect.Controllers.HomeController[0]
User: mark#mark.com, mark#mark.com
info: Microsoft.EntityFrameworkCore.Storage.IRelationalCommandBuilderFactory[1]
Executed DbCommand (4ms) [Parameters=[#__normalizedUserName_0='?'], CommandType='Text', CommandTimeout='30']
SELECT "u"."Id", "u"."AccessFailedCount", "u"."ConcurrencyStamp", "u"."Email", "u"."EmailConfirmed", "u"."LockoutEnabled", "u"."LockoutEnd", "u"."NormalizedEmail", "u"."NormalizedUserName", "u"."PasswordHash", "u"."PhoneNumber", "u"."PhoneNumberConfirmed", "u"."SecurityStamp", "u"."TwoFactorEnabled", "u"."UserName"
FROM "AspNetUsers" AS "u"
WHERE "u"."NormalizedUserName" = #__normalizedUserName_0
LIMIT 1
info: Microsoft.EntityFrameworkCore.Storage.IRelationalCommandBuilderFactory[1]
Executed DbCommand (33ms) [Parameters=[#p0='?', #p1='?', #p2='?', #p17='?', #p3='?', #p4='?', #p18='?', #p5='?', #p6='?', #p7='?', #p8='?', #p9='?', #p10='?', #p11='?', #p12='?', #p13='?', #p14='?', #p15='?', #p16='?'], CommandType='Text', CommandTimeout='30']
INSERT INTO "AspNetUserClaims" ("ClaimType", "ClaimValue", "UserId")
VALUES (#p0, #p1, #p2)
RETURNING "Id";
UPDATE "AspNetUsers" SET "AccessFailedCount" = #p3, "ConcurrencyStamp" = #p4, "Email" = #p5, "EmailConfirmed" = #p6, "LockoutEnabled" = #p7, "LockoutEnd" = #p8, "NormalizedEmail" = #p9, "NormalizedUserName" = #p10, "PasswordHash" = #p11, "PhoneNumber" = #p12, "PhoneNumberConfirmed" = #p13, "SecurityStamp" = #p14, "TwoFactorEnabled" = #p15, "UserName" = #p16
WHERE "Id" = #p17 AND "ConcurrencyStamp" = #p18;
fail: Microsoft.EntityFrameworkCore.DbContext[1]
An exception occurred in the database while saving changes.
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details. ---> Npgsql.PostgresException: 23502: null value in column "Id" violates not-null constraint
at Npgsql.NpgsqlConnector.<DoReadMessageAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
There's lots of logging more but it doesn't seem to add new information. I see the same exception mentioned multiple times throughout the log.
What can I do? Is this a PostgreSQL specific issue? Am I trying to add a claim in the wrong way?
Thanks!
In the migration for Identity, all tables that have a generated Integer id have an annotation for adding auto generation of this id. This annotation is SQL Server specific, like .Annotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn)
The fix for this is to add a Postgres specific annotation to the migration: .Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.SerialColumn). In older versions of Npgsql.EntityFrameworkCore.PostgreSQL it could be that you need to use .Annotation("Npgsql:ValueGeneratedOnAdd", true).
The part of the migration for creating the AspNetRoleClaims table will then look like:
migrationBuilder.CreateTable(
name: "AspNetRoleClaims",
columns: table => new
{
Id = table.Column<int>(nullable: false)
.Annotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn)
.Annotation("Npgsql:ValueGenerationStrategy", NpgsqlValueGenerationStrategy.SerialColumn),
ClaimType = table.Column<string>(nullable: true),
ClaimValue = table.Column<string>(nullable: true),
RoleId = table.Column<string>(nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetRoleClaims", x => x.Id);
table.ForeignKey(
name: "FK_AspNetRoleClaims_AspNetRoles_RoleId",
column: x => x.RoleId,
principalTable: "AspNetRoles",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
It appears that when the Claim is added to the database, the 'RETURNING "Id"' clause of the SQL statement suggests the ID is returned. However, the table does not have an auto incrementing ID column.
I verified this by following the instructions from Adding 'serial' to existing column in Postgres.
The problem is now of course that this should have been taken care of automatically...
hey everyone am trying to insert data in a table using #Query annotation in my spring boot app am getting a postgres Exception :
org.postgresql.util.PSQLException: No results returned by the query
this is my code :
this is the repository
#Query(value="INSERT INTO \"FCT_BY_DEV\"(\"IdDev\", \"IdFonction\") VALUES (?, ?) ",nativeQuery=true)
public String isertfonctionstodev(int dev,int fonction);
this is the controller :
#RequestMapping(value="/function/insert", method = RequestMethod.POST)
public String insererfonctions (int dev,int fonction){
System.out.println("dev="+dev+"fonction="+fonction);
fonctionRepository.isertfonctionstodev(dev, fonction);
System.out.println("********");
return "aaa";
}
am using this service by $http in angularJs
$http.post("/function/insert?dev="+$scope.id+"&fonction="+$scope.idf);
and finaly this is the server log
dev=16006fonction=14
Hibernate: INSERT INTO "FCT_BY_DEV"("IdDev", "IdFonction") VALUES (?, ?)
2016-04-27 16:52:03.204 WARN 7036 --- [nio-8080-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: 02000
2016-04-27 16:52:03.204 ERROR 7036 --- [nio-8080-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Aucun résultat retourné par la requête.
the data is correct and i tried the same query with the same value and it worked why posgres is generating this exception and how can i fixed , thanks to any help
I think modifying queries must be annotared with an extra
#Modifying
This should solve your issue:
#Transactional
#Modifying(clearAutomatically = true)
#Query(value="update policy.tbl_policy set ac_status = 'INACTIVE' where pol_id = :policyId and version_no = :version_no and ac_status = 'ACTIVE'", nativeQuery=true)
public void updateExistingRowbyId(#Param("policyId") Long pol_id, #Param("version_no") Long version_no);
Without #Transactional annotation, following errors may occur:
javax.persistence.TransactionRequiredException: Executing an update/delete query