I have installed apache drill and it is working fine. I am trying to fetch data from MYSQL table in postman REST APi
localhost:8047/query.json - launching properly
mysqlplugin configured successfully
From drill command line, MYSQL command working properly
apache drill> SELECT * FROM mysqlplugin.test.`table` LIMIT 10;
+----+-------+
| id | name |
+----+-------+
| 1 | anil |
| 2 | amit |
| 3 | ankur |
But when I connect REST API to localhost:8047/query.json
using body:
{
"queryType" : "SQL",
"query" : "select * from mysqlplugin.test.`table`;"
}
It gives response:
{
"errorMessage": "HTTP 405 Method Not Allowed"
}
What is the problem?
Related
I have a shell script that uses output from a sql query and based on the value of one column sends out an alert. However i don't think it's capturing the value. although the value is not greater than 0 yet it still sends out an email.
Any idea where i am going wrong? Thanks.
............................................................................
#!/bin/sh
psql -d postgres -U postgres -c "select pid,application_name,pg_wal_lsn_diff(pg_current_wal_lsn(), sent_lsn) sending_lag,pg_wal_lsn_diff(sent_lsn,flush_lsn) receiving_lag,pg_wal_lsn_diff(flush_lsn, replay_lsn) replaying_lag,pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn) total_lag from pg_stat_replication;"| while read total_lag;
do
echo $total_lag
lag1=$(echo $total_lag)
done
if [[ $lag1 -ge 0 ]]
then echo "Current replication lag is $lag1" |mail -s "WARNING!" abcd#mail.com
else
echo "No issue"
fi
............................................................................
this is the output of above query
pid | application_name | sending_lag | receiving_lag | replaying_lag | total_lag
-------+------------------+-------------+---------------+---------------+-----------
27823 | db123 | 0 | 0 | 0 | 0
27824 | db023 | 0 | 0 | 0 | 0
I'm trying to drop a user from a Redshift cluster but receive the following error:
drop user "xxx";
ERROR: user "xxx" cannot be dropped because permission dependency is found
I've installed the admin views and revoked all privileges from all tables and schemas. I cannot find any reference to this specific error. It is also not included in this instructional: https://aws.amazon.com/premiumsupport/knowledge-center/redshift-user-cannot-be-dropped/
select ddl from admin.v_generate_user_grant_revoke_ddl where ddltype='revoke' and grantee='xxx' order by objseq, grantseq desc;
ddl
-----
(0 rows)
select ddl, grantor, grantee from admin.v_generate_user_grant_revoke_ddl where grantee='xxx' and ddltype='grant' and objtype <>'default acl' order by objseq,grantseq;
ddl | grantor | grantee
-----+---------+---------
(0 rows)
select * from pg_user where usename = 'xxx';
usename | usesysid | usecreatedb | usesuper | usecatupd | passwd | valuntil | useconfig
------------+----------+-------------+----------+-----------+----------+----------+-----------
xxx | 110 | f | f | f | xxx | |
(1 row)
select * from pg_default_acl where defacluser=110;
defacluser | defaclnamespace | defaclobjtype | defaclacl
------------+-----------------+---------------+-----------
(0 rows)
The user in not in any groups either. Any guidances is appreciated.
The user had not run queries for at least the last two weeks, he was not very active. His only access was through a Redshift/Excel ODBC setup. I did not check initially, but a day later there were no active sessions for him. I re-ran the drop user command and got the expected result. There must have been some lingering 'something'. For reference I ran this cmd to see who had active session: select * from stv_sessions;. My problem has been resolved by trying again the next day. https://docs.aws.amazon.com/redshift/latest/dg/r_STV_SESSIONS.html
I'm trying to save a file from Azure File Storage into Azure SQL Database table varbinary(max) column (store whole content as advised in this answer). I've tried a few times to adjust my SQL query but without success. Here's the code which results in error 'Bad or inaccessible location specified in external data source "my_Azure_Files".' when it invokes OPENROWSET:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'mypassword123'
GO
CREATE DATABASE SCOPED CREDENTIAL [https://mystorageaccount.file.core.windows.net/]
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = 'sas_token_generated_on_azure_portal';
CREATE EXTERNAL DATA SOURCE my_Azure_Files
WITH (
LOCATION = 'https://mystorageaccount.file.core.windows.net/test',
CREDENTIAL = [https://mystorageaccount.file.core.windows.net/],
TYPE = BLOB_STORAGE
);
Insert into dbo.myTable(targetColumn)
Select BulkColumn FROM OPENROWSET(
BULK 'test.csv',
DATA_SOURCE = 'my_Azure_Files',
SINGLE_BLOB) AS testFile;
CLOSE MASTER KEY;
GO
I'm able to download the test.csv file by a web-browser using the same SAS token and url path. I'm also able to verify that the credential and the external source are successfully created in the database:
+-------------------------------------------------+------------------+-----------------------------------------------------+-------------------------+------------------+---------------------------+---------------+---------------+----------------+--------------------+----------+
| data_source_id | name | location | type_desc | type | resource_manager_location | credential_id | database_name | shard_map_name | connection_options | pushdown |
+-------------------------------------------------+------------------+-----------------------------------------------------+-------------------------+------------------+---------------------------+---------------+---------------+----------------+--------------------+----------+
| 65540 | my_Azure_Files | https://mystorageaccount.file.core.windows.net/test | BLOB_STORAGE | 05/01/1900 00:00 | NULL | 65539 | NULL | NULL | NULL | ON |
| name | principal_id | credential_id | credential_identity | create_date | modify_date | target_type | target_id | | | |
+-------------------------------------------------+------------------+-----------------------------------------------------+-------------------------+------------------+---------------------------+---------------+---------------+----------------+--------------------+----------+
| https://mystorageaccount.file.core.windows.net/ | 1 | 65539 | SHARED ACCESS SIGNATURE | 15/07/2020 13:14 | 15/07/2020 13:14 | NULL | NULL | | | |
When creating SAS on Azure portal I checked all allowed resource types and all allowed permissions, except 'Delete'. I also removed the leading '?' from SAS to use in the SECRET field.
I've tried variations of TYPE = BLOB_STORAGE and TYPE = HADOOP as well as SINGLE_BLOB, SINGLE_CLOB and SINGLE_NCLOB parameters.
Please help me solve my problem.
By following below steps, able to successfully insert into the target table:
While generating the SAS, please select Allowed Resource Type as ‘Container’ and ‘Object’:
Copy the SAS and use below command:
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password#123'
Use SAS token generated without ‘?’ at the start and create Scoped Credentials:
CREATE DATABASE SCOPED CREDENTIAL MyAzureBlobStorageCredential WITH IDENTITY =
'SHARED ACCESS SIGNATURE', SECRET = 'sv=2019-10-
10XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
Create External Data Source referencing your blob path:
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE,
LOCATION = 'https://mystorageaccount.file.core.windows.net'
, CREDENTIAL= MyAzureBlobStorageCredential
);
Run the insert using OPENROWSET:
Insert into dbo.test(name1)
Select BulkColumn FROM OPENROWSET(
BULK 'test/test.csv',
DATA_SOURCE = 'MyAzureBlobStorage',
SINGLE_BLOB) AS testFile;
Can also use Bulk insert:
BULK INSERT dbo.test
FROM 'test/test.csv'
WITH (DATA_SOURCE = 'MyAzureBlobStorage',
FORMAT = 'CSV');
Assuming table dbo.test is already created
I have a common feature which is used to make all soap requests by replacing fields in request.xml.
Feature: Common Soap Feature
Background:
* url 'http://XXX.XXX.XXX.XXX:8080/soap/GetTransaction'
Scenario:
* def soap = read('request.xml')
* replace soap
| token | value |
| ##MTID## | MTID |
| ##MCC_Code## | MCC_Code |
| ##Token## | Token |
| ##Trans_link## | Trans_link |
| ##traceid_lifecycle## | traceid_lifecycle |
| ##TXn_ID## | TXn_ID |
| ##Txn_Type## | Txn_Type |
| ##Bill_Amt## | Bill_Amt |
| ##Bill_Ccy## | Bill_Ccy |
| ##Settle_Amt## | Settle_Amt |
| ##Settle_Ccy## | Settle_Ccy |
| ##Fx_Fee_Fixed## | Fx_Fee_Fixed |
| ##Fx_Fee_Rate## | Fx_Fee_Rate |
| ##FX_Pad## | FX_Pad |
| ##MCC_Pad## | MCC_Pad |
| ##Txn_Amt## | Txn_Amt |
| ##Txn_CCy## | Txn_CCy |
| ##Txn_Ctry## | Txn_Ctry |
| ##Txn_GPS_Date## | Txn_GPS_Date |
| ##GPS_POS_Data## | GPS_POS_Data |
| ##GPS_POS_Capability## | GPS_POS_Capability |
| ##Proc_Code## | Proc_Code |
| ##Authorised_by_GPS## | Authorised_by_GPS |
| ##Txn_Stat_Code## | Txn_Stat_Code |
Given request soap
When soap action
Then status 200
From a caller feature, I can call above common soap feature as following and it works successfully but I can't access soap request's responseTime or othere response values.
Feature: Caller Feature
Background:
* def soap = read('soap.feature')
* def config = { username: 'XXX', password:'XXX', url: 'jdbc:mysql://XXX.XXX.XXX.XXX:32231/XXX', driverClassName: 'com.mysql.jdbc.Driver' }
* def DbUtils = Java.type('com.XXX.XXX.utils.DbUtils')
* def db = new DbUtils(config)
* def result = db.readRows('SELECT public_token FROM card where status="ACTIVE" and type="PHYSICAL" order by id desc limit 1')
* def publicToken = result[0].public_token
* def SampleUtils = Java.type('com.XXX.XXX.utils.SampleUtils')
* def traceIdLifecycle = SampleUtils.randomTraceId()
* def transactionLink = SampleUtils.randomId()
Scenario:
* def transactionId = SampleUtils.randomId()
* def gpsDateTime = SampleUtils.now()
* table parameters
| MTID | MCC_Code | Token | Trans_link | traceid_lifecycle | TXn_ID | Txn_Type | Bill_Amt | Bill_Ccy | Settle_Amt | Settle_Ccy | Fx_Fee_Fixed | Fx_Fee_Rate | FX_Pad | MCC_Pad | Txn_Amt | Txn_CCy | Txn_Ctry | Txn_GPS_Date | GPS_POS_Data | GPS_POS_Capability | Proc_Code | Authorised_by_GPS | Txn_Stat_Code |
| '0100' | '6011' | publicToken | transactionLink | traceIdLifecycle | transactionId | 'A' | '-6.95' | '826' | '0.00' | '' | '0.20' | '0.30' | '0.00' | '0.00' | '0.0000' | '949' | 'TUR' | gpsDateTime | '0051000300000Nx' | '00001001000100000000000010010000000000000012230041' | '010000' | 'N' | 'A' |
* def caller = call soap parameters
* assert caller.responseTime < 400
But the assertion which uses called feature responseTime doesn't work.
How can I access called soap feature response fields?
The data you want to access is within your variable "caller", but caller is an array.
All of the common scripts parameters are in there, the request made, the response, headers and responseTime etc.
To access the time:
assert caller[0].responseTime > 10
To access and validate the response:
* xml soapResponseXml = caller[0].response
* match soapResponseXml /Envelope/Body/AddResponse/AddResult == 4
Here's a working example. The Caller Feature:
Feature: Caller Feature
Background:
* def soap = read('common.feature')
Scenario: soap 1.2
* table parameters
| param_a | param_b |
| 1 | 3 |
* def caller = call soap parameters
* print caller
* xml soapResponseXml = caller[0].response
* match soapResponseXml /Envelope/Body/AddResponse/AddResult == 4
* assert caller[0].responseTime > 10
The Common Feature:
Feature: Common Soap Feature
Background:
* url 'http://www.dneonline.com/calculator.asmx'
Scenario:
* def soap = read('request.xml')
* replace soap
| token | value |
| ##a## | param_a |
| ##b## | param_b |
Given request soap
When soap action 'http://tempuri.org/Add'
Then status 200
And the Request XML template:
<?xml version="1.0" encoding="utf-8"?>
<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
<soap12:Body>
<Add xmlns="http://tempuri.org/">
<intA>##a##</intA>
<intB>##b##</intB>
</Add>
</soap12:Body>
Below is the output from * print caller in the caller feature:
[
{
"param_b": 3,
"responseTime": 846,
"requestMethod": "POST",
"requestParams": null,
"requestUri": "http://www.dneonline.com/calculator.asmx",
"responseStatus": 200,
"__loop": 0,
"__arg": {
"param_a": 1,
"param_b": 3
},
"requestHeaders": {
"SOAPAction": [
"http://tempuri.org/Add"
],
"Content-Type": [
"text/xml"
]
},
"responseHeaders": {
"Cache-Control": [
"private, max-age=0"
],
"Content-Length": [
"323"
],
"Content-Type": [
"application/soap+xml; charset=utf-8"
],
"Server": [
"Microsoft-IIS/7.5"
],
"X-AspNet-Version": [
"2.0.50727"
],
"X-Powered-By": [
"ASP.NET"
],
"Date": [
"Thu, 27 Sep 2018 19:57:50 GMT"
]
},
"requestBody": "<soap12:Envelope xmlns:soap12=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n <soap12:Body>\n <Add xmlns=\"http://tempuri.org/\">\n <intA>1</intA>\n <intB>3</intB>\n </Add>\n </soap12:Body>\n</soap12:Envelope>",
"response": {
"soap:Envelope": {
"_": {
"soap:Body": {
"AddResponse": {
"_": {
"AddResult": "4"
},
"#": {
"xmlns": "http://tempuri.org/"
}
}
}
},
"#": {
"xmlns:soap": "http://www.w3.org/2003/05/soap-envelope",
"xmlns:xsd": "http://www.w3.org/2001/XMLSchema",
"xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance"
}
}
},
"requestTimeStamp": 1538078269984,
"param_a": 1,
"parameters": [
"#ref:java.util.LinkedHashMap"
],
"soap": "<soap12:Envelope xmlns:soap12=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">\n <soap12:Body>\n <Add xmlns=\"http://tempuri.org/\">\n <intA>1</intA>\n <intB>3</intB>\n </Add>\n </soap12:Body>\n</soap12:Envelope>",
"responseCookies": null
}
]
An alternative would be to just call the common script without assigning the result to a variable.
Then in the common script define variables that will be in scope for the caller.
For example:
Feature: Caller Feature
Background:
* def soap = read('common.feature')
Scenario: soap 1.2
* table parameters
| param_a | param_b |
| 1 | 3 |
* call soap parameters
* match soapResponseXml /Envelope/Body/AddResponse/AddResult == 4
* assert soapReponseTime > 10
And
Feature: Common Soap Feature
Background:
* url 'http://www.dneonline.com/calculator.asmx'
Scenario:
* def soap = read('request.xml')
* replace soap
| token | value |
| ##a## | param_a |
| ##b## | param_b |
Given request soap
When soap action 'http://tempuri.org/Add'
Then status 200
* xml soapResponseXml = response
* def soapReponseTime = responseTime
* def soapResponseStatus = responseStatus
It seems that somehow, the responseTime variable is available in the scope of the feature where "When method get" (etc) is called, but it is not actually a variable in the feature scope.
Say you do the "when method get" in a feature called "makecall.feature" which is called from your test feature and its result stored in result. If you do e.g.
* def result = call read("makecall.feature')
* print result.responseTime
you will get an error saying the variable does not exist. If you examine the result object, you won't see it.
However, if you go into makecall.feature and add:
* def timeTaken = responseTime
then you can access the value via result.timeTaken.
I don't know why this is. It's oddball, because e.g. responseHeaders and responseStatus are variables in the feature scope that will be passed back in the return payload, but not responseTime.
It looks like you are doing everything right ! A few suggestions:
is this the latest version of Karate
try to temporarily make the second call in the same feature
as a workaround, you can calculate the time manually using System.currentTimeMillis()
If you think there's a problem, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
My example code output:
time | name | status | s_used | s_max |
+------------+-------------+-----------+------------+-----------+
| 1482222363 | asf | Closed | 0/16 | 0 |
| 1482222363 | as0 | Available | 4/16 | 4 |
I have attached the part of my output which is generated using perl cgi script and mysql database.
My query is how to take denominator value from the column s_used and store only the denominator values in the s_max column using perl.
3.I had attached the following part of code which i tried.
if($i == 4){
if(/s_used/){
print;
}
else{
chomp();
my($num,$s_max)=split /\//,$table_data{2}{'ENTRY'};
print $s_max;
}
}
Code Explanation:
$i == 4 is the column where should I store the variable.
I got time column from the sql database $time, name I got from $table_data{0}{'ENTRY'}, status from $table_data{1}{'ENTRY'}, s_used from $table_Data{2}{'ENTRY'}.
Expected output:
time | name | status | s_used | s_max |
+------------+-------------+-----------+------------+-----------+
| 1482222363 | asf | Closed | 0/16 | 16 |
| 1482222363 | as0 | Available | 4/16 | 16 |
Seems your code "my($num,$s_max)=split /\//,$table_data{2}{'ENTRY'};" is right.
Somehow the value $s_max at the time it's writing to the DB is incorrect. Since you did not post the portion of code to show the part $s_max writing back to the DB, you need to check what value is in $s_max (e.g. printing the $s_max value) at the time right before writing it back to DB. From there, please try to trace back why an incorrect value is assigned to $s_max. Then, the problem would be solved.