Huobi REST API - signing requests - Verification failure - rest

I have problem with Huobi (cryptoexchange) REST API request-signing. Rules are clear (e.g. here). Example for "v1/account/acounts" - GET with no params:
URL params:
AccessKeyId=dbye2sf5t7-d5829459-bf3aee27-67f62&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2020-11-11T12%3A17%3A57
Pre-signing text:
GET\napi.huobi.pro\n/v1/account/accounts\nAccessKeyId=dbye2sf5t7-d5829459-bf3aee27-67f62&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2020-11-11T12%3A17%3A57
Hmac SHA256-Signature of Pre-signing text with private key (verified here):
auZl70i2qsUb7+U9yYEEY1bwzLypWIM7qF1pBAJcvfc=
URL to GET: https://api.huobi.pro/v1/account/accounts?AccessKeyId=dbye2sf5t7-d5829459-bf3aee27-67f62&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2020-11-11T12%3A17%3A57&Signature=auZl70i2qsUb7%2BU9yYEEY1bwzLypWIM7qF1pBAJcvfc%3D
And the response is:
{"status":"error","err-code":"api-signature-not-valid","err-msg":"Signature not valid: Verification failure [校验失败]","data":null}
What I'm making wrong? I have searched thru many code-examples and no error found...

I had this problem with the same endpoint. I'm not sure what code you're using to sign your requests. Here's mine:
class HuobiAuth(requests.auth.AuthBase):
def __init__(self, api_key: str, secret_key: str) -> None:
self.api_key: str = api_key
self.secret_key: str = secret_key
#staticmethod
def create_sign(p_params: Dict, method: str, host_url: str, request_path: str, secret_key: str):
sorted_params = sorted(p_params.items(), key=lambda d: d[0], reverse=False)
# encode_params = urllib.urlencode(sorted_params)
encode_params = urllib.parse.urlencode(sorted_params)
payload = [method, host_url, request_path, encode_params]
payload = '\n'.join(payload)
payload = payload.encode(encoding='UTF8')
secret_key = secret_key.encode(encoding='UTF8')
digest = hmac.new(secret_key, payload, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(digest)
signature = signature.decode()
return signature
def __call__(self, request: requests.PreparedRequest) -> requests.PreparedRequest:
timestamp = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S')
params_to_sign = {'AccessKeyId': self.api_key,
'SignatureMethod': 'HmacSHA256',
'SignatureVersion': '2',
'Timestamp': timestamp}
host_name = urllib.parse.urlparse(request.url).hostname.lower()
params_to_sign['Signature'] = self.create_sign(params_to_sign, request.method, host_name, request.path_url, self.secret_key)
request.url += '?' + urllib.parse.urlencode(params_to_sign)
return request
My original problem was that instead of using request.method I had a hardcoded "POST", which I must have copied from the original source.
Example usage:
requests.post(url, json=your_data, auth=HuobiAuth(key, secret))

Related

How to Yahoo ads conversion tracker name through yahoo api

I want to change Yahoo ads conversion tracker name through API automatically.
But, when I run the following code, that returned an error.
access_token = XXXXXXXXXXXXX
account_id = XXXXXX
conversion_id = XXXXXXX
new_name = "new_name"
header = {"Content-Type": "application/json",\
"Accept": "application/json",\
"Authorization": "Bearer " + access_token}
url = "https://ads-search.yahooapis.jp/api/v8/ConversionTrackerService/set"
data = {'accountId':account_id, 'operand':[{'accountId': account_id, 'conversionTrackerId': conversion_id, "conversionTrackerName": new_name}]}
data = json.dumps(data).encode()
req = urllib.request.Request(url, data=data, method='POST', headers=header)
try:
with urllib.request.urlopen(req) as response:
body = json.loads(response.read())
headers = response.getheaders()
status = response.getcode()
print(headers)
print(status)
print(body)
except urllib.error.URLError as e:
print(e.reason)`
Retuened error is:
{'errors': [{'code': 'L0001', 'message': 'Lower list size.', 'details': [{'requestKey': 'operand', 'requestValue': None}]}], 'rid': '6fab0e1ac60dd2a871831484791976bf', 'rval': None}
I guess the length of "operand" field is 1 and it is right length according to yahoo api document. What I shuold do to fix this error?
I tryied to make length of "operand" field 2. But the result was same.

FastAPI Pytest why does client returns 422

I'm trying to implement testing my post route. It works in my project. I have problems only with pytest.
main.py:
#app.post('/create_service', status_code=status.HTTP_201_CREATED)
async def post_service(
response: Response, service: CreateService, db: Session = Depends(get_db)
):
service_model = models.Service()
service_name = service.name
service_instance = db.query(models.Service).filter(
models.Service.name == service_name
).first()
if service_instance is None:
service_model.name = service_name
db.add(service_model)
db.commit()
serviceversion_model = models.ServiceVersion()
service_instance = db.query(models.Service).filter(
models.Service.name == service_name
).first()
serviceversion_instance = db.query(models.ServiceVersion).filter(
models.ServiceVersion.service_id == service_instance.id
).filter(models.ServiceVersion.version == service.version).first()
if serviceversion_instance:
raise HTTPException(
status_code=400, detail='Version of service already exists'
)
serviceversion_model.version = service.version
serviceversion_model.is_used = service.is_used
serviceversion_model.service_id = service_instance.id
db.add(serviceversion_model)
db.commit()
db.refresh(serviceversion_model)
service_dict = service.dict()
for key in list(service_dict):
if isinstance(service_dict[key], list):
sub_dicts = service_dict[key]
if not sub_dicts:
response.status_code = status.HTTP_400_BAD_REQUEST
return HTTPException(status_code=400, detail='No keys in config')
servicekey_models = []
for i in range(len(sub_dicts)):
servicekey_model = models.ServiceKey()
servicekey_models.append(servicekey_model)
servicekey_models[i].service_id = service_instance.id
servicekey_models[i].version_id = serviceversion_model.id
servicekey_models[i].service_key = sub_dicts[i].get('service_key')
servicekey_models[i].service_value = sub_dicts[i].get('service_value')
db.add(servicekey_models[i])
db.commit()
return 'created'
test_main.py:
def test_create_service(client, db: Session = Depends(get_db)):
key = Key(service_key='testkey1', service_value='testvalue1')
service = CreateService(
name="testname1",
version="testversion1",
is_used=True,
keys=[key, ]
)
response = client.post("/create_service", params={"service": service.dict()})
assert response.status_code == 200
I tried to post service as json, as CreateService instance and finally as params dictionary. I have no errors at response line only with the last one . But I got 422 response status code. What is wrong?
If it can help:
schemas.py
class Key(BaseModel):
service_key: str
service_value: str
class CreateService(BaseModel):
name: str
version: str
is_used: bool
keys: list[Key]
class Config:
orm_mode = True
Don't use params. Use json:
response = client.post("/create_service", json=service.dict())

FastAPI pytest with arguments

I try to test fastAPI get route with pytest and the problem is how i can pass params to client.get
main.py
#app.get('/purpose'):
async def read_purpose(name, date):
"""some logic"""
return {'ok':'ok'}
test.py
client = TestClient(app)
def test_purpose():
response = client.get("/purpose", json={"name":"test_name", "date":"01.01.2020"})
assert response.status_code = 200
My test is failed. it can not find name, and date arguments.
How i can pass this arguments to my test.
Thank you
I have same problem when writing pytest for my first FastAPI demo.
#router.post('/item', tags=['items'], response_model=ShowItem)
async def create_item(item: ItemCreate,
user_id: int,
db: Session = Depends(get_db)):
date_posted = datetime.now().date()
# owner_id = 1
item = Items(**item.dict(),
date_posted=date_posted,
owner_id=user_id)
db.add(item)
db.commit()
db.refresh(item)
return item
You can try "params" instead of "json", because you are passing isolated query parameters
def test_create_item():
# wrong
data = {'title': 'Hot Boat', 'description': 'This is a boat', 'user_id': 1}
resp = client.post('/item', json.dumps(data))
# correct
data = {'title': 'Hot Boat', 'description': 'This is a boat'}
resp = client.post('/item', json.dumps(data), params={"user_id": 2})
assert resp.status_code == 200
Then i can by pass above error.
Try this fix:
client = TestClient(app)
def test_purpose():
response = client.get("/purpose", params={"name":"test_name", "date":"01.01.2020"})
assert response.status_code = 200
More detail refer Question 61383179

How to generate the IAM access tokens for analytics engine requests using python?

The documentation for Analytics Engine provides a link to generate a IAM access tokens using the CLI, but I need to generate the token with an API call. This is the CLI approach:
bx api https://api.ng.bluemix.net
bx login
<enter your credentials>
<If you are part of multiple IBM Cloud accounts, you'll be asked to choose an account for the current session. Also, you'll need to choose an organization and space in IBM Cloud.>
bx iam oauth-tokens
The documentation also states that the Cloud Foundry API is deprecated? How can I generate the IAM access tokens?
Here is the code I created in the end ...
Some utility classes for logging and exceptions:
import requests
import json
from datetime import datetime, timedelta
import logging
import os
class Logger:
def __init__(self):
format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
logging.basicConfig(format=format)
self.ch = logging.StreamHandler()
def get_logger(self, clazz):
logger = logging.getLogger(clazz)
logger.setLevel(os.getenv("LOG_LEVEL", logging.INFO))
return logger
class CloudFoundryException(Exception):
def __init__(self, message, *args):
self.message = message
super(CloudFoundryException, self).__init__(message, *args)
Then a class to do the main work:
class CloudFoundryAPI(object):
def __init__(self, api_key=None, api_key_filename=None, api_endpoint='https://api.ng.bluemix.net', provision_poll_timeout_mins=30):
self.log = Logger().get_logger(self.__class__.__name__)
self.provision_poll_timeout_mins = provision_poll_timeout_mins
assert api_key is not None or api_key_filename is not None, "You must provide a value for api_key or for api_key_filename"
# allow tests to override the api_key_filename parameter
if hasattr(CloudFoundryAPI, 'api_key_filename') and CloudFoundryAPI is not None:
api_key_filename = CloudFoundryAPI.api_key_filename
if api_key_filename is not None:
try:
with open(api_key_filename, 'r') as api_file:
d = json.load(api_file)
try:
self.api_key = d['apikey']
except KeyError:
# The attibute name used to be
self.api_key = d['apiKey']
except:
self.log.error('Error retrieving "apiKey" from file {}'.format(api_key_filename))
raise
else:
self.api_key = api_key
self.api_endpoint = api_endpoint
self.info = self._get_info()
def auth(self):
self.log.debug('Authenticating to CloudFoundry')
url = self.info['authorization_endpoint'] + '/oauth/token'
headers = {
'Content-Type': 'application/x-www-form-urlencoded;charset=utf-8',
'Accept': 'application/x-www-form-urlencoded;charset=utf-8',
'Authorization': 'Basic Y2Y6'
}
data = 'grant_type=password&username=apikey&password={}'.format(self.api_key)
try:
response = requests.post(url, headers=headers, data=data)
response.raise_for_status()
except requests.exceptions.RequestException as e:
self.log.error('Cloud Foundry Auth Response: ' + response.text)
# TODO we should define a custom application exception for this
raise
self.auth_token = response.json()
self.expires_at = datetime.now() + timedelta(seconds=self.auth_token['expires_in']/60)
self.log.debug('Authenticated to CloudFoundry')
def oidc_token(self):
self.log.debug('Retrieving IAM token')
url='https://iam.bluemix.net/identity/token'
data="grant_type=urn:ibm:params:oauth:grant-type:apikey&apikey={}".format(self.api_key)
try:
response = requests.post(url, data=data)
response.raise_for_status()
except requests.exceptions.RequestException as e:
self.log.debug('IAM token response: ' + response.text)
raise
self.oidc_token = response.json()
self.oidc_expires_at = datetime.now() + timedelta(seconds=self.oidc_token['expires_in']/60)
self.log.debug('Retrieved IAM token')
return self.oidc_token
def get_auth_token(self):
if not hasattr(self, 'auth_token') or not hasattr(self, 'expires_at') or datetime.now() > self.expires_at:
self.auth()
return self.auth_token
def get_oidc_token(self):
if not hasattr(self, 'oidc_token') or not hasattr(self, 'oidc_expires_at') or datetime.now() > self.oidc_expires_at:
self.oidc_token()
return self.oidc_token
def _request_headers(self):
auth_token = self.get_auth_token()
access_token = auth_token['access_token']
token_type = auth_token['token_type']
headers = {
'accept': 'application/json',
'authorization': '{} {}'.format(token_type, access_token),
'cache-control': 'no-cache',
'content-type': 'application/json'
}
return headers
def _request(self, url, http_method='get', data=None, description='', create_auth_headers=True):
if create_auth_headers:
headers = self._request_headers()
else:
headers = {}
try:
if http_method == 'get':
response = requests.get(url, headers=headers)
elif http_method == 'post':
response = requests.post(url, headers=headers, data=json.dumps(data))
elif http_method == 'delete':
response = requests.delete(url, headers=headers)
response.raise_for_status()
except requests.exceptions.RequestException as e:
self.log.error('{} : {} {} : {} {}'.format(description, http_method, url, response.status_code, response.text))
raise CloudFoundryException(message=response.text)
try:
self.log.debug('{} : {} {} : {} {}'.format(description, http_method, url, response.status_code, json.dumps(response.json())))
except ValueError:
self.log.debug('{} : {} {} : {} {}'.format(description, http_method, url, response.status_code, response.text))
return response
def _get_info(self):
url = '{}/v2/info'.format(self.api_endpoint)
response = self._request(url=url, http_method='get', description='_get_info', create_auth_headers=False)
return response.json()
You can then use it like so:
cf = CloudFoundryAPI(api_key="xxxx") # or pass api_key_filename
cf.get_auth_token() # get UAA token
cf.get_oidc_token() # get IAM token

Is it possible to use the OpenStack.NET SDK with SoftLayer object storage?

SoftLayer Object Storage is based on the OpenStack Swift object store.
SoftLayer provide SDKs for their object storage in Python, Ruby, Java and PHP, but not in .NET. Searching for .NET SDKs for OpenStack, I came across OpenStack.NET.
Based on this question OpenStack.NET is designed for use with Rackspace by default, but can be made to work with other OpenStack providers using CloudIdentityWithProject and OpenStackIdentityProvider.
SoftLayer provide the following information for connecting to their Object Storage:
Authentication Endpoint
Public: https://mel01.objectstorage.softlayer.net/auth/v1.0/
Private: https://mel01.objectstorage.service.networklayer.com/auth/v1.0/
Username:
SLOS123456-1:email#example.com
API Key (Password):
1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
It's not obvious how this would map to the fields of CloudIdentityWithProject, and OpenStackIdentityProvider but I tried the following and a few other combinations of project name / username / uri:
var cloudIdentity = new CloudIdentityWithProject()
{
ProjectName = "SLOS123456-1",
Username = "email#example.com",
Password = "1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
};
var identityProvider = new OpenStackIdentityProvider(
new Uri("https://mel01.objectstorage.softlayer.net/auth/v1.0/"),
cloudIdentity);
var token = identityProvider.GetToken(null);
However, in all cases I received the following error:
Unable to authenticate user and retrieve authorized service endpoints
Based on reviewing the source code for SoftLayer's other language libraries and for OpenStack.NET, it looks like SoftLayer's object storage uses V1 auth, while OpenStack.NET is using V2 auth.
Based on this article from SoftLayer and this article from SwiftStack, V1 auth uses a /auth/v1.0/ path (like the one provided by SoftLayer), with X-Auth-User and X-Auth-Key headers as arguments, and with the response contained in headers like the following:
X-Auth-Token-Expires = 83436
X-Auth-Token = AUTH_tk1234567890abcdef1234567890abcdef
X-Storage-Token = AUTH_tk1234567890abcdef1234567890abcdef
X-Storage-Url = https://mel01.objectstorage.softlayer.net/v1/AUTH_12345678-1234-1234-1234-1234567890ab
X-Trans-Id = txbc1234567890abcdef123-1234567890
Connection = keep-alive
Content-Length = 1300
Content-Type = text/html; charset=UTF-8
Date = Wed, 14 Oct 2015 01:19:45 GMT
Whereas V2 auth (identity API V2.0) uses a /v2.0/tokens path, with the request and response in JSON objects in the message body.
Based on the OpenStackIdentityProvider class in OpenStack.NET I hacked together my own SoftLayerOpenStackIdentityProvider like this:
using JSIStudios.SimpleRESTServices.Client;
using net.openstack.Core.Domain;
using net.openstack.Providers.Rackspace;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using OpenStack.Authentication;
using System;
using System.Linq;
using System.Collections.Generic;
namespace OpenStackTest1
{
public class SoftLayerOpenStackIdentityProvider : CloudIdentityProvider
{
public SoftLayerOpenStackIdentityProvider(
Uri urlBase, CloudIdentity defaultIdentity)
: base(defaultIdentity, null, null, urlBase)
{
if (urlBase == null)
throw new ArgumentNullException("urlBase");
}
public override UserAccess GetUserAccess(
CloudIdentity identity, bool forceCacheRefresh = false)
{
identity = identity ?? DefaultIdentity;
Func<UserAccess> refreshCallback =
() =>
{
// Set up request headers.
Dictionary<string, string> headers =
new Dictionary<string, string>();
headers["X-Auth-User"] = identity.Username;
headers["X-Auth-Key"] = identity.APIKey;
// Make the request.
JObject requestBody = null;
var response = ExecuteRESTRequest<JObject>(
identity,
UrlBase,
HttpMethod.GET,
requestBody,
headers: headers,
isTokenRequest: true);
if (response == null || response.Data == null)
return null;
// Get response headers.
string authToken = response.Headers.Single(
h => h.Key == "X-Auth-Token").Value;
string storageUrl = response.Headers.Single(
h => h.Key == "X-Storage-Url").Value;
string tokenExpires = response.Headers.Single(
h => h.Key == "X-Auth-Token-Expires").Value;
// Convert expiry from seconds to a date.
int tokenExpiresSeconds = Int32.Parse(tokenExpires);
DateTimeOffset tokenExpiresDate =
DateTimeOffset.UtcNow.AddSeconds(tokenExpiresSeconds);
// Create UserAccess via JSON deseralization.
UserAccess access = JsonConvert.DeserializeObject<UserAccess>(
String.Format(
"{{ " +
" token: {{ id: '{0}', expires: '{1}' }}, " +
" serviceCatalog: " +
" [ " +
" {{ " +
" endpoints: [ {{ publicUrl: '{2}' }} ], " +
" type: 'object-store', " +
" name: 'swift' " +
" }} " +
" ], " +
" user: {{ }} " +
"}}",
authToken,
tokenExpiresDate,
storageUrl));
if (access == null || access.Token == null)
return null;
return access;
};
string key = string.Format("{0}:{1}", UrlBase, identity.Username);
var userAccess = TokenCache.Get(key, refreshCallback, forceCacheRefresh);
return userAccess;
}
protected override string LookupServiceTypeKey(IServiceType serviceType)
{
return serviceType.Type;
}
}
}
Because some of the members of UserAccess (like IdentityToken and Endpoint) have no way to set their fields (the objects have only a default constructor and only read-only members), I had to create the UserAccess object by deserializing some temporary JSON in a similar format as returned by the V2 API.
This works, ie I can now connect like this:
var cloudIdentity = new CloudIdentity()
{
Username = "SLOS123456-1:email#example.com",
APIKey = "1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
};
var identityProvider = new SoftLayerOpenStackIdentityProvider(
new Uri("https://mel01.objectstorage.softlayer.net/auth/v1.0/"),
cloudIdentity);
var token = identityProvider.GetToken(null);
And then get access to files etc like this:
var cloudFilesProvider = new CloudFilesProvider(identityProvider);
var containers = cloudFilesProvider.ListContainers();
var stream = new MemoryStream();
cloudFilesProvider.GetObject("testcontainer", "testfile.dat", stream);
However, is there a better way than this to use SoftLayer Object Storage from .NET?
I briefly also looked at the OpenStack SDK for .NET (a different library to OpenStack.NET), but it too seems to be based on V2 auth.