building REST API in FLASK - rest

A very basic question. I have a FLASK app which has postgresql behind it. There are no ORM for this application. All requests are done via SQL psycopg2 interface.
Now I want to expose certain API's from this application. What would be the best way to proceed.
1> Just like: http://flask-peewee.readthedocs.org/en/latest/rest-api.html
2> or can I do one without the ORM. It seems that ORM for an RESTful API is very useful but in this case I have to have a separate database elements and copy data from the postgres model to the ORM.
any suggestion would be welcome.

I have a similar setup Flask + Postgres and PsycoPG2.
I followed the following tutorials to design and implement the API
I handle errors manually and respond with the appropriate HTTP code
http://blog.luisrei.com/articles/rest.html { Design API}
http://blog.luisrei.com/articles/flaskrest.html { Implement API}

Looks like Flask-Restless is a better choice. Validations, authentication support are much simpler with it.

For non-trivial applications better use flask-classy. Flask-Restless is somewhat limiting, and flask-restful doesn't really give much over flask-classy, besides being more complex.
I personally used flask-restless for some time before moving to flask-classy.

There is bunch of different frameworks on top of flask now.
http://python-eve.org/index.html
http://www.flaskapi.org/

REST API in FLASK and Postman app:
Code:
from flask_sqlalchemy import SQLAlchemy
from flask import Flask,render_template,request,jsonify,json
import psycopg2
import psycopg2.extras
app = Flask(__name__)
db = SQLAlchemy()
conn = psycopg2.connect("postgresql://postgres:postgres#localhost:5432/country")
cr = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
#app.route('/', methods=['GET'])
def test():
return jsonify({
'message': 'Welcome'
})
#app.errorhandler(404)
def page_not_found(e):
return "<h1>404</h1><p>The resource could not be found.</p>", 404
##### Countries ######
#app.route('/country/all', methods=['GET'])
def country():
cr.execute('select * from country')
country = cr.fetchall()
countries = []
for row in country:
countries.append(dict(row))
return jsonify(countries)
if __name__ == '__main__':
app.run(debug=True, port=8080)
Result:
**

Related

Different OpenAPI schema in FastAPI depending on environment

We have a FastApi application that is hosted behind a reverse proxy.
The proxy authenticates the user using Kerberos and adds a X-Remote-User HTTP header to the request.
This header is required by the FastApi application. Here is an example route:
#app.get("/user/me")
async def get_user_me(x_remote_user: str = Header(...)):
return {"User": x_remote_user}
The X-Remote-User header is required for the request which is expected behavior.
When we now open the Swagger Ui, the header is documented and when clicking on "Try it out", we can provide the header value.
This behavior is great for development, but in all other cases it is undesired, because that header is provided by the reverse proxy. For instance, we generate clients using OpenAPI Generator and the clients then all require the X-Remote-User parameter in their requests.
Hence, it would be useful to have a configuration that distinguishes between the environments. If we are behind a reverse proxy, then the generated OpenAPI Schema by FastApi should not include the X-Remote-Header, otherwise if we are in development, it should be included.
What I did so far:
I checked the documentation about security and also some source code of these modules, but I was not able to find a solution.
In the documentation, I read the section Behind a Proxy, but nothing there points me to a potential solution.
I also read about Middleware, but again, no solution.
We could change the generated OpenApi schema. I sketched this in my answer below, but this is not a very elegant solution
Does anyone have a good solution to this problem?
We can use APIKeyHeader to remove the X-Remote-User header from the API signature, but still enforcing the header to be present.
from fastapi.security import APIKeyHeader
apiKey = APIKeyHeader(name="X-Remote-User")
#app.get("/user/me")
async def get_user_me(x_remote_user: str = Depends(apiKey)):
return {"User": x_remote_user}
When the header is not present, we get a "403 Forbidden". If it is present, we retrieve the header value.
The Swagger UI now has a button "Authorize" where we can fill-in the value of the X-Remote-User for testing purposes.
One approach is to generate the OpenApi schema as described in the documentation Extending OpenAPI. After the generation, remove the X-Remote-User from the schema. In the configuration could be a flag that the application it is behind a reverse proxy to execute the code conditionally:
from fastapi import FastAPI
from fastapi.openapi.utils import get_openapi
from MyConfig import Config
app = FastAPI()
#app.get("/items/")
async def read_items():
return [{"name": "Foo"}]
if Config.reverse_proxy:
def custom_openapi():
if app.openapi_schema:
return app.openapi_schema
openapi_schema = get_openapi(
title="Custom title",
version="2.5.0",
description="This is a very custom OpenAPI schema",
routes=app.routes,
)
// remove X-Remote-User here
app.openapi_schema = openapi_schema
return app.openapi_schema
app.openapi = custom_openapi
However this is not a very elegant solution, as we need to parse the Json string and remove the different deeply-nested occurrences of the X-Remote-User header everywhere. This is prone to bugs resulting in an invalid schema. Furthermore it could break if new Rest endpoints are added.
A new param will be soon available for Header, Query and other to exclude elements from the openAPI output: include_in_schema=False
Example:
def test(x_forwarded_for: str = Header(None, include_in_schema=False)):
...
Here the patch state: https://github.com/tiangolo/fastapi/pull/3144

Token-based Authentication with Cornice for Pyramid

I am using the Resources strategy of developing a RESTful API within a Pyramid App.
http://cornice.readthedocs.io/en/latest/resources.html. However I couldn't find an example of adding authentication for the API. Any guidance is greatly appreciated.
As Antoine Leclair pointed out, Cornice relies on Pyramid. You will have to enable an authorization and an authentication policies during your app initialization. For example (here using pyramid-jwt):
from pyramid.config import Configurator
from pyramid.authorization import ACLAuthorizationPolicy
def main():
config = Configurator()
# Pyramid requires an authorization policy to be active.
config.set_authorization_policy(ACLAuthorizationPolicy())
# Enable JWT authentication.
config.include('pyramid_jwt')
config.set_jwt_authentication_policy('secret')
You can also create your own policy, by inheriting from builtin Pyramid classes in pyramid.authentication:
from pyramid.authentication import CallbackAuthenticationPolicy
from pyramid.interfaces import IAuthenticationPolicy
from zope.interface import implementer
#implementer(IAuthenticationPolicy)
class MyAuthenticationPolicy(CallbackAuthenticationPolicy):
def __init__(self, realm='Realm'):
self.realm = realm
def unauthenticated_userid(self, request):
user_id = self._get_credentials(request)
return user_id
def forget(self, request):
return [('WWW-Authenticate', 'MyAuth realm="%s"' % self.realm)]
def _get_credentials(self, request):
authorization = request.headers.get('Authorization', '')
# your own strategy...
# if valid: return user_id
# else return None
Check out existing projects on awesome-pyramid to see if what you need is already there...

How to represent instantiation of DocumentClient in Powershell

Having a hard time figuring out how to do the same thing in powershell as the followings lines:
(in namespace Microsoft.Azure.Documents)
DocumentClient client = new DocumentClient(new Uri("endpoint"), "authKey")
Database database = client.CreateDatabaseQuery().Where(d => d.Id == "collectionName").AsEnumerable().FirstOrDefault()
Can anyone help?
tx
Look here: https://alexandrebrisebois.wordpress.com/2014/08/23/using-powershell-to-seed-azure-documentdb-from-blob-storage/
It shows you how use the authKey endpoint uri in a raw REST request.
Also, study the REST API for DocumentDB here: https://msdn.microsoft.com/en-us/library/azure/dn781481.aspx?f=255&MSPPError=-2147217396.
It'll allow you to look up how to do more operations following the Alexandre's example.
There is also this powershell commandlet DLL that makes many of the operations easy: https://github.com/savjani/Azure-DocumentDB-Powershell-Cmdlets

Qualified element/attribute forms and unqualified forms with Spyne soap server

Is there any way to use elementFormDefault="unqualified" server schema type with Spyne server?
Now my all trials end up with method response result:
<senv:Envelope xmlns:tns="http://test.com/remoteService/"
xmlns:senv="http://schemas.xmlsoap.org/soap/envelope/">
<senv:Body>
<tns:testResponse>
<tns:status>ok</tns:status>
</tns:testResponse>
</senv:Body>
And generated wsdl fragment with "qualified" elementFormDefault :
<xs:schema targetNamespace="http://test.com/remoteService/" elementFormDefault="qualified"></xs:schema>
How to configure method or parameters model to get result like this:
<senv:Envelope xmlns:tns="http://test.com/remoteService/"
xmlns:senv="http://schemas.xmlsoap.org/soap/envelope/">
<senv:Body>
<tns:testResponse>
<status>ok<status>
</tns:testResponse>
</senv:Body>
My goal is to generate result where child element:
<tns:status>ok</tns:status>
will appear without namespace prefix - like this:
<status>ok<status>
If you wonder how to add listener to the event_manager for method_return_string or for another event, see bellow a full example:
from spyne import Application, rpc, ServiceBase, Iterable, Integer, Unicode
from spyne.protocol.soap import Soap11
from spyne.server.wsgi import WsgiApplication
class HelloWorldService(ServiceBase):
#rpc(Unicode, Integer, _returns=Iterable(Unicode))
def say_hello(ctx, name, times):
for i in range(times):
yield u'Hello, %s' % name
def on_method_return_string(ctx):
ctx.out_string[0] = ctx.out_string[0].replace(b'Hello>', b'Good by')
HelloWorldService.event_manager.add_listener('method_return_string',
on_method_return_string)
application = Application([HelloWorldService], 'spyne.examples.hello.soap',
in_protocol=Soap11(validator='lxml'),
out_protocol=Soap11())
wsgi_application = WsgiApplication(application)
if __name__ == '__main__':
import logging
from wsgiref.simple_server import make_server
server = make_server('127.0.0.1', 8000, wsgi_application)
server.serve_forever()
As of Spyne 2.12 this is still the only way to remove namespaces from response variables.
As of 2.10, Spyne does not support this.
The patch would be a bit hairy. Chime in at soap#python.org if you're willing to work on this.
A workaround would be to remove namespace prefixes manually from outgoing documents in a method_return_document hook. If you need to enforce the same for incoming documents as well, you either have to modify the Wsdl as well in a document_built event, or use soft validation (soft validation does not care about namespaces) or no validation at all.

Looking for a way to rewrite Postgresql query before executing it

I have a Postgres client sending queries like
SELECT ... FROM "public"."mycontent" "mycontent"
WHERE (strpos(substring("mycontent"."summary" from 1), 'search_string') + 1 - 1 > 0)
to our Postgres server. I want the client to use my full text search function, but I have no access to the client's code. So I am looking for a way to rewrite all incoming query in the above form to something like:
SELECT ... FROM "public"."mycontent" "mycontent"
WHERE id in full_text_search('search_string')
Note the extraction of the 'search_string', so Postgres Rules cannot be used here because they don't do such extraction. I hope anyone knows of any postgres middleware or proxy that can do query rewrite, or is there any other better idea? Thank you.
I guess I have to answer my own question. I implemented a postgres proxy server for rewriting query, using python gevent socket programming. Note this doesn't work if connection uses SSL.
from gevent import socket, server, Greenlet, joinall
def pipe(source_socket, destination_socket, modify=False):
while True:
try:
data = source_socket.recv(1024)
except socket.error, e:
break
else:
if data:
if modify: data = data.replace("limit 10", "limit 1 ")
destination_socket.send(data)
else:
break
def pg_proxy(client_socket, address):
pg_socket = socket.create_connection(("localhost", 5432))
pg_socket.settimeout(300.0)
client_socket.settimeout(300.0)
joinall((
Greenlet.spawn(pipe, client_socket, pg_socket, modify=True),
Greenlet.spawn(pipe, pg_socket, client_socket, modify=False)
))
pg_socket.close()
client_socket.close()
if __name__ == '__main__':
s = server.StreamServer(("localhost", 5433), pg_proxy)
s.serve_forever()
Ten years later...
There is now something that can do that for you, and it does support SSL (and it's free): Gallium Data. It's a smart database proxy that allows you to change any database request or response to your heart's content. It works with Postgres, MySQL, SQL Server and Mongo.
Disclosure: I am the founder of Gallium Data.
I'm wondoering if tou can rewrite the full_text_search as an SQL function (following postgres sql dialect) executing on the database server.
actually postgres provides a complete set of function to manges strings http://www.postgresql.org/docs/9.0/static/functions-string.html