I'm developing a wiki engine. Since this application can be usefull (at least for my company's private use) in its own it should be able to run as a standalone pyramid application, with its own graphical theme.
However a wiki feature could also be useful as part of a bigger project, and I would like to be able to include it into other pyramid applications.
I already found some pyramid features that could help me to achieve this but first I'm not sure whether it's the best way to do it and second some problems remain open.
Here are the potential issues I currently see:
templates: how to switch between the standalone mode and the hosted mode
host variables: event if we can reuse the host template, some variables may be needed to correctly render the templates but are not set by the guest (the wiki engine) application.
authentication: the guest app defines its own login system (based on pyramid_persona). Can the guest application reuse the host authentication system?
My current idea is to use the config.include() system of pyramid. In the wiki engine, in __init__.py I then define an include(config) method in addition the the main() method used for the standalone mode.
In the host application I then define a variable in the .ini file which points to the template file that the guest should use (ie base_template = hostapp:templates/wikibase.mako)
Inside the guest application, the includeme() method reads the base_template variable and overrides some global config.
Then each guest view work like this:
from pyramid.renderers import render
#view_config(route_name="display_wiki_page", renderer=Globals.base_template)
def view_wiki(request):
"""returns a formatted page content"""
page = request.matchdict['page']
content = get_raw_page_content_from_database(page)
page_formatted = render("wikiengine:templates/page_formatting_template.mako",
{'request': request, 'content': content} )
return {'page_formatted': page_formatted}
So from this point the base template can either be the one from the guest or the host application. Both will contain something like (in mako): ${page_formatted | n }
But this does not solve the problem of necessary host variables for the template to be rendered by the guest code. For example the host may need to have a hot_news variable that need to be displayed on each of the host pages, even the pages that host the wiki.
For this I plan use the event system, and add a subscriber for NewRequest or BeforeRender and set the needed variables here inside the request object.
Is this a correct approach ? Are there examples of what I'm trying to do?
Pyramid's configuration mechanisms make it very easy for clients of a module to override configuration. This is one of the most powerful parts of Pyramid compared to other popular web frameworks.
config.include() is a good approach to solving the problem. It allows the caller to override anything defined within the include.
Assets can be overridden using config.override_assets().
Sharing user information requires your module to either provide the user information or define a contract to which someone can conform allowing them to override your model.
Anyway this is obviously a huge topic. Highly modular apps written on top of pyramid include substanced, kotti, ptah, bookie, etc.
Related
I'm developing an extension for TYPO3 8.7 that queries an API for some data.
The frontend part works, although the API is called live at the moment.
Now I'd like to build a little backend module in which the integrator can at least manage the API credentials. Maybe also storage PIDs and other settings.
I guess storing data like these is usually done using TypoScript(?), but it would be nice to have a interface for storing settings of all kinds.
I rather guess I could create a database table and access it in the backend and frontend on a low level.
But before I do so I'd like to ensure that there is no other, designated way to do so. Maybe interact with the settings array somehow or whatnot.
I thought that storing configuration data would be a common usecase for backend modules. But I could not find any example for this.
Am I mislead about the usage of backend modules somehow?
TYPO3 Provides a Lot of Configuration Options.
in the Extension Manager
this allows you to set settings on a Systemwide Basis.
allows you to Set extension settings. using the ext_conf_template.txt this is easy for extension developers. and only accessible by admins. https://docs.typo3.org/typo3cms/CoreApiReference/ExtensionArchitecture/ConfigurationOptions/Index.html
Typoscrip Contants
this allows you to set configuration on a page basis, this is a great choice if you setup different settings on diffrent pages. or have a mutlipe sites in one TYPO3 Installation. there is an Interface "Constants Editor" in the Template Module which allows editors to set the settings of the constants.
Plugin / Flexform
the most common way an editor configures some settings. but they have to repeat them for every plugin.
Backend Modules
Backend modules are Designed for mor Complex task like manageing a News workflow. or giving inside-view into some process (status reports etc.) of cours they could be used to just store some simple settings. but its uncommen. and clutters the TYPO3 Backend Interface.
AdditionalConfiguration.php / ext_localconf.php this allows you to set very Low Level Configuration. if your configuration needs to be Availible only to Developer or needs to be Present in an Eary Bootstrapping Phase of TYPO3 this is a good choice
I your case, I would create a simple backend module and store the credentials in the Registry. This is easy to explain to the customer and you have the possibility to give access to that module to special user groups. By using the registry there's no need to create custom Tables.
Guide on how to create a backend module : https://docs.typo3.org/typo3cms/ExtbaseFluidBook/10-Outlook/2-Backend-modules.html
Guide on the Registry: https://docs.typo3.org/typo3cms/CoreApiReference/ApiOverview/SystemRegistry/Index.html
For our Web api project we use the following url versioning system:
https://{fqdn}/{apiVersion}/{apiResourceName}/{resourcePath}?{parameters}
for instance we can have something like the following:
https://myapi.mysite.com/v1/customer/2
Now considering above, let say you want to release two versions (live,test) to the customer. One live version (working with live data) and the other one is the test (working with test data for customer development test).
For live one I could easily use the one I mentioned : https://myapi.mysite.com/v1/customer/2 .
How do you name the test version of the above api? what is the test version of a api url version v1? Can specify the test api url?
Also what are best practices for fully qualified domain name of the API {fqdn} when using url versioning?
There are really several ways to do this.
One way, for instance, is to simply use attribute routing to give it a different path. Create a separate method, give it a path of /vtest/customer/2 for example and if users access this /vtest/ version (or v2 or 3 or whatever) then return the test data/new version. See an example in this question
Another way is to host your "test data" API in a different application in your server and have your web.config point to test versions of your database/source data. Using IIS, you'd configure two different applications (one for test, other for live) and the base URL would differ e.g.: https://myapi.mysite.com/appname1/v1/customer/2 vs https://myapi.mysite.com/appname2/v1/customer/2 and your appname could be something like live vs test. Have a look at this simple example
You could also just host them in different servers altogether, which would result in your {fqdn} changing between test and live versions (e.g. server.com/v1/customer/2 vs testserver.com/v1/customer/2) - this is what I do at my current job, and I find it very effective as it isolates live/test data (and API versions) avoiding confusion between them.
I also found this blog post detailing how to do this with namespaces
In other words there isn't just one best/correct way to do what you want, it all boils down to how you (or your company/boss/team) wants to structure and control test vs live data in your APIs. Take a look at these options to see which one is best in your case, hope I was able to help.
I think the title of your question is misleading. The problem you are trying to solve is not versioning (because your client is connecting to the same version of your application: v1). It is about having multiple environments: one for live data and one (or more) for test data.
At my company we solve this issue through the host name. At https://live.mysite.com/api/v1 we host v1 of the API connected to live data. At https://nodex.mysite.com/api/v1 we host v1 of the API connected to test data. Our clients can request new nodes as necessary (e.g. client1-devnode.mysite.com/api/v1 to develop against, and client1-testnode.mysite.com/api/v1 to test against. Each node get it's own set of test data.
Most of the live projects different server for different environments.
Instead of using different version of API endpoints, You should use different servers for different environment like this :
For Prod/live : https://myapi.mysite.com/v1/customer/2
For Test : https://myapi.mysitetest.com/v1/customer/2
For Dev : https://myapi.mysitedev.com/v1/customer/2
You need to configure environment specific properties for different backend endpoints you are hitting. Like : test.properties/dev.properties/live.properties
With my experience in API developing i found that there are 2 way of making server (test/developer)/live
I will show an example with your link type
https://{fqdn}/{apiVersion}/{apiResourceName}/{resourcePath}?{parameters}
In your case you can use or settings based and Link based testing type
What is settings based?
Settings based is that your server for example https://rest.mysite.com/v1/customer/2
will acting as test if you or your customer will set in he's settings server status to test and if as live - status to live.
This method is good in some cases but in order to test and to have live in same time,- this type not recommended.
What is link|URL|URI based?
This method have 2 types of identifying request is test or live
One way is to set test as a parameter https://api.mysite.com/test/v1/customer/2 and without test it goes to live
Second way is to set api to testApi or apiTest for example https://testapi.mysite.com/v1/customer/2 or https://apitest.mysite.com/v1/customer/2 . This way customer have both test and live and he can do testing and having live project too.
And don`t forget for security always check customer and verify before giving live api access.
As an option you may use custom defined header. If request contains custom header -> redirect request to test version of API.
Can someone please point me to the right direction.
I need to be able to host my GWT application in a way that it allows multiple clients to use the same application which could be separated by url's but internally using the same application.
the different sites would probably be seperated by different configurations. eg. different database, different log path etc, etc,
any ideas.?
You could use the following way to arrange your projects :
- my.application.core.project : it holds all the business logic and views for the application except for the entry point
-my.application.customerX.project : it holds only the entry point and the property files used for having the connection to the db, probably customerX specific theme
-my.application.customerY.project : it holds only the entry point and the property files used for having the connection to the db, probably customerY specific theme
Such an organization of the projects would allow you to have a common core that is distributed to each of the customers and also the ability to build on top of the core customer-specific impelementations.
The url's per client can be done with URL rewriting. Be it with an apache server in front of your application and/or in combination with a Filter in your web application.
As for the configuration, logging, and/or database per client you want a solution that doesn't store a file per client on the file system next to your application. Preferable you store client specific settings in one database and have an admin interface to manage it. For the client's data you also don't want a separate database per client, because it doesn't scale well, and would be a maintenance mess if you need to upgrade your application and databases to a newer version. Look for a multitenant architecture.
I admit this is a vague answer, but without specific system and software descriptions it's kind of hard to give a concrete answer. Nevertheless I hope this answer does give you some direction.
I have successfully achieved this by setting up separate directories in tomcat for different clients and then creating soft-links to the main application within that folder. when it comes to database connection properties and other configuration properties, instead of pointing them to the main application I just created them separately.
I'm looking at some (PHP) Frameworks, and I just noticed this in the Laravel documentation:
Like most web-development frameworks, Laravel is designed to protect your application code, bundles, and local storage by placing only files that are necessarily public in the web server's DocumentRoot. This prevents some types of server misconfiguration from making your code (including database passwords and other configuration data) accessible through the web server. It's best to be safe.
I'm familiar with CodeIgniter and CakePHP, as far as I know, these two frameworks don't do this. Should you really split it up and place your core logic outside of the webroot? In my experience, most clients use shared hosting and are not able to change their VirtualHost settings.
What kind of misconfiguration could you possibly do that would output your passwords? When developing, should you really do this?
Yes, keeping only those files which should be publicly accessible in DocumentRoot is a best practice for web application security. Consider:
Every file which is private would need a rule configured with the web server to explicitly block it.
Anyone adding files to the project needs to consider web server security settings. Simply keeping the files in separate directories makes it obvious what's public. And developers don't need to change security configurations.
Separating executable code and static files is a good practice anyway.
Not blocking access to PHP scripts can cause unintended consequences. For example, you may have a script to update some DB records when run manually at the command line, so someone simply guessing a script name can run it over the internet.
Monitoring for and cleaning malicious code written to the public directory is much easier if the real application logic is elsewhere. See Wordpress breakins for an example.
CakePHP supports this - see deployment:
CakePHP applications should have the document root set to the
application’s app/webroot. This makes the application and
configuration files inaccessible through a URL.
I am planning a Catalyst application, that will be deployed over a number of servers. It will be used internally by support staff to control aspects of an operational system that runs on those servers.
The application will run much in the same way on each server, save for a limited amount of site specific behaviours. Some actions will only apply to some servers, and some actions will behave differently on other servers.
Are there any recognized design patterns/practices that enable site-specific customization of a Catalyst application?
I am currently thinking of deploying a site configuration file alongside the application, that will be used to determine what actions to enable, and set parameters that control other action's behaviour. Ideally this customization would happen when the application is loaded by mod_perl (Apache2) - but I am not sure if that would even be possible.
Any suggestions welcome!
Catalyst::Plugin::ConfigLoader has code to help you with site-specific configuration in the form of the MYAPP_CONFIG_LOCAL_SUFFIX environment variable. Since Controllers are Components and config is available at setup_components time, you can do whatever foolery you want with action registration when your controller is compiled. There's not much pre-rolled for it, because everyone's requirements are so different, but it's not exceptionally hard, and there's advice to be found on the mailing list.
You can set templates, or have conditional behaviour in the controllers based on the value of $c->req->host.
I always use the unique combination of $HOSTNAME and $USER to define the specific configuration file to be loaded, e.g.
conf => "my_app_${hostname}_${user}.conf"