How would one go about creating a SAP RFC that runs a transaction with parameters and return its data?
I have seen someone use a PERFORM BDC_DYNPRO and when I run the code through the debugger it seems to run the actual transaction screens. How do you go about setting this up?
There's plenty of RFCs in SAP systems that does exactly that - they're called BAPI functions. Filling parameters can be tricky sometimes and documentation for some of them are not really helpful. Take a look in transaction BAPI to see a list.
You can also create documents in transactions through code using IDOCs, that should be called using built-in IDOC RFCs.
BDCs are not really recommended for what you're trying to achieve, as they simulate the screenflow inside the system and that can consume a lot of resources for some simple tasks (like adding a new item to a document). BDCs also depends on positional references and that can be a pain to implement/maintain. BAPIs are always preferred over BDCs, however, in some cases you don't have BAPIs for a transaction and there's no other solution than using BDCs.
Finally, as I said some BAPIs can be really tricky to implement, so a RFC "wrapper" could be a way of simplifying the integration processes.
Related
At the moment we have one huge API which is used by our backoffice, our frontend, and also our public API.
This causes me a lot of headaches because when building new endpoints I find a lot of application specific logic in the code which I don't necessarily want to include in my endpoint. For example, the code to create a user might contain code to send a welcome email, but because that's not needed for the backoffice endpoint I will then need to add a new endpoint without that logic.
I was thinking about a large refactor to break our code base in to a number of smaller highly specific service APIs, then building a set of small application APIs on top of those.
So for example, an application endpoint to create a new user might do something like this after the refactor:
customerService.createCustomer();
paymentService.chargeCard();
emailService.sendWelcomeEmail();
The application and service APIs will be entirely separate code bases (perhaps a separate code base per service), they may also be built using different languages. They will only interact through REST API calls. They will be on the same local network, so latency shouldn't be a huge issue.
Is this a bad idea? I've never seen/worked on a codebase which has separated the two before, so perhaps there is a better architecture to achieve the flexibility and maintainability I'm looking for?
Advise, links, or comments would all be appreciated.
Your idea of making multiple, well-defined services is sound and really it is the best way to approach this. Going with purely micro-services approach however trendy it might seem, proves to be an overkill most often than not. This is why I'd just redesign the existing API/services properly and follow solid and sound SOA design principles below. Good Resources could be found on both serviceorientation.com and soapatterns.org I've always used them as reference in my career.
Consider what types of services you need
(image from serviceorientation.com)
Entity services are generally your Client, Payment services - e.g. services centered around an entity in your domain. They should be business-agnostic, and be able to be reused in all scenarios. They could be called sometimes by clients directly if sufficient for their needs. They could be called by Task services.
Utility services contain logic you're likely to reuse in other services, but are generally not called by the clients directly. Rather, they'd be called by Task and Entity services. An example might be a Transliteration service.
Task services combine and reuse Entity and Utility services into meaningful tasks. Most often they are not that agnostic and they do implement some specific business logic. They have meaningful business operations and they are what clients mostly call.
Principles to follow when redesigning
I strongly recommend going over this cheat sheet and making sure everything there is covered when you do your redesign. It's great help.
In general, you should make sure that:
Each service has a common context and follows the separation of concerns principle. E.g. Clients service is only for clients related operations, etc.
Each of the Entity and Utility services is business-agnostic and basic enough. So it can be reused in multiple scenarios and context without being changed. Contract must be simple - CRUD and only common operations that make sense in most usage scenarios.
Services follow a common data model - make sure all the data structures you use are used uniformly in all services in order to prevent need for integration efforts in the future and promote combination of services for clients to exploit. If you need to receive a customer that another service returns, this should be happening without the need for transformation
OK, but where to put the non-agnostic logic?
Now, you have multiple options for abstracting business logic whenever you have a need for complex business functionality. It depends on your scenario what you're going to chose:
Leave logic to all clients. Let them combine your simplified services
If there is business logic that is commonly implemented in multiple of your applications and has the potential to be reused heavily you can implement a composite service that reuses multiple existing underlying services and exposing the logic.
Service Composability. Concerns on multiple API calls communication overhead.
Well, this is an age-old question - should you make multiple API calls when they will probably create some communication overhead? The answer is - it depends on how complex your scenario is, how much reuse you expect and how flexible you want to be. Also is speed critical? To what extent? In Service Oriented Architecture though, this is a very common approach - to reuse your existing services and combine them in new configurations as needed. Yes, it does add some overhead, but I've seen implementations in very complex environments, for example Telecoms, where thanks to the use of ESB solutions, message queues, etc the overhead is negligible compared to the benefits. Here is a common architecture approach (image from serviceorientation.com):
The mandatory legacy refactoring heads-up
More often than not, changing the established contract for multiple existing client systems is a messy business and could very well lead to lots of refactoring and need for looking for needle-in-a-stack functionality that's somewhere deep in the (possibly) legacy code. Business logic might be dispersed everywhere. So make sure you're ready and have the controls, time and will to lead this battle.
Hope this helps
Is this a bad idea?
No, but this is a big overall question to be able to provide very specific advice.
I'd like to separate this into 3 areas:
Approach
Design
Technology
Working backwards, the Technology is the final and most-specific part, and totally depends on what your current environment is (platforms, skills), and (hopefully) will be reasonable self-evident to you once the other things are in progress.
The Design that you outlined above seems like a good end-state - having multiple, specific, focused APIs, each with their own responsibility. Again, the details of the design will depend on the skills of you and your organization, and the existing platforms that you have. E.g. if you are already using TIBCO (for example) and have a lot invested (licenses, platforms, tools, people) then leveraging some of their published patterns/designs/templates makes sense; but (probably) not if you don't already have TIBCO exposure.
In the abstract, the REST API services seems like a good starting point - there are a lot of tools and platforms at all levels of the system for security, deployment, monitoring, scalability, etc. If you are NGINX users, they have a lot of (platform-independent) thoughts on how to do this also NGINX blog, including some smart thinking on scalability and performance. If you are more adventurous, and have an smart, eager team, a look at Event-driven architecture - see this
Approach (or Process) is the key thing here. Ultimately, this is a refactoring, though your description of "a large refactor" does scare me a little - put that way, it sounds like you are talking about a big-bang change and calling it refactoring. Perhaps it is just language, but what's in my mind would be "an evolution of the 'one huge API' into multiple, specific, focused APIs (by refactoring the architecture)". One place to start is Martin Fowler, while this book is about refactoring software, the principles and approach are the same, just at a higher-level. Indeed, he talks about just this here
IBM talk about refactoring to microservices and make it sound easy to do in one step, but it never is (outside the lab).
You have an existing API, serving multiple internal and external clients. I will suggest that you'll want to keep this interface solid for these clients - separate your refactoring of the implementation from the additional concerns of liaising with and coordinating external systems/groups. My high-level starting approach would be:
identify a small (3-7) number of related methods on the API
ideally if a significant, limited-scope change is needed anyway with these methods, that is good - business value with the code change
design/specify a new stand-alone API specifically for these methods
at first, clone the existing model/naming/style
code a new service just for these
with proper automated CI/CD testing and deployment practices
with associated monitoring
modify the existing API to have calls to these methods re-direct to call the new service
perhaps have a run-time switch to change between the old implementation and the new implementation
remove the old implementation from codebase
capture issues, assumptions and problems along the way
the first pass will involve a lot of learning about what works and doesn't.
then repeat the process over & over, incorporating improvements each time.
At some point in the future, when appropriate due to other business-driven needs, the API published to the back-end, front-end and/or public clients can change, but that is a whole different project.
As you can see, if the API is huge (1,000 methods => 140 releases) this is a many-months process, and having a reasonably frequent release schedule is important. And there may be no value improving code that works reliably and never changes, so a (potentially) large portion of the existing API may remain, just wrapped by a new API.
Other considerations:
public API? Maybe a new version (significant changes) will be needed sooner than the internal APIs
focus on the methods/services used by it
what parts/services change the most (have the most enhancement requests approved)
these are the bits most likely to change, and could benefit most from a better process/architecture
what are future plans for change and where would the API be impacted
e.g. change to user management, change to payment processors, change to fulfilment systems
e.g. new business plans (new products/services)
consider affected methods in the API
Also see:
Using Microservices for Legacy System Modernization
Migrating From a Monolith to APIs and Microservices
Break the Monolith! Loosely Coupled Architecture Brings DevOps Success
From the CEO’s Desk: Application Modernization – Assess, Strategize, Modernize! 9
[Microservices Architecture As A Large-Scale Refactoring Tool 10
Probably the biggest 4 pieces of advice that I can give is:
think refactoring: small changes that don't affect function
think agile: small increments that are valuable, testable, achievable
think continuous: have a vision for where you will (eventually) get to, then work the process continuously
script & automate the processes from code, documentation, testing, deployment, monitoring...
improving it every time!
you have an application/API that works - keep it working!
That is always the first priority (you just need to work to carve-out time/budget for maintenance)
Not a bad idea at all.
Also what are your looking is microservices arch. and with that the question comes is how you break your system into well defined services.
We use Domain Driven Design Arch. to break our system into microservices and lagom framework , which allows every service to be in diff. code base and event driven arch. between microservices.
Now lets look at your problem at low level: you said a service contains code like creating a user and sending a email and one with just creating a user and there might be other code as well.
First we need to understand how many type of code you are writing:
Domain Object Logic (eg: User Object) -- what parameters are valid and all -- this should be independent of service endpoint and should be encapsulated in one Class like user class and we say it an Aggregate in Domain Driven Design terms
Business Reactions -- like on user creation send a email -- using event driven arch. these type of logics are separated into process managers or sagas which could most cases work conditionally like a for user created externally send a mail and for user created internally send a email , by having extra data in the event
Also the current way you are doing it , how are you handling transaction across services???
I've been a heavy Windsor users for the last several years. Prior to the Fluent Registration API, I would toggle between Xml Registration and huge piles of AddComponent() code. We've been happily using the Fluent Registration API and Installers specifically for quite some time now. I've gotten the impression from various writings like this:
http://docs.castleproject.org/Windsor.XML-Registration-Reference.ashx
That the Xml Registration approach has fallen out of favor and it wouldn't surprise me if it were marked for deprecation at some point in the near future.
Now, for my question: The Fluent Registration API and Installers work swimmingly for auto-wiring scenarios (i.e. when I want Windsor to just figure out how to construct my object graphs). Auto-wiring is the vast majority of IoC use cases out there, but what about when auto-wiring isn't possible? In other words I have multiple implementations of a service and I need to tell Windsor how to construct parts of my object graph. I've done this many times with the Xml Registration approach, but is there a more preferred approach these days? I'm hesitant to go the Xml Registration approach as its future seems uncertain, but I don't know how else to accomplish this with Windsor.
My uses cases are:
System needs to be able to swap implementations at QA-test (i.e.
credit checks and fraud detection processing where we want to test
without a dependency on a credit bureau API)
Provider patterns in our
system where we need to conditionally turn on and off different
implementations at deploy-time.
This all seems very well suited for IoC and we have all the building blocks in place, but want to make sure I'm taking the most future-proof approach with Windsor.
UPDATE:
While I like the feature toggle approach, I recently discovered a Windsor feature that is very useful on this front - Fallback Components. I'm leaving this edit here for anyone that might stumbled across this later.
Configuring your DI container completely through XML is error prone, verbose, and just too painful. The XML configuration possibilities are always a subset of what you can do with code based configuration; code is always more expressive.
Sometimes though your DI configuration depends on deploy-time configurations, but since the number of knobs you need are often fairly small, using a configuration flag is often a much better approach than polluting your configuration file with fully qualified type names.
Or let me put it differently, when you have large amounts of your DI configuration placed in your configuration file because your might want to change them at deploy time, please think again. Many of the changes need testing (by a developer) anyway, so there is no way you want someone from your operations team to fiddle around with that. And when you need a developer to look at it and verify it, what's the advantage of not having to recompile the project? Is this actually any quicker? A developer would still have to start the application anyway.
It is a false sense of flexibility and in fact a poor interface design (xml is the interface for your maintenance and operations department). BTW, are you the person that needs to document how the configuration file should be changed?
Instead of describing the list of fully qualified type names that are valid somewhere in the middle of the xml file, wouldn't it be much easier of all you have to write is "place 'false' in this field to disable ..."?
Here is an example of how to use a configuration switch:
bool detectFraught =
ConfigurationManager.AppSettings["DetectFraud"] != "false";
container.Register(
Component.For(typeof(IFraughtDetector)).ImplementedBy(
detectFraught ? typeof(RealDectector) : typeof(FakeDetector));
See how the configuration switch is now simply a boolean flag. This makes the configuration file much more maintainable, since the configuration is now a simple boolean switch instead of a complete type name (that can be misspelled).
Of course doing the ["DetectFraud"] != "false" isn't that nice by itself, but this can simply be solved by creating a strongly-typed configuration helper.
This answer might help as well. Allows you to dynamically, at runtime, provide an implementation. Though, sounds like you don't need it that dynamically and it's a little less obvious what's going on.
There are no plans to obsolete or remove the XML config support in Windsor.
Yes, you are right, it isn't a preferred approach due to its numerous drawbacks.
Anything you can do in XML can be done in code (note that inverse is not true).
Also keep in mind XML is not all-or-nothing. There are many ways to achieve the scenarios you gave as examples without resorting to registration in XML.
Feature toggles
Conditional compilation
if/else in your installer based on a appSettings flag
others...
I've used each of them in different projects in the past.
Today, I had a training on "Microsoft Workflow Foundation".
While I think the idea is neat, I still see it as a Proof Of Concept and not as a real-life solution. Building an entire application without having to type a single line of code (or only a few of them) seems just wrong.
Have you ever used this technology and if so, can it really fit to big company projects ? What drawbacks/advantages have you got using it ?
I don't see it replacing an entire application, but more likely used in the architecture of an application to make it easier to maintain and build, since business logic is separated out.
In previous application I've built, we used our own workflow solution to make our site easier to maintain and also to add new content and controls quickly without recompiling & redeploying.
MWF can definitely have a place in even small applications, depending on it's needs.
Workflow is just a part of a larger application, just like your entire app isn't WPF or ADO. And it is certainly used in real applications and very useful. Things to look for where it is a good fit are items that behave as small projects in an applications like a sales orders, hiring a new person or building a house. All thee have a start, go through multiple steps, have an ending and you are likely to manage multiple in your application. Describe the "project" steps in one or more workflows and start one for each occurrence.
As this is a very common occurrence I believe far more developers could benefit from using workflow in their applications.
Lately I've been using it to replace those parts of the codebase that are really messy as far as the branching and looping logic is concerned. When you can visualize that stuff in a picture, and just create some custom activities to perform your logic, it makes knowing exactly what is going on much easier.
The entire application wasn't rewritten, for example there's really no reason to do any CRUD type operations with workflow, but its been very helpful for some of the "heavy lifting" operations.
I've been trying to find a good form of RPC to standardize on, but so far I've ran into a ton of walls and was wondering what the stackoverflow communities view was.
My ideal RPC would provide the following:
Somewhat wide support in other languages, in that people shouldn't have to write a custom stack to use our server
Input validation
Ideally, some way to turn the above input validation into some sort of automated documentation to distribute
Clean and maintainable code
I am a fan of the catalyst framework and would prefer to stick to it, but if there is a clearly better alternative for RPC servers I'd be open to that as well.
So far I have looked at the following:
Catalyst::Controller::SOAP
Doesn't appear to support returning of complex data structures, only string('literals'). I could probably serialize data on top of that, but that seems very hacky. It also lets you return a pre-formed XML object, but I couldn't get that to work and it looks like you'd need to re-create a lot of SOAP data structure for it to work.
I do like the idea of WSDLs, but the complexity of the overall spec also makes me wonder how well support for communicating with other languages will be.
Custom POSTing XML based controller
We tried to roll our own by hand in a similar way to how we've seen two other projects do it recently where there is a dispatch url that you post XML to. This lets you have XSD validation/documentation, but required creating a lot more code than we want to maintain at this point.
Catalyst::Plugin::Server::XMLRPC
Gave a warning about using a deprecated method that will be removed in a future version of Catalyst.
No input validation or doc creation, but otherwise the best I've found
JSONRPC
Looks pretty similar to XMLRPC only the module is actually updated. I'll probably go with this next unless someone suggests something better
There also appears to be two different modules for catalyst that do JSONRPC
I realize that REST isn't pure RPC (only a subset), but...
I would recommend the Catalyst::Controller::REST and Catalyst::Action::REST modules. They're frequently updated and the documentation is fairly good. There is also a good (but rather dated) example of using Catalyst::Controller::Rest in the 2006 Catalyst Advent calendar titled Day 9 - Web Services with Catalyst::Action::REST.
FWIW, Catalyst::Controller::SOAP does support returning complex data. Take a look at the documentation http://search.cpan.org/~druoso/Catalyst-Controller-SOAP-1.23/lib/Catalyst/Controller/SOAP.pm, which will show you that you can use a WSDL to describe your service. Also, see http://daniel.ruoso.com/categoria/perl/soap-today.html.en for a more detailed step-by-step process.
i have a big web application running in perl CGI. It's running ok, it's well written, but as it was done in the past, all the html are defined hardcoded in the CGI calls, so as you could imagine, it's hard to mantain, improve and etc. So now i would like to start to add some templating and integrate with a framework (catalyst or CGI::application). My question is: Somebody here has an experience like that? There is any things that i must pay attention for? I'm aware that with both frameworks i can run native CGI scripts, so it's good because i can run both (CGI native ad "frameworked" code) together without any trauma. Any tips?
Write tests first (for example with Test::WWW::Mechanize). Then when you change things you always know if something breaks, and what it is that breaks.
Then extract HTML into templates, and commonly used subs into modules. After that it's a piece of cake to switch to a framework.
In general, go step by step so that you always have a working application.
Extricate the HTML from the processing logic in the CGI script. Identify all code that affects the HTML output, as these are candidates for becoming template variables. Separate that into a HTML file, with the identified parts marked with template variables. Eventually you will be able to refactor the page such that all processing is done at the start of the code and the HTML template just called up at the end of all processing.
In this kind of situation, rewriting from scratch basically, the old code is useful for A) testing, and B) design details. Ideally you'd make a set of tests, for all the basic functionality that you want to replicate, or at least tests that parse the final result pages so you can see the new code is returning the same information for the same inputs.
Design details within the code might be useless, depending on how much the framework handles automatically. If you have a good set of tests, and a straightforward conversion works well, you're done. If the behavior of the new doesn't match the old, you probably need to dig deeper into the "why?" and that'll probably be something odd looking, that doesn't make sense at first glance.
One thing to remember to do first is, find out if anyone has made something similar in the framework you're using. You could save yourself a LOT of time and money.
Here is how I did it using Python instead of Perl, but that should not matter:
Separated out HTML and code into distinct files. I used a template engine for that.
Created functions from the code which rendered a template with a set of parameters.
Organized the functions (which I termed views, inspired by Django) in a sensible way. (Admin views, User views, etc.) The views all follow the same calling convention!
Refactored out the database and request stuff so that the views would only contain view specific code (read: Handling GET, POST requests, etc. but nothing low-level!). Relied heavily on existing libraries for that.
I am here at the moment. :-) The next obvious step is of course:
Write a dispatcher which maps URLs to your views. This will also lead to nicer URLs and nicer 404- and error handling of course.
One of the assumptions that frameworks make is that the urls map to the code. For example in a framework you'll often see the following:
http://app.com/docs/list
http://app.com/docs/view/123
Usually though the old CGI scripts don't work like that, you're more likely to have something like:
http://app.com/docs.cgi?action=view&id=123
To take advantage of the framework you may well need to change all the urls. Whether you can do this, and how you keep old links working, may well form a large part of your decision.
Also frameworks provide support for some sort of ORM (object relational mapper) which abstracts the database calls and lets you only deal with objects. For Catalyst this is usually DBIx::Class. You should evaluate what the cost of switching to this will be.
You'll probably find that you want to do a complete rewrite, with the old code as a reference platform. This may be much less work than you expect. However start with a few toy sites to get a feel for whichever framework/orm/template you decide to go with.