very new to the idea of using web automation with rotating proxies and headers here. I have some basic questions that I can't seem to totally have the answers confirmed from the docs.. Here are some of the questions I most want the answers to for now, please excuse if they are too basic:)
First of all, if you're using web automation for example with Selenium or Playwright, with proxies will it mean that the Selenium or Playwright scripts will be sent to the proxy servers to be used and fetch the data? Thus far, I have only been using paid proxy services that use python requests and do not offer web automation, the problem here is that sometime requests don't return full diplayed data.
Second question would be about TorPy a tool I recently found. As I understand from the manuals it is a layered process in fetching data via multiple free proxies for each request, would this be correct assumption? Many thanks.
I’m new to OPC-UA and came across eclipse milo project. Project seems interesting but there is very little developer help. I am trying to browse code to figure out how to implement Node with historical data. Project has other examples for reference but missing history service example. I tried to modify provided example in ExampleNameSpace.java to enable history on the UaVariableNode but in Prosys OPC UA Client, it doesn't enable "Show History" menu for the Node. Here is what I tried
UaVariableNode node = new UaVariableNode.UaVariableNodeBuilder(server.getNodeMap())
.setNodeId(new NodeId(namespaceIndex, "HelloWorld/Dynamic/" + name))
.setAccessLevel(ubyte(AccessLevel.getMask(AccessLevel.READ_WRITE)))
.setBrowseName(new QualifiedName(namespaceIndex, name))
.setDisplayName(LocalizedText.english(name))
.setDataType(typeId)
.setTypeDefinition(Identifiers.BaseDataVariableType)
**.setHistorizing(true)**
.build();
It will be very helpful if some one who implemented historyService using milo can share example.
UPDATE: Sorry, I should have included other part that I implemented. After reading other stack overflow post, I implemented historyRead function in namespace, which will take care of pulling history reading from datastore. My trouble right now is to indicate OPC client that Node is history capable. Test is to make prosys OPC client to enable "History" menu for the Node. I am probably missing something here.
The Milo Server SDK does not implement historical services for you.
Setting the Historizing attribute is just the tip of the iceberg. Your Namespace also has to override the historyRead (and historyUpdate if you want to support it) methods defined in AttributeHistoryManager and provide implementations. This will be impossible if you're not familiar with how UA history works, which is all defined in Part 11 of the spec.
You'll also have to take responsibility for actually storing the history for any nodes that have their Historizing attribute set, so that the services you implement actually have some data to go and query.
FWIW, developer documentation is a work in progress and should drastically improve in the next couple releases.
History is unlikely to ever be implemented as part of the SDK in such a way that you can just flip a switch and it will start working. It's fairly complicated and an efficient implementation of the services is likely to be coupled to whatever backing store you're using.
I use OmniFocus a ton and I'd really like to be able to connect my data there to other things (Zapier, IFFFT, Beeminder, etc). There's a lot of support for putting data into OmniFocus through these services, but I can't find any support for getting data out of OmniFocus.
In thinking about this, I realized my question isn't really about OmniFocus but rather about building a connector to a service that I don't own. So this is my scenario:
I have data on some publicly accessible web service (in the case of OF, it's Dropbox)
I want to build and host some sort of application that accesses that data and parses it and then provides a REST API that other servers can then query.
Ideally I'd like to make this service available to others - this seems tricky because they have to somehow enable my application to read their data.
I'm a fairly experienced software dev but I have zero experience with web applications or cloud applications. I'm not looking for a super in-depth answer here, but more of a general sketch of how this would work (or a confirmation that this really isn't feasible).
I have heard a lot of people talking recently about middleware, but what is the exact definition of middleware? When I look into middleware, I find a lot of information and some definitions, but while reading these information and definitions, it seems that mostly all 'wares' are in the middle of something. So, are all things middleware?
Or do you have an example of a ware that isn't middleware?
Lets say your company makes 4 different products, your client has another 3 different products from another 3 different companies.
Someday the client thought, why don't we integrate all our systems into one huge system. Ten minutes later their IT department said that will take 2 years.
You (the wise developer) said, why don't we just integrate all the different systems and make them work together? The client manager staring at you... You continued, we will use a Middleware, we will study the Inputs/Outputs of all different systems, the resources they use and then choose an appropriate Middleware framework.
Still explaining to the non tech manager
With Middleware framework in the middle, the first system will produce X stuff, the system Y and Z would consume those outputs and so on.
Middleware is a terribly nebulous term. What is "middleware" in one case won't be in another. In general, you can expect something classed as middleware to have the following characteristics:
Primarily (usually exclusively) software; usually doesn't need any specialized hardware.
If it weren't there, applications that depend on it would have to incorporate it as part of their application and would experience a lot of duplication.
Almost certainly connects two applications and passes data between them.
You'll notice that this is pretty much the same definition as an operating system. So, for instance, a TCP/IP stack or caching could be considered middleware. But your OS could provide the same features, too. Indeed, middleware can be thought of like a special extension to an operating system, specific to a set of applications that depend on it. It just provides a higher-level service.
Some examples of middleware:
distributed cache
message queue
transaction monitor
packet rewriter
automated backup system
Wikipedia has a quite good explanation: http://en.wikipedia.org/wiki/Middleware
It starts with
Middleware is computer software that connects software components or applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact.
What is Middleware gives a few examples.
There are (at least) three different definitions I'm aware of
in business computing, middleware is messaging and integration software between applications and services
in gaming, middleware is pretty well anything that is provided by a third-party
in (some) embedded software systems, middleware provides services that applications use, which are composed out of the functions provided by the hardware abstraction layer - it sits between the application layer and the hardware abstraction layer.
Simply put Middleware is a software component which provides services to integrate disparate systems together.
In an complex enterprise environment, there are a number of challenges when you need to integrate two or more enterprise systems together to talk to each other. Normally these systems do not understand each others language as they are developed on different platforms using different languages (like C++, Java, Cobol, etc.).
So here comes middleware software in picture which provides services like
transformation of messages formats from one app to other,
routing and enriching messages besides taking care of security,
encryption,
validation and
applying different business rules to these messages.
A typical example of middleware is an ESB products like IBM message broker (WMB/IIB), WESB, Datapower XI50, Oracle Fusion, Mule and many others.
Therefore, middleware sits mostly in between the service consuming apps and services provider apps and help these apps to talk to each other.
Middleware is about how our application responds to incoming requests. Middlewares look into the incoming request, and make decisions based on this request. We can build entire applications only using middlewares. For e.g. ASP.NET is a web framework comprising of following chief HTTP middleware components.
Exception/error handling
Static file server
Authentication
MVC
As shown in the above diagram, there are various middleware components in ASP.NET which receive the incoming request, and redirect it to a C# class (in this case a controller class).
Middleware is a general term for software that serves to "glue together" separate, often complex and already existing, programs. Some software components that are frequently connected with middleware include enterprise applications and Web services.
There is a common definition in web application development which is (and I'm making this wording up but it seems to fit): A component which is designed to modify an HTTP request and/or response but does not (usually) serve the response in its entirety, designed to be chained together to form a pipeline of behavioral changes during request processing.
Examples of tasks that are commonly implemented by middleware:
Gzip response compression
HTTP authentication
Request logging
The key point here is that none of these is fully responsible for responding to the client. Instead each changes the behavior in some way as part of the pipeline, leaving the actual response to come from something later in the sequence (pipeline).
Usually, the middlewares are run before some sort of "router", which examines the request (often the path) and calls the appropriate code to generate the response.
Personally, I hate the term "middleware" for its genericity but it is in common use.
Here is an additional explanation specifically applicable to Ruby on Rails.
Middleware stands between web applications and web services that natively can't communicate and often are written in different languages/frameworks.
One such example is OWIN middleware for .NET environment, before owin people were forced to host web apps in a microsoft hosting software called IIS. After owin was developed, it has added capacity to host both in IIS and self host, in IIS was just added support for Owin which acted as an interface. Also it become possible to host .NET web apps on Linux via Mono, which again added support for Owin.
It also added capacity to create Single Page Applications, Owin handling Http request/response context, so on top of owin you can add authentication/authorization logic via OAuth2 for example, you can configure middleware to register a class which contains logic of user authentification (for ex. OAuth2 implementation) or class which contains logic of how to manage http request/response messages, that way you can make one application communicate with other applications/services via different data format (like json, xml, etc if you are targeting web).
Some examples of middleware: CORBA, Remote Method Invocation (RMI),...
The examples mentioned above are all pieces of software allowing you to take care of communication between different processes (either running on the same machine or distributed over e.g. the internet).
From my own experience with webwork, a middleware was stuff between users (the web browser) and the backend database. It was the software that took stuff that users put in (example: orders for iPads, did some magical business logic, i.e. check if there are enough iPads available to fill the order) and updated the backend database to reflect those changes.
It is just a piece of software or a tool on which your application executes and rapplication capabilities with respect to high availability,scalability,integrating with other softwares or systems without you bothering about your application level code changes .
For example : The operating system on which your application runs requires an I.P change , you do not have to worry about it in your code , it is the middleware stack on which you can simple update the configuration.
Example 2 : You experience problems with your runtime memory allocation and feel that the your application usage has increased , you do not have to much about it unless you have a bug or bottleneck in your code , it is easily achievable by tuning middleware software configuration on which your application runs.
Example 3 : You have multiple disparate software and you need them to talk to each other or send data in a common format which is understandable by all the systems then this is where middleware systems comes handy.
Hope the information provided helps.
it is a software layer between the operating system
and applications on each side of a distributed computing system in a network. In fact it connects heterogeneous network and software systems.
If I am not wrong, in software application framework, based on the context, you can consider middleware for the following roles that can be combined in order to perform certain activities in between the user request and the application response.
Adapter
Sanitizer
Validator
I always thought of it as the oldest software I have had to install. The total app used a web server, a database server, and an application server. The web server being the middleware between the data and the app.
Greetings!
I've spent some time researching how to communicate with SAP from a PHP web application, but the documentation is more than spare. The ways I found are:
Creating IDoc files, some sort of proprietary plain-text format. But how do I stick them in SAP when they are created on the webserver?
Coding a Windows Application using the SAP BAPI. This way, you have full control and access - but also a lot of pain.
Accessing the RFC (Remote Function Calls), preferrably via SOAP. This would be the best way to connect to a web app.
What I could not find out is: what is the easiest way to connect? Which versions of SAP support SOAP? Anything I've missed?
Thanks for your help!
How you connect to SAP is dependent on what version of SAP, your SAP license and what you're connecting with and what you are trying to do.
If you are trying to access SAP standard functionality - i.e. Read or Update a SAP Service Order, the standard BAPI should allow you to do what you need. (Note that a BAPI is just an RFC with a standard API, which is released for customer use by SAP).
There are various connectors to SAP, but for newer releases of SAP (2004s and later) many RFC function modules are exposed as webservices, which should be ideal for your purposes.
You may have to ask an ABAP developer (or even a non-developer with SAP experience) to help you find which RFC's are available and whether they are exposed as webservices. (If not an ABAP developer should be able to wrap BAPI as a webservice with a simplified interface).
If you are trying to do something that SAP have not provided prior functionality for, or perhaps access custom developed objects you will definitely need the assistance of an ABAP developer to exposes an RFC to you.
Depending on your system setup there may be some system administration tasks to be done in order to allow you to access the SAP system. Whether by RFC or otherwise. (Useful transactions to search for are SICF and SOAMANAGER). The SAP Basis administrator may also need to set up some RFC connections for you.
If your company uses SAP PI/XI or a non-SAP integration engine, it may be easier to connect but that would be in very limited cases.
I've used IDOCs before for single message transfer, but I'm not sure that it would have any advantage over BAPI's or webservices for any Business Type transactions. Although IDOCs have been around for a while and may be quite stable if standard ones exist.
I have to admit that I've only scratched the surface around connecting to SAP, but I hope this is useful.
The third option is the easiest.
Accessing the RFC (Remote Function
Calls), preferrably via SOAP.