I am working with AWS FireLense using Fluent Bit to forward logs from ECS. There is a parsers file with a JSON parser, and the inputs are dynamically generated by the AWS FireLense image. The generated inputs are both of type forward.
Outside of the AWS specific stuff, I know that some input plugins like Tail have the ability to assign a parser. How do forward, dummy, tcp, etc., know which, if any, parser to use?
For reference, here is a link to a sample partial config section https://github.com/aws-samples/amazon-ecs-firelens-under-the-hood/blob/mainline/generated-configs/fluent-bit/generated_by_firelens.conf
Related
The vscode-antlr4 plugin for VisualStudio Code has a nice call-graph feature which visualizes (as a dendrogram) how grammar (and lexer) rules interact. You can save the graphic as SVG.
Is there a way to export the information as JSON? I wouldn't mind going into the plugin's code to find a way to do it.
My aim is to create reachability graphs for individual rules, i.e. graphs that show from which other rules a particular rule can be reached (transitively). The "calls" and "is-called" information from the call-graph feature would be a nice starting point.
The data for the call graph comes from a source context instance (for each grammar file there's a single source context to manage all details for it). See the function getReferenceGraph, which collects the relations into a map object. You can use that object to generate a JSON object from it. Or you create another function, taking this one as template, to generate the JSON directly, without the overhead required for the UI.
When writing a function for AWS Lambda (in Java/Scala), the handler function can have one of different signatures:
// Raw input / output
def handleRequest(is: InputStream): OutputStream = ???
// Using the AWS dependency
def handleRequest(input: APIGatewayProxyRequestEvent, context: Context): APIGatewayProxyResponseEvent = ???
This is the two possible signatures I know of, at least. Maybe there is more.
The difference is that the first one is a raw response, that needs to be packed into a REST response by the API Gateway with manually configured properties like media type and response code.
The second response seems to utilize the Lambda Proxy Integration and will extract all configuration from the APIGatewayProxyResponse.
Now for my question:
The field body of APIGatewayProxyResponse is of type String.
To me it looks like this POJO is serialized to JSON before being sent to the API Gateway.
This would make it impossible to serve binary data like images or PDF files.
The raw OutputStream cannot carry the information about headers etc. (or can it?), which means I cannot serve multiple different media types.
Of course I could convert images, for example, to Base64. But that is far from optimal.
Is there a way I can serve different (binary and non-binary) media types (with correct headers etc.) in a single AWS Lambda handler? How do I have to configure the API Gateway for that?
Note: This answer is based on reading the docs. I haven't tried it in practice so it may not work.
If you open the "Output Format of a Lambda Function for Proxy Integration" section of the documentation you may see isBase64Encoded field and following text:
The output body is marshalled to the frontend as the method response payload. If body is a binary blob, you can encode it as a Base64-encoded string and set isBase64Encoded to true. Otherwise, you can set it to false or leave it unspecified.
And if you open APIGatewayProxyResponse in the .Net library you can see the IsBase64Encoded property there. It looks like it is just Java API that does not expose this field. But all the rest of the infrastructure should support it. You may also see that a similar field was added to APIGatewayProxyRequestEvent.java at some point but not to APIGatewayProxyResponse. So I think a following workaround should work: create your own APIGatewayProxyResponseEvent class with isBase64Encoded field and use it. Or just extend from the standard com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent and add this field in your subclass. I expect that if you match the naming convention, it should work. It should be marshalled to a JSON after all.
I have a really complicated system which use multiple languages and frameworks (Java Python Scala Bash). In each module I need to retrieve configuration values which are similar and change frequently. Currently I'm maintaining multiple conf files which holds lots of duplicates.
I wonder if there is out of the box RestAPI which can retrieve variables by demand from remote location.
All I manage to find by now are ways to load the entire file from remote source which is half a solution from me:
YAML.parse(open('https://link_to_file/file.yaml'))
My goal, which I fail to find a lead to it, is to make a direct call.
MyRemoteAPI.get("level1.level2.x")
P.S
YAML is not mandatory solution for me, I'm Open for suggestions.
I don't know about an out-of-the-box API, but it's fairly trivial to build. Make a service that will read the YAML file and traverse to the appropriate key. e.g. using a dynamic language like Ruby (+Rails), you could do something like
def value
config = YAML.load_file '/local/path/to/config.yaml'
render plain: config.dig(params[:key].split('.'))
end
dig essentially traverses a structure and safely returns nil if a key isn't found, so this returns the value at the "leaf" of the requested path.
You might also want to cache the structure in memory to prevent constantly reading from the file, e.g. could do something like ##config ||= YAML.parse(open('https://link_to_file/file.yaml')) or config = Rails.cache.fetch('config', expire_in: 1.hour) { ... }. And/or cache the API's HTTP response.
I want to call the API of uima-text-segmenter https://code.google.com/p/uima-text-segmenter/source/browse/trunk/INSTALL?r=22 to run an example.
But I don`t know how to call the API...
the readme said,
With the DocumentAnalyzer, run the following descriptor
`desc/textSegmenter/wst-snowball-C99-JTextTilingAAE.xml` by taking the
uima-examples data as input.
Could anyone give me some code which could be run directly in main func for example?
Thanks a lot!
Long answer:
The link describes how you would set up the application from within the Eclipse UIMA environment. This sort of set-up is typically targeted at subject matter specialists with little or no coding experience. It allows them to work (relatively fast) with UIMA in a declarative way: all data structures and analysis engines (computing blocks within UIMA) are declared in xml (with a GUI on top of it), after which the framework takes care of the rest. In this scenario, you would typically run a UIMA pipeline using a run configuration from within Eclipse (or the included UIMA pipeline runner application). Luckily, UIMA allows you to do exactly the same from code, but I would recommend using UIMAFit (http://uima.apache.org/d/uimafit-current/tools.uimafit.book.html#d5e137) for this purpose instead of of UIMA, as it bundles lots of useful things and coding shortcuts.
Short answer:
Using UIMAFit, you can call Factory methods that create CollectionReader (read input), AnalysisEngine (process input) and Consumer objects (write/do other stuff) from (third-party provided) XML files. Use these methods to construct your pipeline and the SimplePipeline class to run it. To extract the data you need, you would manipulate the CAS object (containing your data) in a Consumer object, possibly with a callback. You could also do this in a Analysis Engine object. I recommend using DKPro's FeaturePathFactory (https://code.google.com/p/dkpro-core-asl/source/browse/de.tudarmstadt.ukp.dkpro.core-asl/trunk/de.tudarmstadt.ukp.dkpro.core.api.featurepath-asl/src/main/java/de/tudarmstadt/ukp/dkpro/core/api/featurepath/FeaturePathFactory.java?spec=svn1811&r=1811) to quickly access the feature you are after.
Code examples:
http://uima.apache.org/d/uimafit-current/tools.uimafit.book.html#d5e137 contains examples, but they all go in the opposite direction (class objects are used in the factory methods, instead of XML files - XML is generated from these classes). Take a look at the UIMAFit API to find the method you need, AnalysisEngineDescription from XML for example: http://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/factory/AnalysisEngineFactory.html#createEngineDescriptionFromPath-java.lang.String-java.lang.Object...-
I'm wondering whether it is at all possible to make the client ask the server for a given string, and incorporate it into another string ?
I don't see how to do that using the async approach.
As far as I know there is no really simple way to do this, because the i18n machanism of GWT replaces strings at compile-time and not at runtime.
You can try one of the following approaches:
Load the i18n in your entrypoint, store all messages in a local Map and create the Label etc, with the values from you cache. PRO: all GWT standard CONS: one request more, before you can show a translated page
Use JSP and no HTML at serverside. Wthin you jsp can create a JSON from your
message.properties and put it into your hostpage. PRO: You can synchronous read te values CONS: You will need to write a JSP which reads the properties for the correnct language, You will need to write a JSNI method to load the translated values.
Rethink, if you need a different way of translation. The built-in i18n will create tranlated versions of your app at compile-tim
I think I would use the second approach.