where to put images uploaded to be viewed in browser [duplicate] - rest

I read here that one should not save the file in the server anyway as it is not portable, transactional and requires external parameters. However, given that I need a tmp solution for tomcat (7) and that I have (relative) control over the server machine I want to know :
What is the best place to save the file ? Should I save it in /WEB-INF/uploads (advised against here) or someplace under $CATALINA_BASE (see here) or ... ? The JavaEE 6 tutorial gets the path from the user (:wtf:). NB : The file should not be downloadable by any means.
Should I set up a config parameter as detailed here ? I'd appreciate some code (I'd rather give it a relative path - so it is at least Tomcat portable) - Part.write() looks promising - but apparently needs a absolute path
I'd be interested in an exposition of the disadvantages of this approach vs a database/JCR repository one
Unfortunately the FileServlet by #BalusC concentrates on downloading files, while his answer on uploading files skips the part on where to save the file.
A solution easily convertible to use a DB or a JCR implementation (like jackrabbit) would be preferable.

Store it anywhere in an accessible location except of the IDE's project folder aka the server's deploy folder, for reasons mentioned in the answer to Uploaded image only available after refreshing the page:
Changes in the IDE's project folder does not immediately get reflected in the server's work folder. There's kind of a background job in the IDE which takes care that the server's work folder get synced with last updates (this is in IDE terms called "publishing"). This is the main cause of the problem you're seeing.
In real world code there are circumstances where storing uploaded files in the webapp's deploy folder will not work at all. Some servers do (either by default or by configuration) not expand the deployed WAR file into the local disk file system, but instead fully in the memory. You can't create new files in the memory without basically editing the deployed WAR file and redeploying it.
Even when the server expands the deployed WAR file into the local disk file system, all newly created files will get lost on a redeploy or even a simple restart, simply because those new files are not part of the original WAR file.
It really doesn't matter to me or anyone else where exactly on the local disk file system it will be saved, as long as you do not ever use getRealPath() method. Using that method is in any case alarming.
The path to the storage location can in turn be definied in many ways. You have to do it all by yourself. Perhaps this is where your confusion is caused because you somehow expected that the server does that all automagically. Please note that #MultipartConfig(location) does not specify the final upload destination, but the temporary storage location for the case file size exceeds memory storage threshold.
So, the path to the final storage location can be definied in either of the following ways:
Hardcoded:
File uploads = new File("/path/to/uploads");
Environment variable via SET UPLOAD_LOCATION=/path/to/uploads:
File uploads = new File(System.getenv("UPLOAD_LOCATION"));
VM argument during server startup via -Dupload.location="/path/to/uploads":
File uploads = new File(System.getProperty("upload.location"));
*.properties file entry as upload.location=/path/to/uploads:
File uploads = new File(properties.getProperty("upload.location"));
web.xml <context-param> with name upload.location and value /path/to/uploads:
File uploads = new File(getServletContext().getInitParameter("upload.location"));
If any, use the server-provided location, e.g. in JBoss AS/WildFly:
File uploads = new File(System.getProperty("jboss.server.data.dir"), "uploads");
Either way, you can easily reference and save the file as follows:
File file = new File(uploads, "somefilename.ext");
try (InputStream input = part.getInputStream()) {
Files.copy(input, file.toPath());
}
Or, when you want to autogenerate an unique file name to prevent users from overwriting existing files with coincidentally the same name:
File file = File.createTempFile("somefilename-", ".ext", uploads);
try (InputStream input = part.getInputStream()) {
Files.copy(input, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
How to obtain part in JSP/Servlet is answered in How to upload files to server using JSP/Servlet? and how to obtain part in JSF is answered in How to upload file using JSF 2.2 <h:inputFile>? Where is the saved File?
Note: do not use Part#write() as it interprets the path relative to the temporary storage location defined in #MultipartConfig(location). Also make absolutely sure that you aren't corrupting binary files such as PDF files or image files by converting bytes to characters during reading/writing by incorrectly using a Reader/Writer instead of InputStream/OutputStream.
See also:
How to save uploaded file in JSF (JSF-targeted, but the principle is pretty much the same)
Simplest way to serve static data from outside the application server in a Java web application (in case you want to serve it back)
How to save generated file temporarily in servlet based web application

I post my final way of doing it based on the accepted answer:
#SuppressWarnings("serial")
#WebServlet("/")
#MultipartConfig
public final class DataCollectionServlet extends Controller {
private static final String UPLOAD_LOCATION_PROPERTY_KEY="upload.location";
private String uploadsDirName;
#Override
public void init() throws ServletException {
super.init();
uploadsDirName = property(UPLOAD_LOCATION_PROPERTY_KEY);
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
// ...
}
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
Collection<Part> parts = req.getParts();
for (Part part : parts) {
File save = new File(uploadsDirName, getFilename(part) + "_"
+ System.currentTimeMillis());
final String absolutePath = save.getAbsolutePath();
log.debug(absolutePath);
part.write(absolutePath);
sc.getRequestDispatcher(DATA_COLLECTION_JSP).forward(req, resp);
}
}
// helpers
private static String getFilename(Part part) {
// courtesy of BalusC : http://stackoverflow.com/a/2424824/281545
for (String cd : part.getHeader("content-disposition").split(";")) {
if (cd.trim().startsWith("filename")) {
String filename = cd.substring(cd.indexOf('=') + 1).trim()
.replace("\"", "");
return filename.substring(filename.lastIndexOf('/') + 1)
.substring(filename.lastIndexOf('\\') + 1); // MSIE fix.
}
}
return null;
}
}
where :
#SuppressWarnings("serial")
class Controller extends HttpServlet {
static final String DATA_COLLECTION_JSP="/WEB-INF/jsp/data_collection.jsp";
static ServletContext sc;
Logger log;
// private
// "/WEB-INF/app.properties" also works...
private static final String PROPERTIES_PATH = "WEB-INF/app.properties";
private Properties properties;
#Override
public void init() throws ServletException {
super.init();
// synchronize !
if (sc == null) sc = getServletContext();
log = LoggerFactory.getLogger(this.getClass());
try {
loadProperties();
} catch (IOException e) {
throw new RuntimeException("Can't load properties file", e);
}
}
private void loadProperties() throws IOException {
try(InputStream is= sc.getResourceAsStream(PROPERTIES_PATH)) {
if (is == null)
throw new RuntimeException("Can't locate properties file");
properties = new Properties();
properties.load(is);
}
}
String property(final String key) {
return properties.getProperty(key);
}
}
and the /WEB-INF/app.properties :
upload.location=C:/_/
HTH and if you find a bug let me know

Related

Can we customize mapping file names in Wiremock?

I am recording the application through Wiremock using JAVA DSL, Do we have the option to customize the mapping file names? instead of getting the filename which is generated from wiremock..
Example: searchpanel_arrivalairport_th-72f9b8b7-076f-4102-b6a8-aa38710fde1b.json (Generated form wiremock using java )
I am expecting the above file name with my desired naming convention like
seacrpanel_airport_LGW.json
Custom filenames can be added by customizing StubMappingJsonRecorder.
I added CustomStubMappingJsonRecorder and override writeToMappingAndBodyFile method.
if(fileName!=null && !fileName.equals("")){
mappingFileName=fileName+"-mapping.json";
bodyFileName=fileName+"-body.json";
}else {
mappingFileName = UniqueFilenameGenerator.generate(request.getUrl(),
"mapping", filed);
bodyFileName = UniqueFilenameGenerator.generate(request.getUrl(), "body",
fileId, ContentTypes.determineFileExtension(request.getUrl(),
response.getHeaders().getContentTypeHeader(), body));
}
There's no easy way to do this at the moment. It is however possible. As #santhiya-ps says you need to write your own implementation of RequestListener, probably using StubMappingJsonRecorder as a template.
You can't extend it and override writeToMappingAndBodyFile as that method is private, but that is the method you probably want to change.
import com.github.tomakehurst.wiremock.common.*;
import com.github.tomakehurst.wiremock.core.*;
import com.github.tomakehurst.wiremock.http.*;
import java.util.List;
import static com.github.tomakehurst.wiremock.core.WireMockApp.*;
class NameTemplateStubMappingJsonRecorder implements RequestListener {
private final FileSource mappingsFileSource;
private final FileSource filesFileSource;
private final Admin admin;
private final List<CaseInsensitiveKey> headersToMatch;
private final IdGenerator idGenerator = new VeryShortIdGenerator();
public NameTemplateStubMappingJsonRecorder(Admin admin) {
this.mappingsFileSource = admin.getOptions().filesRoot().child(MAPPINGS_ROOT);
this.filesFileSource = admin.getOptions().filesRoot().child(FILES_ROOT);
this.admin = admin;
this.headersToMatch = admin.getOptions().matchingHeaders();
}
#Override
public void requestReceived(Request request, Response response) {
// TODO copy StubMappingJsonRecorder changing as required...
}
}
You can then register your RequestListener as so:
WireMockServer wireMockServer = new WireMockServer();
wireMockServer.addMockServiceRequestListener(
new NameTemplateStubMappingJsonRecorder(wireMockServer)
);
wireMockServer.start();
So long as you still store the mapping files in the expected directory (stored in FileSource mappingsFileSource above, which will be ${rootDir}/mappings, where rootDir is configured as explained in Configuration - File Locations) they should be loaded successfully as all files with extension json in that dir are loaded as mappings.
It would be much easier if StubMappingJsonRecorder took a strategy for generating these names - it might be worth creating an issue on the WireMock repo asking for an easier way to do this. I'd suggest getting an agreement on a basic design before raising a PR though.

Dataflow uploading file encoding error

My development environment uses Eclipse OXYGEN, Google Cloud Tools for Eclipse 1.7.0 installed.
I create Google cloud Dataflow Java Project.
There was a problem testing wordcount example.
When reading a file in the bucket, it will be output normally from the log as follows.
The problem occurs when you process data for WordCount and store the data in the bucket.
If you check the saved file, you can see the above picture.
Does dataflow not support Korean language?
here is my TextIO.write Codes
static class WriteData extends PTransform<PCollection<KV<URI, String>>, PDone>
{
private String output;
public WriteData(String output)
{
this.output = output;
}
#Override
public Coder<?> getDefaultOutputCoder()
{
return KvCoder.of(StringDelegateCoder.of(URI.class), StringUtf8Coder.of());
}
#Override
public PDone expand(PCollection<KV<URI, String>> outputfile) {
// TODO Auto-generated method stub
return outputfile
.apply(ParDo.of(new DoFn<KV<URI, String>, String>(){
#ProcessElement
public void processElement(ProcessContext c)
{
output = c.element().getKey().toString();
LOG.info("WRITE DATA : " + c.element().getValue());
c.output(c.element().getValue());
}
}))
.apply(TextIO.write().to(output).withSuffix(".txt"));
}
}
Most of the time, the correct coder can be automatically inferred, but if it doesn't, then make sure you're specifying a coder when reading data.
When you need to specify the coder, you typically need to do it when reading data into your pipeline from an external source (or creating pipeline data from local data), and also when you output pipeline data to an external sink.
For example, you can decode the data to read:
StringUtf8Coder.of().decode(inStream)

Where to place Rythm template files

I am having a weird problem with Rythm templates. Currently, I have these templates placed under
myPrj/src/main/java/resources/templates folder.
And all the Java source code is under myPrj/src/main/java folder.
When I try to render, sometimes Rythm is generating the XML file and sometimes I get the file name as is.
I have the home.template set to "templates" folder:
params.put("home.template", "templates");
String myTemplateString = Rythm.render("MyTemplate.xml", parameters);
Looks like Rythm is not able to locate MyTemplate.xml and resulting in emitting MyTemplate.xml as the output.
Can you please help me on how to solve this problem?? In addition, would appreciate if you can guide me on what should be the appropriate location to place these templates.
home.template is the configuration key to initialize template engine, not the parameter to render your template.
My implementation of your app looks like
public class App {
private static RythmEngine engine;
private static void echo(String msg, Object ... args) {
System.out.println(String.format(msg, args));
}
private static void init() {
echo("initializing rythmengine");
Map<String, Object> conf = new HashMap<String, Object>();
conf.put("home.template", "templates");
engine = new RythmEngine(conf);
echo("engine initialized");
}
private static void render() {
Map<String, Object> params = new HashMap<String, Object>();
params.put("foo", "FOO");
params.put("bar", "BAR");
String result = engine.render("MyTemplate.xml", params);
echo(result);
}
private static void doJob() {
echo("start doing real job now...");
render();
}
public static void main( String[] args ) {
init();
doJob();
}
}
The complete sample code could be found at https://github.com/greenlaw110/Rythm/tree/master/samples/demo_fo_SO_150529. Download the sample code and run mvn compile exec:java to see the result
It seems your problem lies within the path for the home.template. The example on their website might help.
If I'm not mistaken, you should use params.put("home.template", "resources/templates"); rather than params.put("home.template", "templates");.
Generally speaking, this kind of behaviour takes place any time Rythm can't find the template. I found it is best to check both, the path and file name. If necessary, simply use an absolute path to your template to make sure it points to the right directory. After you got the right path, you might want to change it back to be relative.

how to find whole path of Google Apps Drive Documents using deprecated apis 3.0

I am using Google Apps Drive APIs 3.0 . This API is deprecated but it is in maintenance phase.
I want to find a path of Google Drive document.
e.g. test/test1/test2/test3/testDoc.txt
As of now, I am able to retrieve all the documents but without directory path.
I want to show the whole path of a drive document.
I believe, there is no API to retrieve the whole parent path or parent link.
getFolders() method of DocumentListEntry is now deprecated is not able to show the folders path.
I investigated and found that there is one more method getParentsLink() which just shows immediate parent link. It returns List. On which I can not do re-iteration to find its parent link again.
public class MyClass {
private static final String DOCS_BASE_URL = "https://docs.google.com/feeds/";
private static final String DOCS_URL = "/private/full";
private static final String adminEmail = "admin#mytest.com";
private static final String password = "password";
private static final String projectKey = "MyProject";
public static void main(String[] args) {
try {
URL queryUrl = new URL(DOCS_BASE_URL + adminEmail + DOCS_URL);
DocumentQuery docQry = new DocumentQuery(queryUrl);
DocsService docService = new DocsService(projectKey);
docService.setUserCredentials(adminEmail, password);
docQry.setStringCustomParameter("showfolders", "true");
DocumentListFeed docFeed = docService.query(docQry, DocumentListFeed.class);
Iterator<DocumentListEntry> documentEntry = docFeed.getEntries().iterator();
while (documentEntry.hasNext()) {
DocumentListEntry docsEntry = documentEntry.next();
// Complex Logic to find whole directory path.(that I don't understand :P)
}
} catch (Exception exception) {
System.out.println("Error Occured " + exception.getMessage());
}
}
}
Any inputs are welcome.
Thanks.
To solve this you need to stop thinking in terms of folders and paths, and think in terms of labels (aka parents, aka collections). A file can (optionally) have one or more labels/parents/collections. Each parent can in turn have one or more label/parent/collection. So to get the "path" of a file, you need to recursively get the parents of its parent. Remember that a file can have multiple parents, each of which can also have multiple parents, thus a file can have multiple paths.
Taking your example "test/test1/test2/test3/testDoc.txt", assuming you have the ID of testDoc.txt, you can get it's DocumentListEntry, and call getParentLinks which returns a list if URLs for the DocumentListEntry of each of its parents, in your case just "test3". Get the DocumentListEntry for test3, and repeat to get test2, etc.
It might sound complicated, but once you accept that the thing you're calling a folder is not a container of files, but simply a property of the file, it makes more sense.

Mvvmcross binding file url to imageview

I try to bind imageview with local image file. In android, I can use setImageUrl to set image from a file outside resource folder. I read N+1 kitten example and try to use file url instead web url for my project.
The layout of image view
<Mvx.MvxImageView
android:id="#+id/advisor_message_picture"
android:layout_width="64dp"
android:layout_height="64dp"
android:layout_marginLeft="8dp"
android:layout_alignParentRight="true"
local:MvxBind="ImageUrl MessageImage, Converter = Image" />
The converter use to join file name and file directory url. Android view file will set the FileDir
public class ImageConverter : MvxValueConverter<string, string>
{
public static string FileDir;
protected override string Convert(string value, Type targetType, object parameter, CultureInfo culture)
{
return FileDir + "/" + value;
}
}
Update After the answer
I first copy or download to file to the Context.FilesDir.Path and check it with SetImageUrl, the image show up.
view.FindViewById<ImageView>(Resource.Id.advisor_message_picture).SetImageURI( new FileService(_context).CopyFileFromAssetsToStorage("image.png"));
Then I set the FileUrl of converter using same path and file name
ImageConverter.FileDir = FilesDir.Path;
In ViewModel
_messageImage = "image.png";
private string _messageImage;
public string MessageImage
{
get { return _messageImage; }
set { _messageImage = value; RaisePropertyChanged(() => MessageImage); }
}
It works now. The problem is I misunderstood the binding time of viewmodel
For Asset's you can bind using AssetImagePath using the ResourceLoader plugin. However, due to a sticky-fingers editing bug, this custom binding does currently need to added to your Setup - see https://github.com/slodge/MvvmCross/issues/372 for bug details
For files stored using the file plugin (which defaults to Context.FilesDir.Path - see https://github.com/slodge/MvvmCross/blob/v3/Plugins/Cirrious/File/Cirrious.MvvmCross.Plugins.File.Droid/MvxAndroidFileStore.cs#L39), you can use path directly.
For files stored in some custom FileDir determined within your app, you'll need to provide a path relative to Context.FilesDir.Path in order for the plugin to load it.
For further debugging, you could add breakpoints or trace to https://github.com/slodge/MvvmCross/blob/v3/Plugins/Cirrious/DownloadCache/Cirrious.MvvmCross.Plugins.DownloadCache.Droid/MvxAndroidLocalFileImageLoader.cs#L28 - or you could build and register your own IMvxLocalFileImageLoader<Bitmap> implementation that knows about your file paths.